Tenth Week Newsfeed Development Update

At the end of last week, I submitted a major refactoring of the newsfeed component, which now includes a separate subscription component that manages e-mail subscriptions (which can contain more private information than the public ATOM feed).

After receiving feedback on this latest patch, the plan is to commit the Newsfeed. Before I commit, one very important change is to taskify the sendMail method of the mail dispatcher. Obviously, this doesn't just affect the Newsfeed component, but it's very important for newsfeed subscriptions as I discovered after deploying and testing on my jamstage.appspot.com staging instance. The e-mail notification processing is done from within a task, and when a task returns a 500 error the task is repeated. Because the task often sends out many e-mails at once, I hit a OverQuotaError from this task, which re-triggered the task, and it wasn't long before my e-mail inbox was flooded with e-mails.

This has been a good lesson about working with the Task Queue API. It's good that a task will retry if it fails, but this means that only atomic (and idempotent) functionality should be contained within each task, so that if it fails the retry behavior will not bring about undesired behavior.

Once the Newsfeed is commited, I'd like to finish the UpdateLogic which is necessary for automatically compiling subscriptions. This means that we need a list of users with read-access. I describe the problem further in the NewsFeed wiki page:

The subscriptions logic contains an UpdateLogic class that is meant to automatically compile a list of users with read-access to an arbitrary entity. Unfortunately, setting subscriptions manually is the only option right now, since the automatic subscription handler is not nearly where I hoped it would be by now. The idea is very simple - just subscribe users to the entities where they have read-access. But the reality has been much more difficult to do without creating an entirely new infrastructure, and I'm entertaining the idea of simply relying on a manual interface (subscribe-by-star). The takeaway from this is that perhaps the access component should be abstracted to a high-level API, because it appears to have gained dependencies and complexities that prevent it from being useful to the news feed logic.

Finally, I plan to implement a simple PubSub example implementation, and do more in-depth testing (especially of edge cases, etc.)