Friday, July 10, 2009

Statistic module: sixth week update

This is an update of my work on statistic module for Melange.

Basically for the last week I studied JavaScript. It mostly included more advanced programming techniques like anonymous functions (function () {})() or closures, learning jQuery library and learning how it is all constructed in Melange source code. I am very grateful to Mario for all his explanations and for giving me links to some very interesting resources.

So my first JavaScript task was to make 'Task' button on the collect_stats page work in AJAXy way. Before that, it started stats collecting task and the page was reloaded which was completely unnecessary. I changed it so that the Task button generates XMLHttpRequest and also some message is shown after a task is successfully started. Work on that button took me more time that I expected. Anyway, it was my first jQuery job which I did mostly on my own. Before that week JavScript was rather like a strange Java to me and now I am exploring more and more advantages of it. I am sure that my efficiency will improve (and based on this week I am right :-)

Then I started to help with work on statistic dashboard page. Anyway, the first task that I assigned myself came out to be too ambitious. I wanted to fix a bug that when a widget with a pie chart visualization is moved, the pie chart disappears. I could not find a place in code which was responsible for it. The problem was also that only pie chart disappeared while everything with tables was fine. Fortunately this bug does not have a very high priority at this stage, so I put off working on it and moved on to some simpler but more important tasks.

On the backend side we mainly worked on integrating it with frontend. Some methods were added to statistic_chart view - they were needed for generating json responses for frontend ajax calls. Most of these methods were implemented by Mario and I hope it helped him to get more involved in Python side of module.

Apart from direct work on statistic module I created wiki page on introducing Task Queue API in Melange and how I imagine it. Thank you for all comments, I will try to work on that and improve it as soon as possible.

Wednesday, July 8, 2009

GSoC09: Statistics module 5th weekly report: backend/frontend ajax communication

This week has been focused on backend/frontend integration for Statistics dashboard and on helping with survey module bug fixing. Apart from that, I feel this week has been very productive because I've begun to work more actively on the Python backend, and similar beginning has been done by Daniel Hans, who begun an awesome commitment on the frontend Javascript side. I think this is the major step on our projects that have been done after we begun committing in the same repository and organizing the work with meetings and issue tracking.

About my work, the week started introducing Daniel Hans on the depth of advanced Javascript programming in general and, in particular, on the Javascript layer features and architecture for statistics module (and, well, if we'll come up with a good architecture, who knows, it could be the main JS architecture for Melange in the future :)).

On Wednesday I've pushed a new demo instance that was able to get a list of available statistics from the backend and created a new widget in the dashboard out of it. The biggest part of this work has been the generalization of the former code to have the statistic data joined dynamically from an Ajax call instead of hard coded stuff like before.

After that, I kept working on the integration to achieve a real ajaxy communication between frontend and backend. If only I could place a skeleton, then every feature afterwards would be easier to put in place. So, I've begun working on the data model for dashboard and chart and then on the ajax communication. In that way, a dashboard entity is now created for user at the first visit of the statistic dashboard page, and the correct entity is loaded if present.

On Friday, after I worked a bit on the Safari bug for surveys, I've met James Crook, and we talked also about the project, mainly focusing on what has been done and what can be done in the future with the current architecture. On Saturday, then, I've focused on the double quotes bug for survey project (which has been committed on the main repository).

On Sunday, at the end, I've helped fixing Safari 4 appearance bug (at least one of the two) for Survey's take.html template (committed on the main repository as well). About statistics module, I've added support to load/save widgets in backend chart entities. In that way, every time a new widget is created out of a statistic, it's automatically saved on backend. Every time you reload the dashboard the same widgets will appear on the dashboard.

Tuesday, July 7, 2009

GSoC 2009: GHOP - Sixth Week Status update!

Hi everyone,
Another productive week came to an end! As said in the previous week report, I started out this week by completing Bulk Approve and Publishing of tasks for Org Admins, which took over a day for making changes to the way Mentors list of db.Key items being rendered and displayed. This is already available in my demo instance at [0]

After completing it, I started working on Taggable-mixin. Taggable mixin's current architecture supports only 1 tag per Appengine datastore entity. But GHOP tasks require 2 tags per entity, one for difficulty and another for task type. So did some research on how to extend it. Finally, based on some suggestions from the developer of Taggable-mixin, Adam Crossland and Lennie, I wrote property wrappers for creating any number N, tags in the model. Have made all the necessary changes to the extended Taggable-mixin class. The core tagging framework now works as required, but still some code cleanup is needed. Along with that a cool UI for adding, deleting and editing tags must be implemented. Also the extended framework needs to be integrated with the Melange code base.

After this, I started working on Students side of tasks. Till now the UI to view and claim the task is ready. Students can also claim the task. The restriction to claim max number of simultaneous tasks as allowed in the program configuration is also done. Other minor details like storing student reference if already signed up and storing user reference are all done. Few things left out are, task header displaying if the task is already claimed or closed and enforcing a student to sign up before he claims the next task after completing at least one. I will be implementing them this week. Expect the demo instance to be up and running with whatever I have promised in a day or 2.

This week, I will be mostly concentrating on completing the student side of tasks and will be moving on to commenting infrastructure for tasks. I may also spend some time integrating Tags with Melange code base and student datastore entities conversion to support high school level students using Task APIs if everything goes well.

[0] -

Monday, July 6, 2009

Sixth Week Update

It's great to finally see surveys in the wild, and while there are a few rough edges, the feature should serve admirably, and the thorough refactoring that has taken place over the last week and a half highly reduces the amount of technical debt and will make it very easy to add new features and tweaks through subclassing/superclassing.

There are two outstanding requirements for surveys. And yes, they are indeed requirements, as if we are finishing construction of the railroad after the train has already left the station. Fun!

These requirements are 1) nagging notifications and 2) grade activation.

The SurveyRecordGroup entity kind that I've written about before makes both of these much more simple than they would otherwise be. At first, I was thinking that we could just use some logic at runtime to figure out whatever we'd need to send notifications for untaken surveys and to activate grades.

But after a plethora of hypothetical scenarios, it became apparent that unless we created a new entity kind tying together a given survey, a given project, and the status of that project before and after the survey, we'd open ourselves to all kinds of implausible but still possible bugs that might take much longer to find and fix if not nipped in the bud.

For instance, what if one mentor takes the midterm survey, and then the mentor has to drop the project for some reason and another mentor has to take the next survey? Or what if - as will likely happen for GHOP - there is not simply two surveys, but an arbitrary 'n' number of surveys, with an arbitrary choice of statuses for the project in question?

And what about mentors who must take a given Grading Survey more than once for several student projects? How would we avoid any complications or mistakes that might come up, such as giving the wrong grade to a project?

And finally, how could we be sure - positive, in fact - that we're not activating grades for a given set of surveys more than once? If all the students end up getting paid twice - or not at all - I *really* don't want to be the one responsible.

Luckily, SurveyRecordGroup solves this by using the status of the project when the survey is taken, and after the survey is activated. It's very simple and intuitive to use, such as in the following snippet:

# get SurveyRecordGroup for this survey and survey taker
program = survey.scope
for project in program.student_projects.fetch(1000):
this_record_group = SurveyRecordGroup.all().filter(
"project = ", project).filter(
"initial_status = ", project.status).get()

In addition to nagging notifications and activation, the third related feature that I'll be working on this week is a complete series of smoke tests, not that we need it right now, but they'll make it much easier and safer for someone to develop on top of the surveys and make sure they haven't broken anything.

After finishing up these final survey features, I look forward to being able to now dedicate much more attention the news feed that is my primary GSOC project.

I've already submitted patches of News Feed code and the Newsfeed Module wiki page is fairly up to date, but the current state of the news feed feature is a jumping off point, and I'd love to hear any ideas about news feed features that haven't yet been discussed. If you do have any, let me know!