Thursday, June 4, 2009

Google Summer of Code 2009: Statistics module weekly report

So, here we are up and running! The Statistics project, which I'm really happy to share with Daniel, aims on collecting and displaying statistics data out of Melange.

I've spent my first week mainly on research and on the design of the Javascript layer that will be used for the Stats module, studying and testing libraries that will be of some use (hopefully) generally to the whole Melange project, too. The first thing I've done has been writing some code to get a first chart with Google Visualization API using Google API Playground, then I worked on studying how to use jQueryUI to get an iGoogle style dashboard interface for statistics module, finding this great tutorial from NetTuts, that could lead us to a cool interface. Then I had a meeting with Daniel, who had a great patience on giving me guidance on the Python code he started to write for the module backend, and we discussed on how to join our code, then Daniel set up the bitbucket repository he mentioned in his post.

During that meeting and afterwards, I've found Gviz lib and we agreed with Daniel that it would have lead to a smoother code for the communication between Python and Javascript. That library outputs "ready-to-serve" JSONs to Google Visualizations. Now it's already integrated in the main Melange trunk by Pawel for us to use it.

From that library on, I've got to know Google Visualization Query Language, which I thought it would be very powerful for us to show dynamic charts based on dynamic data. Unfortunately, there are no implementations for Python, so I tried to find something to get similar power in JavaScript. In that way I've found three great projects to query over JSON objects: JSONPath, JSONQuery and jLinq. I really like the functional way in which jLinq works, but I'm going to explore in the next week also the "XPath-like" ways, to have a better view on what's out there on JSON objects querying and what could be better for us to integrate.

Apart from that, I've started threads in the dev list and then put in the wiki some thoughts and contributes about a "medium-term" refactoring of Melange JavaScript layer and a draft proposal for JavaScript code style guidelines.

This week I will explore (or, better, I'm exploring, as I'm posting the report late :)) the JSON querying libraries, write on the wiki the design of the JS layer, and, hopefully, get a first skeleton code out of that design.

Newsfeeds Status Update

The Newsfeed concept is very simple - stream updates about things on the site, link to them, and offer ways they can be viewed outside the application.

The Newsfeed logic works by using a onCreate, onUpdate, and onDelete hook to create and save new feed items for a group of receivers, like programs and users that are associated with it. On the view side, these feed items are loaded, cached, and rendered.

There is then a simple HTML template that creates a table containing each update, with a link back to the sender entity, the type of update, and a timesince property. It can also render a payload for each feeditem, which can be any HTML or widget.

The receiver logic needs to be hardcoded for each entity type (or sometimes, groups of subclassed entities). I spent a few days trying to make a universal logic component, but the linkable scope property isn't designed for this kind of threading between entities. So the FeedItem model each stores a sender_key and a receiver_key. A one to many relationship via PolyMorphism would also be possible, but wouldn't necessarily be advantageous from a performance perspective.

The next tasks I'll be working on are tests (and tests for surveys), and then feed export functionality. I'd also like to do some refactoring to make it easier to subclass newsfeed functionality and to propagate "waves" of newsfeed updates as events bubbling up the heirarchy.

Speaking of which, it would be a lot of fun to make a Google Wave integration...

Tuesday, June 2, 2009

My first week update


here is my update on what have been done for statistic module during the first week (May 22 - May 28) of Google Summer of Code. Sorry for posting so late, but as I said on our conference call, I had to prepare a presentation for my classes and was quite busy. As always, when I have a few months for something, I end up doing it on the last night before deadline:-P
Anyway, these days, I worked mainly on backend side of the module.
First of all a new model for keeping stats was designed. It is placed in soc.models.statistic and is designed so that statistics can be collected in batches.
After that I implemented a new view for managing stats. This site just lists all statistics defined for a given program and allows user to collect them batch by batch. It uses standard Melange 'list' view.
The next view displays field values for a particular stats. It does not use any visualization, but shows only raw json strings for collected statistic. It uses standard 'public' view.
Some statistics are very similar to each other in their nature, so on the logic side, I aimed to implement some kind of framework that would make adding similar stats as easy as possible. Some instructions on how to do that have been put on wiki.
Aside from coding, an important thing was my meeting with Mario. I explained him the code I wrote (or I tried, but probably was not so good:) and we discussed some issues about displaying statistics to the users and Mario told me about a few technologies that he is planning for us to use. Jlinq seems to be very powerful, but probably Mario will describe it better:)
We also set up a repository to which we will be uploading our code changes. You can check it out at

Google Summer of Code 2009: First week - status update!

Hello everyone. I wanted to make my first blog post here very special, by narrating the entire story of how I got into Melange development and why I love to work with this brilliant team of Melange developers :P However time constraints (University and other work) never allowed me to write it till now. I will be writing it as soon as I find time to organize the contents for it.

Anyways coming back to the topic, first week status update(23-30, May, 2009). As planned in the schedule, I completed writing code for all the models. This was mostly a translation of the model design document on the wiki, to Google App Engine model code. This is already reviewed and committed into the code base. The new models I introduced were:
  1. ghop.models.task.GHOPTask
  2. ghop.models.program.GHOPProgram
  3. ghop.models.organization.GHOPOrganization
  4. ghop.models.comment.GHOPComment
  5. ghop.models.timeline.GHOPTimeline
  6. ghop.models.work_submission.GHOPWorkSubmission
  7. ghop.models.org_prize_assignment.GHOPOrgPrizeAssignment
I was slightly ahead of schedule, but University monthly tests put a break on the pace. Took a break for 2 days for tests and 1 day for travelling. I am all set now, and hopefully I don't have any other commitments in between except end sems.

I also started working on Views in the mean time. I started with Views for the Program and the Program Admin. I am currently writing the view for GHOP Specific program configuration for Program Admin. Also at the same time, discussions on Melange modular architecture is done and some concrete work is in place. So I am working on redesigning in app/soc/views/helper and making the GHOP a pulg-gable module. So a bit of code re-organization is happening too.

Thats about the previous week. This week (31 May - 6 June, 2009), I will be completing the code re-organization. I will spend most of time working on Program Admin views. Also taggable-mixin code has been checked into the repository. I will take a look at it, and if everything goes as expected, I will complete adding tags support in GHOPTask Models for GHOP Tasks. And finally, if time permits, I will start working on Organization views.