GSoC 2010 Melange Testing Project: Week 12 (Final) of the Coding Period

In this week, I have cleaned up, commited and pushed all the tasks testing code to my online clone. I also worked on adding test cases for soc.logic.cleaning with suggestion of Sverre. Besides, I have also uncovered a couple of bugs in the soc.modules.ghop.tasks.task_update module and sent a patch to the mailing list. Lennard was very kind to help me merge the patch. I have also uncovered one issue with my test code for soc.modules.gsoc.tasks.accept_proposals (set the timelimit parameter too small) and then corrected it. In addition, I have also investigated the diference between the console and html coverage report. It seems that the console one counts statements while the html one counts lines.

When I tried to move the the gaetestbed and pymox modules to buildout dependencies, however, I found the easy_installed versions of both modules have problems (easy_install is used during Melange buildout). For gaetestbed, I found that the gaetestbed on PyPi is an old version which cannot check the html body of an email. So, I emailed the author to ask him to put the latest version onto PyPI. The author (JJ Geewax) was very kind to have helped me do it. For pymox, the easy_installed version available is a quite old version (0.5.0) which has a bug fixed in the later versions. Unfortunately, these newer versions are not available on PyPi, so not easy_install searchable. I reported the issue to the project issue tracker. One of its authors (Steve) was also very kind to helped me to put a newer version (0.5.3) to PyPi. After all these problems have been sorted out, there remains one issue that gaetestbed currently depends on webtest even though you do not need this part of function due to the design of gaetestbed, which the author agreed to resolve in the near future. After consulting Sverre, I finally moved the the gaetestbed and pymox modules to buildout dependenciesi and then commited and pushed the change to my online clone.

In the next week, I will add the experience of how to do mail and taskqueue related tests with gaetestbed to the testing wiki page, make a patch for removing the cron/job related modules and then send it to the list as well as add more test cases for existing Melange modules.

This is the last week of GSoC2010. It was a quite exciting and enjoyable summer. My project is very interesting, my mentor (Sverre), other developers (Lennard, Daniel, Madhusudan, Mario) and GSoC administrator (Carol) are very helpful, and my fellow students (Felix, Pavel and Savitha) are also very nice people to work with. Specially thank my wonderful mentor, Sverre, who is always ready to help me out when I got stuck and without whom I may not complete my project. During this period of time, I have also learned a lot about GAE and Melange, I have also come up with something useful and merged to the trunk, and uncovered several issues. Though the program has ended, it does not mean the end of my project or my involement in Melange. On the contrary, I will continue to work on the testing project after GSoC, i.e. add more test cases for more modules, improve the testing design and code, integrate Felix' data seeder, and so on, which have not been done because of the time limit of GSoC or dependency. I also hope to work for Melange beyond the testing project and hopefully become one core developer of Melange in the future. I also wish to come up some ideas to improve Melange so that I can serve as a mentor for Melange in the next GSoC and experience the excitement of these guys about the upcoming mentor summit :-).

Document Editor: Last week of GSoC coding

Last week of GSoC coding was devoted to... everything :)

I spent 4 days on form saving framework. First of all I had to fix JS code, which sends dirty fields to the server. The problem was as follows: framework was saving only dirty fields of the form. If you loaded the form for the second time, initial state might have been rewritten. This issue was fixed and now form states are consistent. Then I've implemented back-end. I needed to create model which stores temporary fields as a JSON dumped dict. Some logic class which manipulates all the stuff has been implemented. I also had to change createGet, createPost, editGet and editPost methods of the soc.views.models.base.View class to support temporary form saving. Temporary form saving for the model is enabled via "form_autosave" key of the params dict.

After form saving I started testing the code. An issue happened immediately. There was a problem with home documents for different models which derive from soc.models.presence.Presence. Views and logic were not aware of the revision control for the document, so exceptions were raised. This issue is fixed now. I also had to fix soc.models.seed_db.seed function to seed documents in the right way.

I've also refactored diff template, so it can now show arbitrary number of diffs for different revisable fields of the model.

At last I worked at the UI. I've added "Show revisions" button on the edit page for the model. Models that support RC are now distinguished by the "revisable" key of the params dict.

I've also rearranged fields of the Document and StudentProposal forms, so they look as supposed.

Final version of my GSoC contribution was uploaded to the appspot instance http://melange-editor.appspot.com

GSoC coding has come to it's end. And I must admit that it was quite an experience. I've already participated in large software projects and earn my living with development, but GSoC is really memorable. It was cool to participate in such a big and complex open-source project! I want to thank Melange dev team, Carol, fellow GSoCers and my mentor Daniel. They were very helpful and opened. And, you know what? I wish they let me continue contributing :) Come on, guys, I love it!

Document Editor: Week 11

The goals for the last week were to finish RC for StudentProposal model and start working on forms auto-saving. Revision control for StudentProposal is fully functional now. I also made some changes to decouple RC from the model's code.

After that I started working on form auto-saving. The main architectural concern was "what to store". The decision was made to store only changed fields of the form. Then I tried to find some existing solution to figure out dirty form fields. Mario proposed dirtyFields plugin for jQuery. It looked cool, but I found out that it triggers user-defined event for each field change. Furthermore it doesn't provide any means to retrieve collection of changed fields with new values. That's why I decided to develop my own script based on dirtyFields. It's now ready and pushed to my repository clone.

This week I will finish auto-saving for forms, write tests and make code clean-up.

Week 11: Social Features Project

This week I spent mostly on doing the Map View which basically involved writing mostly JS code to retrieve and then filter data based on search terms that are entered in the text input box. Initially, I had planned to allow users to filter based on location, projects etc, later I decided to just let them enter keywords. Internally, I check this keyword against tags as well as locations of users and events to filter.

This way, users do not have to specify what kind of search they want, they can just type any keyword such as 'Python' or 'Australia' to get the relevant filters.

The Google Maps API gave me a bit of problem as it took sometime and debugging to realize some method names have changed completely such as for adding/removing markers etc between V2 and V3, and I had to accordingly change my code.

The latest version with all three features is on my instance.

I have quite a bit of things left to do in the final week of GSoC:

• write tests
  
• write code to catch the exceptions(Application Error) being thrown by Calendars API and re-send requests.
• test the functionality and workflow of user pages, calendars and maps view for bugs and fix them.
  

GSoC 2010 Melange Testing Project: Week 11 of the Coding Period

In this week, I have uncovered a couple of bugs of existing Melange modules with the help of related test cases and sent pathes to the list, which again proves the importance of testing. I have also resolved the error "raise self.__TranslateError(e.application_error, e.error_detail) UnknownQueueError" by adding the path of taskqueue yaml setting file to the instantiation arguments list (root_path) of taskqueue_stub.TaskQueueServiceStub. Besides, I was also looking for a good way to test the mailing and taskqueueing functions in order to test that the correct mail has been sent out and the correct task has been scheduled. During my search, I found gaetestbed which is very simple but serves the purpose well. Although it is not hard to make an equivalent one myself, why not reinvent the wheel if you can make use of an existing one. Its licence is also very generous (GNU V2). So, I added it to Melange tests and also extended its MailTestCase and TaskQueueTestCase and then put them into tests.util module. For MailTestCase, I overrided its assertEmailSent method to add two functions

1. It will print out all sent messages to facilitate debug in case of failure.
2. It accepts an optional argument n which is used to assert exactly n messages satisfying the criteria are sent out.

Later, I plan to add the functions of checking the CC field, allowing to partly match the subject field or even supporting regular expression. For TaskQueueTestCase, I have not added any functions yet. I further used them in my mail and taskqueue related tests, and cleaned up some of my existing testing code and then commited and pushed them to my online clone. In addition, I have also done some new tests on gsoc tasks and statistics etc. and managed to increase the testing coverage of Melange to 59% and the coverage of almost all modules except one (with 39%) over 40%. In fact, the coverage report of the html version which apparently uses a different algorithm from the console one shows the overal testing coverage is already over 70%.

In the next week (the last week of GSoC2010), I will clean up code and commit and push all uncommited testing code, increase the overall coverage printed in console to over 60%, and make the coverage of most modules over 50%. In addition, I will also move the the gaetestbed and pymox modules to buildout dependency and add the experience of how to do mail and taskqueue related tests with gaetestbed to the testing wiki page.

GSoC 2010 Melange Testing Project: Week 10 of the Coding Period

In this week, I worked on some ghop modules with low coverage, e.g. soc.modules.ghop.tasks.task_update and soc.modules.ghop.logic.models.task and managed to increase the testing coverage of Melange to 57%. About the problem that there is no api proxy stub available for service "taskqueue", I found that there is one taskqueue_stub in google.appengine.api.labs.taskqueue, so I registered it to 'taskqueue' service with apiproxy_stub_map.apiproxy.RegisterStub in the testing environment. However, it (taskqueue.py) raises another error, "raise self.__TranslateError(e.application_error, e.error_detail) UnknownQueueError" which is a bit strange. This is the reason why I have not yet checked in the task testing code.

In the next week, in addition to solve the above error and increase the testing coverage of Melange to over 60%, I will also try to find a good way to test the mailing and taskqueueing functions and make sure the correct mail has been sent out and the correct task has been scheduled.

Document Editor: Week 10

As I've already mentioned in the previous post, there was a problem with versioning more then one property of the model. Daniel proposed to store necessary fields as a JSON-formatted dictionary. On tuesday I reimplemented RevisionContent model and several logic methods to support new architecture. Document model, logic and view were refactored too.

Then I started to implement revision control for StudentProposal model. There were several minor issues, but everything is fine now. Revision control for student proposal is almost fully functional now (diff template and delete logic are left). New version of Melange was uploaded to my GAE instance.

On tuesday I plan to finish revisions for student proposals and start implementing auto-saving feature.