Melange Functional Testing

Hi,

This week i did a little bit reading on nose test runner and other articles on functional testing. I was travelling to my home town so i was not able to do much this week, though i have started work on creating test script for Profile Page.

I am learning a lot of new things and that makes me feel good. I am continuously making changes to base module to make it much better. I hope that in few days base module will be good enough to be accepted. As soon as this module will be accepted, i will start sending test scripts. Instead of using TC01 and TC02 for naming test data sheets. I am planning to give them more meaningful names like Gsoc_Student_Registration and Gsoc_Profile. Leo has reviewed my first patch and suggested few changes, i am currently in process of making those changes to make the base module much better. After Profile page, my plan is to create test script for Search page and Events & Timeline page.

GSoc 2011 : Melange Testing Project CP W12 Last Week

Hi,

Only a week is left for GSoC to end and I need to finish testing all the important views as soon as possible. I had been working on views.org_profile, profile_show, accepted_orgs, proposal_review, duplicates and proposal_review this week. Testing views is turning out to be a little more involved and complicated task as compared to testing the logic. I got stuck at some places and so could not submit them for review. I had thought that I would be able to finish the views much earlier and start functional testing but I was not fast enough and also I could not devote 10+ hours per day during the weekends as I had promised in the proposal due to heavy course work, classes and personal issues.

      I will be trying to commit as much tests as possible in this week and work on the remaining tests after GSoC. I aim to work on functional testing and buildbot after GSoC. Lets hope for the best :).

GSoC 2011 Melange Testing Project: CP W11

Hi,

I worked I tested gsoc.views.duplicates module. I have not submitted my tests which I wrote for org_profile, profile_show and student_forms for review as there is a need of some access checks to be defined new for the respective module in the codebase. I was rechecking my tests for org_profile and I felt that the timeline should also be checked if a program is active or not. I put this in the mailing list and Lennard suggested that isProgramActive be renamed to isProgramVisible and a new method isProgramRunning be introduced. Those pages which remain active after gsoc can include isProgramVisible in their access checks and which are current program specific can include isProgramRunning. I submitted the patch for the access checks and Lennard and Sverre reviewed them. I will be accommodating the changes soon.
    
There was also a problem with our coverage module which is outdated and so it does not work with the latest Nose 1.1.2. I reported the issue with Nose and they suggested that we use coverage > 3.0. Leo tried including it in the buildout dependencies so that the coverage module is always the latest but it too did not work. Just replacing the old coverage module with the latest coverage modules works fine.

GSoC 2011 Melange Testing Project: CP W9 & W10

Hi,

I could not write a blog post for the work I did during last week because of the
the university work and some other factors. This post summarises the work I did in last two weeks.

  Last week, I fixed some tests which were failing due to the recent change done to the mentor property. The tests for soc.modules.gsoc.tasks.test_survey_conversion failed because the models were seeded(Model.put()) by explicitly defining the model properties. This method is definitely not a robust one since if we happen to add other required properties in future, the tests will fail again due to the missing properties while seeding. The solution is to use the data seeder which randomly seeds the properties not mentioned in the properties parameter. If there is no data provider for a property, we can patch the data provider to seed such properties and the tests would run fine. So, I edited the tests for gsoc.tasks.survey_conversion and made them to use data seeder to seed the models.

     I also fixed the profile_utils.createStudent() method which had been bugging me while testing gsoc.views.student_forms. My tests failed 1 out every 7-8 test runs. It was because of the failing access checks which resulted because of the number_of_projects property being seeded other than 0 for a student without a project. This problem bugged me for quite sometime and Madhu helped me to figure out that this was actually the problem. Also, I submitted some patches to remove the unused imports in some modules in gsoc.views.

   This week I tested a couple of views namely views.org_profile, views.profile_show and submitted them for review. Leo has reviewed them and I will be fixing them today. I also added createInactiveProfile() and createInactiveStudent() method to profile_utils which I needed while testing views.org_profile. I discovered a defect in DownloadForm request handler in student_forms and I filed a issue for that. The defect was that a student without a project should not be allowed to download the enrollment and tax forms and in the present case a 500 error is returned which should be 403 instead. As told by Sverre, I need to add an access check to fix this.
 
    Since I get very less time during the week days to work on the project, I will be working on Friday, Saturday and Sunday as I don't have classes on these days.

GSoC 2011 Melange Testing Project: CP W8 Mid-term

Hi,
I have completed the logic tests and the task for post mid-term period is to write the tests for all the gsoc.views modules. Leo and I had a discussion after the last conference call and we decided to prioritise which modules should be tested first. A survey of the modules which have tests and which needs to be tested can be found at [0]. Accordingly, I have to first test gsoc.views and gsoc.tasks and then go on to write the functional tests. I am presently working on gsoc.views.student_forms and stopped working on soc.views.helper.access_checker.
I have no experience with Django and I spent some time studying the basics and  then asked Madhu some doubts that I had got while studying the view's code. I will develop pace as I get experience with the views.
    This week was also assigned for mid-term evaluations and Leo has passed me :). My classes are starting from 20th July and I need to manage both gsoc and classes. The last two days went in travelling to the university and the next two days will go in registration and shifting to the new hostel room.

[0] https://spreadsheets.google.com/spreadsheet/ccc?key=0AuENNrqFbxWEdDJMclZySzFUalpsUTd6bzNJTjJGVlE&hl=en_US#gid=0

GSoC 2011 Melange Testing Project: CP W7

Hi

The task for this week was to correct my previous tests and finish the remaining tests in soc.logic and soc.modules.gsoc.logic. I have corrected my previous tests and submitted them again for the review. I have tested all the logic functions which are used at least once in the code base. I have not tested the soc.logic.allocations, soc.logic.lists and soc.logic.helper.notifications because these are not in my priority list for now as I need more familiarity with the code base to test them correctly and efficiently. I will come back at some time in future to test them.

       I have started attempting to write the view tests. I have never written any view tests earlier and my experience with Django is 0.1, but sometimes we have to learn the hard way. I studied many of the new tests written by Sverre. Testing a view seems to be a complicated task and I did not understand many things which I had to ask Madhu. Just before writing this post, I had a discussion with Madhu and I asked him what 'Context' is, what tests_utils.StuboutHelper is for, what soc.views.helper.access_checker.Mutator is used for and a load of other topics. At the time of writing this blog post, I was testing the soc.views.helper.access_checker. It will consume more time to test because there are more than 1100 lines of code to test. After testing this, I will test the gsoc.views.helper.access_checker module.

      I have the target of completing all the view tests before the final evaluations. It will require a huge amount of effort which I am pouring in. Lets hope for the best. :)

GSoC 2011 Melange Testing Project: CP W6

Hi,
In the last conference call, I reported my problems to Sverre that most of my time is spent on reading the code itself and that I had problems in deciding about the use cases when writing tests. My productivity has been very low till now and I have written atmost two tests per week. This week I worked hard and I have written tests for all the modules in gsoc.logic and soc.logic except three modules. Though the tests are under review and they need drastic improvements, I have now become comfortable in deciding about the use cases and writing tests. I had a discussion with Leo and he told that I was unnecessarily using DjangoTestCase. I can use seeder_logic and inherit my test classes from unittest.TestCase and this will improve the performance and speed. I was using DjangoTestCase to avoid instantiating the GSoCOrganization, GSoCTimeline and other entities which I could use directly when inherited from DjangoTestCase. But seeder_logic is more useful and its also used in DjangoTestCase. So, I corrected the test for soc.logic.host and used seeder_logic to seed the entities and it was working fine.
     
I tested a total of 8 modules in this week and the coverage has increased by 4%. The present coverage of the tests in my local copy is 70%. The coverage will increase more when I write tests for the soc.views. 

My work for this week is to correct the previous tests, test the remaining three modules in gsoc.logic and soc.logic and then start the view tests.I hope to finish all the logic tests before the mid-term evaluations.

PS: Mercurial queues are AWESOME. Salute and gratitude to the guy who wrote that extension. :)

GSoC 2011 Melange Testing Project: CP W5

Hi,

This was quite a busy week. Learned a lot and worked a lot. I worked on writing the tests for soc.logic.tags and soc.logic.models.base. I had attempted to test soc.logic.tags during the pre-GSoC era but failed because I did not understand properly what 'scope' and 'fields' parameters were. I asked Madhu what a 'scope' is and he explained it to me that it was a solution adapted by Melange during the early days of development to define a relation such as a GSoC project belongs to a Student. So, a GSoC project entity was scoped under a Student profile. But, as the developers understood more about the Transactionality and other stuff in GAE, they switched to using 'parent' argument of an entity to define the scope. But, scoping will stay for tagging.

   Earlier during the week, after Leo reviewed my patch for soc.logic.accounts, I pushed the code without first updating my local repo. So, I first pushed the patch and then pulled. But, then I could not find the test I had written in the online repository. So, I pushed it again and as a result to this, I ended up creating two heads in the repository. Sverre asked me to take care of the extra head I had created. I did not know how to do that. So, I googled and asked some friends. Some suggested using the 'strip' extension, 'merge' and some 'mercurial queues'.
Ultimately, I asked Madhu how to revert what I had done. He looked at the case and asked me if I do not rebase my patches after pulling. rebase!!!???What is it??. I then asked him what rebase here refers to. He gave a very nice tutorial on the concepts behind rebasing and the two schools of thought which I will never forget, the 'Purists' and the 'Whatever-ists'. :). I like Madhu's way of explaining things. Things get so easily printed in the mind.

   What I learnt about the 'parent' argument from the discussion with Madhu, helped me to extend the old tests for soc.logic.models.base authored by Sverre and Leo. The tests reported errors when I ran because it could not find test_model module which I also thought does not exists and may be it has been removed. So, I defined my own dummy class TestModel extended from soc.models.base.modelWithAttributes, added some more tests to the module and submitted them for review. After this, when I was looking for other module to test, I saw that there was a test_model in old_app.soc.logic.models and this was the reason why it gave ImportError. So, lets see what Leo says :).

 I now plan to re-use the old tests which Leo had written last year. The goal of my project has been re-scoped to write the unit tests until mid-term evaluations. I had written in my proposal that I will start writing functional tests in June. But, there are modules which have to be unit tested before. So, the target until mid-terms is to add as much unit tests as I can.

GSoC 2011 Melange Testing Project : CP W4

Hi!
I had submitted two patches last week: one for test of app.soc.logic.dicts and the other for app.soc.logic.accounts. There were lot of mistakes and issues pointed out in the review. I have corrected soc.logic.accounts and submitted it again for review today. I also missed some functions to test in soc.logic.dicts and will do it this week.
  I wrote the test for app.soc.logic.system this week and I had a long discussion with Madhu whether to use DjangoTestCase or unittest.DjangoTestCase. Madhu suggested using the DjangoTestCase and I insisted that I could test the functions with the help of unittest.TestCase itself. I had to test the soc.logic.system.getHostname(data=None) function and for this I needed a data object which had a 'site.hostname' attribute. After some grepping and exploration, I came to the conclusion that it should be a soc.views.helper.request_data.RequestData() object and the 'site' should be a soc.models.site.Site() entity which Sverre told me. But then I had a problem in instantiating the Site entity and the traceback reported that a link_id property is missing. So, after struggling for some time I told Madhu the problem and he referred me to test_utils.DjangoTestCase which had a site and data attribute which he said I use in my tests. He also said that RequestData entity should be populated and seeded to efficiently test it. My point was that all the entities of RequestData object are set to either False or None and also I do not need all of its attributes in testing. I only need the data.site and data.site.hostname attributes and this I could accomplish without using the DjangoTestCase and directly handling the RequestData and Site objects. Finally he asked me to go ahead and write the test with unittest.TestCase.
    85% of my time is spent on reading codes and there are multiple windows that remain open almost all the time. I have to hibernate my system to avoid opening so many windows again.
   I did not do much coding this week but hope to do more this week. Me and a friend of mine are planning to participate in ACM-ICPC regionals this year. So, we spent most of the time in studying for ICPC only.

GSoC 2011 Melange Testing Project : CP W3

Hi,

For quite a long time I had been working on the online test runner and could not make it work on a live instance. As I had discussed it in my last blog post about my implementation that I intended to exploit some plugins of Nose to run the tests, it failed because of the reason that Nose uses some libraries which are restricted by GAE. So Leo asked me to take my other approach of discovering all the test modules in the tests directory and then make a suite of tests using the unittest.loadTestsFromName  method. I used the os.walk() built-in function to first list all the modules with their absolute paths and then changed the paths into a dotted notation so that these paths could be easily used by unittest.loadTestsFromName to load the tests. I have used a lot of code from the gaeunit source specifically which is related to the browser interface and the test runner. The test runner had to be modified so as to fit in with melange code. I also had to include the tests directory, gaetestbed, nose, webtest and  mox libraries in the app folder so that tests can easily use them when running on a live instance.
    The online test runner runs on my local development environment but it shows errors that are due to the bugs in the runner itself. Below is the video of the melange tests running on my local machine through the browser.

I have stopped working on the runner as of now. But in the end, I discovered that I have extended some of the features of gaeunit. It has been 2 years since gaeunit was last released, so I was planning to contact the gaeunit developers if I can develop for them.

     During the last week conference call, Sverre asked me to treat the online runner as a side project and concentrate on my testing project now. So, I have started working on my project and wrote the test for app.soc.logic.accounts and submitted it for review. At the time of writing this post, I have also almost completed the test for app.soc.logic.dicts with a few methods left to test.

GSoC 2011 Melange Testing Project : CP W2

My task for this week was to implement an online test runner which enables us to run our unit tests online rather than on the local machine. I have written the necessary modules which provide a very basic functionality and features for now. More features can be added and improved with time. Leo had suggested me the gaeunit Open Source project to study and see how they do it.

I outline the basic process of how gaeunit does the magic :-

• gaeunit defines two main request handler classes - MainTestPageHandler and JsonTestRunHandler. The request to run the tests is handled by the MainTestPageHandler class which first validates the URI arguments and renders an html /plain report according to the 'format' argument. Our interest lies in the generation of the html report.
• When the request is to render an HTML report, gaeunit first creates a suite of all the tests found according to the 'name' parameter passed to it in the URIs. 'name' can be any package, module or class. This suite of tests is then converted into a JSON format with structure {"module_name": {
  "class_name": [list_of_methods]}} and passed with the HTML response to the browser.
• As soon as the response is loaded in the browser, it triggers a javascript whose task is to iterate through the JSON object and pass the (module_name, class_name, method_name) tuple to the server by generating an asynchronous XMLHttpRequest which then waits for the data to be sent by the server.
• The GAE server upon receiving the tuple runs the JsonTestRunner and the test result is collected in a unittest.TestResult object. This TestResult object is then again converted into a JSON object and returned to the browser. 
• The browser then sends the data to the appropriate fields in the web page

My Implementation:

My implementation does not work presently and it needs lot of debugging. I moved the tests folder to melange/app/ so that GAE finds it easily. I wrote two modules online_run.py and online_tester.py. online_run is colleague of tests.run.py with lot of code borrowed from it to implement the test runner for online testing.

      online_tester implements classes to handle and manage all the tasks right from initiating the tests to printing the test reports. I thought of exploiting the 'testid' and 'collect' plugins of nose. Running the tests with these plugins only collects the tests without actually running the tests and assigns an integer id to all of them. We can run a specific test next time by just passing the integer ids. So I have defined three classes MainTestPageHandler, PrepareForTests and RunTests. MainTestPageHandler just renders a view where users can select which tests to run. The arguments are passed as POST arguments to the PrepareForTests handler which is supposed to collect the tests and assign IDs to them. The number of collected tests is then passed to the browser with HTML source which contains a javascript . This javascript passes the integer IDs to RunTests which makes use of the runner in online_run module to execute the tests and returns the HTML output to response.out stream.

This implementation is not working  and I asked Madhu for help. He said may be I have messed with the appengine paths. I have to try again keeping the tests in app/ and making the necessary changes in my scripts.

Also the paths like '/home/praveen/testmelange/app/tests' which I have included in the python paths according to my system will not work on GAE because we do not know about their file systems.

You can take a look at the error reported by my implementation at my instance[1]
Just choose 'Run all tests' and hit Run.

[1] http://melangetesting.appspot.com/runtest
[2] online_run.py    http://tinypaste.com/da4b5e
[3] online_tester.py http://tinypaste.com/1f3b88

GSoC 2011 Melange Testing Project : Coding Period Week 1

The coding period has officially started and I have been rather slow and falling behind the schedule. I still have not started writing a test but I will try to give more time to the project in a couple of days. 
    It was quite an eventful week. I have been waiting for the welcome package by google and it was already May 23, so I just asked on the GSoC students mailing lists if anyone has already received it. Some of the students were very furious about starting such a thread as Carol had stated the deadline to be May 27. What I believed was that its not necessary that everyone will receive the package after 23. I also made a remark of why he bothered to read and reply if he was not interested. I was surprised to see that only one or two previous GSoC students from the 4k members strong mailing list care to answer constructively. It was encouraging to read Phillip Herron's comment which I quote below:

"Just be patient they _will_ come from last years GSoC it just takes
time if you consider the fact they have to individually create like 1000
of these things and get them each shipped to each of our address's
just be patient. Its a great feeling when it comes :) "

This is the reason why I love Melange. You are never restricted but re-directed if you are going wrong. I still remember when Mario told me that its bad collaboration when I commented a little offensively on the efforts of an aspiring student during the pre-GSoC time. I am also amazed by Carol's patience at answering the same  set of questions again and again asked by students on the gsoc mailing list. Anyway, I learnt from the criticism that I should be patient and keep good relations and collaboration with fellow students.

    I am going slow at the project and falling behind the schedule. I have not written a single line of usable code till now. My project is concerned with writing tests for the logic and views  and as I stated in my last post that I shall start with app.soc.logic modules, so I have generated a directory structure of app.soc.logic which is basically a tree like structure of the files in the directory. All the previous tests have been moved into tests/old_app directory as they conflicted with the new framework. My aim is to re-use the old tests and make the necessary changes with respect to the new framework. I will start with the app.soc.logic.allocations.py  module.

    I had a meeting with my mentor Leo yesterday i.e. on 25 May '11 and I discussed with him a few concepts related to testing.I asked hime what should the basic approach be while attempting to write the tests. Basically, we should first understand what a specific function does, think about all its use cases and then we test each of the use cases.
   I also have to implement a view to run the tests online rather than running the tests locally. I did not exactly know how to implement this. But Leo suggested that there are already some open source projects like gaeunit which do that. So, I referred to the gaeunit project and its just a single module which implements that. My approach was to run the tests using the python environment of GAE and somehow capture the test results but there is already a TestResult class in unittest module which returns an object for such manipulation. So, I need to make use of this class and tasks to implement my view.
   So, my goal for this week is to write the view and implement the test for app.soc.logic.allocations.py module at least.

 

GSoC 2011 Melange Testing Project : CBP

About one week is left for the start of the official coding period. I have been unable to concentrate much on my project as there were end-semester exams till 13th May and after that I was busy in packaging all the stuff as we have to empty our rooms before going for the summer vacations. I played with my ACM account and explored all the facilities that I could make use of. I found many good books related to programming in the Online Books section. It would have been great if there were more video tutorials available for the students from those which are available for professional members.
I had a meeting with my mentor Leo on 16th May and we discussed about the project and I have set coding plans for the first three weeks during the official coding period. I will start with the tests for soc.logic which require a basic knowledge of Django and GAE. Leo suggested reusing the old tests and making changes according to the new framework.
I have not much to write as I have not done anything significant. I have settled down now and seriously started working on the project today. My next post would be more concrete and specific.

GSoC 2011 Melange Testing Project : Community Bonding Period

Its been two days since the GSoC 2011 results were declared and I am still unable to tell myself that I am now a Google Summer of Code student. :) . A vote of thanks and gratitude from the very depth of my heart to the Melange developers.

Python is awesome and I am now in love with it. My project is on testing and frankly speaking I have never studied about software testing before. The only idea about testing which I had was during solving exercise problems proposed in Google's Python Class videos. The coding problems had to be solved and then their output tested.

I explored about the Melange testing project more and more. I did not understand many of the terms like 'Test Driven Development', 'Continuous Integration' etc mentioned on the Melange's ideas page, but I googled and learnt about them, followed online tutorials and resources linked on the ideas page. I joined the melange-soc-dev mailing list and asked Sverre if I understood correctly what the project is aimed at. He said 'Sounds about right :) ' and then he pointed me to the 'Getting Started' guide. Since then there was no turning back and I devoted most of my time getting familiar with the framework and the testing modules. I explored the Melange's blog and read the posts of previous GSoC students. Leo and Savitha very well explained how they tried to understand the codebase. I did not know about the tools like tree and pylint.pyreverse before.

Then I decided to write a test for one of the modules in app.soc.logic and chose the 'validate' module. The test was simple but difficult to write if one does not know about the organisation of the modules and how the functions were used. I took me two days to get a feel of the code and get familiar with the directory structure. I finished the test, learnt and fixed the style issues and finally signed the CLA and the module was integrated into the codebase. That was my first contribution to Melange as a pre-GSoC exercise. This gave a boost to my confidence to contribute more to Melange.

I knew there were many students competing for the project and I decided to write my proposal differently so as to have an edge over other students. I decided to show a graph of the failures and errors that occurred everyday, the data which I collected everyday by running tests over a period of ten days. I also thought of showing a test code vs production code analysis. April 8 was the last date of submitting the proposals and I was busy with exams till April 5. Melange was the only organisation on which I had concentrated since the beginning and I knew I had to write a good proposal to get selected and dedicated all the 3 days in writing the proposal. The 'graph' and 'the code analysis' do not reveal anything groundbreaking but that was what my mind could think of then.

I regularly followed all the discussions on IRC, melange-soc-dev and gsoc lists. There were criticisms about the tinyMce editor and asked Sverre if the editor size can be increased because it seemed too cluttered. He asked me to do it. I read the tinyMce documentation and found many more helpful buttons can be added to the toolbar. Mario helped me in writing the configurations for the 'basic' and 'advanced' editor settings. I had never learnt jQuery before and I believed that I should not say " I don't know jQuery, so I can not do it". Instead I followed some tutorials and then it was easy. I also thought "Hey! you applied for the testing project and you are working on the editor."

It was a great experience working hard to bag the project. It is the first time I am contributing to an Open Source project in a major way and to see my name in the Author's List, Mario's Code Swarm video and the commit messages fills me with a feeling that I find difficult to describe.

Yesterday I attended my first conference call with the Melange developers and heard the sweet voice of Carol :P . I thought the meeting would be very formal but all of them were very jovial. Sadly, I could not introduce myself to other members as the mic was not working. Hope to fix it soon so that I can communicate during the meeting on Monday. I have exams starting Tuesday next week, so will discuss about the project plan with Leo after 15 May 2011. I will get a week of time before May 23 to read about GAE, Django and Software testing.

Melange community is really the 'awesome-st ' community and I wish to meet the developers and have fun with them at least once in my life. :) :)

GSoC 2010 Melange Testing Project: Week 7 of the Coding Period

Last week, I found that Python coverage does not track the properties definition of Melange models which makes the coverage of models still quite low. So, I first tried to exclude models from coverage report. This is done by stubbing out nose.plugins.cover.Coverage.wantModuleCoverage with a method which offers the option to skip packages for testing coverage report through pymox stubout. Later, however, I remembered that coverage should be able to track class variables and module variables. I reckoned that the problem may be that these Melange models have been loaded before the coverage starts tracking. So, I further used pymox stubout to stub out nose.plugins.cover.Coverage.begin so that it loads Melange after the testing coverage starts. It works! All the class variables and module variables of Melange models, logic and views can be tracked by coverage and the code coverage now is over 50%.

Then I further worked on various views, e.g. sponsor, program, student, organization, etc and aim to increase the overall code coverage to over 60% next week.

GSoC 2010 Melange Testing Project: Week 6 of the Coding Period

In this week, I first tried to add test coverage to Melange models, i.e. did model tests. However, I found that Python coverage does not track the properties definition of models which makes the coverage of models still quite low. When I discussed with Sverre, he suggested me to exclude models in the coverage report and start to test other modules with lowest coverage. After rechecking the coverage report, I decided to do view tests for sponsor, org, org_admin, mentors, student and so on.

Then my hard drive was corrupt:(. Fortunately, I have the online clone on Google servers, so I have not lost much work. It just took me some time to reinstall the operating system and resetup the environments. Due to the problem, I have only finished most parts of the sponsor view tests and some parts of org and student tests. Next week, I will definitely work harder to catch up.

GSoC 2010 Melange Testing Project: Week 5 of the Coding Period

As I had planned last week, I added content to the wiki page GettingStartedOnTesting [0] with the aim to provide a guideline on writing tests for Melange this week. Sverre has had a look at it and thinks that it should be extended, though it looks good.

I have also added test cases for a couple of logic modules this week and am currently working on testing the views of sponsor, program and student. Sverre thought the test coverage is still low and suggests me to focus the modules with low coverage next week, e.g. do some tests on the models. So, I will move my focus on testing the models next week.

[0] http://code.google.com/p/soc/wiki/GettingStartedOnTesting

GSoC 2010 Melange Testing Project: Week 4 of the Coding Period

In the last week, I further added more test cases to the user view test to test the views for administrator users. In order to test the operations of administrator users, an XSRF middleware token has to be manually generated and added to the post data so that the operation can pass authentication. Then, I added more test cases to the user logic test. This week, I will try to add test cases for more logic and view pairs.

One big thing for Melange next month is that each commit will be accompanied with tests. In order to contribute to the movement, I am preparing for a wiki page explaining the current testing environment (e.g. datastore), test runners (e.g. how to only run a single test case), available tools (e.g. pymox and Django test client), utilities (e.g. StuboutHelper), and so on.

GSoC 2010 Melange Testing Project: Week 3 of the Coding Period

Last week, I finally got Django test client working with the help of Sverre (my mentor). In this week, however, another problem arose. For some urls, e.g. /cron/poke, the response returned by the test client is a Django http response object; for other urls, e.g. /user/show/a_user, however, the response is a dict. After failing to figure out what had caused the problem, I asked Sverre for help again. Sverre suggested that it might be due to the stubout code (pymox) in tests/app/soc/views/models/test_base.py. After I commented the code, however, the problem still did not go away. Later, Sverre found that the problem was due to that the fact that the stubout in tests/app/soc/modules/ghop/views/models/test_task.py has not been unset. So, he added the unset code to tearDown methods and also helped me reviewed/improved and merged my code of refactoring the stubout code into a helper class. Thank you very much, Sverre!

After solving these problems, I started to do view tests for soc.views.models.user using Django test client. I have almost finished it. Yesterday, I pushed the code into my online clone. Today, I asked Sverre to have a quick look to see if it is alright and he was satisfied with it. Though I am still behind my original schedule, I am confident that I can catch up soon. This is because I have solved all major obstructions and have finished adding tests for one logic (base) and one view (user) so that I can follow similar patterns to add tests for other logic and views very quickly.

This week, I also came across the trace module of python when I tried to find another problem I met. It can be used either in command line with python -m option or in your program. This module save you time to code a call back function for sys.settrace by offering some common functions. One useful function is to trace the execution flow of your program. It traces every statement, so be aware the output could be quite large. For example, the output of tracing a single statement of Django test client in Melange testing environment is more than 100MB. Another disadvantage is that the output only shows the module name without package path which could confuse you when you have many modules with the same name or you have a big project and do not know the package of every module. Another useful function is to trace calls. The output of this one is quite small compared with the first function and also has full package path. This function, however, is not perfect either. The problem is that the calling functions are ordered by their module name rather than by their calling order which is arguably more useful. And you have no options to change the order. Besides, you can only use it in command line.

GSoC 2010 Melange Testing Project: Week 2 of the Coding Period

In this week, I had planned to add test cases for app.soc.views.models.base.View as well as app.soc.logic.models.User.Logic and app.soc.views.models.User.View, app.soc.logic.models.Role.Logic and app.soc.views.models.Role.View.

As I mentioned in my project proposal, I want to use Django test client which acts as a dummy Web browser to test views of Melange because of its ease of use and good integration with Django (no surprise since it was developed by the Django team). So, I first tried to get Django test client working.

After many attempts, however, I still could not make it work. Then, I found that I could not even run Melange locally suddenly due to the error (google.appengine.api.yaml_errors.EmptyConfigurationFile). I first tired to solve the problem by checking the tracktrace and code. Unfortunately, however, I could not find the problem. I thought that I might have accidentally changed or removed some important files. So, I rebuilt my local Melange but it still did not work. I also tried to re-clone and rebuild another Melange, but it was still in a vain. At that time, I realized that I had to fall back on my last resort, asking my mentor Sverre :-) . Awesomely, Sverre found that the problem was due to the corrupted .appcfg_nag file and he then solved the problem by removing it.

After this problem was solved, I tried the test client again. However, it still did not work. So, I asked Sverre again and found that it is because I had used the trailing slash at the end of the url following Django's tradition while Melange uses a url pattern without the trailing slash. Finally, Django test client works!!! Hope I had asked Sverre earlier :-).

Because of these problems I encountered, I have not finished this week's work as planned. Fortunately, however, I have got Django test client working. In the next week, I will try to speed up my work, finish this week's work as soon as possible and then start next week's work load.