Tuesday, August 2, 2011

GSoC 2011 Integration with External APIs: Same Origin Policy - W10

Browsers have some restrictions for cross-domain Javascript requests due to some security issues. For "List exporting" from client side, we need to make cross-domain requests to Google from user. Before this jsonp was only standart that i know as a solution. Jsonp allows loading a json from a remote web server. But i knew GData protocol supports XML format data exchange. I tried to find a solution to load a cross-domain XML source. No solution was neat enough to apply.

Script Approach
Then i thought GData should support JSON of course, and found related document :) We should just have to add a "alt=json" parameter to url GET params. Same page also was providing a method to access resources from a cross-domain with JavaScript. We should call a special script that calls our callback with related json as parameter when page is loaded. Here is an example script:

<script src="http://www.google.com/calendar/feeds/developer-calendar@google.com/public/full?alt=json-in-script&callback=listEvents"></script>

which returns a javascript response with a form in which json response is embed.

listEvents({"version":"1.0" ..... })

This provides another way for making cross-domain requests. In this solution server must support this method with a special effort to form a javascript response instead of just interpretting request as a regular json request.

Http Access Control
This approach was good but unfortunately not enough for our purpose because we need to set Authentication header with request. So i returned back to jsonp. I thought it should be possible to make a jsonp request and add a "Authentication" header with request like that:

jQuery.ajax({url: a_cross_domain_url+"&callback=?", data: data, headers: {"Authentication": auth_header}, success: a_callback)

This was almost working until i encountered with another cross-domain restriction. Sending headers with a cross-domain request was limited to another standard. I read this document about "Cross-Origin Resource Sharing". As far as i understand from document, For sending headers with request, this is defining a way for browser to first ask to server if it allows given headers, then makes actual request. This sounds good but because it's a new recommendation and some browsers may not support it (and perhaps GData server too) , i skipped this. Firefox supports this after 3.5 version.

At the end
I learned some stuff  but still was not happy as there wasn't a way sending OAuth header with request. GData documentation contains AuthSub authentication over Javascript but i couldn't find anything related OAuth with JavaScript in which we will provide OAuth header. Then, while i was hopelessly playing with OAuth playground, saw that choice in form:


Put oauth params in:  



This was just i was looking for. We were able to pass oauth params with url instead of header. This was something not documented in GData OAuth documentation. I was lucky to find this out.


At the and my approach was shaped: Make a jsonp request, put oauth params inside url.

Note: This didn't work either, to see my final solution to make Javascript requests see next post by me.

Monday, August 1, 2011

GSoC 2011 GUI Overhaul: Document List for Admin

Hi everyone,

Lets talk about document list. Carol asked this on issue 1257. To implement such list, I borrowed the view from accepted_orgs.py. Lets name the list "DocumentList" and the page "DocumentListPage". I only added two visible cols, the title and shortname, for DocumentList. When the row is clicked, it will go to the edit document page view. I also created the test for document list inside test_document.py. The test will make sure, the list will be shown to the host (based on checkAccess method). Okay here's the class for DocumentList and DocumentListPage:

class DocumentList(Template):
"""Template for list of documents.
"""

def __init__(self, request, data):
self.request = request
self.data = data
r = data.redirect

list_config = lists.ListConfiguration()
list_config.addColumn('title', 'Title',
lambda e, *args: e.title.strip())
list_config.addSimpleColumn('link_id', 'Link ID', hidden=True)
list_config.addColumn('short_name', 'Short Name',
lambda e, *args: e.short_name.strip())
list_config.setRowAction(
lambda e, *args: r.document(e).urlOf('edit_gsoc_document'))

list_config.setDefaultPagination(False)
list_config.setDefaultSort('title')

self._list_config = list_config

def context(self):
description = 'List of documents for %s' % (
self.data.program.name)

list = lists.ListConfigurationResponse(
self.data, self._list_config, 0, description)

return {
'lists': [list],
}

def getListData(self):
idx = lists.getListIndex(self.request)
if idx == 0:
fields = {'scope': self.data.program}
response_builder = lists.QueryContentResponseBuilder(
self.request, self._list_config, document_logic, fields)
return response_builder.build()
else:
return None

def templatePath(self):
return 'v2/modules/gsoc/document/_document_list.html'


class DocumentListPage(RequestHandler):
"""View for the list documents page.
"""

def templatePath(self):
return 'v2/modules/gsoc/document/document_list.html'

def djangoURLPatterns(self):
return [
url(r'documents/%s$' % url_patterns.PROGRAM, self,
name='list_gsoc_documents'),
]

def checkAccess(self):
self.check.isHost()

def jsonContext(self):
list_content = DocumentList(self.request, self.data).getListData()

if not list_content:
raise AccessViolation(
'You do not have access to this data')
return list_content.content()

def context(self):
return {
'page_name': "Documents for %s" % self.data.program.name,
'document_list': DocumentList(self.request, self.data),
'program_select': ProgramSelect(self.data, 'list_gsoc_documents'),
}

And here's the test for document list:
class ListDocumentTest(DjangoTestCase):
"""Test document list page.
"""

def setUp(self):
self.init()
self.data.createHost()

def testListDocument(self):
url = '/gsoc/documents/' + self.gsoc.key().name()
response = self.client.get(url)
self.assertGSoCTemplatesUsed(response)

response = self.getListResponse(url, 0)
self.assertIsJsonResponse(response)
data = response.context['data']['']
self.assertEqual(1, len(data))

Sunday, July 31, 2011

GSoC 2011 Integration with External APIs: Popup Blocking Policy - W10

This week, unfortunately i was not productive. I had quite problems mainly based on "same origin policy". Tonight, story of the week seems ending nice, because i finally resolved how to make cross-domain requests with JavaScript. I will write two blog posts this week, this one for another problem "popup blocking problem" and next one for "same origin policy". If you don't have extra time for reading two blog posts, i suggest you read next one, because it's the main story of the week for me.

Popup Blocker Policy
I want to mention this before "same origin policy" problem. We had to use a window popup in OAuth authentication. Popup was used to open immediately after user clicks an element whom click event is connected to an authentication required function. This authentication required function was either opening popup, or running its own logic if user was already authenticated. This was how it was working:
 
User (before page loads):
- User requests a page which uses melange.gdata.core.js on user side. This script provides gdata login methods and gdata API requests for JS.

Server:
- Page indicates melange.gdata.core.js dependecy in its template by using our dependecy *mechanism*. 
- Page also needs to init melange.gdata.core.js with some context values those tell script some initial values: gdata_is_logged_in, oauth_redirect_url.

User (after page loads):
- If a page specific Javascript function  has a logic that needs authentication to GData, for example for the proposal page:

In proposal/base.js:

function sync() {
    //code that nees authentication
}
authenticated_sync = melange.gdata.core.createAuthorizedFunction(sync);
button.click(authenticated_sync)


This is working in peace for proposal page which adds melange.gdata.core.js to dependencies in it's template and inits it with some context value which is produced by a context helper (A helper which creates gdata_is_logged_in and oauth_redirect_url values). But for the "List Exporting" this was quite redundant workforce to init melange.gdata.core.js in every template because a big per cent of Melange pages contain lists. We did not want any extra configuration for every list page. Indicating list dependecy should be enough. So we tried to skip initialization of melange.gdata.core.js until user expresses he wants to use an authorization required feature. We tried to make this initilaziation with a JSON request. For example when user clicks "Export" button, it asks server for initialization values and with respect to response, performs required authentication process if required. Every thing seemed fine until i realized tha, request time delay was causing popup window to be blocked by browser's popup blocker. If we want to open a popup it should be immediately after a click event occured, but we can't know if we need to open a popup without asking it to server (so without waiting).

So we needed to change this mechanism. Script melange.gdata.core.js must be aware of gdata_is_logged_in value before click event. We solved this by moving gdata initialization into Melange's general javascript config initialization which works for every page. So with little redundancy (because we don't need to calculate is_logged_in value for every page), we were able to init it without extra initialization code for every page that uses melange.gdata.core.js. If we can assume %90 of Melange pages contains lists, this is not a big CPU usage redundacy.

I don't know if this popup blocking policy is browser dependent or not. But from my experiences i had the conclusion that most guarantee way of opening a popup is doing immediately after a click event is occured.