How to implement JSONP to download XML resource

According to the same-origin security policy, web page can freely share images, stylesheets, scripts… but web fonts and AJAX requests are only accessible from the same domain.

Sometimes this is not desired behaviour, such as when getting a json / xml file from a different domain, using AJAX. In such cases, it all comes down to two solutions :

1- The source sets its header to allow specific domain cross-origin resource sharing (CORS). In PHP, this is done with header function :

<?php
header('Access-Control-Allow-Origin: *');

2- Use JSONP.

JSONP is short for JSON padding, a well-known “workaround” to solve CORS. It only works with GET.

To send AJAX requests using JSONP, the server must first knows how to respond to JSONP requests. It usually detects JSONP requests using callback parameter.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
<?php
// compute result
[...]

// output result
JsonpHelper::outputXML($result);

// json helper class - credits : @andibraeu
class JsonpHelper {
	private static function is_valid_callback($subject)
	{
	    $identifier_syntax
	      = '/^[$_\p{L}][$_\p{L}\p{Mn}\p{Mc}\p{Nd}\p{Pc}\x{200C}\x{200D}]*+$/u';
	    $reserved_words = array('break', 'do', 'instanceof', 'typeof', 'case',
	      'else', 'new', 'var', 'catch', 'finally', 'return', 'void', 'continue', 
	      'for', 'switch', 'while', 'debugger', 'function', 'this', 'with', 
	      'default', 'if', 'throw', 'delete', 'in', 'try', 'class', 'enum', 
	      'extends', 'super', 'const', 'export', 'import', 'implements', 'let', 
	      'private', 'public', 'yield', 'interface', 'package', 'protected', 
	      'static', 'null', 'true', 'false');
	    return preg_match($identifier_syntax, $subject)
	        && ! in_array(mb_strtolower($subject, 'UTF-8'), $reserved_words);
	}

	public static function outputXML($string) {
		# JSON if no callback
		if( ! isset($_GET['callback']) )
		    exit($string);
		$string = str_replace("\n", " ", str_replace("'", '"', $string));
		# JSONP if valid callback
		if(JsonpHelper::is_valid_callback($_GET['callback']))
		    exit("{$_GET['callback']}('$string')");
		# Otherwise, bad request
		header('status: 400 Bad Request', true, 400);
	}
}

Line 29 is dedicated to replace all single-quote characters to double-quote, and new lines to simple space. This is to be able to pass xml document as valid javasript string parameter.

Then, the client can use jQuery to get the document from a different domain (it works on same domain as well, which is convenient for testing & development) :

1
2
3
4
5
6
7
8
9
$.ajax({
	url : source_url,
	dataType : 'jsonp',  // if dataType is set to 'jsonp' 
			    // a 'callback' parameter will be appended to the request url
	success: function(data) {
		$data = $($.parseXML(data));
		items = $data.find('item');
	}
});

Something worth noticing on the client side is on line 5 : the data needs to be converted to xml first, with $.parseXMLbefore converting to jQuery object. Converting the result string directly to jQuery won’t work : $data = $(data);.

That’s about it ! To know more about JSONP, please refer to its wiki https://en.wikipedia.org/wiki/JSONP.

References :

API Integration

My FOSSASIA project is taking a big transitioning step from the starting plan. In milestone 3 and 4, rather than focusing on statistical retrieval and API documentation wepapp, my mentors – Mario (@mario) and Andreas (@andibraeu) are guiding me towards integrating fossasia api data in other services. The main goal remains the same though : getting more people to use and to know about the API.

Integrating our set of data into external services can be tricky, but this is also what makes the task so exciting. Currently, we’re already started the integration process into loklak, a distributed searching & harvesting server. You can find out more about loklak via their well detailed about page. But the plan doesn’t stop there. We’re targeting popular web platforms : WordPress, Drupal, Github Pages.. so that users can access our data with ease, and use our services / plugins in their websites within simple steps. That would be a big win for us, so I’m very impatient to get going with the code to brings these ideas to life.

References :

Get Facebook comments with large user pictures

On the /user edge, user pictures url are not returned by default, and when users ask for it, the response image is not in biggest format. This query should get the largest user pictures associated with post :

	/user{picture.type(large)}

There are other use cases where this syntax applies as well

  • get comments of a feed
	/feed?fields='from{picture.type(large)}'
  • get feeds of an user
	/userid/feed?fields={from{picture.type(large)}}

Examples tested with Facebook Graph API > v2.0, using Graph Explorer and PHP SDK.

A glance at FOSSASIA Calendar API

I am a frequent user of popular social platforms API : Facebook, Twitter, Instagram.. Among others, Facebook Graph API is the API I’m most familiarized with, and despite its lack of documentation, Graph API makes me from time to time admire the engineering mind of those who built it. It’s simple, has a clear design, and communicates a consistent philosophy. I inspired a lot from it to implement Calendar API, a simple web service at FOSSASIA.

Why naming it API, and not web service ? I don’t know, I was always confused about the two, and as far as I concerns, they are the same. The only difference might be the word ‘web’ that tights web service to HTTP, and the scenario involving two distant machines, while API is somewhat more general (an API call of an in-app library for example).

This is my first time building a web service, and so far there wasn’t any technical challenge : it’s just simple & straightforward PHP script with echo $results; in the end. Mainly because the hard part is still ahead : getting people to know about it, and maintaining it while there are (hopefully) enough clients who care to yell at API developers when they do something wrong. The API I built supposed to parse the merged ics file of all communities event feeds, and expose it to users, in various formats, with filters and options.

Don’t worry if the last sentence doesn’t make sense. We’ll dive in the details right away.

There are currently two possible formats : .ics and .json. Users set the format with format parameter, for e.g. format=icsics format is suited for developers who want to import events to their calendar app / wordpress plugin. json format is for front-end developers who want to build a web / mobile web without the hassle of parsing ics feed.

A few filter parameters are possible to refine search results : source to filter by community, fromto to filter by events time, limit to limit number of events returned, and fields to custom select event fields. Finally, sort parameter is to order the results by certain criteria. Currently only datetime ordering (ascending or descending) are available.

As I said, the Calendar API is inspired a lot from Facebook API. Those are the ideas I pickpocketed from their design :

  • multiple values parameter. With source and fields, multiple values can be provided, separated using comma.
  • fields parameter. The motivation for event field filter is because the event has lots of different fields, and different users are interested in different information.

There is also the “linked edge” characteristic of Graph API that I find very interesting : connection between related data make them more useful. It is done that way mainly because the API reflects Facebook own data model, but there is nothing thats prevents unconnected data to be presented in a connected way. So I’ll hope to implement something similar for FOSSASIA Calendar API in the futur.

(Right now the API serves events from Freifunk community, not FOSSASIA, mostly because there are not enough event feeds from FOSSASIA community to collect. So if you want to help us, you can add ics feed of your open-source community to FOSSASIA API. Here’s how).

References :

Optimize React-Redux App

React is advertised as quite fast out of the box. In extreme cases when your app hits performance problem, you just have to optimize the re-rendering process by shallow comparing React props and state inside shouldComponentUpdatelifecycle function. The strategy for optimizing seems quite straight-forward.

However I found that this strategy is less clear in a React/Redux environment. Here are some advices that you should considering when dealing with your app performance:

Use Immutable data

This one should be a no-brainer with any React app. shouldComponentUpdate can only be useful when all the props can be shallow-compared. So as a rule of thumb, your components props can only be of 3 types:

  • Immutable
  • Primitive type (number, boolean, etc.)
  • Function type (e.g. callback event)

I didn’t mention how you should handle React state. That is because Redux uses a separate store as application state. But sometimes, you need to use both React’s state and Redux’s store. In that case, both props and state are shallow-compared so state elements should follow mentioned rules as well.

Do not use ownProps

When you create stateful components (containers) using connect function from react-redux library, shouldComponentUpdate is declared implicitly, so you normally don’t have to implement it yourself.

However the implementation of shouldComponentUpdate is such that if you uses ownProps in your mapStateToProps or mergeProps function, it will always return false, so the component will always be updated.

This means that if you want your component to be optimized, you can’t use information from passed props to get other information from the store. Why the implementation has to be this way is still unclear to me.

Avoid connect on frequently-called elements

connect function wraps the component inside another component with lots of decorated elements. These heavy overheads should be avoided on components that are called a lot in your application, for e.g. an element of a list.

Use Performance Tools

Finally, here are some excellent tools from the React team. Use them to identify the bottleneck in your React application: