How to Geolocate Tweets for Emoji Heatmapper App using LokLak API

Geolocating tweets i.e., getting the location of the tweets posted, is one of the major task in the app Emoji Heatmapper. As I have to plot a marker on the map according to the query searched and the location tracked from the tweet containing that query. This is easily done in the app using the LokLak Search API.
Social media, such as Twitter, are filled with millions of micro messages by people around the world. My geolocation task is to get the geographic location with a Twitter message (having emoji for Emoji Heatmapper) based on the information we have about the user and the message. Twitter provides various data APIs, and tweets are obtained as JSON objects that include the tweet text along with metadata, such as location coordinates (lat and long).

Solutions

So there are few ways to geolocate tweets:

  • Place object from tweets Tweets usually delivered by the Twitter API include a JSON “Place” object which has a location associated with the tweet. These can have fields such as the country and city related to the place, as well as lat and long coordinates. Twitter Users have the option to tag their twitter statuses i.e., tweets with a place; the tagging can also be done automatically based on matches to the user’s current GPS position, if the user allows this (if GPS is switched ON). For the tweets containing place objects, the geolocation has already been done by Twitter.
  • Coordinates from tweets Some tweets are tagged with the coordinates (lat and long) of the user when the message was written, based on the user’s current GPS position. Reverse geocoding using APIs from Google Maps can be done to know the detailed information about the place.
  • Location from user profile Many users provide a location in their profile, a field with values such as “IND”. These locations are mostly always the same(i.e.,static), corresponding to the user’s primary location rather than the location at the time of the message posting. We assume that the users are not avid travelers.
  • Content-based Geo-location Geo-location can also be done on a message or set of messages based on the textual content of the messages.A user’s primary location can be detected based on their dialect or the mention of regional issues like sports teams, for example, as well as the mention of landmarks.

Implementation

So, here comes about the usage of LokLak search API in Emoji Heatmapper app:
I have used the LokLak Search API to get the tweets, which contains the query(i.e., emoji) which is being searched for and the location of the tweets. In LokLak we perform geolocation using location information from tweet metadata and user profiles.
Try this simple query:

http://loklak.org/api/search.json?q=😄

Which shows the data related to search query in which we also have geolocation information. Below is the output of the query, in which a single Twitter status is displayed which has the search query in it and the highlighted part shows the information about the location:

 {
      "timestamp": "2017-06-01T17:46:41.874Z",
      "created_at": "2017-06-01T17:46:25.000Z",
      "screen_name": "CerdaEnterprise",
      "text": "\"These boots were made for wearing... & that's just what we'll do.\" 🎶😄 Coral + guest pins = 💟 http://Www.TfjByCei.com https://pic.twitter.com/469uVWumSN",
      "link": "https://twitter.com/CerdaEnterprise/status/870335741561151488",
      "id_str": "870335741561151488",
      "canonical_id": "",
      "parent": "",
      "source_type": "TWITTER",
      "provider_type": "SCRAPED",
      "retweet_count": 0,
      "favourites_count": 0,
      "place_name": "Made",
      "place_id": "",
      "text_length": 155,
      "place_context": "ABOUT",
      "place_country": "Netherlands",
      "place_country_code": "NL",
      "place_country_center": [
        -3.5756949584817903,
        26.74012558382941
      ],
      "location_point": [
        4.7930598188159195,
        51.67666995510302
      ],
      "location_radius": 0,
      "location_mark": [
        4.793226608645116,
        51.676940788684036
      ],
      "location_source": "ANNOTATION",
      "hosts": [
        "www.tfjbycei.com",
        "pic.twitter.com"
      ],
      "hosts_count": 2,
      "links": [
        "http://Www.TfjByCei.com",
        "https://pic.twitter.com/469uVWumSN"
      ],
      "links_count": 2,
      "unshorten": {},
      "images": [
        "https://pbs.twimg.com/media/DBQNqkDUQAAkntY.jpg",
        "https://pic.twitter.com/469uVWumSN"
      ],
      "images_count": 2,
      "audio": [],
      "audio_count": 0,
      "videos": [],
      "videos_count": 0,
      "mentions": [],
      "mentions_count": 0,
      "hashtags": [],
      "hashtags_count": 0,
      "classifier_language": "english",
      "classifier_language_probability": 3.246518732845919E-16,
      "without_l_len": 96,
      "without_lu_len": 96,
      "without_luh_len": 96,
      "user": {
        "appearance_first": "2017-06-01T17:46:41.874Z",
        "profile_image_url_https": "https://pbs.twimg.com/profile_images/769877281497944064/PgsRK9C7_bigger.jpg",
        "screen_name": "CerdaEnterprise",
        "user_id": "529052548",
        "name": "The Fashion Junkies",
        "appearance_latest": "2017-06-01T17:46:41.874Z"
      }
    }

As you can see the above JSON output from the “Search API of LokLak”, the location we get in the JSON output i.e., location_mark/location_point is used to plot the point on the map. If in case the location from where the tweet posted is missing, location from the user profile is used.

Location is pointed with a marker on the map as shown below:

You can find the information about Emoji Heatmapper app here and even can try it out:

http://apps.loklak.org/details.html?q=emojiHeatmapper.

Resources

Published by

Kavitha Nair

Open Source Enthusiast and Wanderlust.