Using Data Access Object to Store Information

We often need to store the information received from the network to retrieve that later. Although we can store and read data directly but by using data access object to store information enables us to do data operations without exposing details of the database. Using data access object is also a best practice in software engineering. Recently I modified Connfa app to store the data received in Open Event format. In this blog, I describe how to use data access object.

The goal is to abstract and encapsulate all access to the data and provide an interface. This is called Data Access Object pattern. In a nutshell, the DAO “knows” which data source (that could be a database, a flat file or even a WebService) to connect to and is specific for this data source. It makes no difference in applications when it accesses a relational database or parses xml files (using a DAO). The DAO is usually able to create an instance of a data object (“to read data”) and also to persist data (“to save data”) to the datasource.

Consider the example from Connfa app in which get the tracks from API and store them in SQL database. We use DAO to create a layer between model and database. Where AbstractEntityDAO is an abstract class which have the functions to perform CRUD operation. We extend it to implement them in our DAO model. Here is TrackDAO structure,

public class TrackDao extends AbstractEntityDAO<Track, Long> {

    public static final String TABLE_NAME = "table_track";

    @Override
    protected String getSearchCondition() {
        return "_id=?";
    }
    
    ...
}

Find the complete class to see the detailed methods to implement search conditions, get key columns, create instance etc.  here.

Here is a general method to get the data from the database. Where getFacade() for the given layer element, this method returns the requested facade object to represent the passed in layer element.

public List<ClassToSave> getAllSafe() {
   ILAPIDBFacade facade = getFacade();
   try {
       facade.open();
       return getAll();

   } finally {
       facade.close();
   }
}

Now we can create an instance to use these methods instead of directly using SQL operations. This functions gets the data and sort it accordingly.

private TrackDao mTrackDao;
 public List<Track> getTracks() {
   List<Track> tracks = mTrackDao.getAllSafe();
   Collections.sort(tracks, new Comparator<Track>() {
       @Override
       public int compare(Track track, Track track2) {
           return Double.compare(track.getOrder(), track2.getOrder());
       }
   });
   return tracks;
}

References:

 

Receiving Data From the Network Asynchronously

We often need to receive information from networks in Android apps. Although it is a simple process but the problem arises when it is done through the main user interaction thread. To understand this problem better consider using an app which shows some text downloaded from the online database. As the user clicks a button to display a text it may take some time to get the HTTP response. So what does an activity do in that time? It creates a lag in the user interface and makes the app to stop responding. Recently I implemented this in Connfa App to download data in Open Event Format.

To solve that problem we use concurrent thread to download the data and send the result back to be processed separating it from main user interaction thread. AsyncTask allows you to perform background operations and publish results on the UI thread without having to manipulate threads and/or handlers.

AsyncTask is designed to be a helper class around Thread and Handler and does not constitute a generic threading framework. AsyncTasks should ideally be used for short operations (a few seconds at the most). If you need to keep threads running for long periods of time, it is highly recommended you use the various APIs provided by the java.util.concurrent package such as Executor, ThreadPoolExecutor and FutureTask.

An asynchronous task is defined by a computation that runs on a background thread and whose result is published on the UI thread. An asynchronous task is defined by 3 generic types, called Params, Progress and Result, and 4 steps, called onPreExecute, doInBackground, onProgressUpdate and onPostExecute.

In this blog, I describe how to download data in an android app with AsyncTask using OkHttp library.

First, you need to add dependency for OkHttp Library,

compile 'com.squareup.okhttp:okhttp:2.4.0' 


Extend AsyncTask class and implement all the functions in the class by overriding them to modify them accordingly. Here I declare a class named AsyncDownloader extending AsyncTask. We took the string parameter, integer type progress and string type result to return instead which will be our JSON data and param is our URL. Instead of using get the function of AsyncTask we implement an interface to receive data so the main user interaction thread does not have to wait for the return of the result.

public class AsyncDownloader extends AsyncTask<String, Integer, String> {
    private String jsonData = null;

    public interface JsonDataSetter {
        void setJsonData(String str);
    }


    JsonDataSetter jsonDataSetterListener;

    public AsyncDownloader(JsonDataSetter context) {

        this.jsonDataSetterListener = context;
    }

    @Override
    protected void onPreExecute() {
        super.onPreExecute();
    }

    @Override
    protected String doInBackground(String... params) {

        String url = params[0];

        OkHttpClient client = new OkHttpClient();
        Request request = new Request.Builder()
                .url(url)
                .build();
        Call call = client.newCall(request);

        Response response = null;

        try {
            response = call.execute();

            if (response.isSuccessful()) {
                jsonData = response.body().string();

            } else {
                jsonData = null;
            }

        } catch (IOException e) {
            e.printStackTrace();
        }


        return jsonData;
    }

    @Override
    protected void onPostExecute(String jsonData) {
        super.onPostExecute(jsonData);
        jsonDataSetterListener.setJsonData(jsonData);
    }
}


We download the data in doInBackground making a request using OkHttp library and send the result back in onPostExecute method using our interface method. See above to check out the complete class implementation.

You can make an instance to AsyncDownloader in an activity. Implement interface to receive that data.

Tip: Always close AsyncTask after using to avoid many concurrent tasks at one time.

Find the complete code here.

References:

Advantage of Open Event Format over xCal, pentabarf/frab XML and iCal

Event apps like Giggity and Giraffe use event formats like xCal, pentabarf/frab XML, iCal etc. In this blog, I present some of the advantages of using FOSSASIA’s Open Event data format over other formats. I added support for Open Event format in these two apps so I describe the advantages and improvements that were made with respect to them.

  • The main problem that is faced in Giggity app is that all the data like social links, microlocations, the link for the logo file etc., can not be fetched from a single file, so a separate raw file is being added to provide this data. Our Open Event format provides all this information from the single URL that could be received from the server so no need to use any separate file.
  • Here is the pentabarf format data for FOSSASIA 2016 conference excluding sessions. Although it provides all the necessary information it leaves the information for logo URL, details for longitude and latitude for microlocations (rooms) and links to social media and website. While the open event format provides all the missing details including some extra information like language, comments etc. See FOSSASIA 2016 Open Event format sample.
<conference>
<title>FOSSASIA 2016</title>
<subtitle/>
<venue>Science Centre Road</venue>
<city>Singapore</city>
<start>2016-03-18</start>
<end>2016-03-20</end>
<days>3</days>
<day_change>09:00:00</day_change>
<timeslot_duration>00:00:00</timeslot_duration>
</conference>
  • The parsing of received file format gets very complicated in case of iCal, xCal etc. as tags needs to be matched to get the data. Howsoever there are various libraries available for parsing JSON data. So we can create simply an array list of the received data to send it to the adapter. See this example for more information of working code. You can also see the parser for iCal to compare the complexity of the code.
  • The other more common problem is the structure of the formats received is sometimes it becomes complicated to define the sub parts of a single element. For example for the location we define latitude and longitude separately while in iCal format it is just separated by a comma. For example for

iCal

GEO:1.333194;103.736132

JSON

{
           "id": 1,
           "name": "Stage 1",
           "floor": 0,
           "latitude": 37.425420,
           "longitude": -122.080291,
           "room": null
}

And the information provided is more detailed.

  • Open Event format is well documented and it makes it easier for other developers to work on it. Find the documentation here.

References:

 

Displaying Multiple Markers on Map

Connfa and Open Event Android app provide the facility to see the locations on Google map by displaying the marker on places where the sessions will be taking place. As there are many sessions in a conference we need to display marker for each location. In this blog, we learn how to display multiple markers on a map using Google map API with the reference of Connfa app.
First, you need to set up the Google map API in your app. Find the detailed tutorial for that here where you can follow to get an API key and you’ll need a Gmail account for that. Then you can define a fragment like this in your XML layout where the map will appear.

<FrameLayout
   android:id="@+id/fragmentHolder"
   android:layout_width="match_parent"
   android:layout_height="match_parent"
   android:layout_below="@+id/layoutInfo"
   android:background="@android:color/white" />

Now you can define the in the fragment to make map in this FrameLayout named “fragmentHolder”,

private void replaceMapFragment() {
   CustomMapFragment mapFragment = CustomMapFragment.newInstance(LocationFragment.this);
   LocationFragment
           .this.getChildFragmentManager()
           .beginTransaction()
           .replace(R.id.fragmentHolder, mapFragment)
           .commitAllowingStateLoss();
}

We receive the data about locations in a List containing longitude, latitude, name and room number of the locations. Find the sample result in Open Event format. Here is how we fill in the locations,

for (int i = 0; i < locations.size(); i++) {
   Location location = locations.get(i);
   LatLng position = new LatLng(location.getLat(), location.getLon());
   mGoogleMap.addMarker(new MarkerOptions().position(position));

   if (i == 0) {
       CameraPosition camPos = new CameraPosition(position, ZOOM_LEVEL, TILT_LEVEL, BEARING_LEVEL);
       mGoogleMap.moveCamera(CameraUpdateFactory.newCameraPosition(camPos));
       fillTextViews(location);
   }
}

The CameraPostition defines the properties like Zoom level which can be adjusted by changing the parameters. You can define the like this,

private static final int ZOOM_LEVEL = 15;
private static final int TILT_LEVEL = 0;
private static final int BEARING_LEVEL = 0;

Find the complete implementation here. Here is the screenshot for the final result.

References:

  • Google Map APIs Documentation – https://developers.google.com/maps/documentation/android-api/map-with-marker

Checking Image Size to Avoid Crash in Android Apps

In Giggity app a user can create a shortcut for the event by clicking on “home shortcut” button in the navigation drawer. Open Event format provides the logo URL in the return data so we do not need to provide it separately in the app’s raw file.

Sometimes the image can be too big to be put on screen as icon for shortcut. In this blog I describe a very simple method to check if we should use the image or not to avoid the crash and pixelation due to resolution.

We can store the image received in bitmap format. A bitmap is a type of memory organization or image file format used to store digital images. The term bitmap comes from the computer programming terminology, meaning just a map of bits, a spatially mapped array of bits. By storing it in bitmap format we can easily get the necessary information about the image to check if it is suitable for use.

We can use the BitmapFactory class which provides several decoding methods like (decodeByteArray(), decodeFile(), decodeResource(), etc.) for creating a Bitmap from various sources. Choose the most appropriate decode method based on your image data source. These methods attempt to allocate memory for the constructed bitmap and therefore can easily result in an OutOfMemory exception. Each type of decode method has additional signatures that let you specify decoding options via the BitmapFactory.Options class. Setting the inJustDecodeBounds property to true while decoding avoids memory allocation, returning null for the bitmap object but setting outWidth, outHeight and outMimeType. This technique allows you to read the dimensions and type of the image data prior to construction (and memory allocation) of the bitmap.

BitmapFactory.Options options = new BitmapFactory.Options();
options.inJustDecodeBounds = true;
BitmapFactory.decodeResource(getResources(), R.id.myimage, options);
int imageHeight = options.outHeight;
int imageWidth = options.outWidth;
String imageType = options.outMimeType;

To avoid java.lang.OutOfMemory exceptions, check the dimensions of a bitmap before decoding it, unless you absolutely trust the source to provide you with predictably sized image data that comfortably fits within the available memory.

So here is the particular example from Giggity app, it avoids crash on the recieving a large image for the icon. So once we store the the image in bitmap format we check if the height and width of the icon is exceeding the maximum limit.

public Bitmap getIconBitmap() {

 InputStream stream = getIconStream();
 Bitmap ret = null;

 if (stream != null) {
 ret = BitmapFactory.decodeStream(stream);
 if (ret == null) {
 Log.w("getIconBitmap", "Discarding unparseable file");
 return null;
 }
 if (ret.getHeight() > 512 || ret.getHeight() != ret.getWidth()) {
 Log.w("getIconBitmap", "Discarding, icon not square or >512 pixels");
 return null;
 }
 if (!ret.hasAlpha()) {
 Log.w("getIconBitmap", "Discarding, no alpha layer");
 return null;
 }
 }
 
 return ret;
}

If it does then we can avoid the icon. In this case we check if the icon is more than 512 pixels in height and width. If it is so then we could avoid it.

We could also check if the icon has a transparent background by using “hasAlpha” so we could have uniformity in the icons displayed on the screen. In the final result you can see the icon of the TUBIX 2017 conference added on the screen as it was following all those defined criterias.

Now that the image dimensions are known, they can be used to decide if the full image should be loaded into memory or if a subsampled version should be loaded instead. Here are some factors to consider:

  • Estimated memory usage of loading the full image in memory.
  • Amount of memory you are willing to commit to loading this image given any other memory requirements of your application.
  • Dimensions of the target ImageView or UI component that the image is to be loaded into.
  • Screen size and density of the current device.

For example, it’s not worth loading a 1024×768 pixel image into memory if it will eventually be displayed in a 128×96 pixel thumbnail in an ImageView.

 

References:

Importing the Open Event format in Giraffe

Giraffe is a personal conference schedule tool for Android. Most conferences, bar camps and similar events offer their plan of sessions and talks in the iCal format for importing into your calendar. However importing a whole session plan into your standard calendar renders it pretty much useless for anything else. Giraffe allows users to import the schedule into a separate list giving you a simple overview on what happens on the conference. Besides the session, title, date and  time it also lists the speaker, location and description if available in the iCal URL. Sessions can be bookmarked and the list can be filtered by favourites and upcoming talks.

Recently I added the support for Open Event JSON format along with iCal. In this blog I describe the simple steps you need to follow to see the event that is created in the Open Event server in the Giraffe app. The initial steps are similar to Giggity app,

 1. Go to your event dashboard

2. Click on the export button.

3. Select sessions from the dashboard and copy the URL.

 

4. Click on the “Giraffe” button on the toolbar and paste the link in the box following. App will ask you to paste it when the first time you open it. Here the app loads the data and checks few initial character to see which kind of data is received. Find my other blog post to solve that problem here.

The app uses separate data models for iCal and JSON to store the informations received and then save them in SQL database for CRUD options. See the database activity here

   

5. Now you can see the sessions. Click on them to see more information or bookmark them if needed. The data is loaded from the database so when app is offline so we don’t need to worry about connection once the data is being loaded.

   

Resources:

 

How can you get an access to Instagram API?

unnamed.png

First of all you need to know that Instagram API uses OAuth 2.0 protocol. OAuth 2.0 provides a specific authorization flow for web apps, desktop apps and mobile apps. Instagram requires authentication before getting information from their API, but don’t be afraid it’s very simple.

Pre Requirements:

  • Created account in Instagram
  • Registered Client(You can create your own client here)

Instagram Developer Documentation.png

Requirements:

CLIENT_ID -79e1a142dbeabd57a3308c52ad43e31d
CLIENT_SECRET -34a6834081c44c20bd11e0a112a6adg1
REDIRECT_URI - http://127.0.0.1:8001/iCallback

You can get above information from https://www.instagram.com/developer/clients/manage/

CODE - You need to open page https://api.instagram.com/oauth/authorize/?client_id=CLIENT-ID&redirect_uri=REDIRECT-URI&response_type=code
https://api.instagram.com/oauth/authorize/?client_id=79e1a142dbeabd57a3308c52ad43e31d&redirect_uri=http://127.0.0.1:8001/iCallback&response_type=code

You will be redirected to

http://your-redirect-uri?code=CODE

In my case it looks like this:

http://127.0.0.1:8001/iCallback?code=2e122f3d76e8125b8b4982f16ed623c2

Now we have all information to get access token!

 curl -F 'client_id=CLIENT_ID'  
      -F 'client_secret=CLIENT_SECRET' 
      -F 'grant_type=authorization_code'  
      -F 'redirect_uri=REDIRECT_URI' 
      -F 'code=CODE'  https://api.instagram.com/oauth/access_token

if everything is ok you should receive

 { "access_token": "fb2e77d.47a0479900504cb3ab4a1f626d174d2d",
   "user": { "id": "1574083",
   "username": "rafal_kowalski",
   "full_name": "Rafal Kowalski",
   "profile_picture": "..." } } 

In Open Event we used it to get all media from instagram – to use it  as for example a background in an event details’ page

curl 'https://api.instagram.com/v1/users/self/media/recent/?access_token=ACCESS_TOKEN'

 

How do we work? Agile

It’s not typical team. We don’t meet face-to-face each other on daily basis, one trouble is that we have to work from different time zones, and we have different operating system, the way of working, or even culture. But we are FOSSASIA Open Event team of six amazing developers so it does not discourage us to achieve our goals.

But even if we experience all these things we have to learn how to work together successfully. We are trying, and I think it is becoming better and better every day, the Agile methodology.

So first of all, before coding started, we had prepared user stories in issues. Every issue had been very well documented, and we had divided each issue to smaller ones. It brings us a lot of benefits, because it doesn’t matter where you work. If sub issue is empty you can take it. Due to the fact that issue is very well documented you don’t need to even ask someone what to do. This case saves a lot of time. I know that writing clear issues is very boring and It could seemed not to be beneficial, but it’s. We are still trying to improve our performance and we are looking for new opportunities where and how we can improve our work, without lost quality of our development process.

Then, we have one week sprint (milestones). It helps us to control our work and we know where are we, what have we done, and what is still to do. We know if our speed is enough or we have to move faster. And it also enables us to see our progress.Screen Shot 2016-06-25 at 11.27.08.png

Moreover, we have daily scrums. We answer three following questions – what did you do yesterday, what do you plan to do today and what is preventing you from achieving your goals. Furthermore, we follow continues integration system. We push code to our common Open Event repository on daily basis. It helps us to control our code and fix bugs quickly. I am sure we will continue along this path, and successfully finish to code an amazing Open Event web app.

Migrations make us crazy!

Our Open Event team is not small, so problems with migrations occur very often. I’d like to describe how to solve these common problems and avoid Contributors frustration. Because more of us didn’t know how to solve this problems at the beginning so we wasted a lot of time to find a source of problem.

The most common mistake is that we forget run migration on Heroku. Developer is sometimes surprised because something works for him but he forgets to run migration on server. These small mistakes lead to huge problems and at that time our app throws a lots of bugs related to database. We can often see “Internal server error”.Screen Shot 2016-06-17 at 22.55.25.png So if developer changes table he has to run migration!

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db migrate

But above command Quite often doesn’t solve our problem, because We face other problems while updating DB for example

alembic.util.exc.CommandError: Multiple head revisions are present for given argument ‘head’; please specify a specific target revision, ‘<branchname>@head’ to narrow to a specific head, or ‘heads’ for all heads

this problem is caused by two developers which push code to Github with migrations without merging two heads to achieve one head.

So to solve this problem you only have to know ids of two heads

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db heads

e38935822969 (head)
f55fde3d62b1 (head)

Then you have to merge it

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db merge e38935822969 f55fde3d62b1

Generating /vagrant/migrations/versions/ecb671d1eb4b_.py … done

Upgrade DB

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db upgrade

INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 00ea66754d06 -> d84f976530e1, empty message
INFO [alembic.runtime.migration] Running upgrade d84f976530e1 -> 1b3e4f5f56bd, empty message
INFO [alembic.runtime.migration] Running upgrade 1b3e4f5f56bd -> e38935822969, empty message
INFO [alembic.runtime.migration] Running upgrade e38935822969, f55fde3d62b1 -> ecb671d1eb4b, empty message

And finally run migrations

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db migrate
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.ddl.postgresql] Detected sequence named ‘role_id_seq’ as owned by integer column ‘role(id)’, assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named ‘microlocation_id_seq’ as owned by integer column ‘microlocation(id)’, assuming SERIAL and omitting….