Adding Sentry Integration in Open Event Orga Android App

Sentry is a service that allows you to track events, issues and crashes in your apps and provide deep insights with context about them. This blog post will discuss how we implemented it in Open Event Orga App (Github Repo).

Configuration

First, we need to include the gradle dependency in build.gradle
compile ‘io.sentry:sentry-android:1.3.0’
Now, our project uses proguard for release builds which obfuscates the code and removes unnecessary class to shrink the app. For the crash events to make sense in Sentry dashboard, we need proguard mappings to be uploaded every time release build is generated. Thankfully, this is automatically handled by sentry through its gradle plugin, so to include it, we add this in our project level build.gradle in dependencies block

classpath 'io.sentry:sentry-android-gradle-plugin:1.3.0'

 

And then apply the plugin by writing this at top of our app/build.gradle

apply plugin: 'io.sentry.android.gradle'

 

And then configure the options for automatic proguard configuration and mappings upload

sentry {
   // Disables or enables the automatic configuration of proguard
   // for Sentry.  This injects a default config for proguard so
   // you don't need to do it manually.
   autoProguardConfig true

   // Enables or disables the automatic upload of mapping files
   // during a build.  If you disable this you'll need to manually
   // upload the mapping files with sentry-cli when you do a release.
   autoUpload false
}

 

We have set the autoUpload to false as we wanted Sentry to be an optional dependency to the project. If we turn it on, the build will crash if sentry can’t find the configuration, which we don’t want to happen.

Now, as we want Sentry to configurable, we need to set Sentry DSN as one of the configuration options. The easiest way to externalize configuration is to use environment variables. There are other methods to do it given in the official documentation for config https://docs.sentry.io/clients/java/config/

Lastly, for proguard configuration, we also need 3 other config options, namely:

defaults.project=your-project
defaults.org=your-organisation
auth.token=your-auth-token

 

For getting the auth token, you need to go to https://sentry.io/api/

Now, the configuration is complete and we’ll move to the code

Implementation

First, we need to initialise the sentry instance for all further actions to be valid. This is to be done when the app starts, so we add it in onCreate method Application class of our project by calling this method

// Sentry DSN must be defined as environment variable
// https://docs.sentry.io/clients/java/config/#setting-the-dsn-data-source-name
Sentry.init(new AndroidSentryClientFactory(getApplicationContext()));

 

Now, we’re all set to send crash reports and other events to our Sentry server. This would have required a lot of refactoring if we didn’t use Timber for logging. We are using default debug tree for debug build and a custom Timber tree for release builds.

if (BuildConfig.DEBUG)
   Timber.plant(new Timber.DebugTree());
else
   Timber.plant(new ReleaseLogTree());

 

The ReleaseLogTree extends Timber.Tree which is an abstract class requiring you to override this function:

@Override
protected void log(int priority, String tag, String message, Throwable throwable) {

 }

 

This function is called whenever there is a log event through Timber and this is where we send reports through Sentry. First, we return from the function if the event priority is debug or verbose

if(priority == Log.DEBUG || priority == Log.VERBOSE)
   return;

 

If the event if if info priority, we attach it to sentry bread crumb

if (priority == Log.INFO) {
    Sentry.getContext().recordBreadcrumb(new BreadcrumbBuilder()
          .setMessage(message)
          .build());
}

 

Breadcrumbs are stored and only send with an event. What event comprises for us is the crash event or something we want to be logged to dashboard whenever the user does it. But since info events are just user interactions throughout the app, we don’t want to crowd the issue dashboard with them. However, we want to understand what user was doing before the crash happened, and that is why we use bread crumbs to store the events and only send them attached to a crash event. Also, only the last 100 bread crumbs are stored, making it easier to parse through them.

Now, if there is an error event, we want to capture and send it to the server

if (priority == Log.ERROR) {
   if (throwable == null)
       Sentry.capture(message);
   else
       Sentry.capture(throwable);
}

 

Lastly, we want to set Sentry context to be user specific so that we can easily track and filter through issues based on the user. For that, we create a new class ContextManager with two methods:

  • setOrganiser: to be called at login
  • clearOrganiser: to be called at logout

public void setOrganiser(User user) {
   Map<String, Object> userData = new HashMap<>();
   userData.put("details", user.getUserDetail());
   userData.put("last_access_time", user.getLastAccessTime());
   userData.put("sign_up_time", user.getSignupTime());

   Timber.i("User logged in - %s", user);
   Sentry.getContext().setUser(
       new UserBuilder()
       .setEmail(user.getEmail())
       .setId(String.valueOf(user.getId()))
       .setData(userData)
       .build()
   );
}

 

In this method, we put all the information about the user in the context so that every action from here on is attached to this user.

public void clearOrganiser() {
   Sentry.clearContext();
}

 

And here, we just clear the sentry context.

This concludes the implementation of our sentry client. Now all Timber log events will through sentry and appropriate events will appear on the sentry dashboard. To read more about sentry features and Timber, visit these links:

Sentry Java Documentation (check Android section)

https://docs.sentry.io/clients/java/

Timber Library

https://github.com/JakeWharton/timber

Implementing Attendee Detail BottomSheet UI in Open Event Orga App

In Open Event Orga App (Github Repo), we allow the option to check the attendee details before checking him/her in or out. Originally, a dialog was shown showing the attendee details, which did not contain much information about the attendee, ticket or the order. The disadvantage of such design was also that it was tied to only one view. We couldn’t show the check in dialog elsewhere in the app, like during QR scanning. So we had to switch back to the attendee view for showing the check in dialog. We decided to create a usable detached component in the form of a bottom sheet containing all required information. This blog will outline the procedure we employed to design the bottom sheet UI.

The attendee check in dialog looked like this:

So, first we decide what we need to show on the check in bottom sheet:

  • Attendee Name
  • Attendee Email
  • Attendee Check In Status
  • Order Status ( Completed, Pending, etc )
  • TIcket Type ( Free, Paid, Donation )
  • Ticket Price
  • Order Date
  • Invoice Number
  • Order ‘Paid Via’

As we are using Android Data Binding in our layout, we’ll start by including the variables required in the layout. Besides the obvious attendee variable, we need presenter instance to handle the check in and check out of the attendee and DateUtils class to parse the order date. Additionally, to handle the visibility of views, we need to include the View class too

<data>
   <import type="org.fossasia.openevent.app.utils.DateUtils" />
   <import type="android.view.View" />

   <variable
       name="presenter"
       type="org.fossasia.openevent.app.event.checkin.contract.IAttendeeCheckInPresenter" />

   <variable
       name="checkinAttendee"
       type="org.fossasia.openevent.app.data.models.Attendee" />
</data>

 

Then, we make the root layout to be CoordinatorLayout and add a NestedScrollView inside it, which contains a vertical linear layout in it. This vertical linear layout will contain our fields.

Note: For brevity, I’ll skip most of the layout attributes from the blog and only show the ones that correspond to the text

Firstly, we show the attendee name:

<TextView
   style="@style/TextAppearance.AppCompat.Headline"
   android:text='@{attendee.firstName + " " + attendee.lastName }'
   tools:text="Name" />

 

The perks of using data binding can be seen here, as we are using string concatenation in layout itself. Furthermore, data binding also handles null checks for us if we add a question mark at the end of the variable name ( attendee.firstName? ).

But our server ensures that both these fields are not null, so we skip that part.

Next up, we display the attendee email

<TextView
   android:text="@{ checkinAttendee.email }"
   tools:text="[email protected]" />

 

And then the check in status of the attendee

<TextView
   android:text="@{ checkinAttendee.checkedIn ? @string/checked_in : @string/checked_out }"
   android:textColor="@{ checkinAttendee.checkedIn ? @color/light_green_500 : @color/red_500 }"
   tools:text="CHECKED IN" />

 

Notice that we dynamically change the color and text based on the check in status of the attendee

Now we begin showing the fields with icons to their left. You can use Compound Drawable to achieve this effect, but we use vector drawables which are incompatible with compound drawables on older versions of Android, so we use a horizontal LinearLayout instead.

The first field is the order status denoting if the order is completed or in transient state

<LinearLayout android:orientation="horizontal">

   <ImageView app:srcCompat="@drawable/ic_transfer" />
   <TextView android:text="@{ checkinAttendee.order.status }" />
</LinearLayout>

 

Now, again for keeping the snippets relevant, I’ll skip the icon portion and only show the text binding from now on.

Next, we include the type of ticket attendee has. There are 3 types of ticket supported in Open Event API – free, paid, donation

<TextView
   android:text="@{ checkinAttendee.ticket.type }"  />

 

Next, we want to show the price of the ticket, but only when the ticket is of paid type.

I’ll include the previously omitted LinearLayout part in this snippet because it is the view we control to hide or show the field

<LinearLayout
   android:visibility='@{ checkinAttendee.ticket.type.equalsIgnoreCase("paid") ? View.VISIBLE : View.GONE }'>

   <ImageView app:srcCompat="@drawable/ic_coin" />
   <TextView
       android:text='@{ "$" + checkinAttendee.ticket.price }'
       tools:text="3.78" />
</LinearLayout>

 

As you can see, we are showing this layout only if the ticket type equals paid

The next part is about showing the date on which the order took place

<TextView
   android:text="@{ DateUtils.formatDateWithDefault(DateUtils.FORMAT_DAY_COMPLETE, checkinAttendee.order.completedAt) }" />

 

Here we are using internal DateUtils method to format the date into complete date time from the ISO 8601 standard date present in the order object

Now, we show the invoice number of the order

<TextView
   android:text="@{ checkinAttendee.order.invoiceNumber }" />

 

Lastly, we want to show how the ticket was paid for via

<LinearLayout
   android:visibility='@{ checkinAttendee.order.paidVia.equalsIgnoreCase("free") ? View.GONE : View.VISIBLE }'>

   <ImageView app:srcCompat="@drawable/ic_ray" />
   <TextView  android:text="@{ checkinAttendee.order.paidVia }" />
</LinearLayout>

 

Notice that here too we are controlling the visibility of the layout container and only showing it if the ticket type is paid

This ends our vertical linear layout showing the fields about attendee detail. Now, we add a floating action button to toggle the check in status of attendee

<FrameLayout
   android:layout_gravity="top|end">

   <android.support.design.widget.FloatingActionButton
       android:layout_gravity="center"
       android:onClick="@{() -> presenter.toggleCheckIn() }"
       app:backgroundTint="@{ checkinAttendee.checkedIn ? @color/red_500 : @color/light_green_500 }"
       app:srcCompat="@{ checkinAttendee.checkedIn ? @drawable/ic_checkout : @drawable/ic_checkin }"
       app:tint="@android:color/white" />

   <ProgressBar
       android:layout_gravity="center" />

</FrameLayout>

 

We have used a FrameLayout to wrap a FAB and progress bar together in top end of the bottom sheet. The progress bar shows the indeterminate progress of the toggling of attendee status. And you can see the click binder on FAB triggering the presenter method toggleCheckIn() and how the background color and icon change according to the check in status of the attendee.

This wraps up our layout design. Now we just have to create a BottomSheetDialogFragment, inflate this layout in it and bind the attendee variable and we are all set. The result with all fields visible looks like this:

To learn more about bottom sheet and android data binding, please refer to these links:

Invalidating user login using JWT in Open Event Orga App

User authentication is an essential part of Open Event Orga App (Github Repo), which allows an organizer to log in and perform actions on the event he/she organizes. Backend for the application, Open Event Orga Server sends an authentication token on successful login, and all subsequent privileged API requests must include this token. The token is a JWT (Javascript Web Token) which includes certain information about the user, such as identifier and information about from when will the token be valid, when will it expire and a signature to verify if it was tampered.

Parsing the Token

Our job was to parse the token to find two fields:

  • Identifier of user
  • Expiry time of the token

We stored the token in our shared preference file and loaded it from there for any subsequent requests. But, the token expires after 24 hours and we needed our login model to clear it once it has expired and shown the login activity instead.

To do this, we needed to parse the JWT and compare the timestamp stored in the exp field with the current timestamp and determine if the token is expired. The first step in the process was to parse the token, which is essentially a Base 64 encoded JSON string with sections separated by periods. The sections are as follows:

  • Header ( Contains information about algorithm used to encode JWT, etc )
  • Payload ( The data in JWT – exp. Iar, nbf, identity, etc )
  • Signature ( Verification signature of JWT )

We were interested in payload and for getting the JSON string from the token, we could have used Android’s Base64 class to decode the token, but we wanted to unit test all the util functions and that is why we opted for a custom Base64 class for only decoding our token.

So, first we split the token by the period and decoded each part and stored it in a SparseArrayCompat

public static SparseArrayCompat<String> decode(String token) {
   SparseArrayCompat<String> decoded = new SparseArrayCompat<>(2);

   String[] split = token.split("\\.");
   decoded.append(0, getJson(split[0]));
   decoded.append(1, getJson(split[1]));

   return decoded;
}

 

The getJson function is primarily decoding the Base64 string

private static String getJson(String strEncoded) {
   byte[] decodedBytes = Base64Utils.decode(strEncoded);
   return new String(decodedBytes);
}

The decoded information was stored in this way

0={"alg":"HS256","typ":"JWT"},  1={"nbf":1495745400,"iat":1495745400,"exp":1495745800,"identity":344}

Extracting Information

Next, we create a function to get the expiry timestamp from the token. We could use GSON or Jackson for the task, but we did not want to map fields into any object. So we simply used JSONObject class which Android provides. It took 5 ms on average to parse the JSON instead of 150 ms by GSON

public static long getExpiry(String token) throws JSONException {
   SparseArrayCompat<String> decoded = decode(token);

   // We are using JSONObject instead of GSON as it takes about 5 ms instead of 150 ms taken by GSON
   return Long.parseLong(new JSONObject(decoded.get(1)).get("exp").toString());
}

 

Next, we wanted to get the ID of user from token to determine if a new user is logging in or an old one, so that we can clear the database for new user.

public static int getIdentity(String token) throws JSONException {
   SparseArrayCompat<String> decoded = decode(token);

   return Integer.parseInt(new JSONObject(decoded.get(1)).get("identity").toString());
}

Validating the token

After this, we needed to create a function that tells if a stored token is expired or not. With all the right functions in place, it was just a matter of comparing current time with the stored timestamp

public static boolean isExpired(String token) {
   long expiry;

   try {
       expiry = getExpiry(token);
   } catch (JSONException jse) {
       return true;
   }

   return System.currentTimeMillis()/1000 >= expiry;
}

 

Since the token provides timestamp from epoch in terms of seconds, we needed to divide the current time in milliseconds by 1000 and the function returned true if current timestamp was greater than the expiry time of token.

After writing a few unit tests for both functions, we just needed to plug them in our login model at the time of authentication.

At the time of starting of the application, we use this function to check if a user is logged in or not:

public boolean isLoggedIn() {
   String token = utilModel.getToken();

   return token != null && !JWTUtils.isExpired(token);
}

 

So, if there is no token or the token is expired, we do not automatically login the user and show the login screen.

Implementing login

The next task were

  • Sequest the server to login
  • Store the acquired token
  • Delete database if it is a new user

Before implementing the above logic, we needed to implement a function to determine if the person logging in is previous user, or new one. For doing so, we first loaded the saved user from our database, if the query is empty, surely it is a new user logging in. So we return false, and if there is a user in the database, we match its ID with the logged in user’s ID:

public Single<Boolean> isPreviousUser(String token) {
   return databaseRepository.getAllItems(User.class)
       .first(EMPTY)
       .map(user -> !user.equals(EMPTY) && user.getId() == JWTUtils.getIdentity(token));
}

 

We have added a default user EMPTY in the first operator so that RxJava returns it if there are no users in the database and then we simply map the user to a boolean denoting if they are same or different using the EMPTY user and getIdentity method from JWTUtils

Finally, we use all this information to implement our self contained login request:

eventService
   .login(new Login(username, password))
   .flatMapSingle(loginResponse -> {
       String token = loginResponse.getAccessToken();
       utilModel.saveToken(token);

       return isPreviousUser(token);
   })
   .flatMapCompletable(isPrevious -> {
       if (!isPrevious)
           return utilModel.deleteDatabase();

       return Completable.complete();
   });

 

Let’s see what is happening here. A request using username and password is made to the server which returns a login response containing a JWT, which we store for future use. Next, we flatMapSingle to the Single returned by the isPreviousUser method. And we finally clear the database if it is not a previous user.

Creating these self contained models help reduce complexity in presenter or view layer and all data is handled in one layer making presenter layer model agnostic.

To learn more about JWT and some of the Rx operators I mentioned here, please visit these links:

Shrinking Model Classes Boilerplate in Open Event Android Projects Using Jackson and Lombok

JSON is the de facto standard format used for REST API communication, and for consuming any of such API on Android apps like Open Event Android Client and Organiser App, we need Plain Old Java Objects, or POJOs to map the JSON attributes to class properties. These are called models, for they model the API response or request. Basic structure of these models contain

  • Private properties representing JSON attributes
  • Getters and Setters for these properties used to change the object or access its data
  • A toString() method which converts object state to a string, useful for logging and debugging purposes
  • An equals and hashcode method if we want to compare two objects

These can be easily and automatically be generated by any modern IDE, but add unnecessarily to the code base for a relatively simple model class, and are also difficult to maintain. If you add, remove, or rename a method, you have to change its getters/setters, toString and other standard data class methods.

There are a couple of ways to handle it:

  • Google’s Auto Value: Creates Immutable class builders and creators, with all standard methods. But, as it generates a new class for the model, you need to add a library to make it work with JSON parsers and Retrofit. Secondly, there is no way to change the object attributes as they are immutable. It is generally a good practice to make your models immutable, but if you are manipulating their data in your application and saving it in your database, it won’t be possible except than to create a copy of that object. Secondly, not all database storage libraries support it
  • Kotlin’s Data Classes: Kotlin has a nice way to made models using data classes, but it has certain limitations too. Only parameters in primary constructor will be included in the data methods generated, and for creating a no argument constructor (required for certain database libraries and annotation processors), you either need to assign default value to each property or call the primary constructor filling in all the default values, which is a boilerplate of its own. Secondly, sometimes 3rd party libraries are needed to work correctly with data classes on some JSON parsing frameworks, and you probably don’t want to just include Kotlin in your project just for data classes
  • Lombok: We’ll be talking about it later in this blog
  • Immutables, Xtend Lang, etc

This is one kind of boilerplate, and other kind is JSON property names. As we know Java uses camelcase notation for properties, and JSON attributes are mostly:

  • Snake Cased: property_name
  • Kebab Cased: property-name

Whether you are using GSON, Jackson or any other JSON parsing framework, it works great for non ambiguous property names which match as same in both JSON and Java, but requires special naming attributes for translating JSON attributes to camelcase and vice versa. Won’t you want your parser to intelligently convert property_name to propertyName without having you write the tedious mapping which is not only a boilerplate, but also error prone in case your API changes and you forget to update the annotations, or make a spelling mistake, as they are just non type-safe strings.

These boilerplates cause serious regressions during development for what should be a simple Java model for a simple API response. These both kinds of boilerplate are also related to each other as all JSON parsers look for getters and setters for private fields, so there are two layers of potential errors in modeling JSON to Java Models. This should be a lot simpler than it is. So, in this blog, we’ll see how we can configure our project to be 0 boilerplate tolerant and error free. We reduced approximately 70% boilerplate using this configuration in our projects. For example, the Event model class we had (our biggest) reduced from 590 lines to just 74!

We will use a simpler class for our example here:

public class CallForPapers {

    private String announcement;
    @JsonProperty("starts-at")
    private String startsAt;
    private String privacy;
    @JsonProperty("ends-at")
    private String endsAt;

    // Getters and Setters

    @Override
    public String toString() {
        return "CallForPapers{" +
                "announcement='" + announcement + '\'' +
                ", startsAt='" + startsAt + '\'' +
                ", privacy='" + privacy + '\'' +
                ", endsAt='" + endsAt + '\'' +
                '}';
    }
}

 

Note that getters and setters have been omitted for brevity. The actual class is 57 lines long

As you can see, we are using @JsonProperty annotation to properly map the starts-at attribute to startsAt property and similarly on endsAt. First, we’ll remove this boilerplate from our code. Note that this seems a bit overkill for 2 attributes, but imagine the time you’ll save by not having to maintain 100s of attributes for the whole project.

Jackson is smart enough to map different naming styles to one another in both serializing and deserializing. This is done by using Naming Strategy class in Jackson. There is an option to globally configure it, but I found that it did not work for our case, so we had to apply it to each model. It can be simply done by adding another annotation on the top of your class declaration and removing the JsonProperty attribute from your fields

@JsonNaming(PropertyNamingStrategy.KebabCaseStrategy.class)
public class CallForPapers {

    private String announcement;
    private String startsAt;
    private String privacy;
    private String endsAt;

    // Getters and Setters

    @Override
    public String toString() {
        return "CallForPapers{" +
                "announcement='" + announcement + '\'' +
                ", startsAt='" + startsAt + '\'' +
                ", privacy='" + privacy + '\'' +
                ", endsAt='" + endsAt + '\'' +
                '}';
    }
}

 

Our class looks like this now. But be careful to properly name your getters and setters because now, Jackson will map attributes by method names, so if you name the setter for startsAt -> setStartsAt it will automatically understand that the attribute to be mapped is “starts-at”. But, if the method name is something else, then it won’t be able to correctly map the fields. If your properties are not private, then Jackson may instead use them to map fields, so be sure to name your public properties in a correct manner.

Note: If your API does not use kebab case, there are plenty of other options or naming strategies present in Jackson, one example will be

  • PropertyNamingStrategy.SnakeCaseStrategy for attributes like “starts_at”

 Needless to say, this will only work if your API uses a uniform naming strategy

Now we have removed quite a burden from the development lifecycle, but 70% of class is still getters, setters, toString and other data methods. Now, we’ll configure lombok to automatically generate these for us. First, we’ll need to add lombok in our project by adding provided dependency in build.gradle and sync the project

provided 'org.projectlombok:lombok:1.16.18'

 

And now you’d want to install Lombok plugin in Android Studio by going to Files > Settings > Plugins and searching and installing Lombok Plugin and restarting the IDE

 

After you have restarted the IDE, navigate to your model and add @Data annotation at the top of your class and remove all getters/setters, toString, equals, hashcode and if the plugin was installed correctly and lombok was installed from the gradle dependencies, these will be automatically generated for you at build time without any problem. A way for you to see the generated methods is to the structure perspective in the Project Window.

 

There are many more fun tools in lombok and more fine grained control options are provided. Our class looks like this now

@Data
@JsonNaming(PropertyNamingStrategy.KebabCaseStrategy.class)
public class CallForPapers {
    private String announcement;
    private String startsAt;
    private String privacy;
    private String endsAt;
}

 

Reduced to 16 lines (including imports and package). Now, there are some corner cases that you want to iron out for the integration between lombok and Jackson to work correctly.

Lombok uses property names for generating its getters and setters. But there’s a different convention for handling booleans. For the sake of simplicity, we’ll only talk about primitive boolean. You can check out the links below to learn more about class type. The primitive boolean property of the standard Java format, for example hasSessions will generate hasSessions and getter and setHasSessions. Jackson is smart but it expects a getter named getHasSessions creating problems in serialization. Similarly, for a property name isComplete, generate getter and setter will be isComplete and setComplete, creating a problem in deserialization too. Actually, there are ways how Jackson can get boolean values mapped correctly with these getters/setters, but that method needs to rename property itself, changing the getters and setters generated by Lombok. There is actually a way to tell Lombok to not generate this format of getter/setter. You’d need to create a file named lombok.config in your project directory app/ and write this in it

lombok.anyConstructor.suppressConstructorProperties = true
lombok.addGeneratedAnnotation = false
lombok.getter.noIsPrefix = true

 

There are some other settings in it that make it configured for Android specific project

There are some known issues in Android related to Lombok. As lombok itself is an annotation processor, and there is no order for annotation processors to run, it may create problems with other annotation processors. Dagger had issues with it until they fixed it in their later versions. So you might need to check out if any of your libraries depend upon the lombok generated code like getters and setters. Certain database libraries use that and Android Data Binding does too. Currently, there is no solution to the problem as they will throw an error about not finding a getter/setter because they ran before lombok. A possible workaround is to make properties public so that instead of using getters and setters, these libraries use them instead. This is not a good practice, but as this is a data class and you are already creating getters and setters for all fields, this is not a security vulnerability.

There are tons of options for both Jackson and Lombok with a lot of features to help the development process, so be sure to check out these links:

Smart Data Loading in Open Event Android Orga App

In any API centric native application like the Open Event organizer app (Github Repo), there is a need to access data through network, cache it for later use in a database, and retrieve data selectively from both sources, network and disk. Most of Android Applications use SQLite (including countless wrapper libraries on top of it) or Realm to manage their database, and Retrofit has become a de facto standard for consuming a REST API. But there is no standard way to manage the bridge between these two for smart data loading. Most applications directly make calls to DB or API service to selectively load their data and display it on the UI, but this breaks fluidity and cohesion between these two data sources. After all, both of these sources manage the same kind of data.

Suppose you wanted to load your data from a single source without having to worry from where it is coming, it’d require a smart model or repository which checks which kind of data you want to load, check if it is available in the DB, load it from there if it is. And if it not, call the API service to load the data and also save it in the DB there itself. These smart models are self contained, meaning they handle the loading logic and also handle edge cases and errors and take actions for themselves about storing and retrieving the data. This makes presentation or UI layer free of data aspects of the application and also remove unnecessary handling code unrelated to UI.

From the starting of Open Event Android Orga Application planning, we proposed to create an efficient MVP based design with clear separation of concerns. With use of RxJava, we have created a single source repository pattern, which automatically handles connection, reload and database and network management. This blog post will discuss the implementation of the AbstractObservableBuilder class which manages from which source to load the data intelligently

Feature Set

So, first, let’s discuss what features should our AbstractObservableBuilder class should have:

  • Should take two inputs – disk source and network source
  • Should handle force reload from server
  • Should handle network connection logic
  • Should load from disk observable if data is present
  • Should load from network observable if data is not present is disk observable

This constitutes the most basic data operations done on any API based Android application. Now, let’s talk about the implementation

Implementation

Since our class will be of generic type, we will used variable T to denote it. Firstly, we have 4 declarations globally

private IUtilModel utilModel;
private boolean reload;
private Observable<T> diskObservable;
private Observable<T> networkObservable;

 

  • UtilModel tells us if the device is connected to internet or not
  • reload tells if the request should bypass database and fetch from network source only
  • diskObservable is the database source of the item to be fetched
  • networkObservable is the network source of the same item

Next is a very simple implementation of the builder pattern methods which will be used to set these variable fields

@Inject
public AbstractObservableBuilder(IUtilModel utilModel) {
    this.utilModel = utilModel;
}

AbstractObservableBuilder<T> reload(boolean reload) {
    this.reload = reload;

    return this;
}

AbstractObservableBuilder<T> withDiskObservable(Observable<T> diskObservable) {
    this.diskObservable = diskObservable;

    return this;
}

AbstractObservableBuilder<T> withNetworkObservable(Observable<T> networkObservable) {
    this.networkObservable = networkObservable;

    return this;
}

 

UtilModel is the required dependency, and so is added as a constructor parameter.

All right, all variables are set up, now we need to create the build function to actually create the observable:

@NonNull
public Observable<T> build() {
    if (diskObservable == null || networkObservable == null)
        throw new IllegalStateException("Network or Disk observable not provided");

    return Observable
            .defer(getReloadCallable())
            .switchIfEmpty(getConnectionObservable())
            .compose(applySchedulers());
}

Reloading Logic

First of all, we check if the caller forgot to add disk or network source and throw an exception if it is actually so. Next, we use defer operator to defer the call to getReloadCallable() so that this function is not executed until this observable is subscribed. Some articles over the internet directly use combine operators from Rx to make things easy, but this lazy calling is the most efficient way to do things because no actual call will be made to observables.

Secondly, you can easily test the behaviour in unit tests, by verifying that

  • no call to the network observable was made if the data inside disk observable was present; or
  • call to network observable was made even if there was data in disk observable if the reload request was made

These tests would not have been possible if we did not employ the lazy call technique because the calls to the observables and utilModel would have been made before the subscription to this model happen, in order to create this observable eagerly.

Now, let’s see what getReloadCallable does

@NonNull
private Callable<Observable<T>> getReloadCallable() {
    return () -> {
        if (reload)
            return Observable.empty();
        else
            return diskObservable
                .doOnNext(item -> 
                    Timber.d("Loaded %s From Disk on Thread %s",
                    item.getClass(), Thread.currentThread().getName()));
    };
}

 

This function’s role is to return the disk observable if the request is not a reload call, or else return an empty observable, so that force network request happens to reload the data

So it returns a Callable which encapsulates this logic, and besides that, it also adds a log if loading from disk about the type of item loaded and the thread it was loaded on

Connection and Database Switch Logic

In the next chain of operation, we make a switchIfEmpty call to getConnectionObservable(). Because of the above reloading logic, switchIfEmpty serves 2 purpose here, it changes to API call:

  • if db does not contain data
  • If it is a reload call

The observable we switch to is returned by getConnectionObservable() and its purpose is to check if the device is connected to the internet, and if it is, to forward the network request and if it is not, then return an Error Observable.

@NonNull
private Observable<T> getConnectionObservable() {
    if (utilModel.isConnected())
        return networkObservable
            .doOnNext(item -> Timber.d("Loaded %s From Network on Thread %s",
                item.getClass(), Thread.currentThread().getName()));
    else
        return Observable.error(new Throwable(Constants.NO_NETWORK));
}

We use util model to determine if we are connected to internet and take action accordingly. As you can see, here too, we log about the data being loaded and the thread information.

Threading

Lastly, we want to ensure that all processing happens on correct threads, and for that, we call compose with an Observable Transformer to make all requests happen on I/O scheduler and the data is received on Android’s Main Thread

@NonNull
private <V> ObservableTransformer<V, V> applySchedulers() {
    return observable -> observable
        .subscribeOn(Schedulers.io())
        .observeOn(AndroidSchedulers.mainThread());
}

And that’s all it takes to create a reactive, generic and reusable data handler for disk and network based operations. In the repository pattern we have employed in the Open Event Android Orga Application, all our data switching and handling code is delegated to it, with unit tests and integration tests testing the individual and cross component working in all cases.

If you want to learn more about other implementations, you can read these articles

Dynamic Ticket Analysis UI using Data Binding in Open Event Android Orga App

Any event manager application has the responsibility to show the analytics about the event to the organiser and in Open Event Android Orga App (Github Repo), we wanted to achieve a way to display the analytics of total and sold tickets with the data present to us.
To analyse, we have a list of tickets, which are divided into 3 categories:

  • Free
  • Paid
  • Donation

Our goal was to show information about total tickets and the amount of sold tickets per category. This blog will focus on the dynamic UI creation for the ticket analysis component of the Event Details Dashboard using Android Layout Data Binding. By using Data Binding, we not only reduced the amount of Java Boilerplate code we would have to write, but also accomplished UI reuse in just XML which wouldn’t have been possible without it. You’ll see in a moment what I mean.

Properties

So first, we’d need to define some properties which will be bound in the UI. These properties are declared in the Event model and their type is ObservableLong provided by the Android DataBinding package. The reason why we are using these instead of primitives is because these fields being Observable, will update the UI as soon as they are updated, without requiring the programmer to set the View Property at all.

There are six fields, 3 for total tickets of each type and 3 for sold tickets

public final ObservableLong freeTickets = new ObservableLong();
public final ObservableLong paidTickets = new ObservableLong();
public final ObservableLong donationTickets = new ObservableLong();

public final ObservableLong soldFreeTickets = new ObservableLong();
public final ObservableLong soldPaidTickets = new ObservableLong();
public final ObservableLong soldDonationTickets = new ObservableLong();

Some more advantages we get from using these are the batch view update and the use of computed properties in UI. Imagine having a TextView display the amount of free tickets and a progress bar showing the percentage of free tickets sold. Traditionally, you’d have to set the text and compute the percentage and set the progress bar as the data changes, whereas you can just use the fields in layout as is in both TextView and ProgressBar with the computations required and they’ll work in harmony.

We have leveraged this feature to show the analytics component of tickets with a

  • Ticket Type
  • Circular Progress Bar
  • Sold Tickets
  • Total Tickets

All using the XML layout and databinding

Ticket Component

For each ticket component, we have 4 variables, namely

  • Ticket Type Name
  • Total Amount
  • Completed Amount (Sold Tickets)
  • Color

First 3 are fairly self explanatory, the color attribute we used in our component needs a little bit of description. We decided to give each ticket category its own color for circular progress bar for aesthetics. So, we need each component to have its own color attribute too. But this is not a normal android color ID or a hex. We needed 2 variants of the same color to show in the circular progress to discern the total and completed part. As we are using Material Color Palette, which has a color divided by intensities, we used 500 variant for completed portion and 100 (lighter) variant for the background of circular progress.

Let’s look at the layout now:

<data>
    <variable name="color" type="String" />
    <variable name="ticketName" type="String" />
    <variable name="total" type="long" />
    <variable name="completed" type="long" />
</data>

<LinearLayout
     android:orientation="vertical">
    <TextView
        android:text="@{ticketName}" />
    <FrameLayout>
        <com.mikhaellopez.circularprogressbar.CircularProgressBar
            app:circular_progress_color="@{color}"
            app:progress_with_animation="@{total == 0 ? 0 : (int) ((completed*100)/total)}" />

        <LinearLayout
            android:orientation="horizontal">

            <TextView
                android:text="@{completed}" />
            <TextView
                android:text='@{"/" + total}' />

        </LinearLayout>
    </FrameLayout>

    <TextView
        android:text='@{(total == 0 ? 0 : (int) ((completed*100)/total)) + "%"}' />

</LinearLayout>

Note: The layout snippet is not complete. Only attribute names to be discussed in the blog are shown for brevity

As you can see, after the data variable declarations, we have a CardView first showing the ticket name on top, and then we have a FrameLayout wrapping the circular progress and a textview showing the Sold/Total tickets.

Circular Progress Bar

Let’s discuss the circular progress first, we have used this library to create a circular progress bar, the two other attributes circular_progress_color and progress_with_animation are specific to Open Event Orga Application and we have created custom adapters for them:

@BindingAdapter("progress_with_animation")
public static void bindCircularProgress(CircularProgressBar circularProgressBar, int progress) {
    circularProgressBar.setProgressWithAnimation(progress, 500);
}

@BindingAdapter("circular_progress_color")
public static void bindCircularProgressColor(CircularProgressBar circularProgressBar, String colorName) {
    Context context = circularProgressBar.getContext();
    Resources resources = circularProgressBar.getResources();

    int color = ContextCompat.getColor(context, resources.getIdentifier(colorName + "_500", "color", context.getPackageName()));
    int bgColor = ContextCompat.getColor(context, resources.getIdentifier(colorName + "_100", "color", context.getPackageName()));

    circularProgressBar.setColor(color);
    circularProgressBar.setBackgroundColor(bgColor);
}
  • progress_with_animation sets the provided integer value as the progress of the circular progress bar with an animation of 500 ms
  • circular_progress_color finds the 100 and 500 variant of the color name string provided and sets them as background and foreground color of the progress bar

These are the color definitions we have used in the app:

<color name="light_blue_100">#B3E5FC</color>
<color name="light_blue_500">#03A9F4</color>
<color name="purple_100">#E1BEE7</color>
<color name="purple_500">#9C27B0</color>
<color name="red_100">#ffcdd2</color>
<color name="red_500">#f44336</color>

As you can that if we pass purple as the color name, it’ll load purple_100 and purple_500 and set it as corresponding background and foreground color

Other Properties

Now, let’s talk about other properties like the progress value :

  • total == 0 ? 0 : (int) ((completed*100)/total)The conditional is used to prevent divide by zero error.
    The same expression is used to display the circular progress and percentage text in the TextView at the bottom of the layout
  • completed and “/” + total are used to in TextViews of different sizes to create a nice design with completed/total format

This completes our ticket component design and now we’ll see how to reuse this component to display different ticket types.

Composite Layout

To use the ticket component, we just include the layout and bind specific variables from Event model to create a dynamic layout like this:

<data>
    <variable
        name="event"
        type="org.fossasia.openevent.app.data.models.Event" />
</data>

<LinearLayout
    android:orientation="vertical">

    <TextView
        android:text="@string/tickets" />

    <LinearLayout
        android:orientation="horizontal">

        <include
            layout="@layout/ticket_analytics_item"
            bind:color='@{"light_blue"}'
            bind:completed="@{event.soldFreeTickets}"
            bind:ticketName="@[email protected]/ticket_free}"
            bind:total="@{event.freeTickets}" />

        <include
            layout="@layout/ticket_analytics_item"
            bind:color='@{"purple"}'
            bind:completed="@{event.soldPaidTickets}"
            bind:ticketName="@[email protected]/ticket_paid}"
            bind:total="@{event.paidTickets}" />

        <include
            layout="@layout/ticket_analytics_item"
            bind:color='@{"red"}'
            bind:completed="@{event.soldDonationTickets}"
            bind:ticketName="@[email protected]/ticket_donation}"
            bind:total="@{event.donationTickets}" />
    </LinearLayout>

</LinearLayout>

The layout consists of a horizontal with 3 equally divided ticket components,

  • Free Ticket Component -> Light Blue
  • Paid Ticket Component -> Purple
  • Donation Ticket Component -> Red

This is how it looks on a device

 

So this is how data binding made us accomplish easily which would have been a very convoluted solution using traditional ID based view binding. For more info about data binding, refer to these sites:

https://developer.android.com/topic/libraries/data-binding/index.html

http://www.vogella.com/tutorials/AndroidDatabinding/article.html

Data Binding in Android

Implementing Barcode Scanning in Open Event Android Orga App using RxJava

One of the principal goals of Open Event Orga App (Github Repo) is to let the event organizer to scan the barcode identifier of an attendee at the time of entry in the event, in order to quickly check in that attendee. Although there are several scanning APIs available throughout the web for Android Projects, we chose Google Vision API as it handles multitude of formats and orientations automatically with little to no configuration, integrates seamlessly with Android, and is provided by Google itself, ensuring great support in future as well. Currently, the use case of our application is:

  • Scan barcode from the camera feed
  • Detect and show barcode information on screen
  • Iterate through the attendee list to match the detected code with unique attendee identifier
  • Successfully return to the caller with scanned attendee’s information so that a particular action can be taken

There are several layers of optimisations done in the Orga Application to make the user interaction fluid and concise. I’ll be sharing a few in this blog post regarding the configuration and handling of bursty data from camera source efficiently. To see the full implementation, you can visit the Github Repository of the project using the link provided above.

Configuration

The configuration of our project was done through Dagger 2 following the dependency injection pattern, which is not the focus of this post, but it is always recommended that you follow separation of concerns in your project and create a separate module for handling independent works like barcode scanner configuration. Normally, people would create factories with scanner, camera source and processors encapsulated. This enables the caller to have control over when things are initialized.

Our configuration provides us two initialised objects, namely, CameraSource and BarcodeDetector

@Provides
BarcodeDetector providesBarCodeDetector(Context context, Detector.Processor<Barcode> processor) {
    BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(context)
        .setBarcodeFormats(Barcode.QR_CODE)
        .build();

    barcodeDetector.setProcessor(processor);

    return barcodeDetector;
}

@Provides
CameraSource providesCameraSource(Context context, BarcodeDetector barcodeDetector) {
    return new CameraSource
        .Builder(context, barcodeDetector)
        .setRequestedPreviewSize(640, 480)
        .setRequestedFps(15.0f)
        .setAutoFocusEnabled(true)
        .build();
}

The fields needed to create the objects are provided as arguments to the provider functions as all the dependencies required to construct these objects. Now focusing on the Detector.Processor requirement of the BarcodeDetector is the classic example on non injectable code. This is because the processor is to be supplied by the activity or any other object which wants to receive the callback with the detected barcode data. This means we could inject it at the time of creation of the Activity or Presenter itself. We could easily overcome by adding a constructor to this dagger module containing the Barcode.Processor at the time of injection, but that would violate our existing 0 configuration based model where we just get the required component from the Application class and inject it. So, we wrapped the the processor into a PublishSubject

@Provides
@Singleton
@Named("barcodeEmitter")
PublishSubject<Notification<Barcode>> providesBarcodeEmitter() {
    return PublishSubject.create();
}

@Provides
@Singleton
Detector.Processor<Barcode> providesProcessor(@Named("barcodeEmitter") PublishSubject<Notification<Barcode>> emitter) {
    return new Detector.Processor<Barcode>() {
        @Override
        public void release() {
            // No action to be taken
        }

        @Override
        public void receiveDetections(Detector.Detections<Barcode> detections) {
            SparseArray<Barcode> barcodeSparseArray = detections.getDetectedItems();
            if (barcodeSparseArray.size() == 0)
                emitter.onNext(Notification.createOnError(new Throwable()));
            else
                emitter.onNext(Notification.createOnNext(barcodeSparseArray.valueAt(0)));
        }
    };
}

This solves 2 of our problems, not only now all these dependencies are injectable with 0 configurations, but also our stream of barcodes is now reactive.

Note that not everyone is in favour of using Singleton, but you can decrease the scope using your own annotation. We prefer not creating life cycle bound objects, those are hard to manage and can cause potential memory leaks, and the creation of an anonymous inner class object every time listener activates is not good for memory too.

Also, note that Singleton classes will cause memory leaks too if you don’t release their reference at the time of destruction of life cycle bound object

Notice how the type of PublishSubject is not just a barcode, but Notification which wraps the bar code. That’s because we want to send both the value and error streams down uninterrupted to the caller. Otherwise, the data stream would have stopped on the emission of first onError call. Here, we detect the barcodeSparseArray size and accordingly send error or first value to the PublishSubject which will be accordingly subscribed by the activity or presenter

Handling Bursty Data

barcodeEmitter.subscribe(barcodeNotification -> {
    if (barcodeNotification.isOnError()) {
        presenter.onBarcodeDetected(null);
    } else {
        presenter.onBarcodeDetected(barcodeNotification.getValue());
    }
});

Here is how we are subscribing the notification emitter and passing the appropriate value to the presenter to handle, null if it is an error and the value if it is the next emission.

Note that you must dispose the disposable returned by the subscribe method on the subject when the Activity is to be destroyed or else it will keep the reference to the anonymous inner class created with the lambda for barcodeNotification and cause a memory leak

Now, let’s see how the presenter handles this data for:

  • Hiding and showing the barcode panel when barcode is on the screen accordingly
  • Showing the data extracted from the barcode scanner

These things can be implemented in a very standard way with a few conditionals, but most developers forget the fact that the data emission rate is enormous when concerning with live feed of data. In the Open Event Orga app, we have reduced it to 15 FPS as it is more than enough to scan barcodes for our use case, but it is still huge. The continuous stream of nulls and barcode data is useless to us unless it changes.

A little explanation about nulls and values here: You must have noticed above the conditions when we pass null and value, but I’ll explain again. A value will be passed if there is a detected barcode on screen, and null will be passed if there is no barcode detected. The Google Vision API will keep sending the same value for barcode at 15 FPS and so we’ll get this redundant stream of nulls and values which we should not concern with processing as this will load the CPU unnecessarily.

There are only 2 cases where we need to process it:

  • Null changes to Value -> Show barcode panel
    Value changes to null -> Hide barcode panel
  • Value changes irrespective of nulls -> Show barcode data on UI and search through the attendee identifiers

So, here too we’ll create 2 PublishSubject objects

private PublishSubject<Boolean> detect = PublishSubject.create();
private PublishSubject<String> data = PublishSubject.create();

And we’ll configure them both in this way to receive data on each barcode emission in the presenter:

public void onBarcodeDetected(Barcode barcode) {
   detect.onNext(barcode == null);
   if (barcode != null)
       data.onNext(barcode.displayValue);
}

This will make data only receive non-null changes and detect receive a boolean notifying if the current detected barcode was null or not.

Now, we see how each of these subjects is configured to pass the emissions downstream:

data.distinctUntilChanged()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(this::processBarcode);

This one is pretty straightforward. It’ll only send data downstream if its value has changed from the previous emission, disregarding nulls. So, for a stream like this

A A A A A null null null null A A A A null B B B B B B null null null B B A A null A

It will only emit:

A                                                                         B                                                  A

Which is actually what we want, as we only need to process and show the distinct barcode data and match it with our attendee list. Now, with thousands of attendees, the first method would have triggered unnecessary and time-consuming computations over and over again on same identifiers with little gaps of time, which would have created mediocre results even in a multi-threaded environment. The second one saves us from repetitive calculations and also gives us enormous gaps between emissions, which is optimal for our use case.

The second case is not so obvious because we can’t ignore nulls here as we have to show and hide UI based on them. This means that unlike our previous stream if we just use distinctUntilChanged, it will look like this:

A A A A A null null null null A A A A null B B B B B B null null null B B A A null A

f                 t                               f               t      f                    t                        f             t       f

This is because, if you remember, we were sending down emissions of barcode == null on each emission for this Subject. So, in this case, as you may see, some of the values are so close enough that it will not be discernable in UI and also annoy users who’ll see the panel pop up for milliseconds before vanishing or vice-versa. The perfect operation for this case will be debounce

detect.distinctUntilChanged()
    .debounce(150, TimeUnit.MILLISECONDS)
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(receiving -> scanQRView.showBarcodePanel(!receiving));

This operator will drop any emission in the window of 150ms succession and only pick up those emissions which are 150ms apart from each other. Now, 150 ms is not a magic number, it is picked through hit and trial and what works best for your case. Lower the value and you will pick up more changes downstream, increase the value and you might miss the required events.

This makes our stream somewhat like this, cleaning out the cluttered events

f                 t                               f                                           t                        f

This is the screenshot of the implementation:  

 

And an animated gif of the scanning process:

This is all for this blog, you may use many other operators from the arsenal of RxJava, whatever fits your use case. As I have presumed the knowledge about Subjects, MVP and a little bit of Dagger in this post, I’ll link some of the resources where you can find more information about these:

http://reactivex.io/documentation/subject.html

https://github.com/ReactiveX/RxJava/wiki/Subject

http://reactivex.io/documentation/operators/distinct.html

http://www.vogella.com/tutorials/Dagger/article.html

https://antonioleiva.com/mvp-android/

Testing Asynchronous Code in Open Event Orga App using RxJava

In the last blog post, we saw how to test complex interactions through our apps using stubbed behaviors by Mockito. In this post, I’ll be talking about how to test RxJava components such as Observables. This one will focus on testing complex situations using RxJava as the library itself provides methods to unit test your reactive streams, so that you don’t have to go out of your way to set contraptions like callback captors, and implement your own interfaces as stubs of the original one. The test suite (kind of) provided by RxJava also allows you to test the fate of your stream, like confirming that they got subscribed or an error was thrown; or test an individual emitted item, like its value or with a predicate logic of your own, etc. We have used this heavily in Open Event Orga App (Github Repo) to detect if the app is correctly loading and refreshing resources from the correct source. We also capture certain triggers happening to events like saving of data on reloading so that the database remains in a consistent state. Here, we’ll look at some basic examples and move to some complex ones later. So, let’s start.

public class AttendeeRepositoryTest {

    private AttendeeRepository attendeeRepository;

    @Before
    public void setUp() {
        testDemo = new TestDemo();
    }

    @Test
    public void shouldReturnAttendeeByName() {
        // TODO: Implement test
    }

}

 

This is our basic test class setup with general JUnit settings. We’ll start by writing our tests, the first of which will be to test that we can get an attendee by name. The attendee class is a model class with firstName and lastName. And we will be checking if we get a valid attendee by passing a full name. Note that although we will be talking about the implementation of the code which we are writing tests for, but only in an abstract manner, meaning we won’t be dealing with production code, just the test.

So, as we know that Observables provide a stream of data. But here, we are only concerned with one attendee. Technically, we should be using Single, but for generality, we’ll stick with Observables.

So, a person from the background of JUnit would be tempted to write this code below.

Attendee attendee = attendeeRepository.getByAttendeeName("John Wick")
    .blockingFirst();

assertEquals("John Wick", attendee.getFirstName() + attendee.getLastName());

 

So, what this code is doing is blocking the thread till the first attendee is provided in the stream and then checking that the attendee is actually John Wick.

While this code works, this is not reactive. With the reactive way of testing, not only you can test more complex logic than this with less verbosity, but it naturally provides ways to test other behaviors of Reactive streams such as subscriptions, errors, completions, etc. We’ll only be covering a few. So, let’s see the reactive version of the above test.

attendeeRepository.getByAttendeeName("John Wick")
    .firstElement()
    .test()
    .assertNoErrors()
    .assertValue(attendee -> "John Wick".equals(
        attendee.getFirstName() + attendee.getLastName()
    ));

 

So clean and complete. Just by calling test() on the returned observable, we got this whole suite of testing methods with which, not only did we test the name but also that there are no errors while getting the attendee.

Testing for Network Error on loading of Attendees

OK, so let’s move towards a more realistic test. Suppose that you call this method on AttendeeRepository, and that you can fetch attendees from the network. So first, you want to handle the simplest case, that there should be an error if there is no connection. So, if you have (hopefully) set up your project using abstractions for the model using MVP, then it’ll be a piece of cake to test this. Let’s suppose we have a networkUtil object with a method isConnected.

The NetworkUtil class is a dependency of AttendeeRepository and we have set it up as a mock in our test using Mockito. If this is sounding somewhat unfamiliar, please read my previous article “The Joy of Testing with MVP”.

So, our test will look like this

@Test
public void shouldStreamErrorOnNetworkDown() {
    when(networkUtils.isConnected()).thenReturn(false);
    
    attendeeRepository.getAttendees()
        .test()
        .assertErrorMessage("No Network");
}

 

Note that, if you don’t define the mock object’s behavior like I have here, attendeeRepository will likely throw an NPE as it will be calling isConnected() on an undefined object.

With RxJava, you get a whole lot of methods for each use case. Even for checking errors, you get to assert a particular Throwable, or a predicate defining an operation on the Throwable, or error message as I have shown in this case.

Now, if you run this code, it’ll probably fail. That’s because if you are testing this by offloading the networking task to a different thread by using subscribeOn observeOn methods, the test body may be detached from Main Thread while the requests complete. Furthermore, if testing in an application made for Android, you would have use AndroidSchedulers.mainThread(), but as it is an Android dependency, the test will fail. Well actually, crash. There were some workarounds by creating abstractions for even RxJava schedulers, but RxJava2 provides a very convenient method to override the default schedulers in the form of RxJavaPlugins. Similarly, RxAndroidPlugins is present in the rx-android package. Let’s suppose you have the plan to use Schedulers.io() for asynchronous work and want to get the stream on Android’s Main Thread, meaning you use AndroidSchedulers.mainThread() in the observeOn method. To override these schedulers to Schedulers.trampoline() which queues your tasks and performs them one by one, like the main thread, your setUp will include this:

RxJavaPlugins.setIoSchedulerHandler(scheduler ->  Schedulers.trampoline());
RxAndroidPlugins.setInitMainThreadSchedulerHandler(scheduler -> Schedulers.trampoline());

 

And if you are not using isolated tests and need to resume the default scheduler behavior after each test, then you’ll need to add this in your tearDown method

RxJavaPlugins.reset();
RxAndroidPlugins.reset();

Testing for Correct loading of Attendees

Now that we have tested that our Repository is correctly throwing an error when the network is down, let’s test that it correctly loads attendees when the network is connected. For this, we’ll need to mock our EventService to return attendees when queried, since we don’t want our unit tests to actually hit the servers.

So, we’ll need to keep these things in mind:

  • Mock the network until it shows that it is connected to the Internet
  • Mock the EventService to return attendees when queried
  • Call the getter on the attendeeRepository and test that it indeed returned a list of attendees

For these conditions, our test will look like this:

@Test
public void shouldLoadAttendeesSuccessfully() {
    List<Attendee> attendees = Arrays.asList(
        new Attendee(),
        new Attendee(),
        new Attendee()
    );

    when(networkUtils.isConnected()).thenReturn(true);
    when(eventService.getAttendees()).thenReturn(Observable.just(attendees));

    attendeeRepository.getAttendees()
        .test()
        .assertValues(attendees.toArray(new Attendee[attendees.size()]));
}

 

The assertValues function asserts that these values were emitted by the observable. And if you want to be terser, you can even verify that in fact EventService’s getAttendees function was called by

verify(eventService).getAttendees();

 

But the problem in this way is that the getAttendees function returns an observable and just calling it does not necessarily means that it was subscribed, emitting the results, hence we need to test to ensure that it was indeed subscribed. If we call the normal test() function on the observable, it is already subscribed, making the result of testSubscribed always true. In order to test that correctly, let’s look at our final use case.

Testing for saving of Attendees

In the Open Event Orga App, we have strived to create self-sufficient and intelligent classes, thus, our repository is also built this way. It detects that new attendees are loaded from the server and saves them in the database. Now we’d want to test this functionality.

In this test, there is an added dependency of DatabaseRepository for saving the attendees, which we will mock. The conditions for this test will be:

  • Network is connected
  • EventService returns attendees
  • DatabaseRepository mocks the saving of attendees

For DatabaseRepository’s save method, we’ll be returning a Completable, which will notify when the saving of data is completed. The primary purpose of this test will be to assert that this completable is indeed subscribed when the attendee loading is triggered. This will not only ensure that the correct function to save the attendees is called, but also that it is indeed triggered and not just left hanging after the call. So, our test will look like this.

@Test
public void shouldSaveAttendeesInDatabase() {
    List<Attendee> attendees = Arrays.asList(
        new Attendee(),
        new Attendee(),
        new Attendee()
    );

    TestObserver testObserver = TestObserver.create();
    Completable completable = Completable.complete()
        .doOnSubscribe(testObserver::onSubscribe);

    when(networkUtils.isConnected()).thenReturn(true);
    when(databaseRepository.save(attendees)).thenReturn(completable);
    when(eventService.getAttendees()).thenReturn(Observable.just(attendees));

    attendeeRepository.getAttendees()
        .test()
        .assertNoErrors();

    testObserver.assertSubscribed();
}

 

Here, we have created a separate test observable and set it to be subscribed when the Completable is subscribed and we have returned that Completable when the save method is called. In the last, we have asserted that the test observer is indeed subscribed.

You can create more complex use cases and assert subscriptions, errors, the emptiness of a stream and much more, by using the built-in test functionalities of RxJava2. So, that’s all for this blog, you can visit these links for more details on unit testing RxJava

http://fedepaol.github.io/blog/2015/09/13/testing-rxjava-observables-subscriptions/

https://www.infoq.com/articles/Testing-RxJava

The Joy of Testing with MVP in Open Event Orga App

Testing applications is hard, and testing Android Applications is harder. The natural way an Android Developer codes is completely untestable. We are born and molded into creating God classes – Activities and perform every bit of logic inside them. Some thought that introduction to Fragments will introduce a little bit of modularity, but we proved otherwise by shifting to God Fragments. When the natural urge of an Android Developer to

  • apply logic,
  • load UI,
  • handle concurrency (hopefully, not using AsyncTask),
  • load data – from the network; disk; and cache,
  • and manage the state of the Activity/Fragment

finally, meets with a new form of revelation that he/she should test his/her application, all of the concepts acquired are shattered in a moment. The person goes to Android Docs to see why he started coding that way and realizes that even the system is flawed – Android Documentation is full of examples promoting God Activities, which they have used to show the use of API for only one reason – brevity. It’s not uncommon for us to steal some code from StackOverflow or Android Docs, but we don’t realize that they are presented there without an application environment or structure, all the required component just glued together to get it functionally complete, so that reader does not have to configure it anymore. Why would an answer from StackOverflow load a barcode scanner using a builder? Or build a ContentProvider to show how to query sorted data from SQLite? Or use a singleton for your provider classes? The simple answer is, they won’t. But this doesn’t mean you shouldn’t too.

The first thought that enters developer’s mind when he gets his hand on a piece of code is to paste it in the correct position and get it to work. There is always this moment of hesitation where conscience rings a bell saying, “What are you doing? Is it the right way to do it? How many lines till you stop overloading this Activity? It’s 2000 lines already”. But it dies as soon as we see the feature is working. And why test something which we have confirmed to work, right? I just wrote this code to show a progress bar while the data loads and hide it when it is done. I can see this working, what will I achieve in painfully writing a test for it, mocking the loading conditions and all. Wrong! Unit tests which test your trivial utils like Date Modification, String Parsing, etc are good and needed but are so trivial that they are hard to go wrong, and if they are, they are easy to fix as they have single usage throughout the app, and it is easy to spot bugs and fix them.

The real problem is testing of your app over dynamic conditions, where you have to emulate them so you can see if your app is making the right decisions. The progress bar working example may work for now, but what if it breaks over refactoring, or you wrote the same code elsewhere and forget to hide the progress bar? Simply copying a well-written test and changing 2-3 class names will fail the build and tell you what’s wrong. A well-contracted app can even contain tests to check that there’ll be no memory leaks. But none of this is possible if everything is jumbled into single Activity with callbacks, loaders, UI handling, business logic, etc. You can’t even think that where to begin. In this post, I will briefly discuss, how to design and test applications using MVP pattern.

MVP to the Rescue

Before moving ahead, I must put a disclaimer saying that MVP is a design pattern which follows a very opinionated implementation. The extent to which you want to refactor your app and the number of abstractions you are willing to do are in your hand. There aren’t any golden rules where your implementation will fail to be called as MVP. Remember, the client doesn’t care about architecture, it’s for you, so whatever makes your life and testing easier, works. Also, I’ll use an axiom related to testing here, “Test until fear turns into boredom”. You can apply the same to your MVP implementation. Testing and design patterns are here to eradicate your fear of failure, not to bore you.

So, in this guide, I’ll be using a use case of a QR Code Scanner Activity built using MVP pattern which we have employed in Open Event Orga Application (Github Repo). Because we are focusing on the test pattern and how to make it easy for us to test our app logic, I have omitted the model part from the MVP equation. The reason being that the possible model in the application would have been the camera loader or barcode initializer and the caveats associated with following this is that both these modules rely heavily on the Android specific view classes, namely, SurfaceView and other lifecycle methods. You could always create your way around it to include them in a separate model, but it won’t help us in writing unit tests for them, because firstly, they aren’t our logic to test, and secondly, they can’t be tested in a unit test (those models would have probably just implemented certain setters and getters).

So, the main purpose of our QR Code Scanner class is to scan a QR code and match it with a list of identifiers, and if there is a match, return it successfully to the caller. In this specific example, the identifiers will the ticket IDs of event attendees and the caller will be event organizer scanning QR codes to check the attendee in. The use case sounds simple, but has several mini use cases and dependencies of itself, which we have to take into our account while designing the View and Presenter class. Let’s discuss them one by one:

App State – Start: Activity starts, loads camera

  • Permission Granted: detects that the app already has Camera Permission,
    • starts scanning
  • Permission Absent: detects that the app doesn’t have Camera Permission,
    • Denied: asks for it, denied, shows error
    • Accepted: asks for it, granted, starts scanning

App State – QR Code Detected: Starts parsing them

  • Attendees are not present: Attendees aren’t present because of some reason, either due to internal error or have not loaded yet
    • stops parsing
  • Attendees are present:
    • QR Code does not match with any attendee : Do nothing
    • QR Code matches with one of the attendee : Send attendee to caller

App State: Camera or containing View is getting destroyed

  • Release Camera

There can be much more internal data flows, but this much is sufficient for our example. So, let’s start defining our contracts using the above knowledge. First, we will design our View. So what should be our strategy? Always think of presenters and views to be mapped in a 1:1 relation. They can have conversations with each other, the difference is that the view is only allowed to talk to the presenter, but presenter may talk to models too. So, the view is going to get all of its information from the presenter and can only take action when presenter tells it to, essentially saying that the view is dumb and passive. The presenter can talk to view and model(s), meaning the collection of information presenter has does not have to come from the view only. In fact, the less dependent presenter has to be on view, the better. The main motive of MVP is to make our logic less dependent on views.

Presenter Contract

In our example, presenter relies on the view to tell it when a certain event happens, they can be lifecycle callbacks, permission grants/denies, camera load/destroy or anything purely related to the Android implementation. So, we generally know from the start what kind of information presenter needs from a View, so let’s design our presenter first.

public interface IScanQRPresenter {

    void attach(long eventId, IScanQRView scanQRView);

    void start();

    void detach();

    void cameraPermissionGranted(boolean granted);

    void onBarcodeDetected(Barcode barcode);

    void onScanStarted();

    void onCameraLoaded();

    void onCameraDestroyed();

}

 

The methods are self-explanatory. Don’t worry if you didn’t get why we defined certain hooks like onScanStarted() or why we didn’t define onScanStopped(). These kinds of details will reveal themselves as you develop your components. You can skip to next section if you don’t want to know why we did it.

Basically, you should define a callback for events which are not reliable to be synchronized. What? Let me explain. Let’s say you got your camera permission and requested the view to start scanning, but you have to wait till the camera is loaded as it is not a synchronous call (and it shouldn’t be or your main thread will block). So, instead of that, our presenter will request the camera to load, go in an idle state, wait for the onCameraLoaded() call, and then request the scan to start, and since it is also not a synchronous work, it will go into idle state again and wait for onScanStarted() and then further its work. Whenever the camera is destroyed, the onCameraDestroyed() callback will be called and we will stop the scanning, and since there is nothing to be done after that, we won’t wait till scanning has stopped, thus dropping the need of onScanStopped() callback.

View Contract

View contract will come naturally to you once you have understood the data flow. It will contain the commands presenter will issue on the view and also, the requests for data that view holds. So, this will be our view.

public interface IScanQRView {

    boolean hasCameraPermission();

    void requestCameraPermission();

    void showPermissionError(String error);

    void onScannedAttendee(Attendee attendee);

    void showBarcodePanel(boolean show);

    void showBarcodeData(@NonNull String data);

    void showProgressBar(boolean show);

    void loadCamera();

    void startScan();

    void stopScan();

}

 

Almost all commands are straightforward but let me quickly explain showBarcodePanel(boolean) and showBarcodeData(String). They are used to display the currently visible barcode data to the user.

So, with our implementations set and data flow in place, let’s start writing tests for the feature. Yes, we’ll write tests without implementation and then you’ll see how easy it will be to write your views and presenters with only one goal to mind, to make the tests pass. If your tests are written correctly and cover everything, you should feel confident that your app will work without even seeing the actual implementation, because that is what tests are for. And by making passive and dumb views, imagine how light your instrumentation tests will be. Just check that the individual methods in view implementation are working as expected and you are done! No need to test logic or complex interactions, etc because you have got it covered in the unit tests themselves. This is not only a benefit of data flow tests but also a best practice. You always want to follow DRY, Don’t Repeat Yourself, even while testing.

Tests

So, we will start writing our tests now and you’ll realize how easy it is and all the hard work of abstraction and designing will pay off.

Attach Tests

So, firstly, we will test if the presenter calls appropriate methods when it is attached

@Test
public void shouldLoadAttendeesAutomatically() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));

    scanQRPresenter.start();

    verify(eventRepository).getAttendees(eventId, false);
}

@Test
public void shouldLoadCameraAutomatically() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));

    scanQRPresenter.start();

    verify(scanQRView).loadCamera();
}

 

Here, we are using Mockito to mock our EventDataRepository to return locally defined attendees instead of doing an actual network call. Then we are calling attach on presenter in each method and then we verify in the first test that the presenter is calling getAttendees on the EventDataRepository, and in the second test that it is requesting the view to load the camera.

Note that in the implementation of attach function, both loading of Camera and attendee loading will take place, but it is best practice to test them separately so that when a test fails, we know why it did

Detach Tests

@Test
public void shouldDetachViewOnStop() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));

    scanQRPresenter.start();

    assertNotNull(scanQRPresenter.getView());

    scanQRPresenter.detach();

    assertNull(scanQRPresenter.getView());
}

@Test
public void shouldNotAccessViewAfterDetach() {
    scanQRPresenter.detach();

    scanQRPresenter.start();
    scanQRPresenter.onCameraLoaded();
    scanQRPresenter.cameraPermissionGranted(false);
    scanQRPresenter.cameraPermissionGranted(true);
    scanQRPresenter.onScanStarted();
    scanQRPresenter.onBarcodeDetected(barcode2);
    scanQRPresenter.onCameraDestroyed();

    verifyZeroInteractions(scanQRView);
}

 

In detach tests, we are verifying that after attaching and view not being null, the detach method call makes the presenter leave the reference to the view making it null. And in the second test, we do all possible interactions after detaching and confirm that no call whatsoever was made on the view. Tests like these enforce to avoid memory leaks and check for any NullPointerExceptions that may happen after the view was made null.

Note: This does not mean that memory leaks will not happen if this test passes. You can cause memory leaks by giving the view reference to any long living object, not just the presenter. This just ensures that presenter will not hold the view reference after detach and won’t reference to the null view in future.

Permission Tests

@Test
public void shouldStartScanOnCameraLoadedIfPermissionPresent() {
    when(scanQRView.hasCameraPermission()).thenReturn(true);

    scanQRPresenter.onCameraLoaded();

    verify(scanQRView).startScan();
}

@Test
public void shouldAskPermissionOnCameraLoadedIfPermissionsAbsent() {
    when(scanQRView.hasCameraPermission()).thenReturn(false);

    scanQRPresenter.onCameraLoaded();

    verify(scanQRView).requestCameraPermission();
}

@Test
public void shouldStartScanningOnPermissionGranted() {
    scanQRPresenter.cameraPermissionGranted(true);

    verify(scanQRView).startScan();
}

@Test
public void shouldShowErrorOnPermissionDenied() {
    scanQRPresenter.cameraPermissionGranted(false);

    verify(scanQRView).showPermissionError(matches("(.*permission.*denied.*)|(.*denied.*permission.*)"));
}

The permission tests are straightforward:

Implicit Permission Handling

  1. If the view already has the camera permission and camera has loaded, start scanning
  2. If the view does not have camera permission and the camera has loaded, request the permission

Request Handling

  1. If the request was granted, start scanning
  2. If the permission was denied, show the permission error. Here, I have used regex to match that the error message contains permission and denied words, you can use anyString() from Mockito for more flexibility or a specific message for more tight testing

Camera Destroy Test

@Test
public void shouldStopScanOnCameraDestroyed() {
    scanQRPresenter.onCameraDestroyed();

    verify(scanQRView).stopScan();
}

Pretty simple, stop the scan on destruction of camera

Flow Tests

You can also test that the callback flow happens in order so that not only the unit tests work but also the implementation logic is correct.

/**
 * Checks that the flow of commands happen in order
 */
@Test
public void shouldFollowFlowOnImplicitPermissionGrant() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));
    when(scanQRView.hasCameraPermission()).thenReturn(true);

    scanQRPresenter.start();

    InOrder inOrder = inOrder(scanQRView);
    inOrder.verify(scanQRView).loadCamera();
    scanQRPresenter.onCameraLoaded();
    inOrder.verify(scanQRView).startScan();
}

@Test
public void shouldShowProgressInBetweenImplicitPermissionGrant() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));
    when(scanQRView.hasCameraPermission()).thenReturn(true);

    scanQRPresenter.start();

    InOrder inOrder = inOrder(scanQRView);
    inOrder.verify(scanQRView).showProgressBar(true);
    scanQRPresenter.onCameraLoaded();
    scanQRPresenter.onScanStarted();
    inOrder.verify(scanQRView).showProgressBar(false);
}

Here, we are verifying two things, first that if we have the request granted implicitly, we load the camera and start the scan in order. This test isn’t that useful as we already tested that loading of the camera is done on attach and scan is started when the camera is loaded. In fact, this is an example of breaking the DRY rule. Even though it doesn’t hurt to include this, it also doesn’t help as it does not cover anything that hasn’t already been tested.

The second test is important and tests that progress bar is correctly shown and hidden after certain communications have taken place and the scan has started. Similarly, we can also test the progress bar behavior over all of the possible combinations of cases that can happen. The code snippets below show the tests:

@Test
public void shouldShowProgressInBetweenImplicitPermissionDenyRequestGrant() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));
    when(scanQRView.hasCameraPermission()).thenReturn(false);

    scanQRPresenter.start();

    InOrder inOrder = inOrder(scanQRView);
    inOrder.verify(scanQRView).showProgressBar(true);
    scanQRPresenter.onCameraLoaded();
    scanQRPresenter.cameraPermissionGranted(true);
    scanQRPresenter.onScanStarted();
    inOrder.verify(scanQRView).showProgressBar(false);
}

@Test
public void shouldShowProgressInBetweenImplicitPermissionDenyRequestDeny() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));
    when(scanQRView.hasCameraPermission()).thenReturn(false);

    scanQRPresenter.start();

    InOrder inOrder = inOrder(scanQRView);
    inOrder.verify(scanQRView).showProgressBar(true);
    scanQRPresenter.onCameraLoaded();
    scanQRPresenter.cameraPermissionGranted(false);
    inOrder.verify(scanQRView).showProgressBar(false);
}

QR Code Detection Tests

@Test
public void shouldNotSendAnyBarcodeIfAttendeesAreNull() {
    sendNullInterleaved();

    verify(scanQRView, never()).onScannedAttendee(any(Attendee.class));
}

@Test
public void shouldNotSendAttendeeOnWrongBarcodeDetection() {
    scanQRPresenter.setAttendees(attendees);
    sendNullInterleaved();

    verify(scanQRView, never()).onScannedAttendee(any(Attendee.class));
}

@Test
public void shouldSendAttendeeOnCorrectBarcodeDetection() {
    // Somehow the setting in setUp is not working, a workaround till fix is found
    RxJavaPlugins.setComputationSchedulerHandler(scheduler -> Schedulers.trampoline());

    scanQRPresenter.setAttendees(attendees);

    barcode1.displayValue = "test4-91";
    scanQRPresenter.onBarcodeDetected(barcode1);

    verify(scanQRView).onScannedAttendee(attendees.get(3));
}

 

Lastly, the core tests of our presenter. To explain the tests, let me show you the sendNullInterleaved() method

private void sendNullInterleaved() {
    sendBarcodeBurst(barcode1);
    sendBarcodeBurst(null);
    sendBarcodeBurst(barcode1);
    sendBarcodeBurst(null);
    sendBarcodeBurst(barcode2);
    sendBarcodeBurst(barcode2);
    sendBarcodeBurst(null);
    sendBarcodeBurst(barcode1);
    sendBarcodeBurst(null);
}

 

So what it basically does is send some barcodes interleaved with null (no barcode detected) to the presenter using the onBarcodeDetected method to emulate the real camera sending barcode values with sometimes sending null whenever no barcode is in view.

The first test simply checks that no attendee is sent if the attendee list is null. Seems pretty obvious. The second one checks that if barcode does not match with any attendee’s identifier, it should not send any attendee as well. It does this by setting an arbitrary list of attendees with different identifiers and sending non-matching barcodes to the presenter. Lastly, the successful test, where a single matching barcode is sent to the presenter and it should send that particular attendee with matching identifier.

Phew! Quite a lot of tests and we are done. The tests were not very large and mostly self-explanatory. Now, you just have to implement the view and presenter methods till all the tests light up to be green and you are done! You have created a feature using MVP design and implemented it using test driven development.

So start testing and get a seal of reliability and confidence about your code and the satisfaction of seeing the green bar fill up.

Intro to concurrency and Refactoring Open Event Android using RxJava

Functional reactive programming seems to have taken the whole development world by storm. It’s one of the hottest thing even after 2 years of constant traction in the communities of several programming languages, where different implementations of the specifications defined by Rx or Reactive Extensions have changed the paradigm of programming for many professional and enthusiast developers.

RxJava is no exception, not only has it been widely adopted by Android and Java developers unanimously, but also received attention of well known and top developers of both communities. The reason of its success is the fluent API with heavy toolset it provides from the Functional Programming paradigm and its ease and natural ability to handle concurrency on different levels based on the type of operation being performed, i.e., computations, I/O, etc. It basically takes away the several constraints and concurrency related checklists developers had to maintain while working with thread management. So, now, developers can’t make an excuse for using database operations on the Main Thread because offloading it on another thread is hard.

So, in this blog post, I will be detailing the process of converting the existing synchronous code of your app into a performant reactive code without breaking the structure of your project, like we did in Open Event Android (Github Repo). Before starting, I have assumed that you know how to add RxJava dependency to your project as it is covered in many other blog posts and the documentation is also very clear. Secondly, you should also add RxAndroid dependency as it contains the Scheduler needed to work on Android’s Main Thread. So, Let’s start.

Current State

Currently, our code loads the queries from database synchronously on Main Thread using the SQLiteDatabase for an Android application. This is how it looks like –

As we can see, we are directly returning the loaded results to the caller. This is called synchronous call, meaning the caller will block till the function is returned, and can’t move further to do anything else. It basically waits for the function to return, which may take hundreds of milliseconds to seconds based on the function it performs.

New Android version crash the applications that perform Network interactions on the main thread but no such restriction for disk based operations is there, making it hard to enforce best performance practices. Before RxJava, there were interfaces made for different kinds of objects, passed in as parameters of the db request function, which created a new thread and performed operations and when completed, returned back the results to the main thread using the postOnUiThread method, so that the views could update themselves. The interface implementations passed are called callbacks because they call a particular function that you provide back when the asynchronous operation is completed. Even the calling of callback function is delegated on the implementor and may result in undesired effects. The query done in this fashion is called an asynchronous query because the execution of this takes place in parallel with main thread and is not synchronised with the main thread. It may take up forever to complete, complete even before the main thread moved on to next operation or even return when the main thread was completed and done waiting for it and destroyed. This will result in a weird crash even when the application was closed, because the returned function will try to update the views which are not even there.

Problems like these made Android Devs lazy and compromise with the performance of their application. Not anymore! RxJava is here to solve half of our problems. You see, RxJava does provide a solution to achieve effortless concurrency but does not ensure thread safety, memory contention, race conditions, deadlocks and other concurrency related issues for you. These you must code up for yourself.

So, after the introduction of Rx and its dire need in Android projects, we will move on to a basic procedure to convert any synchronous code to asynchronous call using RxJava Observable.

Let’s subscribe

The Observable class in RxJava is the most used and standard stream class you will use. Observable handles a stream of object and passes them as they arrive to the Subscriber attached to it. As you may guess, for a stream of data that arrives in a non deterministic fashion ( we don’t know when it will arrive ), we require an asynchronous query, and this is where RxJava excels at. You can configure an Observable to wait for result in one thread so that main thread doesn’t block and deliver result on another thread. You can either create a new thread or use certain pre configured schedulers for basic type of operations :

  1. Schedulers.newThread() : Creates a new thread for each request
  2. Schedulers.io() : For I/O bound work like a network call, database access
  3. Schedulers.computation() : For heavy computations
  4. AndroidSchedulers.mainThread() : For returning to UI thread of Android ( Present in RxAndroid )

There are other types of Schedulers like Schedulers.trampoline(), etc that are used for other purposes like testing, but the above ones are most commonly used ones and we’ll be using Schedulers.computation() for loading the SQLite query on the thread from Computation Thread Pool and AndroidSchedulers.mainThread() for delivering the result on UI thread.

Using Computation instead of I/O because I/O uses unbounded executor, meaning it continues adding threads to the thread pool, which isn’t good. So, we use computation instead. You can create your own bounded executor and pass it as a scheduler

The basic operation of passing an object to a subscriber is :

Observable.just(getEventDetails())
        .subscribe(new Consumer<Event>() {
          @Override
          public void accept(@NonNull Event event) throws Exception {
              Log.d("EVENT", event.toString());
          }
      });

 

Using lambda notation, we get a terse form of the same :

Observable.just(getEventDetails())
 	.subscribe(event -> Log.d("EVENT", event.toString()));

We’ll be using lambda notations from now on.

In the above example, we are just loading and passing the Event object to the subscriber below who logs it. But this is not asynchronous, everything gets executed on main thread. The above code is equivalent to :

Event event = getEventDetails();
Log.d("EVENT", event.toString());

 

So why use it, you say? Well, we can still get a lot of goodies from functional programming this way. For example,

String track = "Android";

Observable.fromIterable(getSessionList())
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .toList()
    .subscribe(titles -> Log.d("Titles", titles.toString()));

 

What this code does is, take a list of sessions and emit each session at a time, filter out the ones which don’t have Android as their track name, take out their titles and puts them in a list and gives it to subscriber.

Now imagine doing it in plain Java. Create a list of string, loop through each session, check track, push title to that list and this much when this example is the most basic of use cases of RxJava.

But how to achieve concurrency. If the there are 10000 sessions, this code will take huge time even if sessions are in memory and not loaded from database. So we will listen to these list events on computation thread.

Observable.fromIterable(getSessionList())
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .subscribeOn(Schedulers.computation())
    .subscribe(titles -> adapter.setItems(titles));

 

That’s it. Now each filtering and mapping and converging to a list is done on another thread.

If you want to listen to each session one at a time, and not all at once when it is completed, you can remove toList() operator

But now, our app will crash! Because when we deliver the result to subscriber, we are still on computation thread, so we need to come back to Main Thread because Android Views are not thread safe, meaning they cannot be accessed from any thread other than UI thread. So in order to do that, we just use observeOn() operator :

Observable.fromIterable(getSessionList())
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .toList()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(titles -> adapter.setItems(titles));

 

Still, our code has a critical problem, the mapping and filtering takes place on background thread, but the loading of session list still takes place on UI thread because it is loaded first and then passed to Observable

Observable methods like just, from, fromIterable, etc all take object from the current thread, meaning passing the object to these functions will not occur on the Scheduler you have supplied. This is very basic programming concept that language parses rightmost parameter first but usually is misunderstood in terms of Rx programming.

So, what do we do? We use fromCallable which waits till the containing function returns and then operates on it

Observable.fromCallable(this::getSessionList)
    .flatMapIterable(sessions -> sessions)
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .toList()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(titles -> adapter.setItems(titles));

 

We’re done! We have changed our synchronous database call to an asynchronous call.

Another use case is when you just have to do an operation asynchronously and not return anything, then fromCallable won’t work as it expects some return value to operate on, instead use Completable

Completable.fromAction(this::clearDatabase)
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(() -> {
        // Completed
        showToast("Success");
    });

 

Note that here we use method reference to call a function, you can just pass in a lambda or Action implementation to do some in place work like this

Completable.fromAction(() -> {
    doSomeStuff();
    // ...
    doOtherStuff(); 
}).subscribeOn(Schedulers.computation())
  .observeOn(AndroidSchedulers.mainThread())
  .subscribe(() -> {
      // Completed
      showToast("Success");
  });

 

Now, you can wrap all your slow methods into observable or completable without changing any code structure and your code will look like this :

On parting note, a trick to avoid repeated subscribeOn observeOn :

private <T> ObservableTransformer<T, T> applySchedulers() {
    return upstream -> upstream.subscribeOn(Schedulers.computation())
                               .observeOn(AndroidSchedulers.mainThread());
}

 

Create this function and just call compose on each Observable and call the function inside that, passing the transformer., like it is shown in the picture above

That’s it for now. Have a happy and lag free day!