Smart Data Loading in Open Event Android Orga App

In any API centric native application like the Open Event organizer app (Github Repo), there is a need to access data through network, cache it for later use in a database, and retrieve data selectively from both sources, network and disk. Most of Android Applications use SQLite (including countless wrapper libraries on top of it) or Realm to manage their database, and Retrofit has become a de facto standard for consuming a REST API. But there is no standard way to manage the bridge between these two for smart data loading. Most applications directly make calls to DB or API service to selectively load their data and display it on the UI, but this breaks fluidity and cohesion between these two data sources. After all, both of these sources manage the same kind of data.

Suppose you wanted to load your data from a single source without having to worry from where it is coming, it’d require a smart model or repository which checks which kind of data you want to load, check if it is available in the DB, load it from there if it is. And if it not, call the API service to load the data and also save it in the DB there itself. These smart models are self contained, meaning they handle the loading logic and also handle edge cases and errors and take actions for themselves about storing and retrieving the data. This makes presentation or UI layer free of data aspects of the application and also remove unnecessary handling code unrelated to UI.

From the starting of Open Event Android Orga Application planning, we proposed to create an efficient MVP based design with clear separation of concerns. With use of RxJava, we have created a single source repository pattern, which automatically handles connection, reload and database and network management. This blog post will discuss the implementation of the AbstractObservableBuilder class which manages from which source to load the data intelligently

Feature Set

So, first, let’s discuss what features should our AbstractObservableBuilder class should have:

  • Should take two inputs – disk source and network source
  • Should handle force reload from server
  • Should handle network connection logic
  • Should load from disk observable if data is present
  • Should load from network observable if data is not present is disk observable

This constitutes the most basic data operations done on any API based Android application. Now, let’s talk about the implementation

Implementation

Since our class will be of generic type, we will used variable T to denote it. Firstly, we have 4 declarations globally

private IUtilModel utilModel;
private boolean reload;
private Observable<T> diskObservable;
private Observable<T> networkObservable;

 

  • UtilModel tells us if the device is connected to internet or not
  • reload tells if the request should bypass database and fetch from network source only
  • diskObservable is the database source of the item to be fetched
  • networkObservable is the network source of the same item

Next is a very simple implementation of the builder pattern methods which will be used to set these variable fields

@Inject
public AbstractObservableBuilder(IUtilModel utilModel) {
    this.utilModel = utilModel;
}

AbstractObservableBuilder<T> reload(boolean reload) {
    this.reload = reload;

    return this;
}

AbstractObservableBuilder<T> withDiskObservable(Observable<T> diskObservable) {
    this.diskObservable = diskObservable;

    return this;
}

AbstractObservableBuilder<T> withNetworkObservable(Observable<T> networkObservable) {
    this.networkObservable = networkObservable;

    return this;
}

 

UtilModel is the required dependency, and so is added as a constructor parameter.

All right, all variables are set up, now we need to create the build function to actually create the observable:

@NonNull
public Observable<T> build() {
    if (diskObservable == null || networkObservable == null)
        throw new IllegalStateException("Network or Disk observable not provided");

    return Observable
            .defer(getReloadCallable())
            .switchIfEmpty(getConnectionObservable())
            .compose(applySchedulers());
}

Reloading Logic

First of all, we check if the caller forgot to add disk or network source and throw an exception if it is actually so. Next, we use defer operator to defer the call to getReloadCallable() so that this function is not executed until this observable is subscribed. Some articles over the internet directly use combine operators from Rx to make things easy, but this lazy calling is the most efficient way to do things because no actual call will be made to observables.

Secondly, you can easily test the behaviour in unit tests, by verifying that

  • no call to the network observable was made if the data inside disk observable was present; or
  • call to network observable was made even if there was data in disk observable if the reload request was made

These tests would not have been possible if we did not employ the lazy call technique because the calls to the observables and utilModel would have been made before the subscription to this model happen, in order to create this observable eagerly.

Now, let’s see what getReloadCallable does

@NonNull
private Callable<Observable<T>> getReloadCallable() {
    return () -> {
        if (reload)
            return Observable.empty();
        else
            return diskObservable
                .doOnNext(item -> 
                    Timber.d("Loaded %s From Disk on Thread %s",
                    item.getClass(), Thread.currentThread().getName()));
    };
}

 

This function’s role is to return the disk observable if the request is not a reload call, or else return an empty observable, so that force network request happens to reload the data

So it returns a Callable which encapsulates this logic, and besides that, it also adds a log if loading from disk about the type of item loaded and the thread it was loaded on

Connection and Database Switch Logic

In the next chain of operation, we make a switchIfEmpty call to getConnectionObservable(). Because of the above reloading logic, switchIfEmpty serves 2 purpose here, it changes to API call:

  • if db does not contain data
  • If it is a reload call

The observable we switch to is returned by getConnectionObservable() and its purpose is to check if the device is connected to the internet, and if it is, to forward the network request and if it is not, then return an Error Observable.

@NonNull
private Observable<T> getConnectionObservable() {
    if (utilModel.isConnected())
        return networkObservable
            .doOnNext(item -> Timber.d("Loaded %s From Network on Thread %s",
                item.getClass(), Thread.currentThread().getName()));
    else
        return Observable.error(new Throwable(Constants.NO_NETWORK));
}

We use util model to determine if we are connected to internet and take action accordingly. As you can see, here too, we log about the data being loaded and the thread information.

Threading

Lastly, we want to ensure that all processing happens on correct threads, and for that, we call compose with an Observable Transformer to make all requests happen on I/O scheduler and the data is received on Android’s Main Thread

@NonNull
private <V> ObservableTransformer<V, V> applySchedulers() {
    return observable -> observable
        .subscribeOn(Schedulers.io())
        .observeOn(AndroidSchedulers.mainThread());
}

And that’s all it takes to create a reactive, generic and reusable data handler for disk and network based operations. In the repository pattern we have employed in the Open Event Android Orga Application, all our data switching and handling code is delegated to it, with unit tests and integration tests testing the individual and cross component working in all cases.

If you want to learn more about other implementations, you can read these articles

Continue ReadingSmart Data Loading in Open Event Android Orga App

Dynamic Ticket Analysis UI using Data Binding in Open Event Android Orga App

Any event manager application has the responsibility to show the analytics about the event to the organiser and in Open Event Android Orga App (Github Repo), we wanted to achieve a way to display the analytics of total and sold tickets with the data present to us.
To analyse, we have a list of tickets, which are divided into 3 categories:

  • Free
  • Paid
  • Donation

Our goal was to show information about total tickets and the amount of sold tickets per category. This blog will focus on the dynamic UI creation for the ticket analysis component of the Event Details Dashboard using Android Layout Data Binding. By using Data Binding, we not only reduced the amount of Java Boilerplate code we would have to write, but also accomplished UI reuse in just XML which wouldn’t have been possible without it. You’ll see in a moment what I mean.

Properties

So first, we’d need to define some properties which will be bound in the UI. These properties are declared in the Event model and their type is ObservableLong provided by the Android DataBinding package. The reason why we are using these instead of primitives is because these fields being Observable, will update the UI as soon as they are updated, without requiring the programmer to set the View Property at all.

There are six fields, 3 for total tickets of each type and 3 for sold tickets

public final ObservableLong freeTickets = new ObservableLong();
public final ObservableLong paidTickets = new ObservableLong();
public final ObservableLong donationTickets = new ObservableLong();

public final ObservableLong soldFreeTickets = new ObservableLong();
public final ObservableLong soldPaidTickets = new ObservableLong();
public final ObservableLong soldDonationTickets = new ObservableLong();

Some more advantages we get from using these are the batch view update and the use of computed properties in UI. Imagine having a TextView display the amount of free tickets and a progress bar showing the percentage of free tickets sold. Traditionally, you’d have to set the text and compute the percentage and set the progress bar as the data changes, whereas you can just use the fields in layout as is in both TextView and ProgressBar with the computations required and they’ll work in harmony.

We have leveraged this feature to show the analytics component of tickets with a

  • Ticket Type
  • Circular Progress Bar
  • Sold Tickets
  • Total Tickets

All using the XML layout and databinding

Ticket Component

For each ticket component, we have 4 variables, namely

  • Ticket Type Name
  • Total Amount
  • Completed Amount (Sold Tickets)
  • Color

First 3 are fairly self explanatory, the color attribute we used in our component needs a little bit of description. We decided to give each ticket category its own color for circular progress bar for aesthetics. So, we need each component to have its own color attribute too. But this is not a normal android color ID or a hex. We needed 2 variants of the same color to show in the circular progress to discern the total and completed part. As we are using Material Color Palette, which has a color divided by intensities, we used 500 variant for completed portion and 100 (lighter) variant for the background of circular progress.

Let’s look at the layout now:

<data>
    <variable name="color" type="String" />
    <variable name="ticketName" type="String" />
    <variable name="total" type="long" />
    <variable name="completed" type="long" />
</data>

<LinearLayout
     android:orientation="vertical">
    <TextView
        android:text="@{ticketName}" />
    <FrameLayout>
        <com.mikhaellopez.circularprogressbar.CircularProgressBar
            app:circular_progress_color="@{color}"
            app:progress_with_animation="@{total == 0 ? 0 : (int) ((completed*100)/total)}" />

        <LinearLayout
            android:orientation="horizontal">

            <TextView
                android:text="@{completed}" />
            <TextView
                android:text='@{"/" + total}' />

        </LinearLayout>
    </FrameLayout>

    <TextView
        android:text='@{(total == 0 ? 0 : (int) ((completed*100)/total)) + "%"}' />

</LinearLayout>

Note: The layout snippet is not complete. Only attribute names to be discussed in the blog are shown for brevity

As you can see, after the data variable declarations, we have a CardView first showing the ticket name on top, and then we have a FrameLayout wrapping the circular progress and a textview showing the Sold/Total tickets.

Circular Progress Bar

Let’s discuss the circular progress first, we have used this library to create a circular progress bar, the two other attributes circular_progress_color and progress_with_animation are specific to Open Event Orga Application and we have created custom adapters for them:

@BindingAdapter("progress_with_animation")
public static void bindCircularProgress(CircularProgressBar circularProgressBar, int progress) {
    circularProgressBar.setProgressWithAnimation(progress, 500);
}

@BindingAdapter("circular_progress_color")
public static void bindCircularProgressColor(CircularProgressBar circularProgressBar, String colorName) {
    Context context = circularProgressBar.getContext();
    Resources resources = circularProgressBar.getResources();

    int color = ContextCompat.getColor(context, resources.getIdentifier(colorName + "_500", "color", context.getPackageName()));
    int bgColor = ContextCompat.getColor(context, resources.getIdentifier(colorName + "_100", "color", context.getPackageName()));

    circularProgressBar.setColor(color);
    circularProgressBar.setBackgroundColor(bgColor);
}
  • progress_with_animation sets the provided integer value as the progress of the circular progress bar with an animation of 500 ms
  • circular_progress_color finds the 100 and 500 variant of the color name string provided and sets them as background and foreground color of the progress bar

These are the color definitions we have used in the app:

<color name="light_blue_100">#B3E5FC</color>
<color name="light_blue_500">#03A9F4</color>
<color name="purple_100">#E1BEE7</color>
<color name="purple_500">#9C27B0</color>
<color name="red_100">#ffcdd2</color>
<color name="red_500">#f44336</color>

As you can that if we pass purple as the color name, it’ll load purple_100 and purple_500 and set it as corresponding background and foreground color

Other Properties

Now, let’s talk about other properties like the progress value :

  • total == 0 ? 0 : (int) ((completed*100)/total)The conditional is used to prevent divide by zero error.
    The same expression is used to display the circular progress and percentage text in the TextView at the bottom of the layout
  • completed and “/” + total are used to in TextViews of different sizes to create a nice design with completed/total format

This completes our ticket component design and now we’ll see how to reuse this component to display different ticket types.

Composite Layout

To use the ticket component, we just include the layout and bind specific variables from Event model to create a dynamic layout like this:

<data>
    <variable
        name="event"
        type="org.fossasia.openevent.app.data.models.Event" />
</data>

<LinearLayout
    android:orientation="vertical">

    <TextView
        android:text="@string/tickets" />

    <LinearLayout
        android:orientation="horizontal">

        <include
            layout="@layout/ticket_analytics_item"
            bind:color='@{"light_blue"}'
            bind:completed="@{event.soldFreeTickets}"
            bind:ticketName="@{@string/ticket_free}"
            bind:total="@{event.freeTickets}" />

        <include
            layout="@layout/ticket_analytics_item"
            bind:color='@{"purple"}'
            bind:completed="@{event.soldPaidTickets}"
            bind:ticketName="@{@string/ticket_paid}"
            bind:total="@{event.paidTickets}" />

        <include
            layout="@layout/ticket_analytics_item"
            bind:color='@{"red"}'
            bind:completed="@{event.soldDonationTickets}"
            bind:ticketName="@{@string/ticket_donation}"
            bind:total="@{event.donationTickets}" />
    </LinearLayout>

</LinearLayout>

The layout consists of a horizontal with 3 equally divided ticket components,

  • Free Ticket Component -> Light Blue
  • Paid Ticket Component -> Purple
  • Donation Ticket Component -> Red

This is how it looks on a device

 

So this is how data binding made us accomplish easily which would have been a very convoluted solution using traditional ID based view binding. For more info about data binding, refer to these sites:

https://developer.android.com/topic/libraries/data-binding/index.html

http://www.vogella.com/tutorials/AndroidDatabinding/article.html

Continue ReadingDynamic Ticket Analysis UI using Data Binding in Open Event Android Orga App

Implementing Barcode Scanning in Open Event Android Orga App using RxJava

One of the principal goals of Open Event Orga App (Github Repo) is to let the event organizer to scan the barcode identifier of an attendee at the time of entry in the event, in order to quickly check in that attendee. Although there are several scanning APIs available throughout the web for Android Projects, we chose Google Vision API as it handles multitude of formats and orientations automatically with little to no configuration, integrates seamlessly with Android, and is provided by Google itself, ensuring great support in future as well. Currently, the use case of our application is:

  • Scan barcode from the camera feed
  • Detect and show barcode information on screen
  • Iterate through the attendee list to match the detected code with unique attendee identifier
  • Successfully return to the caller with scanned attendee’s information so that a particular action can be taken

There are several layers of optimisations done in the Orga Application to make the user interaction fluid and concise. I’ll be sharing a few in this blog post regarding the configuration and handling of bursty data from camera source efficiently. To see the full implementation, you can visit the Github Repository of the project using the link provided above.

Configuration

The configuration of our project was done through Dagger 2 following the dependency injection pattern, which is not the focus of this post, but it is always recommended that you follow separation of concerns in your project and create a separate module for handling independent works like barcode scanner configuration. Normally, people would create factories with scanner, camera source and processors encapsulated. This enables the caller to have control over when things are initialized.

Our configuration provides us two initialised objects, namely, CameraSource and BarcodeDetector

@Provides
BarcodeDetector providesBarCodeDetector(Context context, Detector.Processor<Barcode> processor) {
    BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(context)
        .setBarcodeFormats(Barcode.QR_CODE)
        .build();

    barcodeDetector.setProcessor(processor);

    return barcodeDetector;
}

@Provides
CameraSource providesCameraSource(Context context, BarcodeDetector barcodeDetector) {
    return new CameraSource
        .Builder(context, barcodeDetector)
        .setRequestedPreviewSize(640, 480)
        .setRequestedFps(15.0f)
        .setAutoFocusEnabled(true)
        .build();
}

The fields needed to create the objects are provided as arguments to the provider functions as all the dependencies required to construct these objects. Now focusing on the Detector.Processor requirement of the BarcodeDetector is the classic example on non injectable code. This is because the processor is to be supplied by the activity or any other object which wants to receive the callback with the detected barcode data. This means we could inject it at the time of creation of the Activity or Presenter itself. We could easily overcome by adding a constructor to this dagger module containing the Barcode.Processor at the time of injection, but that would violate our existing 0 configuration based model where we just get the required component from the Application class and inject it. So, we wrapped the the processor into a PublishSubject

@Provides
@Singleton
@Named("barcodeEmitter")
PublishSubject<Notification<Barcode>> providesBarcodeEmitter() {
    return PublishSubject.create();
}

@Provides
@Singleton
Detector.Processor<Barcode> providesProcessor(@Named("barcodeEmitter") PublishSubject<Notification<Barcode>> emitter) {
    return new Detector.Processor<Barcode>() {
        @Override
        public void release() {
            // No action to be taken
        }

        @Override
        public void receiveDetections(Detector.Detections<Barcode> detections) {
            SparseArray<Barcode> barcodeSparseArray = detections.getDetectedItems();
            if (barcodeSparseArray.size() == 0)
                emitter.onNext(Notification.createOnError(new Throwable()));
            else
                emitter.onNext(Notification.createOnNext(barcodeSparseArray.valueAt(0)));
        }
    };
}

This solves 2 of our problems, not only now all these dependencies are injectable with 0 configurations, but also our stream of barcodes is now reactive.

Note that not everyone is in favour of using Singleton, but you can decrease the scope using your own annotation. We prefer not creating life cycle bound objects, those are hard to manage and can cause potential memory leaks, and the creation of an anonymous inner class object every time listener activates is not good for memory too.

Also, note that Singleton classes will cause memory leaks too if you don’t release their reference at the time of destruction of life cycle bound object

Notice how the type of PublishSubject is not just a barcode, but Notification which wraps the bar code. That’s because we want to send both the value and error streams down uninterrupted to the caller. Otherwise, the data stream would have stopped on the emission of first onError call. Here, we detect the barcodeSparseArray size and accordingly send error or first value to the PublishSubject which will be accordingly subscribed by the activity or presenter

Handling Bursty Data

barcodeEmitter.subscribe(barcodeNotification -> {
    if (barcodeNotification.isOnError()) {
        presenter.onBarcodeDetected(null);
    } else {
        presenter.onBarcodeDetected(barcodeNotification.getValue());
    }
});

Here is how we are subscribing the notification emitter and passing the appropriate value to the presenter to handle, null if it is an error and the value if it is the next emission.

Note that you must dispose the disposable returned by the subscribe method on the subject when the Activity is to be destroyed or else it will keep the reference to the anonymous inner class created with the lambda for barcodeNotification and cause a memory leak

Now, let’s see how the presenter handles this data for:

  • Hiding and showing the barcode panel when barcode is on the screen accordingly
  • Showing the data extracted from the barcode scanner

These things can be implemented in a very standard way with a few conditionals, but most developers forget the fact that the data emission rate is enormous when concerning with live feed of data. In the Open Event Orga app, we have reduced it to 15 FPS as it is more than enough to scan barcodes for our use case, but it is still huge. The continuous stream of nulls and barcode data is useless to us unless it changes.

A little explanation about nulls and values here: You must have noticed above the conditions when we pass null and value, but I’ll explain again. A value will be passed if there is a detected barcode on screen, and null will be passed if there is no barcode detected. The Google Vision API will keep sending the same value for barcode at 15 FPS and so we’ll get this redundant stream of nulls and values which we should not concern with processing as this will load the CPU unnecessarily.

There are only 2 cases where we need to process it:

  • Null changes to Value -> Show barcode panel
    Value changes to null -> Hide barcode panel
  • Value changes irrespective of nulls -> Show barcode data on UI and search through the attendee identifiers

So, here too we’ll create 2 PublishSubject objects

private PublishSubject<Boolean> detect = PublishSubject.create();
private PublishSubject<String> data = PublishSubject.create();

And we’ll configure them both in this way to receive data on each barcode emission in the presenter:

public void onBarcodeDetected(Barcode barcode) {
   detect.onNext(barcode == null);
   if (barcode != null)
       data.onNext(barcode.displayValue);
}

This will make data only receive non-null changes and detect receive a boolean notifying if the current detected barcode was null or not.

Now, we see how each of these subjects is configured to pass the emissions downstream:

data.distinctUntilChanged()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(this::processBarcode);

This one is pretty straightforward. It’ll only send data downstream if its value has changed from the previous emission, disregarding nulls. So, for a stream like this

A A A A A null null null null A A A A null B B B B B B null null null B B A A null A

It will only emit:

A                                                                         B                                                  A

Which is actually what we want, as we only need to process and show the distinct barcode data and match it with our attendee list. Now, with thousands of attendees, the first method would have triggered unnecessary and time-consuming computations over and over again on same identifiers with little gaps of time, which would have created mediocre results even in a multi-threaded environment. The second one saves us from repetitive calculations and also gives us enormous gaps between emissions, which is optimal for our use case.

The second case is not so obvious because we can’t ignore nulls here as we have to show and hide UI based on them. This means that unlike our previous stream if we just use distinctUntilChanged, it will look like this:

A A A A A null null null null A A A A null B B B B B B null null null B B A A null A

f                 t                               f               t      f                    t                        f             t       f

This is because, if you remember, we were sending down emissions of barcode == null on each emission for this Subject. So, in this case, as you may see, some of the values are so close enough that it will not be discernable in UI and also annoy users who’ll see the panel pop up for milliseconds before vanishing or vice-versa. The perfect operation for this case will be debounce

detect.distinctUntilChanged()
    .debounce(150, TimeUnit.MILLISECONDS)
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(receiving -> scanQRView.showBarcodePanel(!receiving));

This operator will drop any emission in the window of 150ms succession and only pick up those emissions which are 150ms apart from each other. Now, 150 ms is not a magic number, it is picked through hit and trial and what works best for your case. Lower the value and you will pick up more changes downstream, increase the value and you might miss the required events.

This makes our stream somewhat like this, cleaning out the cluttered events

f                 t                               f                                           t                        f

This is the screenshot of the implementation:  

 

And an animated gif of the scanning process:

This is all for this blog, you may use many other operators from the arsenal of RxJava, whatever fits your use case. As I have presumed the knowledge about Subjects, MVP and a little bit of Dagger in this post, I’ll link some of the resources where you can find more information about these:

http://reactivex.io/documentation/subject.html

https://github.com/ReactiveX/RxJava/wiki/Subject

http://reactivex.io/documentation/operators/distinct.html

http://www.vogella.com/tutorials/Dagger/article.html

https://antonioleiva.com/mvp-android/

Continue ReadingImplementing Barcode Scanning in Open Event Android Orga App using RxJava

Testing Asynchronous Code in Open Event Orga App using RxJava

In the last blog post, we saw how to test complex interactions through our apps using stubbed behaviors by Mockito. In this post, I’ll be talking about how to test RxJava components such as Observables. This one will focus on testing complex situations using RxJava as the library itself provides methods to unit test your reactive streams, so that you don’t have to go out of your way to set contraptions like callback captors, and implement your own interfaces as stubs of the original one. The test suite (kind of) provided by RxJava also allows you to test the fate of your stream, like confirming that they got subscribed or an error was thrown; or test an individual emitted item, like its value or with a predicate logic of your own, etc. We have used this heavily in Open Event Orga App (Github Repo) to detect if the app is correctly loading and refreshing resources from the correct source. We also capture certain triggers happening to events like saving of data on reloading so that the database remains in a consistent state. Here, we’ll look at some basic examples and move to some complex ones later. So, let’s start.

public class AttendeeRepositoryTest {

    private AttendeeRepository attendeeRepository;

    @Before
    public void setUp() {
        testDemo = new TestDemo();
    }

    @Test
    public void shouldReturnAttendeeByName() {
        // TODO: Implement test
    }

}

 

This is our basic test class setup with general JUnit settings. We’ll start by writing our tests, the first of which will be to test that we can get an attendee by name. The attendee class is a model class with firstName and lastName. And we will be checking if we get a valid attendee by passing a full name. Note that although we will be talking about the implementation of the code which we are writing tests for, but only in an abstract manner, meaning we won’t be dealing with production code, just the test.

So, as we know that Observables provide a stream of data. But here, we are only concerned with one attendee. Technically, we should be using Single, but for generality, we’ll stick with Observables.

So, a person from the background of JUnit would be tempted to write this code below.

Attendee attendee = attendeeRepository.getByAttendeeName("John Wick")
    .blockingFirst();

assertEquals("John Wick", attendee.getFirstName() + attendee.getLastName());

 

So, what this code is doing is blocking the thread till the first attendee is provided in the stream and then checking that the attendee is actually John Wick.

While this code works, this is not reactive. With the reactive way of testing, not only you can test more complex logic than this with less verbosity, but it naturally provides ways to test other behaviors of Reactive streams such as subscriptions, errors, completions, etc. We’ll only be covering a few. So, let’s see the reactive version of the above test.

attendeeRepository.getByAttendeeName("John Wick")
    .firstElement()
    .test()
    .assertNoErrors()
    .assertValue(attendee -> "John Wick".equals(
        attendee.getFirstName() + attendee.getLastName()
    ));

 

So clean and complete. Just by calling test() on the returned observable, we got this whole suite of testing methods with which, not only did we test the name but also that there are no errors while getting the attendee.

Testing for Network Error on loading of Attendees

OK, so let’s move towards a more realistic test. Suppose that you call this method on AttendeeRepository, and that you can fetch attendees from the network. So first, you want to handle the simplest case, that there should be an error if there is no connection. So, if you have (hopefully) set up your project using abstractions for the model using MVP, then it’ll be a piece of cake to test this. Let’s suppose we have a networkUtil object with a method isConnected.

The NetworkUtil class is a dependency of AttendeeRepository and we have set it up as a mock in our test using Mockito. If this is sounding somewhat unfamiliar, please read my previous article “The Joy of Testing with MVP”.

So, our test will look like this

@Test
public void shouldStreamErrorOnNetworkDown() {
    when(networkUtils.isConnected()).thenReturn(false);
    
    attendeeRepository.getAttendees()
        .test()
        .assertErrorMessage("No Network");
}

 

Note that, if you don’t define the mock object’s behavior like I have here, attendeeRepository will likely throw an NPE as it will be calling isConnected() on an undefined object.

With RxJava, you get a whole lot of methods for each use case. Even for checking errors, you get to assert a particular Throwable, or a predicate defining an operation on the Throwable, or error message as I have shown in this case.

Now, if you run this code, it’ll probably fail. That’s because if you are testing this by offloading the networking task to a different thread by using subscribeOn observeOn methods, the test body may be detached from Main Thread while the requests complete. Furthermore, if testing in an application made for Android, you would have use AndroidSchedulers.mainThread(), but as it is an Android dependency, the test will fail. Well actually, crash. There were some workarounds by creating abstractions for even RxJava schedulers, but RxJava2 provides a very convenient method to override the default schedulers in the form of RxJavaPlugins. Similarly, RxAndroidPlugins is present in the rx-android package. Let’s suppose you have the plan to use Schedulers.io() for asynchronous work and want to get the stream on Android’s Main Thread, meaning you use AndroidSchedulers.mainThread() in the observeOn method. To override these schedulers to Schedulers.trampoline() which queues your tasks and performs them one by one, like the main thread, your setUp will include this:

RxJavaPlugins.setIoSchedulerHandler(scheduler ->  Schedulers.trampoline());
RxAndroidPlugins.setInitMainThreadSchedulerHandler(scheduler -> Schedulers.trampoline());

 

And if you are not using isolated tests and need to resume the default scheduler behavior after each test, then you’ll need to add this in your tearDown method

RxJavaPlugins.reset();
RxAndroidPlugins.reset();

Testing for Correct loading of Attendees

Now that we have tested that our Repository is correctly throwing an error when the network is down, let’s test that it correctly loads attendees when the network is connected. For this, we’ll need to mock our EventService to return attendees when queried, since we don’t want our unit tests to actually hit the servers.

So, we’ll need to keep these things in mind:

  • Mock the network until it shows that it is connected to the Internet
  • Mock the EventService to return attendees when queried
  • Call the getter on the attendeeRepository and test that it indeed returned a list of attendees

For these conditions, our test will look like this:

@Test
public void shouldLoadAttendeesSuccessfully() {
    List<Attendee> attendees = Arrays.asList(
        new Attendee(),
        new Attendee(),
        new Attendee()
    );

    when(networkUtils.isConnected()).thenReturn(true);
    when(eventService.getAttendees()).thenReturn(Observable.just(attendees));

    attendeeRepository.getAttendees()
        .test()
        .assertValues(attendees.toArray(new Attendee[attendees.size()]));
}

 

The assertValues function asserts that these values were emitted by the observable. And if you want to be terser, you can even verify that in fact EventService’s getAttendees function was called by

verify(eventService).getAttendees();

 

But the problem in this way is that the getAttendees function returns an observable and just calling it does not necessarily means that it was subscribed, emitting the results, hence we need to test to ensure that it was indeed subscribed. If we call the normal test() function on the observable, it is already subscribed, making the result of testSubscribed always true. In order to test that correctly, let’s look at our final use case.

Testing for saving of Attendees

In the Open Event Orga App, we have strived to create self-sufficient and intelligent classes, thus, our repository is also built this way. It detects that new attendees are loaded from the server and saves them in the database. Now we’d want to test this functionality.

In this test, there is an added dependency of DatabaseRepository for saving the attendees, which we will mock. The conditions for this test will be:

  • Network is connected
  • EventService returns attendees
  • DatabaseRepository mocks the saving of attendees

For DatabaseRepository’s save method, we’ll be returning a Completable, which will notify when the saving of data is completed. The primary purpose of this test will be to assert that this completable is indeed subscribed when the attendee loading is triggered. This will not only ensure that the correct function to save the attendees is called, but also that it is indeed triggered and not just left hanging after the call. So, our test will look like this.

@Test
public void shouldSaveAttendeesInDatabase() {
    List<Attendee> attendees = Arrays.asList(
        new Attendee(),
        new Attendee(),
        new Attendee()
    );

    TestObserver testObserver = TestObserver.create();
    Completable completable = Completable.complete()
        .doOnSubscribe(testObserver::onSubscribe);

    when(networkUtils.isConnected()).thenReturn(true);
    when(databaseRepository.save(attendees)).thenReturn(completable);
    when(eventService.getAttendees()).thenReturn(Observable.just(attendees));

    attendeeRepository.getAttendees()
        .test()
        .assertNoErrors();

    testObserver.assertSubscribed();
}

 

Here, we have created a separate test observable and set it to be subscribed when the Completable is subscribed and we have returned that Completable when the save method is called. In the last, we have asserted that the test observer is indeed subscribed.

You can create more complex use cases and assert subscriptions, errors, the emptiness of a stream and much more, by using the built-in test functionalities of RxJava2. So, that’s all for this blog, you can visit these links for more details on unit testing RxJava

http://fedepaol.github.io/blog/2015/09/13/testing-rxjava-observables-subscriptions/

https://www.infoq.com/articles/Testing-RxJava

Continue ReadingTesting Asynchronous Code in Open Event Orga App using RxJava

The Joy of Testing with MVP in Open Event Orga App

Testing applications is hard, and testing Android Applications is harder. The natural way an Android Developer codes is completely untestable. We are born and molded into creating God classes – Activities and perform every bit of logic inside them. Some thought that introduction to Fragments will introduce a little bit of modularity, but we proved otherwise by shifting to God Fragments. When the natural urge of an Android Developer to

  • apply logic,
  • load UI,
  • handle concurrency (hopefully, not using AsyncTask),
  • load data – from the network; disk; and cache,
  • and manage the state of the Activity/Fragment

finally, meets with a new form of revelation that he/she should test his/her application, all of the concepts acquired are shattered in a moment. The person goes to Android Docs to see why he started coding that way and realizes that even the system is flawed – Android Documentation is full of examples promoting God Activities, which they have used to show the use of API for only one reason – brevity. It’s not uncommon for us to steal some code from StackOverflow or Android Docs, but we don’t realize that they are presented there without an application environment or structure, all the required component just glued together to get it functionally complete, so that reader does not have to configure it anymore. Why would an answer from StackOverflow load a barcode scanner using a builder? Or build a ContentProvider to show how to query sorted data from SQLite? Or use a singleton for your provider classes? The simple answer is, they won’t. But this doesn’t mean you shouldn’t too.

The first thought that enters developer’s mind when he gets his hand on a piece of code is to paste it in the correct position and get it to work. There is always this moment of hesitation where conscience rings a bell saying, “What are you doing? Is it the right way to do it? How many lines till you stop overloading this Activity? It’s 2000 lines already”. But it dies as soon as we see the feature is working. And why test something which we have confirmed to work, right? I just wrote this code to show a progress bar while the data loads and hide it when it is done. I can see this working, what will I achieve in painfully writing a test for it, mocking the loading conditions and all. Wrong! Unit tests which test your trivial utils like Date Modification, String Parsing, etc are good and needed but are so trivial that they are hard to go wrong, and if they are, they are easy to fix as they have single usage throughout the app, and it is easy to spot bugs and fix them.

The real problem is testing of your app over dynamic conditions, where you have to emulate them so you can see if your app is making the right decisions. The progress bar working example may work for now, but what if it breaks over refactoring, or you wrote the same code elsewhere and forget to hide the progress bar? Simply copying a well-written test and changing 2-3 class names will fail the build and tell you what’s wrong. A well-contracted app can even contain tests to check that there’ll be no memory leaks. But none of this is possible if everything is jumbled into single Activity with callbacks, loaders, UI handling, business logic, etc. You can’t even think that where to begin. In this post, I will briefly discuss, how to design and test applications using MVP pattern.

MVP to the Rescue

Before moving ahead, I must put a disclaimer saying that MVP is a design pattern which follows a very opinionated implementation. The extent to which you want to refactor your app and the number of abstractions you are willing to do are in your hand. There aren’t any golden rules where your implementation will fail to be called as MVP. Remember, the client doesn’t care about architecture, it’s for you, so whatever makes your life and testing easier, works. Also, I’ll use an axiom related to testing here, “Test until fear turns into boredom”. You can apply the same to your MVP implementation. Testing and design patterns are here to eradicate your fear of failure, not to bore you.

So, in this guide, I’ll be using a use case of a QR Code Scanner Activity built using MVP pattern which we have employed in Open Event Orga Application (Github Repo). Because we are focusing on the test pattern and how to make it easy for us to test our app logic, I have omitted the model part from the MVP equation. The reason being that the possible model in the application would have been the camera loader or barcode initializer and the caveats associated with following this is that both these modules rely heavily on the Android specific view classes, namely, SurfaceView and other lifecycle methods. You could always create your way around it to include them in a separate model, but it won’t help us in writing unit tests for them, because firstly, they aren’t our logic to test, and secondly, they can’t be tested in a unit test (those models would have probably just implemented certain setters and getters).

So, the main purpose of our QR Code Scanner class is to scan a QR code and match it with a list of identifiers, and if there is a match, return it successfully to the caller. In this specific example, the identifiers will the ticket IDs of event attendees and the caller will be event organizer scanning QR codes to check the attendee in. The use case sounds simple, but has several mini use cases and dependencies of itself, which we have to take into our account while designing the View and Presenter class. Let’s discuss them one by one:

App State – Start: Activity starts, loads camera

  • Permission Granted: detects that the app already has Camera Permission,
    • starts scanning
  • Permission Absent: detects that the app doesn’t have Camera Permission,
    • Denied: asks for it, denied, shows error
    • Accepted: asks for it, granted, starts scanning

App State – QR Code Detected: Starts parsing them

  • Attendees are not present: Attendees aren’t present because of some reason, either due to internal error or have not loaded yet
    • stops parsing
  • Attendees are present:
    • QR Code does not match with any attendee : Do nothing
    • QR Code matches with one of the attendee : Send attendee to caller

App State: Camera or containing View is getting destroyed

  • Release Camera

There can be much more internal data flows, but this much is sufficient for our example. So, let’s start defining our contracts using the above knowledge. First, we will design our View. So what should be our strategy? Always think of presenters and views to be mapped in a 1:1 relation. They can have conversations with each other, the difference is that the view is only allowed to talk to the presenter, but presenter may talk to models too. So, the view is going to get all of its information from the presenter and can only take action when presenter tells it to, essentially saying that the view is dumb and passive. The presenter can talk to view and model(s), meaning the collection of information presenter has does not have to come from the view only. In fact, the less dependent presenter has to be on view, the better. The main motive of MVP is to make our logic less dependent on views.

Presenter Contract

In our example, presenter relies on the view to tell it when a certain event happens, they can be lifecycle callbacks, permission grants/denies, camera load/destroy or anything purely related to the Android implementation. So, we generally know from the start what kind of information presenter needs from a View, so let’s design our presenter first.

public interface IScanQRPresenter {

    void attach(long eventId, IScanQRView scanQRView);

    void start();

    void detach();

    void cameraPermissionGranted(boolean granted);

    void onBarcodeDetected(Barcode barcode);

    void onScanStarted();

    void onCameraLoaded();

    void onCameraDestroyed();

}

 

The methods are self-explanatory. Don’t worry if you didn’t get why we defined certain hooks like onScanStarted() or why we didn’t define onScanStopped(). These kinds of details will reveal themselves as you develop your components. You can skip to next section if you don’t want to know why we did it.

Basically, you should define a callback for events which are not reliable to be synchronized. What? Let me explain. Let’s say you got your camera permission and requested the view to start scanning, but you have to wait till the camera is loaded as it is not a synchronous call (and it shouldn’t be or your main thread will block). So, instead of that, our presenter will request the camera to load, go in an idle state, wait for the onCameraLoaded() call, and then request the scan to start, and since it is also not a synchronous work, it will go into idle state again and wait for onScanStarted() and then further its work. Whenever the camera is destroyed, the onCameraDestroyed() callback will be called and we will stop the scanning, and since there is nothing to be done after that, we won’t wait till scanning has stopped, thus dropping the need of onScanStopped() callback.

View Contract

View contract will come naturally to you once you have understood the data flow. It will contain the commands presenter will issue on the view and also, the requests for data that view holds. So, this will be our view.

public interface IScanQRView {

    boolean hasCameraPermission();

    void requestCameraPermission();

    void showPermissionError(String error);

    void onScannedAttendee(Attendee attendee);

    void showBarcodePanel(boolean show);

    void showBarcodeData(@NonNull String data);

    void showProgressBar(boolean show);

    void loadCamera();

    void startScan();

    void stopScan();

}

 

Almost all commands are straightforward but let me quickly explain showBarcodePanel(boolean) and showBarcodeData(String). They are used to display the currently visible barcode data to the user.

So, with our implementations set and data flow in place, let’s start writing tests for the feature. Yes, we’ll write tests without implementation and then you’ll see how easy it will be to write your views and presenters with only one goal to mind, to make the tests pass. If your tests are written correctly and cover everything, you should feel confident that your app will work without even seeing the actual implementation, because that is what tests are for. And by making passive and dumb views, imagine how light your instrumentation tests will be. Just check that the individual methods in view implementation are working as expected and you are done! No need to test logic or complex interactions, etc because you have got it covered in the unit tests themselves. This is not only a benefit of data flow tests but also a best practice. You always want to follow DRY, Don’t Repeat Yourself, even while testing.

Tests

So, we will start writing our tests now and you’ll realize how easy it is and all the hard work of abstraction and designing will pay off.

Attach Tests

So, firstly, we will test if the presenter calls appropriate methods when it is attached

@Test
public void shouldLoadAttendeesAutomatically() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));

    scanQRPresenter.start();

    verify(eventRepository).getAttendees(eventId, false);
}

@Test
public void shouldLoadCameraAutomatically() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));

    scanQRPresenter.start();

    verify(scanQRView).loadCamera();
}

 

Here, we are using Mockito to mock our EventDataRepository to return locally defined attendees instead of doing an actual network call. Then we are calling attach on presenter in each method and then we verify in the first test that the presenter is calling getAttendees on the EventDataRepository, and in the second test that it is requesting the view to load the camera.

Note that in the implementation of attach function, both loading of Camera and attendee loading will take place, but it is best practice to test them separately so that when a test fails, we know why it did

Detach Tests

@Test
public void shouldDetachViewOnStop() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));

    scanQRPresenter.start();

    assertNotNull(scanQRPresenter.getView());

    scanQRPresenter.detach();

    assertNull(scanQRPresenter.getView());
}

@Test
public void shouldNotAccessViewAfterDetach() {
    scanQRPresenter.detach();

    scanQRPresenter.start();
    scanQRPresenter.onCameraLoaded();
    scanQRPresenter.cameraPermissionGranted(false);
    scanQRPresenter.cameraPermissionGranted(true);
    scanQRPresenter.onScanStarted();
    scanQRPresenter.onBarcodeDetected(barcode2);
    scanQRPresenter.onCameraDestroyed();

    verifyZeroInteractions(scanQRView);
}

 

In detach tests, we are verifying that after attaching and view not being null, the detach method call makes the presenter leave the reference to the view making it null. And in the second test, we do all possible interactions after detaching and confirm that no call whatsoever was made on the view. Tests like these enforce to avoid memory leaks and check for any NullPointerExceptions that may happen after the view was made null.

Note: This does not mean that memory leaks will not happen if this test passes. You can cause memory leaks by giving the view reference to any long living object, not just the presenter. This just ensures that presenter will not hold the view reference after detach and won’t reference to the null view in future.

Permission Tests

@Test
public void shouldStartScanOnCameraLoadedIfPermissionPresent() {
    when(scanQRView.hasCameraPermission()).thenReturn(true);

    scanQRPresenter.onCameraLoaded();

    verify(scanQRView).startScan();
}

@Test
public void shouldAskPermissionOnCameraLoadedIfPermissionsAbsent() {
    when(scanQRView.hasCameraPermission()).thenReturn(false);

    scanQRPresenter.onCameraLoaded();

    verify(scanQRView).requestCameraPermission();
}

@Test
public void shouldStartScanningOnPermissionGranted() {
    scanQRPresenter.cameraPermissionGranted(true);

    verify(scanQRView).startScan();
}

@Test
public void shouldShowErrorOnPermissionDenied() {
    scanQRPresenter.cameraPermissionGranted(false);

    verify(scanQRView).showPermissionError(matches("(.*permission.*denied.*)|(.*denied.*permission.*)"));
}

The permission tests are straightforward:

Implicit Permission Handling

  1. If the view already has the camera permission and camera has loaded, start scanning
  2. If the view does not have camera permission and the camera has loaded, request the permission

Request Handling

  1. If the request was granted, start scanning
  2. If the permission was denied, show the permission error. Here, I have used regex to match that the error message contains permission and denied words, you can use anyString() from Mockito for more flexibility or a specific message for more tight testing

Camera Destroy Test

@Test
public void shouldStopScanOnCameraDestroyed() {
    scanQRPresenter.onCameraDestroyed();

    verify(scanQRView).stopScan();
}

Pretty simple, stop the scan on destruction of camera

Flow Tests

You can also test that the callback flow happens in order so that not only the unit tests work but also the implementation logic is correct.

/**
 * Checks that the flow of commands happen in order
 */
@Test
public void shouldFollowFlowOnImplicitPermissionGrant() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));
    when(scanQRView.hasCameraPermission()).thenReturn(true);

    scanQRPresenter.start();

    InOrder inOrder = inOrder(scanQRView);
    inOrder.verify(scanQRView).loadCamera();
    scanQRPresenter.onCameraLoaded();
    inOrder.verify(scanQRView).startScan();
}

@Test
public void shouldShowProgressInBetweenImplicitPermissionGrant() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));
    when(scanQRView.hasCameraPermission()).thenReturn(true);

    scanQRPresenter.start();

    InOrder inOrder = inOrder(scanQRView);
    inOrder.verify(scanQRView).showProgressBar(true);
    scanQRPresenter.onCameraLoaded();
    scanQRPresenter.onScanStarted();
    inOrder.verify(scanQRView).showProgressBar(false);
}

Here, we are verifying two things, first that if we have the request granted implicitly, we load the camera and start the scan in order. This test isn’t that useful as we already tested that loading of the camera is done on attach and scan is started when the camera is loaded. In fact, this is an example of breaking the DRY rule. Even though it doesn’t hurt to include this, it also doesn’t help as it does not cover anything that hasn’t already been tested.

The second test is important and tests that progress bar is correctly shown and hidden after certain communications have taken place and the scan has started. Similarly, we can also test the progress bar behavior over all of the possible combinations of cases that can happen. The code snippets below show the tests:

@Test
public void shouldShowProgressInBetweenImplicitPermissionDenyRequestGrant() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));
    when(scanQRView.hasCameraPermission()).thenReturn(false);

    scanQRPresenter.start();

    InOrder inOrder = inOrder(scanQRView);
    inOrder.verify(scanQRView).showProgressBar(true);
    scanQRPresenter.onCameraLoaded();
    scanQRPresenter.cameraPermissionGranted(true);
    scanQRPresenter.onScanStarted();
    inOrder.verify(scanQRView).showProgressBar(false);
}

@Test
public void shouldShowProgressInBetweenImplicitPermissionDenyRequestDeny() {
    when(eventRepository.getAttendees(eventId, false))
        .thenReturn(Observable.fromIterable(attendees));
    when(scanQRView.hasCameraPermission()).thenReturn(false);

    scanQRPresenter.start();

    InOrder inOrder = inOrder(scanQRView);
    inOrder.verify(scanQRView).showProgressBar(true);
    scanQRPresenter.onCameraLoaded();
    scanQRPresenter.cameraPermissionGranted(false);
    inOrder.verify(scanQRView).showProgressBar(false);
}

QR Code Detection Tests

@Test
public void shouldNotSendAnyBarcodeIfAttendeesAreNull() {
    sendNullInterleaved();

    verify(scanQRView, never()).onScannedAttendee(any(Attendee.class));
}

@Test
public void shouldNotSendAttendeeOnWrongBarcodeDetection() {
    scanQRPresenter.setAttendees(attendees);
    sendNullInterleaved();

    verify(scanQRView, never()).onScannedAttendee(any(Attendee.class));
}

@Test
public void shouldSendAttendeeOnCorrectBarcodeDetection() {
    // Somehow the setting in setUp is not working, a workaround till fix is found
    RxJavaPlugins.setComputationSchedulerHandler(scheduler -> Schedulers.trampoline());

    scanQRPresenter.setAttendees(attendees);

    barcode1.displayValue = "test4-91";
    scanQRPresenter.onBarcodeDetected(barcode1);

    verify(scanQRView).onScannedAttendee(attendees.get(3));
}

 

Lastly, the core tests of our presenter. To explain the tests, let me show you the sendNullInterleaved() method

private void sendNullInterleaved() {
    sendBarcodeBurst(barcode1);
    sendBarcodeBurst(null);
    sendBarcodeBurst(barcode1);
    sendBarcodeBurst(null);
    sendBarcodeBurst(barcode2);
    sendBarcodeBurst(barcode2);
    sendBarcodeBurst(null);
    sendBarcodeBurst(barcode1);
    sendBarcodeBurst(null);
}

 

So what it basically does is send some barcodes interleaved with null (no barcode detected) to the presenter using the onBarcodeDetected method to emulate the real camera sending barcode values with sometimes sending null whenever no barcode is in view.

The first test simply checks that no attendee is sent if the attendee list is null. Seems pretty obvious. The second one checks that if barcode does not match with any attendee’s identifier, it should not send any attendee as well. It does this by setting an arbitrary list of attendees with different identifiers and sending non-matching barcodes to the presenter. Lastly, the successful test, where a single matching barcode is sent to the presenter and it should send that particular attendee with matching identifier.

Phew! Quite a lot of tests and we are done. The tests were not very large and mostly self-explanatory. Now, you just have to implement the view and presenter methods till all the tests light up to be green and you are done! You have created a feature using MVP design and implemented it using test driven development.

So start testing and get a seal of reliability and confidence about your code and the satisfaction of seeing the green bar fill up.

Continue ReadingThe Joy of Testing with MVP in Open Event Orga App

Intro to concurrency and Refactoring Open Event Android using RxJava

Functional reactive programming seems to have taken the whole development world by storm. It’s one of the hottest thing even after 2 years of constant traction in the communities of several programming languages, where different implementations of the specifications defined by Rx or Reactive Extensions have changed the paradigm of programming for many professional and enthusiast developers.

RxJava is no exception, not only has it been widely adopted by Android and Java developers unanimously, but also received attention of well known and top developers of both communities. The reason of its success is the fluent API with heavy toolset it provides from the Functional Programming paradigm and its ease and natural ability to handle concurrency on different levels based on the type of operation being performed, i.e., computations, I/O, etc. It basically takes away the several constraints and concurrency related checklists developers had to maintain while working with thread management. So, now, developers can’t make an excuse for using database operations on the Main Thread because offloading it on another thread is hard.

So, in this blog post, I will be detailing the process of converting the existing synchronous code of your app into a performant reactive code without breaking the structure of your project, like we did in Open Event Android (Github Repo). Before starting, I have assumed that you know how to add RxJava dependency to your project as it is covered in many other blog posts and the documentation is also very clear. Secondly, you should also add RxAndroid dependency as it contains the Scheduler needed to work on Android’s Main Thread. So, Let’s start.

Current State

Currently, our code loads the queries from database synchronously on Main Thread using the SQLiteDatabase for an Android application. This is how it looks like –

As we can see, we are directly returning the loaded results to the caller. This is called synchronous call, meaning the caller will block till the function is returned, and can’t move further to do anything else. It basically waits for the function to return, which may take hundreds of milliseconds to seconds based on the function it performs.

New Android version crash the applications that perform Network interactions on the main thread but no such restriction for disk based operations is there, making it hard to enforce best performance practices. Before RxJava, there were interfaces made for different kinds of objects, passed in as parameters of the db request function, which created a new thread and performed operations and when completed, returned back the results to the main thread using the postOnUiThread method, so that the views could update themselves. The interface implementations passed are called callbacks because they call a particular function that you provide back when the asynchronous operation is completed. Even the calling of callback function is delegated on the implementor and may result in undesired effects. The query done in this fashion is called an asynchronous query because the execution of this takes place in parallel with main thread and is not synchronised with the main thread. It may take up forever to complete, complete even before the main thread moved on to next operation or even return when the main thread was completed and done waiting for it and destroyed. This will result in a weird crash even when the application was closed, because the returned function will try to update the views which are not even there.

Problems like these made Android Devs lazy and compromise with the performance of their application. Not anymore! RxJava is here to solve half of our problems. You see, RxJava does provide a solution to achieve effortless concurrency but does not ensure thread safety, memory contention, race conditions, deadlocks and other concurrency related issues for you. These you must code up for yourself.

So, after the introduction of Rx and its dire need in Android projects, we will move on to a basic procedure to convert any synchronous code to asynchronous call using RxJava Observable.

Let’s subscribe

The Observable class in RxJava is the most used and standard stream class you will use. Observable handles a stream of object and passes them as they arrive to the Subscriber attached to it. As you may guess, for a stream of data that arrives in a non deterministic fashion ( we don’t know when it will arrive ), we require an asynchronous query, and this is where RxJava excels at. You can configure an Observable to wait for result in one thread so that main thread doesn’t block and deliver result on another thread. You can either create a new thread or use certain pre configured schedulers for basic type of operations :

  1. Schedulers.newThread() : Creates a new thread for each request
  2. Schedulers.io() : For I/O bound work like a network call, database access
  3. Schedulers.computation() : For heavy computations
  4. AndroidSchedulers.mainThread() : For returning to UI thread of Android ( Present in RxAndroid )

There are other types of Schedulers like Schedulers.trampoline(), etc that are used for other purposes like testing, but the above ones are most commonly used ones and we’ll be using Schedulers.computation() for loading the SQLite query on the thread from Computation Thread Pool and AndroidSchedulers.mainThread() for delivering the result on UI thread.

Using Computation instead of I/O because I/O uses unbounded executor, meaning it continues adding threads to the thread pool, which isn’t good. So, we use computation instead. You can create your own bounded executor and pass it as a scheduler

The basic operation of passing an object to a subscriber is :

Observable.just(getEventDetails())
        .subscribe(new Consumer<Event>() {
          @Override
          public void accept(@NonNull Event event) throws Exception {
              Log.d("EVENT", event.toString());
          }
      });

 

Using lambda notation, we get a terse form of the same :

Observable.just(getEventDetails())
 	.subscribe(event -> Log.d("EVENT", event.toString()));

We’ll be using lambda notations from now on.

In the above example, we are just loading and passing the Event object to the subscriber below who logs it. But this is not asynchronous, everything gets executed on main thread. The above code is equivalent to :

Event event = getEventDetails();
Log.d("EVENT", event.toString());

 

So why use it, you say? Well, we can still get a lot of goodies from functional programming this way. For example,

String track = "Android";

Observable.fromIterable(getSessionList())
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .toList()
    .subscribe(titles -> Log.d("Titles", titles.toString()));

 

What this code does is, take a list of sessions and emit each session at a time, filter out the ones which don’t have Android as their track name, take out their titles and puts them in a list and gives it to subscriber.

Now imagine doing it in plain Java. Create a list of string, loop through each session, check track, push title to that list and this much when this example is the most basic of use cases of RxJava.

But how to achieve concurrency. If the there are 10000 sessions, this code will take huge time even if sessions are in memory and not loaded from database. So we will listen to these list events on computation thread.

Observable.fromIterable(getSessionList())
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .subscribeOn(Schedulers.computation())
    .subscribe(titles -> adapter.setItems(titles));

 

That’s it. Now each filtering and mapping and converging to a list is done on another thread.

If you want to listen to each session one at a time, and not all at once when it is completed, you can remove toList() operator

But now, our app will crash! Because when we deliver the result to subscriber, we are still on computation thread, so we need to come back to Main Thread because Android Views are not thread safe, meaning they cannot be accessed from any thread other than UI thread. So in order to do that, we just use observeOn() operator :

Observable.fromIterable(getSessionList())
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .toList()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(titles -> adapter.setItems(titles));

 

Still, our code has a critical problem, the mapping and filtering takes place on background thread, but the loading of session list still takes place on UI thread because it is loaded first and then passed to Observable

Observable methods like just, from, fromIterable, etc all take object from the current thread, meaning passing the object to these functions will not occur on the Scheduler you have supplied. This is very basic programming concept that language parses rightmost parameter first but usually is misunderstood in terms of Rx programming.

So, what do we do? We use fromCallable which waits till the containing function returns and then operates on it

Observable.fromCallable(this::getSessionList)
    .flatMapIterable(sessions -> sessions)
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .toList()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(titles -> adapter.setItems(titles));

 

We’re done! We have changed our synchronous database call to an asynchronous call.

Another use case is when you just have to do an operation asynchronously and not return anything, then fromCallable won’t work as it expects some return value to operate on, instead use Completable

Completable.fromAction(this::clearDatabase)
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(() -> {
        // Completed
        showToast("Success");
    });

 

Note that here we use method reference to call a function, you can just pass in a lambda or Action implementation to do some in place work like this

Completable.fromAction(() -> {
    doSomeStuff();
    // ...
    doOtherStuff(); 
}).subscribeOn(Schedulers.computation())
  .observeOn(AndroidSchedulers.mainThread())
  .subscribe(() -> {
      // Completed
      showToast("Success");
  });

 

Now, you can wrap all your slow methods into observable or completable without changing any code structure and your code will look like this :

On parting note, a trick to avoid repeated subscribeOn observeOn :

private <T> ObservableTransformer<T, T> applySchedulers() {
    return upstream -> upstream.subscribeOn(Schedulers.computation())
                               .observeOn(AndroidSchedulers.mainThread());
}

 

Create this function and just call compose on each Observable and call the function inside that, passing the transformer., like it is shown in the picture above

That’s it for now. Have a happy and lag free day!

Continue ReadingIntro to concurrency and Refactoring Open Event Android using RxJava