GSoC

One of the principal goals of Open Event Orga App (Github Repo) is to let the event organizer to scan the barcode identifier of an attendee at the time of entry in the event, in order to quickly check in that attendee. Although there are several scanning APIs available throughout the web for Android Projects, we chose Google Vision API as it handles multitude of formats and orientations automatically with little to no configuration, integrates seamlessly with Android, and is provided by Google itself, ensuring great support in future as well. Currently, the use case of our application is:

• Scan barcode from the camera feed
• Detect and show barcode information on screen
• Iterate through the attendee list to match the detected code with unique attendee identifier
• Successfully return to the caller with scanned attendee’s information so that a particular action can be taken

There are several layers of optimisations done in the Orga Application to make the user interaction fluid and concise. I’ll be sharing a few in this blog post regarding the configuration and handling of bursty data from camera source efficiently. To see the full implementation, you can visit the Github Repository of the project using the link provided above.

Configuration

The configuration of our project was done through Dagger 2 following the dependency injection pattern, which is not the focus of this post, but it is always recommended that you follow separation of concerns in your project and create a separate module for handling independent works like barcode scanner configuration. Normally, people would create factories with scanner, camera source and processors encapsulated. This enables the caller to have control over when things are initialized.

Our configuration provides us two initialised objects, namely, CameraSource and BarcodeDetector

@Provides
BarcodeDetector providesBarCodeDetector(Context context, Detector.Processor<Barcode> processor) {
    BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(context)
        .setBarcodeFormats(Barcode.QR_CODE)
        .build();

    barcodeDetector.setProcessor(processor);

    return barcodeDetector;
}

@Provides
CameraSource providesCameraSource(Context context, BarcodeDetector barcodeDetector) {
    return new CameraSource
        .Builder(context, barcodeDetector)
        .setRequestedPreviewSize(640, 480)
        .setRequestedFps(15.0f)
        .setAutoFocusEnabled(true)
        .build();
}

The fields needed to create the objects are provided as arguments to the provider functions as all the dependencies required to construct these objects. Now focusing on the Detector.Processor requirement of the BarcodeDetector is the classic example on non injectable code. This is because the processor is to be supplied by the activity or any other object which wants to receive the callback with the detected barcode data. This means we could inject it at the time of creation of the Activity or Presenter itself. We could easily overcome by adding a constructor to this dagger module containing the Barcode.Processor at the time of injection, but that would violate our existing 0 configuration based model where we just get the required component from the Application class and inject it. So, we wrapped the the processor into a PublishSubject

@Provides
@Singleton
@Named("barcodeEmitter")
PublishSubject<Notification<Barcode>> providesBarcodeEmitter() {
    return PublishSubject.create();
}

@Provides
@Singleton
Detector.Processor<Barcode> providesProcessor(@Named("barcodeEmitter") PublishSubject<Notification<Barcode>> emitter) {
    return new Detector.Processor<Barcode>() {
        @Override
        public void release() {
            // No action to be taken
        }

        @Override
        public void receiveDetections(Detector.Detections<Barcode> detections) {
            SparseArray<Barcode> barcodeSparseArray = detections.getDetectedItems();
            if (barcodeSparseArray.size() == 0)
                emitter.onNext(Notification.createOnError(new Throwable()));
            else
                emitter.onNext(Notification.createOnNext(barcodeSparseArray.valueAt(0)));
        }
    };
}

This solves 2 of our problems, not only now all these dependencies are injectable with 0 configurations, but also our stream of barcodes is now reactive.

Note that not everyone is in favour of using Singleton, but you can decrease the scope using your own annotation. We prefer not creating life cycle bound objects, those are hard to manage and can cause potential memory leaks, and the creation of an anonymous inner class object every time listener activates is not good for memory too.

Also, note that Singleton classes will cause memory leaks too if you don’t release their reference at the time of destruction of life cycle bound object

Notice how the type of PublishSubject is not just a barcode, but Notification which wraps the bar code. That’s because we want to send both the value and error streams down uninterrupted to the caller. Otherwise, the data stream would have stopped on the emission of first onError call. Here, we detect the barcodeSparseArray size and accordingly send error or first value to the PublishSubject which will be accordingly subscribed by the activity or presenter

Handling Bursty Data

barcodeEmitter.subscribe(barcodeNotification -> {
    if (barcodeNotification.isOnError()) {
        presenter.onBarcodeDetected(null);
    } else {
        presenter.onBarcodeDetected(barcodeNotification.getValue());
    }
});

Here is how we are subscribing the notification emitter and passing the appropriate value to the presenter to handle, null if it is an error and the value if it is the next emission.

Note that you must dispose the disposable returned by the subscribe method on the subject when the Activity is to be destroyed or else it will keep the reference to the anonymous inner class created with the lambda for barcodeNotification and cause a memory leak

Now, let’s see how the presenter handles this data for:

• Hiding and showing the barcode panel when barcode is on the screen accordingly
• Showing the data extracted from the barcode scanner

These things can be implemented in a very standard way with a few conditionals, but most developers forget the fact that the data emission rate is enormous when concerning with live feed of data. In the Open Event Orga app, we have reduced it to 15 FPS as it is more than enough to scan barcodes for our use case, but it is still huge. The continuous stream of nulls and barcode data is useless to us unless it changes.

A little explanation about nulls and values here: You must have noticed above the conditions when we pass null and value, but I’ll explain again. A value will be passed if there is a detected barcode on screen, and null will be passed if there is no barcode detected. The Google Vision API will keep sending the same value for barcode at 15 FPS and so we’ll get this redundant stream of nulls and values which we should not concern with processing as this will load the CPU unnecessarily.

There are only 2 cases where we need to process it:

• Null changes to Value -> Show barcode panel
  Value changes to null -> Hide barcode panel
• Value changes irrespective of nulls -> Show barcode data on UI and search through the attendee identifiers

So, here too we’ll create 2 PublishSubject objects

private PublishSubject<Boolean> detect = PublishSubject.create();
private PublishSubject<String> data = PublishSubject.create();

And we’ll configure them both in this way to receive data on each barcode emission in the presenter:

public void onBarcodeDetected(Barcode barcode) {
   detect.onNext(barcode == null);
   if (barcode != null)
       data.onNext(barcode.displayValue);
}

This will make data only receive non-null changes and detect receive a boolean notifying if the current detected barcode was null or not.

Now, we see how each of these subjects is configured to pass the emissions downstream:

data.distinctUntilChanged()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(this::processBarcode);

This one is pretty straightforward. It’ll only send data downstream if its value has changed from the previous emission, disregarding nulls. So, for a stream like this

A A A A A null null null null A A A A null B B B B B B null null null B B A A null A

It will only emit:

A                                                                         B                                                  A

Which is actually what we want, as we only need to process and show the distinct barcode data and match it with our attendee list. Now, with thousands of attendees, the first method would have triggered unnecessary and time-consuming computations over and over again on same identifiers with little gaps of time, which would have created mediocre results even in a multi-threaded environment. The second one saves us from repetitive calculations and also gives us enormous gaps between emissions, which is optimal for our use case.

The second case is not so obvious because we can’t ignore nulls here as we have to show and hide UI based on them. This means that unlike our previous stream if we just use distinctUntilChanged, it will look like this:

A A A A A null null null null A A A A null B B B B B B null null null B B A A null A

f                 t                               f               t      f                    t                        f             t       f

This is because, if you remember, we were sending down emissions of barcode == null on each emission for this Subject. So, in this case, as you may see, some of the values are so close enough that it will not be discernable in UI and also annoy users who’ll see the panel pop up for milliseconds before vanishing or vice-versa. The perfect operation for this case will be debounce

detect.distinctUntilChanged()
    .debounce(150, TimeUnit.MILLISECONDS)
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(receiving -> scanQRView.showBarcodePanel(!receiving));

This operator will drop any emission in the window of 150ms succession and only pick up those emissions which are 150ms apart from each other. Now, 150 ms is not a magic number, it is picked through hit and trial and what works best for your case. Lower the value and you will pick up more changes downstream, increase the value and you might miss the required events.

This makes our stream somewhat like this, cleaning out the cluttered events

f                 t                               f                                           t                        f

This is the screenshot of the implementation:  

And an animated gif of the scanning process:

This is all for this blog, you may use many other operators from the arsenal of RxJava, whatever fits your use case. As I have presumed the knowledge about Subjects, MVP and a little bit of Dagger in this post, I’ll link some of the resources where you can find more information about these:

http://reactivex.io/documentation/subject.html

https://github.com/ReactiveX/RxJava/wiki/Subject

http://reactivex.io/documentation/operators/distinct.html

http://www.vogella.com/tutorials/Dagger/article.html

https://antonioleiva.com/mvp-android/