Performing Database Migrations using DbFlow

In Open Event Organizer Android App we decided to add database migrations for every change in database while development. Two of the reasons behind this –

  1. The users have some version of the app installed and then during development the developers had to modify the database, let’s say, add a table or modify existing ones. This makes the existing database incompatible with the new app. On adding database migration the upgradation of database takes place automatically and prevent the situation of reinstalling the app.
  2. Migrations makes it possible to rollback or upgrade to some particular database state. Thus, help in debugging certain changes in isolation.

Let’s discuss the implementation details. Consider the scenario when a user adds a new table named SpeakersCall. For creating migration for this change we need to add migration class inside OrgaDatabase class annotated with @Database. We will break it down to look closely at each step.

  1. Use @Migration annotation in DbFlow library and specify the new database version (by incrementing the existing version) and the database class.
  2. Extend BaseMigration and override migrate method.
  3. The logic used inside the migrate method can be different for different tasks. In the present case we first need to delete any existing table (if exists) with the name SpeakersCall and then recreate that table in database.
  4. Create an array of java classes which are created/modified.
  5. We wrap the SQL query inside a Database Wrapper class which prevents it from running recursively.
  6. FlowManager uses reflection to look up and construct the generated database holder class used in defining the structure for all databases used in this application. So we will getModelAdapter for the class to be recreated and use creation query returned by Model Adapter and execute it.
@Migration(version = 15, database = OrgaDatabase.class)
public static class MigrationTo15 extends BaseMigration {

@Override
public void migrate(@NonNull DatabaseWrapper databaseWrapper) {
Timber.d(“Running migration for DB version 14”);

Class<?>[] recreated = new Class[] {SpeakersCall.class};

for (Class<?> recreate: recreated) {
ModelAdapter modelAdapter = FlowManager.getModelAdapter(recreate);
databaseWrapper.execSQL(DROP_TABLE + modelAdapter.getTableName());
databaseWrapper.execSQL(modelAdapter.getCreationQuery());
}
}
}

Similarly, we can write migration for changing a column of table(s).

Use Of DBFlowChangeListener for Automatic Updation of FAQ List in the Open Event Orga App

In the Open Event Orga App, whenever a faq is created in via the CreatefaqFragment, the new faqs aren’t updated in the FAQ’s list. The user has to swipe down and refresh to see the newly created FAQ. This seems to be a major problem and hence to solve this issue a DBFlowChangeListener class has been created.

Following blog post shows the implementation of the ChangeListener class.

Steps of Implementation

In the FaqListPresenter constructor pass an argument of the type DatabaseChangeListener<Faq> and assign it to local variable faqChangeListener.

@Inject
public FaqListPresenter(FaqRepository faqRepository, DatabaseChangeListener<Faq> faqChangeListener) {
  this.faqRepository = faqRepository;
  this.faqChangeListener = faqChangeListener;
}

A method listenChanges( ) is made in the Presenter that will initiate the listening of database. Following code shows the listenChanges( ) . startListening( ) is a method in the DBFlowDatabaseChangeListener which is accessed with the help of DatabaseChangeListener interface.

private void listenChanges() {
  
  faqChangeListener.startListening();
  
  faqChangeListener.getNotifier()
      .compose(dispose(getDisposable()))
      .map(DbFlowDatabaseChangeListener.ModelChange::getAction)
      .filter(action -> action.equals(BaseModel.Action.INSERT) || action.equals(BaseModel.Action.DELETE))
      .subscribeOn(Schedulers.io())
      .subscribe(faqModelChange -> loadFaqs(false), Logger::logError);
}

In the startListening( )  in DatabaseChangeListener, DirectModelNotifier is used to get notified of database changes. It is ultimately registered for any kind of model change as shown in the following code snippet. ReplaySubject is one of the 4 kinds of subjects ( Publish Subject, Replay Subject, Behavior Subject, Async Subject) that RxJava provides. It emits all the items of the source Observable, regardless of when the subscriber subscribes.

public void startListening() {
  if (disposable == null || disposable.isDisposed())
      replaySubject = ReplaySubject.create();

  modelModelChangedListener = new DirectModelNotifier.ModelChangedListener<T>() {

      @Override
      public void onTableChanged(@Nullable Class<?> aClass, @NonNull BaseModel.Action action) {
          replaySubject.onNext(new ModelChange<>(null, action));
      }

      @Override
      public void onModelChanged(@NonNull T model, @NonNull BaseModel.Action action) {
          replaySubject.onNext(new ModelChange<>(model, action));
      }
  };

  DirectModelNotifier.get().registerForModelChanges(classType, modelModelChangedListener);
}

When the faqChangeListener is subscribed it calls the loadFaqs( ) , it ultimately loads the updated values from the database. Hence all the updated values are now displayed in the FaqListFragment.

public void loadFaqs(boolean forceReload) {
  getFaqSource(forceReload)
      .compose(dispose(getDisposable()))
      .compose(progressiveErroneousRefresh(getView(), forceReload))
      .toList()
      .compose(emptiable(getView(), faqs))
      .subscribe(Logger::logSuccess, Logger::logError);
}

Hence the use of DBFlowListener makes it possible to communicate between 2 fragments. Its usability has also been exploited in Tracks, Tickets, Sponsors fragments as well.

References:

  1. RaizLab’s DBFLow:      https://github.com/Raizlabs/DBFlow
  2. Docs related to DBFlow:             https://agrosner.gitbooks.io/dbflow/content/Observability.html
  3. Subjects used in RxJavahttps://blog.mindorks.com/understanding-rxjava-subject-publish-replay-behavior-and-async-subject-224d663d452f

Persistence Layer in Open Event Organizer Android App

Open Event Organizer is an Event Managing Android App with the core features of Attendee Check In by QR Code Scan and Data Sync with the Open Event API Server. As an event can be large, so the app will be dealing with a large amount of a data. Hence to avoid repetitive network requests for fetching the data, the app maintains a local database containing all the required data and the database is synced with the server. Android provides android.database.sqlite package which contains the API needed to use the database on the Android. But it is really not a good practice to use the sqlite queries everywhere in the app. So there comes a persistence layer. A persistence layer works between the database and the business logic. Open Event Organizer uses Raizlabs’s DbFlow, an ORM based Android Database Library for the same. I will be talking about its implementation through the app in this blog.

First of all, you declare the base class of the database which is used to create the database by Android for the app. You declare all the base constants here. The class looks like:

@Database(
   name = OrgaDatabase.NAME,
   version = OrgaDatabase.VERSION,
   ...
)
public class OrgaDatabase {
   public static final String NAME = "orga_database";
   public static final int VERSION = 2;
   ...
}

OrgaDatabase.java
app/src/main/java/org/fossasia/openevent/app/data/db/configuration/OrgaDatabase.java

Initialise the database in the Application class using FlowManager provided by the library. Choose the Application class to do this to ensure that the library finds the generated code in the DbFlow.

FlowManager.init(
   new FlowConfig.Builder(context)
       .addDatabaseConfig(
           new DatabaseConfig.Builder(OrgaDatabase.class)
           ...
           .build()
       )
       .build());

OrgaApplication.java
app/src/main/java/org/fossasia/openevent/app/OrgaApplication.java

The database is created now. For tables creation, DbFlow uses model classes which must be annotated using the annotations provided by the library. The basic annotations are – @Table, @PrimaryKey, @Column, @ForeignKey etc.

For example, the Attendee class in the app looks like:

@Table(database = OrgaDatabase.class)
public class Attendee ... {

   @PrimaryKey
   public long id;

   @Column
   public boolean checkedIn;
   ...
   ...
   @ForeignKey(
       onDelete = ForeignKeyAction.CASCADE,
       onUpdate = ForeignKeyAction.CASCADE)
   public Order order;
   ...
}

Attendee.java
app/src/main/java/org/fossasia/openevent/app/data/models/Attendee.java

This will create a table named attendee with the columns and relationships annotated. Now comes the part of accessing data from the database. Open Event App uses RxJava’s support to the DbFlow library which enables async data accessing. The getItems method from DataBaseRepository looks like:

public <T> Observable<T> getItems(Class<T> typeClass, SQLOperator... conditions) {
   return RXSQLite.rx(SQLite.select()
       .from(typeClass)
       .where(conditions))
       .queryList()
       .flattenAsObservable(items -> items);
}

 

The method returns an observable emitting the items from the result. For data saving, the method looks like:

DatabaseDefinition database = FlowManager.getDatabase(OrgaDatabase.class);
FastStoreModelTransaction<T> transaction = FastStoreModelTransaction
   .insertBuilder(FlowManager.getModelAdapter(itemClass))
   .addAll(items)
   .build();
database.executeTransaction(transaction);

 

And for updating data, the method looks like:

ModelAdapter<T> modelAdapter = FlowManager.getModelAdapter(classType);
modelAdapter.update(item);

DatabaseRepository.java
app/src/main/java/org/fossasia/openevent/app/data/db/DatabaseRepository.java

DbFlow provides DirectModelNotifier which is used to get notified of the database change anywhere in the app. Open Event App uses PublishSubjects to send notifications on database change event. The implementation of the DatabaseChangeListener in the app looks like:

public class DatabaseChangeListener<T> ... {
   private PublishSubject<ModelChange<T>> publishSubject = PublishSubject.create();
   private DirectModelNotifier.ModelChangedListener<T> modelModelChangedListener;
   ...
   public void startListening() {
       modelModelChangedListener = new DirectModelNotifier.ModelChangedListener<T>() {
           @Override
           public void onTableChanged(@Nullable Class<?> aClass, @NonNull BaseModel.Action action) {
               // No action to be taken
           }
           @Override
           public void onModelChanged(@NonNull T model, @NonNull BaseModel.Action action) {
               publishSubject.onNext(new ModelChange<>(model, action));
           }
       };
       DirectModelNotifier.get().registerForModelChanges(classType, modelModelChangedListener);
   }
   ...
}

DatabaseChangeListener.java
app/src/main/java/org/fossasia/openevent/app/data/db/DatabaseChangeListener.java

The class is used in the app to get notified of the data change and to update the required local data fields using data from item emitted by the publishSubject of the class. This is used in the app where same data is accessed at more than one places. For example, There are two fragments – AttendeesFragment and AttendeeCheckInFragment from which an attendee’s check in status is toggled. So when the status is toggled from AttendeeCheckInFragment, the change must be updated in the AttendeesFragment’s attendees list. This is carried out using DatabaseChangeListener in the AttendeesPresenter which provides attendees list to the AttendeesFragment. And on the change in the attendee’s check in status, AttendeePresenter’s attendeeListener listens for the change and update the attendee in the list accordingly.

Links:
1. Raizlabs’s DbFlow , an ORM Android Database Library Github Repo Link
2. DbFlow documentation
3. Android database managing API android.database.sqlite