Restoring State after Orientation Change in Loklak Wok Android

During orientation change i.e. from portrait to landscape mode in Android, the current activity restarts again. As the activity restarts again, all the defined variables loose their previous value, for example the scroll position of a RecyclerView, or the data in the rows of RecyclerView etc. Just imagine a user searched some tweets in Loklak Wok Android, and as the user’s phone is in “Auto rotation” mode, the orientation changes from portrait to landscape. As a result of this, the user loses the search result and has to do the search again. This leads to a bad UX.

Saving state in onSavedInstanceState

The state of the app can be saved by inserting values in a Bundle object in onSavedInstanceState callback. Inserting values is same as adding elements to a Map in Java. Methods like putDouble, putFloat, putChar etc. are used where the first parameter is a key and the second parameter is the value we want to insert.

@Override
public void onSaveInstanceState(Bundle outState) {
   if (mLatitude != null && mLongitude != null) {
       outState.putDouble(PARCELABLE_LATITUDE, mLatitude);
       outState.putDouble(PARCELABLE_LONGITUDE, mLongitude);
   }
...
}

 

The values can be retrieved back when onCreate or onCreateView of the Activity or Fragment is called. Bundle object in the callback parameter is checked, whether it is null or not, if not the values are retrieved back using the keys provided at the time of inserting. The latitude and longitude of a location in TweetPostingFragment are retrieved in the same fashion

public void onViewCreated(View view, @Nullable Bundle savedInstanceState) {
   ...
   if (savedInstanceState != null) { // checking if bundle is null
       // extracting from bundle
       mLatitude = savedInstanceState.getDouble(PARCELABLE_LATITUDE);
       mLongitude = savedInstanceState.getDouble(PARCELABLE_LONGITUDE);
       // use extracted value
   }
}

Restoring Custom Objects, using Parcelable

But what if we want to restore custom object(s). A simple option can be serializing the objects using the native Java Serialization or libraries like Gson. The problem in these cases is performance, they are quite slow. Parcelable can be used, which leads the pack in performance and moreover it is provided by Android SDK, on top of that, it is simple to use.

The objects of class which needs to be restored implements Parcelable interface and the class must provide a static final object called CREATOR which implements Parcelable.Creator interface.

writeToParcel and describeContents method need to be override to implement Parcelable interface. In writeToParcel method the member variables are put inside the parcel, in our case describeContents method is not used, so, simply 0 is returned. Status class which stores the data of a searched tweet implements parcelable.

@Override
public int describeContents() {
   return 0;
}

@Override
public void writeToParcel(Parcel dest, int flags) {
   dest.writeString(mText);
   dest.writeInt(mRetweetCount);
   dest.writeInt(mFavouritesCount);
   dest.writeStringList(mImages);
   dest.writeParcelable(mUser, flags);
}

 

NOTE: The order in which variables are pushed into Parcel needs to be maintained while variables are extracted from the parcel to recreate the object. This is the reason why no “key” is required to push data into a parcel as we do in bundle.

The CREATOR object implements the creation of object from a Parcel. The CREATOR object overrides two methods createFromParcel and newArray. createFromParcel is the method in which we implement the way an object is created from a parcel.

public static final Parcelable.Creator<Status> CREATOR = new Creator<Status>() {
   @Override
   public Status createFromParcel(Parcel source) {
       return new Status(source); // a private constructor to create object from parcel
   }

   @Override
   public Status[] newArray(int size) {
       return new Status[size];
   }
};

 

The private constructor, note that the order in which variables were pushed is maintained while retrieving the values.

private Status(Parcel source) {
   mText = source.readString();
   mRetweetCount = source.readInt();
   mFavouritesCount = source.readInt();
   mImages = source.createStringArrayList();
   mUser = source.readParcelable(User.class.getClassLoader());
}

 

The status objects are restored the same way, latitude and longitude were restored. putParcelableArrayList in onSaveInstaceState and getParcelableArrayList in onCreateView methods are used to push into Bundle object and retrieve from Bundle object respectively.

@Override
public void onSaveInstanceState(Bundle outState) {
   super.onSaveInstanceState(outState);
   ArrayList<Status> searchedTweets = mSearchCategoryAdapter.getStatuses();
   outState.putParcelableArrayList(PARCELABLE_SEARCHED_TWEETS, searchedTweets);
   ...
}


// retrieval of the pushed values in bundle
@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container,
                            Bundle savedInstanceState) {
   ...
   if (savedInstanceState != null) {
       ...
       List<Status> searchedTweets =
               savedInstanceState.getParcelableArrayList(PARCELABLE_SEARCHED_TWEETS);
       mSearchCategoryAdapter.setStatuses(searchedTweets);
   }
   ...
   return view;
}

Resources:

Testing Presenter of MVP in Loklak Wok Android

Imagine working on a large source code, and as a new developer you are not sure whether the available source code works properly or not, you are surrounded by questions like, Are all these methods invoked properly or the number of times they need to be invoked? Being new to source code and checking manually already written code is a pain. For cases like these unit-tests are written. Unit-tests check whether the implemented code works as expected or not. This blog post explains about implementation of unit-tests of Presenter in a Model-View-Presenter (MVP) architecture in Loklak Wok Android.

Adding Dependencies to project

In app/build.gradle file

defaultConfig {
   ...
   testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}

dependencies {
   ...
   androidTestCompile 'org.mockito:mockito-android:2.8.47'
   androidTestCompile 'com.android.support:support-annotations:25.3.1'
   androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2.2'
}

Setup for Unit-Tests

The presenter needs a realm database and an implementation of LoklakAPI interface. Along with that a mock of the View is required, so as to check whether the methods of View are called or not.

The LoklakAPI interface can be mocked easily using Mockito, but the realm database can’t be mocked. For this reason an in-memory realm database is created, which will be destroyed once all unit-test are executed. As the presenter is required for each unit-test method we instantiate the in-memory database before all the tests start i.e. by annotating a public static method with @BeforeClass, e.g. setDb method.

@BeforeClass
public static void setDb() {
   Realm.init(InstrumentationRegistry.getContext());
   RealmConfiguration testConfig = new RealmConfiguration.Builder()
           .inMemory()
           .name("test-db")
           .build();
   mDb = Realm.getInstance(testConfig);
}

 

NOTE: The in-memory database should be closed once all unit-tests are executed. So, for closing the databasse we create a public static method annotated with @AfterClass, e.g. closeDb method.

@AfterClass
public static void closeDb() {
   mDb.close();
}

 

Now, before each unit-test is executed we need to do some setup work like instantiating the presenter, a mock instance of API interface generated  by using mock static method and pushing in some sample data into the database. Our presenter uses RxJava and RxAndroid which depend on IO scheduler and MainThread scheduler to perform tasks asynchronously and these schedulers are not present in testing environment. So, we override RxJava and RxAndroid to use trampoline scheduler in place of IO and MainThread so that our test don’t encounter NullPointerException. All this is done in a public method annotated with @Before e.g. setUp.

@Before
public void setUp() throws Exception {
   // mocking view and api
   mMockView = mock(SuggestContract.View.class);
   mApi = mock(LoklakAPI.class);

   mPresenter = new SuggestPresenter(mApi, mDb);
   mPresenter.attachView(mMockView);

   queries = getFakeQueries();
   // overriding rxjava and rxandroid
   RxJavaPlugins.setIoSchedulerHandler(scheduler -> Schedulers.trampoline());
   RxAndroidPlugins.setMainThreadSchedulerHandler(scheduler -> Schedulers.trampoline());

   mDb.beginTransaction();
   mDb.copyToRealm(queries);
   mDb.commitTransaction();
}

 

Some fake suggestion queries are created which will be returned as observable when API interface is mocked. For this, simply two query objects are created and added to a List after their query parameter is set. This is implemented in getFakeQueries method.

private List<Query> getFakeQueries() {
   List<Query> queryList = new ArrayList<>();

   Query linux = new Query();
   linux.setQuery("linux");
   queryList.add(linux);

   Query india = new Query();
   india.setQuery("india");
   queryList.add(india);

   return queryList;
}

 

After that, a method is created which provides the created fake data wrapped inside an Observable as implemented in getFakeSuggestionsMethod method.

private Observable<SuggestData> getFakeSuggestions() {
   SuggestData suggestData = new SuggestData();
   suggestData.setQueries(queries);
   return Observable.just(suggestData);
}

 

Lastly, the mocking part is implemented using Mockito. This is really simple, when and thenReturn static methods of mockito are used for this. The method which would provide the fake data is invoked inside when and the fake data is passed as a parameter to thenReturn. For example, stubSuggestionsFromApi method

private void stubSuggestionsFromApi(Observable observable) {
   when(mApi.getSuggestions(anyString())).thenReturn(observable);
}

Finally, Unit-Tests

All the tests methods must be annotated with @Test.

Firstly, we test for a successful API request i.e. we get some suggestions from the Loklak Server. For this, getSuggestions method of LoklakAPI is mocked using stubSuggestionFromApi method and the observable to be returned is obtained using getFakeSuggestions method. Then, loadSuggestionFromAPI method is called, the one that we need to test. Once loadSuggestionFromAPI method is invoked, we then check whether the method of the View are invoked inside loadSuggestionFromAPI method, this is done using verify static method. The unit-test is implemented in testLoadSuggestionsFromApi method.

@Test
public void testLoadSuggestionsFromApi() {
   stubSuggestionsFromApi(getFakeSuggestions());

   mPresenter.loadSuggestionsFromAPI("", true);

   verify(mMockView).showProgressBar(true);
   verify(mMockView).onSuggestionFetchSuccessful(queries);
   verify(mMockView).showProgressBar(false);
}

 

Similarly, a failed network request for obtaining is suggestions is tested using testLoadSuggestionsFromApiFail method. Here, we pass an IOException throwable – wrapped inside an Observable – as parameter to stubSuggestionsFromApi.

@Test
public void testLoadSuggestionsFromApiFail() {
   Throwable throwable = new IOException();
   stubSuggestionsFromApi(Observable.error(throwable));

   mPresenter.loadSuggestionsFromAPI("", true);
   verify(mMockView).showProgressBar(true);
   verify(mMockView).showProgressBar(false);
   verify(mMockView).onSuggestionFetchError(throwable);
}

 

Lastly, we test if our suggestions are saved in the database by counting the number of saved suggestions and asserting that, in testSaveSuggestions method.

@Test
public void testSaveSuggestions() {
   mPresenter.saveSuggestions(queries);
   int count = mDb.where(Query.class).findAll().size();
  // queries is the List that contains the fake suggestions
   assertEquals(queries.size(), count);
}

Resources:

MVP in Loklak Wok Android using Dagger2

MVP stands for Model-View-Presenter, one of the most popular and commonly used design pattern in android apps. Where “Model” refers to data source, it can be a SharedPreference, Database or data from a Network call. Going by the word, “View” is the user interface and finally “Presenter”, it’s a mediator between model and view. Whatever events occur in a view are passed to presenter and the presenter fetches the data from the model and finally passes it back to the view, where the data is populated in ViewGroups. Now, the main question, why it is so widely used? One of the obvious reason is the simplicity to implement it and it completely separates the business logic, so, easy to write unit-tests. Though it is easy to implement, its implementation requires a lot of boilerplate code, which is one of its downpoints. But, using Dagger2 the boilerplate code can be reduced to a great extent. Let’s see how Dagger2 is used in Loklak Wok Android to implement MVP architecture.

Adding Dagger2 to the project

In app/build.gradle file

dependencies {
   ...
   compile 'com.google.dagger:dagger:2.11'
    annotationProcessor 'com.google.dagger:dagger-compiler:2.11'
}

 

Implementation

First a contract is created which defines the behaviour or say the functionality of View and Presenter. Like showing a progress bar when data is being fetched, or the view when the network request is successful or it failed. The contract should be easy to read and going by the names of the method one should be able to know the functionality of methods. For tweet search suggestions, the contract is defined in SuggestContract interface.

public interface SuggestContract {

   interface View {

       void showProgressBar(boolean show);

       void onSuggestionFetchSuccessful(List<Query> queries);

       void onSuggestionFetchError(Throwable throwable);
   }

   interface Presenter {

       void attachView(View view);

       void createCompositeDisposable();

       void loadSuggestionsFromAPI(String query, boolean showProgressBar);

       void loadSuggestionsFromDatabase();

       void saveSuggestions(List<Query> queries);

       void suggestionQueryChanged(Observable<CharSequence> observable);

       void detachView();
   }
}

 

A SuggestPresenter class is created which implements the SuggestContract.Presenter interface. I will not be explaining how each methods in SuggestPresenter class is implemented as this blog solely deals with implementing MVP. If you are interested you can go through the source code of SuggestPresenter. Similarly, the view i.e. SuggestFragment implements SuggestContract.View interface.

So, till this point we have our presenter and view ready. The presenter needs to access the model and the view requires to have an instance of presenter. One way could be instantiating an instance of model inside presenter and an instance of presenter inside view. But, this way model, view and presenter would be coupled and that defeats our purpose. So, we just INJECT model into presenter and presenter into view using Dagger2. Injecting here means Dagger2 instantiates model and presenter and provides wherever they are requested.

ApplicationModule provides the required dependencies for accessing the “Model” i.e. a Loklak API client and realm database instance. When we want Dagger2 to provide a dependency we create a method annotated with @Provides as providesLoklakAPI and providesRealm.

@Provides
LoklakAPI providesLoklakAPI(Retrofit retrofit) {
   return retrofit.create(LoklakAPI.class);
}

@Provides
Realm providesRealm() {
   return Realm.getDefaultInstance();
}

 

If we look closely providesLoklakAPI method requires a Retrofit instance i.e. a to create an instance of LoklakAPI the required dependency is Retrofit, which is fulfilled by providesRetrofit method. Always remember that whenever a dependency is required, it should not be instantiated at the required place, rather it should be injected by Dagger2.

@Provides
Retrofit providesRetrofit() {
   Gson gson = Utility.getGsonForPrivateVariableClass();
   return new Retrofit.Builder()
           .baseUrl(mBaseUrl)
           .addCallAdapterFactory(RxJava2CallAdapterFactory.create())
           .addConverterFactory(GsonConverterFactory.create(gson))
           .build();
}

 

As the ApplicationModule class provides these dependencies the class is annotated with @Module.

@Module
public class ApplicationModule {

   private String mBaseUrl;

   public ApplicationModule(String baseUrl) {
       this.mBaseUrl = baseUrl;
   }
   
   
   // retrofit, LoklakAPI, realm @Provides methods
}


After preparing the source to provide the dependencies, it’s time we request the dependencies.

Dependencies are requested simply by using @Inject annotation e.g. in the constructor of SuggestPresenter @Inject is used, due to which Dagger2 provides instance of LoklakAPI and Realm for constructing an object of SuggestPresenter.

public class SuggestPresenter implements SuggestContract.Presenter {

   private final Realm mRealm;
   private LoklakAPI mLoklakAPI;
   private SuggestContract.View mView;
   ...

   @Inject
   public SuggestPresenter(LoklakAPI loklakAPI, Realm realm) {
       this.mLoklakAPI = loklakAPI;
       this.mRealm = realm;
       ...
   }
   
   // implementation of methods defined in contract
}


@Inject can be used on the fields also. When @Inject is used with a constructor the class also becomes a dependency provider, this way creating a method with @Provides is not required in a Module class.

Now, it’s time to connect the dependency providers and dependency requesters. This is done by creating a Component interface, here ApplicationComponent. The component interface defines where are the dependencies required. This is only for those cases where dependencies are injected by using @Inject for the member variables. So, we define a method inject with a single parameter of type SuggestFragment, as the Presenter needs to be injected in SuggestFragment.

@Component(modules = ApplicationModule.class)
public interface ApplicationComponent {


   void inject(SuggestFragment suggestFragment);

}

 

The component interface is instantiated in onCreate method of LoklakWokApplication class, so that it is accessible all over the project.

public class LoklakWokApplication extends Application {

   private ApplicationComponent mApplicationComponent;

   @Override
   public void onCreate() {
       super.onCreate();
      ...
       mApplicationComponent = DaggerApplicationComponent.builder()
               .applicationModule(new ApplicationModule(Constants.BASE_URL_LOKLAK))
               .build();
   }

   public ApplicationComponent getApplicationComponent() {
       return mApplicationComponent;
   }
   
   ...
}


NOTE: DaggerApplicationComponent is created after building the project. So, AndroidStudio will show “Cannot resolve symbol …”, thus build the project : Build > Make Module ‘app’.

Finally, in the onCreateView callback of SuggestFragment we call inject method of DaggerApplicationComponent to tell Dagger2 that SuggestFragment is requesting dependencies.

@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container,
                        Bundle savedInstanceState) {
...   
   LoklakWokApplication application = (LoklakWokApplication) getActivity().getApplication();
   application.getApplicationComponent().inject(this);
   suggestPresenter.attachView(this);

   return rootView;
}

Resources:

Animations in Loklak Wok Android

Imagine an Activity popping out of nowhere suddenly in front of the user. And even more irritating, the user doesn’t even know whether a button was clicked. Though these are very small animation implementations but these animations enhance the user experience to a new level. This blog deals with the animations in Loklak Wok Android, a peer message harvester of Loklak Server.

Activity transition animation

Activity transition is applied when we move from a current activity to a new activity or just go back to an old activity by pressing back button.

In Loklak Wok Android, when user navigates for search suggestions from TweetHarvestingActivity to SuggestActivity, the new activity i.e. SuggestActivity comes from right side of the screen and the old one i.e. TweetHarvestingActivity leaves the screen through the left side. This is an example of left-right activity transition. For implementing this, two xml files which define the animations are created, enter.xml and exit.xml are created.

<set
   xmlns:android="http://schemas.android.com/apk/res/android"
   android:shareInterpolator="false">

   <translate
       android:duration="500"
       android:fromXDelta="100%"
       android:toXDelta="0%"/>
</set>

 

NOTE: The entering activity comes from right side, that’s why android:fromXDelta parameter is set to 100% and as the activity finally stays at extreme left, android:toXDelta parameter is set to 0%.

As the current activity, in this case TweetHarvestingActivity, leaves the screen from left to the negative of left. So, in exit.xml the android:fromXDelta parameter is set to 0% and android:toXDelta parameter is set to -100%.

Now, that we are done with defining the animations in xml, it’s time we apply the animations, which is really easy. The animations are applied by invoking Activity.overridePendingTransition(enterAnim, exitAnim) just after the startActivity method. For example, in openSuggestActivity

private void openSuggestActivity() {
   Intent intent = new Intent(getActivity(), SuggestActivity.class);
   startActivity(intent);
   getActivity().overridePendingTransition(R.anim.enter, R.anim.exit);
}

 

Touch Selectors

Using touch selectors background color of a button or any clickable can be changed, this way a user can see that the clickable responded to the click. The background is usually light accent color or a lighter shade of the icon present in button.

There are three states involved while a clickable is touched, pressed, activated and selected. And a default state, i.e. the clickable is not clicked. The background color of each state is defined in a xml file like media_button_selector, which is present in drawable directory.

<selector xmlns:android="http://schemas.android.com/apk/res/android">

   <item android:drawable="@color/media_button_touch_selector_backgroud" android:state_pressed="true"/>
   <item android:drawable="@color/media_button_touch_selector_backgroud" android:state_activated="true"/>
   <item android:drawable="@color/media_button_touch_selector_backgroud" android:state_selected="true"/>

   <item android:drawable="@android:color/transparent"/>
</selector>

 

The selector is applied by setting it as the background of a clickable, for example, touch selector applied on Location image button present in fragment_tweet_posting.xml .

<ImageButton
   android:layout_width="40dp"
   android:layout_height="40dp"
   
   android:background="@drawable/media_button_selector" />

 

Notice the change in the background color of the buttons when clicked.

Resources:

Some youtube videos for getting started:

Avoiding Nested Callbacks using RxJS in Loklak Scraper JS

Loklak Scraper JS, as suggested by the name, is a set of scrapers for social media websites written in NodeJS. One of the most common requirement while scraping is, there is a parent webpage which provides links for related child webpages. And the required data that needs to be scraped is present in both parent webpage and child webpages. For example, let’s say we want to scrape quora user profiles matching search query “Siddhant”. The matching profiles webpage for this example will be https://www.quora.com/search?q=Siddhant&type=profile which is the parent webpage, and the child webpages are links of each matched profiles.

Now, a simplistic approach is to first obtain the HTML of parent webpage and then synchronously fetch the HTML of child webpages and parse them to get the desired data. The problem with this approach is that, it is slower as it is synchronous.

A different approach can be using request-promise-native to implement the logic in asynchronous way. But, there are limitations with this approach. The HTML of child webpages that needs to be fetched can only be obtained after HTML of parent webpage is obtained and number of child webpages are dynamic. So, there is a request dependency between parent and child i.e. if only we have data from parent webpage we can extract data from child webpages. The code would look like this

request(parent_url)
   .then(data => {
       ...
       request(child_url)
           .then(data => {
               // again nesting of child urls
           })
           .catch(error => {

           });
   })
   .catch(error => {

   });

 

Firstly, with this approach there is callback hell. Horrible, isn’t it? And then we don’t know how many nested callbacks to use as the number of child webpages are dynamic.

The saviour: RxJS

The solution to our problem is reactive extensions in JavaScript. Using rxjs we can obtain the required data without callback hell and asynchronously!

The promise-request object of the parent webpage is obtained. Using this promise-request object an observable is generated by using Rx.Observable.fromPromise. flatmap operator is used to parse the HTML of the parent webpage and obtain the links of child webpages. Then map method is used transform the links to promise-request objects which are again transformed into observables. The returned value – HTML – from the resulting observables is parsed and accumulated using zip operator. Finally, the accumulated data is subscribed. This is implemented in getScrapedData method of Quora JS scraper.

getScrapedData(query, callback) {
   // observable from parent webpage
   Rx.Observable.fromPromise(this.getSearchQueryPromise(query))
     .flatMap((t, i) => { // t is html of parent webpage
       // request-promise object of child webpages
       let profileLinkPromises = this.getProfileLinkPromises(t);
       // request-promise object to observable transformation
       let obs = profileLinkPromises.map(elem => Rx.Observable.fromPromise(elem));

       // each Quora profile is parsed
       return Rx.Observable.zip( // accumulation of data from child webpages
         ...obs,
         (...profileLinkObservables) => {
           let scrapedProfiles = [];
           for (let i = 0; i < profileLinkObservables.length; i++) {
             let $ = cheerio.load(profileLinkObservables[i]);
             scrapedProfiles.push(this.scrape($));
           }
           return scrapedProfiles; // accumulated data returned
         }
       )
     })
     .subscribe( // desired data is subscribed
       scrapedData => callback({profiles: scrapedData}),
       error => callback(error)
     );
 }

 

Resources:

Posting Tweet from Loklak Wok Android

Loklak Wok Android is a peer harvester that posts collected tweets to the Loklak Server. Not only it is a peer harvester, but also provides users to post their tweets from the app. Images and location of the user can also be attached in the tweet. This blog explains

Adding Dependencies to the project

In app/build.gradle:

apply plugin: 'com.android.application'
apply plugin: 'me.tatarka.retrolambda'

android {
   ...
   packagingOptions {
       exclude 'META-INF/rxjava.properties'
   }
}

dependencies {
   ...
   compile 'com.google.code.gson:gson:2.8.1'

   compile 'com.squareup.retrofit2:retrofit:2.3.0'
   compile 'com.squareup.retrofit2:converter-gson:2.3.0'
   compile 'com.squareup.retrofit2:adapter-rxjava2:2.3.0'

   compile 'io.reactivex.rxjava2:rxjava:2.0.5'
   compile 'io.reactivex.rxjava2:rxandroid:2.0.1'
}

 

In build.gradle project level:

dependencies {
   classpath 'com.android.tools.build:gradle:2.3.3'
   classpath 'me.tatarka:gradle-retrolambda:3.2.0'
}

 

Implementation

User first authorize the application, so that they are able to post tweet from the app. For posting tweet statuses/update API endpoint of twitter is used and for attaching images with tweet media/upload API endpoint is used.

As, photos and location can be attached in a tweet, for Android Marshmallow and above we need to ask runtime permissions for camera, gallery and location. The related permissions are mentioned in Manifest file first

<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
// for location
<uses-feature android:name="android.hardware.location.gps"/>
<uses-feature android:name="android.hardware.location.network"/>

 

If, the device is using an OS below Android Marshmallow, there will be no runtime permissions, the user will be asked permissions at the time of installing the app.

Now, runtime permissions are asked, if the user had already granted the permission the related activity (camera, gallery or location) is started.

For camera permissions, onClickCameraButton is called

@OnClick(R.id.camera)
public void onClickCameraButton() {
   int permission = ContextCompat.checkSelfPermission(
           getActivity(), Manifest.permission.CAMERA);
   if (isAndroidMarshmallowAndAbove && permission != PackageManager.PERMISSION_GRANTED) {
       String[] permissions = {
               Manifest.permission.CAMERA,
               Manifest.permission.WRITE_EXTERNAL_STORAGE,
               Manifest.permission.READ_EXTERNAL_STORAGE
       };
       requestPermissions(permissions, CAMERA_PERMISSION);
   } else {
       startCameraActivity();
   }
}

 

To start the camera activity if the permission is already granted, startCameraActivity method is called

private void startCameraActivity() {
   Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
   File dir = getActivity().getExternalFilesDir(Environment.DIRECTORY_PICTURES);
   mCapturedPhotoFile = new File(dir, createFileName());
   Uri capturedPhotoUri = getImageFileUri(mCapturedPhotoFile);
   intent.putExtra(MediaStore.EXTRA_OUTPUT, capturedPhotoUri);
   startActivityForResult(intent, REQUEST_CAPTURE_PHOTO);
}

 

If the user decides to save the photo clicked from camera activity, the photo should be saved by creating a file and its uri is required to display the saved photo. The filename is created using createFileName method

private String createFileName() {
   String timeStamp = new SimpleDateFormat("ddMMyyyy_HHmmss").format(new Date());
   return "JPEG_" + timeStamp + ".jpg";
}

 

and uri is obtained using getImageFileUri

private Uri getImageFileUri(File file) {
   if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) {
       return Uri.fromFile(file);
   } else {
       return FileProvider.getUriForFile(getActivity(), "org.loklak.android.provider", file);
   }
}

 

Similarly, for the gallery, onClickGalleryButton method is implemented to ask runtime permissions and launch gallery activity if the permission is already granted.

@OnClick(R.id.gallery)
public void onClickGalleryButton() {
   int permission = ContextCompat.checkSelfPermission(
           getActivity(), Manifest.permission.WRITE_EXTERNAL_STORAGE);
   if (isAndroidMarshmallowAndAbove && permission != PackageManager.PERMISSION_GRANTED) {
       String[] permissions = {
               Manifest.permission.WRITE_EXTERNAL_STORAGE,
               Manifest.permission.READ_EXTERNAL_STORAGE
       };
       requestPermissions(permissions, GALLERY_PERMISSION);
   } else {
       startGalleryActivity();
   }
}

 

For starting the gallery activity, startGalleryActivity is used

private void startGalleryActivity() {
   Intent intent = new Intent();
   intent.setType("image/*");
   intent.setAction(Intent.ACTION_GET_CONTENT);
   intent.putExtra(Intent.EXTRA_ALLOW_MULTIPLE, true);
   startActivityForResult(
           Intent.createChooser(intent, "Select images"), REQUEST_GALLERY_MEDIA_SELECTION);
}

 

And finally for location onClickAddLocationButton is implemented

@OnClick(R.id.location)
public void onClickAddLocationButton() {
   int permission = ContextCompat.checkSelfPermission(
           getActivity(), Manifest.permission.ACCESS_FINE_LOCATION);
   if (isAndroidMarshmallowAndAbove && permission != PackageManager.PERMISSION_GRANTED) {
       String[] permissions = {Manifest.permission.ACCESS_FINE_LOCATION};
       requestPermissions(permissions, LOCATION_PERMISSION);
   } else {
       getLatitudeLongitude();
   }
}

 

If, the permission is already granted getLatitudeLongitude is called. Using LocationManager last known location is tried to obtain, if there is no last known location, current location is requested using a LocationListener.

private void getLatitudeLongitude() {
   mLocationManager =
           (LocationManager) getActivity().getSystemService(Context.LOCATION_SERVICE);

   // last known location from network provider
   Location location = mLocationManager.getLastKnownLocation(LocationManager.NETWORK_PROVIDER);
   if (location == null) { // last known location from gps
       location = mLocationManager.getLastKnownLocation(LocationManager.GPS_PROVIDER);
   }

   if (location != null) { // last known loaction available
       mLatitude = location.getLatitude();
       mLongitude = location.getLongitude();
       setLocation();
   } else { // last known location not available
       mLocationListener = new TweetLocationListener();
       // current location requested
       mLocationManager.requestLocationUpdates("gps", 1000, 1000, mLocationListener);
   }
}

 

TweetLocationListener implements a LocationListener that provides the current location. If GPS is disabled, settings is launched so that user can enable GPS. This is implemented in onProviderDisabled callback of the listener.

private class TweetLocationListener implements LocationListener {

   @Override
   public void onLocationChanged(Location location) {
       mLatitude = location.getLatitude();
       mLongitude = location.getLongitude();
       setLocation();
   }

   @Override
   public void onStatusChanged(String s, int i, Bundle bundle) {

   }

   @Override
   public void onProviderEnabled(String s) {

   }

   @Override
   public void onProviderDisabled(String s) {
       Intent intent = new Intent(Settings.ACTION_LOCATION_SOURCE_SETTINGS);
       startActivity(intent);
   }
}

 

If the user is asked for permissions, onRequestPermissionResult callback is invoked, if the permission is granted then the respective activities are opened or latitude and longitude are obtained.

@Override
public void onRequestPermissionsResult(
       int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {
   boolean isResultGranted = grantResults[0] == PackageManager.PERMISSION_GRANTED;
   switch (requestCode) {
       case CAMERA_PERMISSION:
           if (grantResults.length > 0 && isResultGranted) {
               startCameraActivity();
           }
           break;
       case GALLERY_PERMISSION:
           if (grantResults.length > 0 && isResultGranted) {
               startGalleryActivity();
           }
           break;
       case LOCATION_PERMISSION:
           if (grantResults.length > 0 && isResultGranted) {
               getLatitudeLongitude();
           }
   }
}

 

Since, the camera and gallery activities are started to obtain a result i.e. photo(s). So, onActivityResult callback is called

@Override
public void onActivityResult(int requestCode, int resultCode, Intent data) {
   switch (requestCode) {
       case REQUEST_CAPTURE_PHOTO:
           if (resultCode == Activity.RESULT_OK) {
               onSuccessfulCameraActivityResult();
           }
           break;
       case REQUEST_GALLERY_MEDIA_SELECTION:
           if (resultCode == Activity.RESULT_OK) {
               onSuccessfulGalleryActivityResult(data);
           }
           break;
       default:
           super.onActivityResult(requestCode, resultCode, data);
   }
}

 

If the result of Camera activity is success i.e. the image is saved by the user. The saved image is displayed in a RecyclerView in TweetPostingFragment. This is implemented in onSuccessfulCameraActivityResult mehtod

private void onSuccessfulCameraActivityResult() {
   tweetMultimediaContainer.setVisibility(View.VISIBLE);
   Bitmap bitmap = BitmapFactory.decodeFile(mCapturedPhotoFile.getAbsolutePath());
   mTweetMediaAdapter.clearAdapter();
   mTweetMediaAdapter.addBitmap(bitmap);
}

 

For a gallery activity, if a single image is selected then the uri of image can be obtained using getData method of an Intent. If multiple images are selected, the uri of images are stored in ClipData. After uris of images are obtained, it is checked if more than 4 images are selected as Twitter allows at most 4 images in a tweet. If more than 4 images are selected than the uris of extra images are removed. Using the uris of the images, the file is obtained and then from file Bitmap is obtained which is displayed in RecyclerView. This is implemented in onSuccessfulGalleryActivityResult

private void onSuccessfulGalleryActivityResult(Intent intent) {
   tweetMultimediaContainer.setVisibility(View.VISIBLE);
   Context context = getActivity();

   // get uris of selected images
   ClipData clipData = intent.getClipData();
   List<Uri> uris = new ArrayList<>();
   if (clipData != null) {
       for (int i = 0; i < clipData.getItemCount(); i++) {
           ClipData.Item item = clipData.getItemAt(i);
           uris.add(item.getUri());
       }
   } else {
       uris.add(intent.getData());
   }

   // remove of more than 4 images
   int numberOfSelectedImages = uris.size();
   if (numberOfSelectedImages > 4) {
       while (numberOfSelectedImages-- > 4) {
           uris.remove(numberOfSelectedImages);
       }
       Utility.displayToast(mToast, context, moreImagesMessage);
   }

   // get bitmap from uris of images
   List<Bitmap> bitmaps = new ArrayList<>();
   for (Uri uri : uris) {
       String filePath = FileUtils.getPath(context, uri);
       Bitmap bitmap = BitmapFactory.decodeFile(filePath);
       bitmaps.add(bitmap);
   }

   // display images in RecyclerView
   mTweetMediaAdapter.setBitmapList(bitmaps);
}

 

Now, to post images with tweet, first the ID of the image needs to be obtained using media/upload API endpoint, a multipart post request and then the obtained ID(s) is passed as the value of “media_ids” in statuses/update API endpoint. Since, there can be more than one image, a single observable is created for each image. The bitmap is converted to raw bytes for the multipart post request. As the process includes a network request and converting bitmap to bytes – a heavy resource consuming task which shouldn’t be on the main thread -, so an observable is created for the same as a result of which the tasks are performed concurrently i.e. in a separate thread.

private Observable<String> getImageId(Bitmap bitmap) {
   return Observable
           .defer(() -> {
               // convert bitmap to bytes
               ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
               bitmap.compress(Bitmap.CompressFormat.JPEG, 100, byteArrayOutputStream);
               byte[] bytes = byteArrayOutputStream.toByteArray();
               RequestBody mediaBinary = RequestBody.create(MultipartBody.FORM, bytes);
               return Observable.just(mediaBinary);
           })
           .flatMap(mediaBinary -> mTwitterMediaApi.getMediaId(mediaBinary, null))
           .flatMap(mediaUpload -> Observable.just(mediaUpload.getMediaIdString()))
           .subscribeOn(Schedulers.newThread());
}

 

The tweet is posted when the “Tweet” button is clicked by invoking onClickTweetPostButton mehtod

@OnClick(R.id.tweet_post_button)
public void onClickTweetPostButton() {
   String status = tweetPostEditText.getText().toString();

   List<Bitmap> bitmaps = mTweetMediaAdapter.getBitmapList();
   List<Observable<String>> mediaIdObservables = new ArrayList<>();
   for (Bitmap bitmap : bitmaps) { // observables for images is created
       mediaIdObservables.add(getImageId(bitmap));
   }

   if (mediaIdObservables.size() > 0) {
       // Post tweet with image
       postImageAndTextTweet(mediaIdObservables, status);
   } else if (status.length() > 0) {
       // Post text only tweet
       postTextOnlyTweet(status);
   } else {
       Utility.displayToast(mToast, getActivity(), tweetEmptyMessage);
   }
}

 

Tweet containing images are posted by calling postImageAndTextTweet, once the tweet data is obtained, the data is cross posted to loklak server. The image IDs are obtained concurrently by using the zip operator.

private void postImageAndTextTweet(List<Observable<String>> imageIdObservables, String status) {
   mProgressDialog.show();
   ConnectableObservable<StatusUpdate> observable = Observable.zip(
           imageIdObservables,
           mediaIdArray -> {
               String mediaIds = "";
               for (Object mediaId : mediaIdArray) {
                   mediaIds = mediaIds + String.valueOf(mediaId) + ",";
               }
               return mediaIds.substring(0, mediaIds.length() - 1);
           })
           .flatMap(imageIds -> mTwitterApi.postTweet(status, imageIds, mLatitude, mLongitude))
           .subscribeOn(Schedulers.io())
           .publish();

   Disposable postingDisposable = observable
           .subscribeOn(Schedulers.io())
           .observeOn(AndroidSchedulers.mainThread())
           .subscribe(this::onSuccessfulTweetPosting, this::onErrorTweetPosting);
   mCompositeDisposable.add(postingDisposable);

   // cross posting to loklak server   
   Disposable crossPostingDisposable = observable
           .flatMap(this::pushTweetToLoklak)
           .subscribeOn(Schedulers.io())
           .observeOn(AndroidSchedulers.mainThread())
           .subscribe(
                   push -> {},
                   t -> Log.e(LOG_TAG, "Cross posting failed: " + t.toString())
           );
   mCompositeDisposable.add(crossPostingDisposable);

   Disposable publishDisposable = observable.connect();
   mCompositeDisposable.add(publishDisposable);
}

 

In case of only text tweets, the text is obtained from editText and mediaIds are passed as null. And once the tweet data is obtained it is cross posted to loklak_server. This is executed by calling postTextOnlyTweet

private void postTextOnlyTweet(String status) {
   mProgressDialog.show();
   ConnectableObservable<StatusUpdate> observable =
           mTwitterApi.postTweet(status, null, mLatitude, mLongitude)
           .subscribeOn(Schedulers.io())
           .publish();

   Disposable postingDisposable = observable
           .subscribeOn(Schedulers.io())
           .observeOn(AndroidSchedulers.mainThread())
           .subscribe(this::onSuccessfulTweetPosting, this::onErrorTweetPosting);
   mCompositeDisposable.add(postingDisposable);


   // cross posting to loklak server
   Disposable crossPostingDisposable = observable
           .flatMap(this::pushTweetToLoklak)
           .subscribeOn(Schedulers.io())
           .observeOn(AndroidSchedulers.mainThread())
           .subscribe(
                   push -> Log.e(LOG_TAG, push.getStatus()),
                   t -> Log.e(LOG_TAG, "Cross posting failed: " + t.toString())
           );
   mCompositeDisposable.add(crossPostingDisposable);

   Disposable publishDisposable = observable.connect();
   mCompositeDisposable.add(publishDisposable);
}

 

Resources

Implementing 3 legged Authorization in Loklak Wok Android for Twitter

Loklak Wok Android is a peer harvester that posts collected tweets to the Loklak Server. Not only it is a peer harvester, but also provides users to post their tweets from the app. Posting tweets from the app requires users to authorize the Loklak Wok app, the client app created https://apps.twitter.com/ . This blog explains in detail about the authorization process.

Adding Dependencies to the project

In app/build.gradle:

apply plugin: 'com.android.application'
apply plugin: 'me.tatarka.retrolambda'

android {
   ...
   packagingOptions {
       exclude 'META-INF/rxjava.properties'
   }
}

dependencies {
   ...
   compile 'com.google.code.gson:gson:2.8.1'

   compile 'com.squareup.retrofit2:retrofit:2.3.0'
   compile 'com.squareup.retrofit2:converter-gson:2.3.0'
   compile 'com.squareup.retrofit2:adapter-rxjava2:2.3.0'

   compile 'io.reactivex.rxjava2:rxjava:2.0.5'
   compile 'io.reactivex.rxjava2:rxandroid:2.0.1'
}

 

In build.gradle project level:

dependencies {
   classpath 'com.android.tools.build:gradle:2.3.3'
   classpath 'me.tatarka:gradle-retrolambda:3.2.0'
}

 

Steps of Authorization

Step 1: Create client app in Twitter

Create a twitter client app at https://apps.twitter.com/. Provide the mandatory entries and also Callback url (would be used in next steps). Then go to “Keys and Access Token” and save your consumer key and consumer secret. In case you want to use Twitter API for yourself, click on “Create my access token”, which provides access token and access token secret.

Step 2: Obtaining a request token

Using the “consumer key” and “consumer secret” request token is obtained by sending a POST request to oauth/request_token. As Twitter API are Oauth1 based the sent request needs to be signed by generating oauth_signature. The oauth_signature is generated by intercepting the network request sent by retrofit rest API client, the oauth interceptor used in Loklak Wok Android is a modified version of this snippet. The retrofit TwitterAPI interface is defined

public interface TwitterAPI {

   String BASE_URL = "https://api.twitter.com/";

   @POST("/oauth/request_token")
   Observable<ResponseBody> getRequestToken();

   @FormUrlEncoded
   @POST("/oauth/access_token")
   Observable<ResponseBody> getAccessTokenAndSecret(@Field("oauth_verifier") String oauthVerifier);
}

 

And the retrofit REST client is implemented in TwitterRestClient. createTwitterAPIWithoutAccessToken method returns a twitter API client which can be called without providing access keys, this is used as we don’t have access tokens right now.

public static TwitterAPI createTwitterAPIWithoutAccessToken() {
   if (sWithoutAccessTokenRetrofit == null) {
       sLoggingInterceptor.setLevel(HttpLoggingInterceptor.Level.BODY);
       // uncomment to debug network requests
       // sWithoutAccessTokenClient.addInterceptor(sLoggingInterceptor);
       sWithoutAccessTokenRetrofit = sRetrofitBuilder
               .client(sWithoutAccessTokenClient.build()).build();
   }
   return sWithoutAccessTokenRetrofit.create(TwitterAPI.class);
}

 

So, getRequestToken method is used to obtain the request token, if the request is successful oauth_token is returned.

@OnClick(R.id.twitter_authorize)
public void onClickTwitterAuthorizeButton(View view) {
   mTwitterApi.getRequestToken()
           .subscribeOn(Schedulers.io())
           .observeOn(AndroidSchedulers.mainThread())
           .subscribe(this::parseRequestTokenResponse, this::onFetchRequestTokenError);
}

 

Step 3: Redirecting the user

Using the oauth_token obtained in Step 2, the user is redirected to login page using WebView.

private void setAuthorizationView() {
   ...
   webView.setVisibility(View.VISIBLE);
   webView.loadUrl(mAuthorizationUrl);
}

 

A WebView client is created by extending WebViewClient, this is used to keep track of which webpage is opened by overriding shouldOverrideUrlLoading.

@Override
public boolean shouldOverrideUrlLoading(WebView view, String url) {
   if (url.contains("github")) {
       String[] tokenAndVerifier = url.split("&");
       mOAuthVerifier = tokenAndVerifier[1].substring(tokenAndVerifier[1].indexOf('=') + 1);
       getAccessTokenAndSecret();
       return true;
   }
   return false;
}

 

As the link provided in callback url while creating our twitter app is a github page. The WebViewClient checks if it is a github page or not. If yes, then it parses the oauth_verifier from the github url.

Step 4: Converting the request token to an access token

A new rest client is created using the access token obtained in step 2, as implemented in createTwitterAPIWithAccessToken method.

public static TwitterAPI createTwitterAPIWithAccessToken(String token) {
   TwitterOAuthInterceptor withAccessTokenInterceptor =
           sInterceptorBuilder.accessToken(token).accessSecret("").build();
   OkHttpClient withAccessTokenClient = new OkHttpClient.Builder()
           .addInterceptor(withAccessTokenInterceptor)
           //.addInterceptor(loggingInterceptor) // uncomment to debug network requests
           .build();
   Retrofit withAccessTokenRetrofit = sRetrofitBuilder.client(withAccessTokenClient).build();
   return withAccessTokenRetrofit.create(TwitterAPI.class);
}

 

Now, to obtain access token and access token secret oauth_verifier obtained in step 3 is passed as a parameter to getAccessTokenAndSecret method defined in TwitterAPI interface which calls oauth/access_token endpoint from the rest client created above. This is implemented in getAccessTokenAndSecret method of WebViewClient class

private void getAccessTokenAndSecret() {
   mTwitterApi = TwitterRestClient.createTwitterAPIWithAccessToken(mOauthToken);
   mTwitterApi.getAccessTokenAndSecret(mOAuthVerifier)
           .flatMap(this::saveAccessTokenAndSecret)
           ....
}

 

Finally the obtained access_token and access_token_secret is saved in SharedPreference so that it can be used to call other Twitter API endpoints as in saveAccessTokenAndSecret

private Observable<Integer> saveAccessTokenAndSecret(ResponseBody responseBody)
       throws IOException {
   String[] responseValues = responseBody.string().split("&");

   String token = responseValues[0].substring(responseValues[0].indexOf("=") + 1);
   SharedPrefUtil.setSharedPrefString(getActivity(), OAUTH_ACCESS_TOKEN_KEY, token);
   mOauthToken = token; // here access_token that would be used for API calls

   String tokenSecret = responseValues[1].substring(responseValues[1].indexOf("=") + 1);
   SharedPrefUtil.setSharedPrefString(
           getActivity(), OAUTH_ACCESS_TOKEN_SECRET_KEY, tokenSecret);
   mOauthTokenSecret = tokenSecret;
   return Observable.just(1);
}

 

Resources:

Implementing Tweet Search feature in Loklak Wok Android

Loklak Wok Android is a peer harvester that posts collected tweets to the Loklak Server. Along with that tweets can be searched using the app. This post describes how search API endpoint and TabLayout is used to implement the tweet searching feature.

Adding Dependencies to the project

This feature uses Retrofit2, Reactive extensions(RxJava2, RxAndroid and Retrofit RxJava adapter) and RetroLambda (for Java lambda support in Android).

In app/build.gradle:

apply plugin: 'com.android.application'
apply plugin: 'me.tatarka.retrolambda'

android {
   ...
   packagingOptions {
       exclude 'META-INF/rxjava.properties'
   }
}

dependencies {
   ...
   compile 'com.google.code.gson:gson:2.8.1'

   compile 'com.squareup.retrofit2:retrofit:2.3.0'
   compile 'com.squareup.retrofit2:converter-gson:2.3.0'
   compile 'com.squareup.retrofit2:adapter-rxjava2:2.3.0'

   compile 'io.reactivex.rxjava2:rxjava:2.0.5'
   compile 'io.reactivex.rxjava2:rxandroid:2.0.1'
}

 

In build.gradle project level:

dependencies {
   classpath 'com.android.tools.build:gradle:2.3.3'
   classpath 'me.tatarka:gradle-retrolambda:3.2.0'
}

 

Implementation

The search API endpoint is defined in LoklakApi interface which would provide the tweet search result.

public interface LoklakApi {

   @GET("api/search.json")
   Observable<Search> getSearchedTweets(
           @Query("q") String query,
           @Query("filter") String filter,
           @Query("count") int count);
}

 

The POJOs (Plain Old Java Objects) for the result of search API endpoint are obtained using jsonschema2pojo, Gson uses POJOs to convert JSON to Java objects and vice-versa.

The REST client is created by Retrofit2 and is implemented in RestClient class. The Gson converter and RxJava adapter for retrofit is added in the retrofit builder. create method is called to generate the API methods(retrofit implements LoklakApi Interface).

public class RestClient {

   private RestClient() {
   }

   private static void createRestClient() {
       sRetrofit = new Retrofit.Builder()
               .baseUrl(BASE_URL)
               // gson converter
               .addConverterFactory(GsonConverterFactory.create(gson))
               // retrofit adapter for rxjava
               .addCallAdapterFactory(RxJava2CallAdapterFactory.create())
               .build();
   }

   private static Retrofit getRetrofitInstance() {
       if (sRetrofit == null) {
           createRestClient();
       }
       return sRetrofit;
   }

   public static <T> T createApi(Class<T> apiInterface) {
       // create method to generate API methods
       return getRetrofitInstance().create(apiInterface);
   }

}

 

As search API endpoint provides filter parameter which can be used to filter out tweets containing images and videos. So, the tweets are displayed in three categories i.e. latest, images and videos.

The tweets of different category are displayed using a ViewPager. The fragments in ViewPager are inflated by a class that extends FragmentPagerAdapter. SearchFragmentPagerAdapter extends FragmentPagerAdapter, at least two methods getItem and getCount needs to be overridden. Going by the name of methods, getItem provides ith fragment to the  ViewPager and based on the value returned by getCount number of tabs are inflated in TabLayout, a ViewGroup to display fragments in ViewPager in an elegant way. For better UI, the names (here the category of tweets) are displayed, for which we override getPageTitle method.

public class SearchFragmentPagerAdapter extends FragmentPagerAdapter {

   private List<Fragment>  mFragmentList = new ArrayList<>();
   private List<String> mFragmentNameList = new ArrayList<>();

   public SearchFragmentPagerAdapter(FragmentManager fm) {
       super(fm);
   }

   @Override
   public Fragment getItem(int position) {
       return mFragmentList.get(position);
   }

   @Override
   public int getCount() {
       return mFragmentList.size();
   }

   @Override
   public CharSequence getPageTitle(int position) {
       return mFragmentNameList.get(position);
   }

   public void addFragment(Fragment fragment, String pageTitle) {
       mFragmentList.add(fragment);
       mFragmentNameList.add(pageTitle);
   }
}

 

For easy understanding an analogy with RecyclerView can be made. The TabLayout here functions as a RecyclerView, ViewPager does the work of LayoutManager and FragmentPagerAdapter is analogous to RecyclerView.Adapter.

Now, the fragments which contain the categorical tweets are inflated in the parent fragment. Firstly, the ViewPager of TabLayout is set. Then fragments and their names are added to the FragmentPagerAdapter using the addFragment method implemented in SearchFragmentAdapter class above and finally the created adapter is set as the adapter of ViewPager, which is implemented in setupWithViewPager method.

@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container,
                        Bundle savedInstanceState) {
   // Inflate the layout for this fragment
   View rootView = inflater.inflate(R.layout.fragment_search, container, false);
   ButterKnife.bind(this, rootView);
   ...
   tabLayout.setupWithViewPager(viewPager);
   setupViewPager(viewPager);

   return rootView;
}

private void setupViewPager(ViewPager viewPager) {
   FragmentManager fragmentManager = getActivity().getSupportFragmentManager();
   SearchFragmentPagerAdapter pagerAdapter = new SearchFragmentPagerAdapter(fragmentManager);
   pagerAdapter.addFragment(SearchCategoryFragment.newInstance("", mQuery), "LATEST");
   pagerAdapter.addFragment(SearchCategoryFragment.newInstance("image", mQuery), "PHOTOS");
   pagerAdapter.addFragment(SearchCategoryFragment.newInstance("video", mQuery), "VIDEOS");
   viewPager.setAdapter(pagerAdapter);
}

 

SearchCategoryFragment are child fragments displayed as tabs in TabLayout. These child fragments are created using newInstance method which takes two parameters, category of tweets and the tweet search query respectively, the reason a constructor with these parameters are not used is that during a orientation change only the default constructor i.e. with no parameters is restored by Android system. So, these parameters are stored in a data structure called Bundle, once the fragment object is created using the default parameter the arguments present in the bundle are passed to fragment using setArguments method. These parameter are retrieved in onCreate lifecycle callback method of fragment which are used to fetch search results.

public static SearchCategoryFragment newInstance(String category, String query) {
   Bundle args = new Bundle();
   // query and category stored in bundle
   args.putString(Constants.TWEET_SEARCH_SUGGESTION_QUERY_KEY, query);
   args.putString(TWEET_SEARCH_CATEGORY_KEY, category);
   // fragment with default constructor created
   SearchCategoryFragment fragment = new SearchCategoryFragment();
   // arguments in bundle are passed to fragment
   fragment.setArguments(args);
   return fragment;
}

@Override
public void onCreate(@Nullable Bundle savedInstanceState) {
   super.onCreate(savedInstanceState);
   Bundle bundle = getArguments();
   if (bundle != null) {
       // arguments retrieved
       mTweetSearchCategory = bundle.getString(TWEET_SEARCH_CATEGORY_KEY);
       mSearchQuery = bundle.getString(Constants.TWEET_SEARCH_SUGGESTION_QUERY_KEY);
   }
}

 

As we have search query and category we can now obtain the search result and pass the obtained result – a List of type Status – to the adapter of RecyclerView which shows the tweets beautifully inside a CardView. The adapter and LayoutManager of RecyclerView are instantiated and set in onCreateView lifecycle callback method. Finally, network request is sent by calling fetchSearchedTweets method to obtain the search results.

@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container,
                        Bundle savedInstanceState) {
   // Inflate the layout for this fragment
   View view = inflater.inflate(R.layout.fragment_search_category, container, false);
   ButterKnife.bind(this, view);

   mSearchCategoryAdapter = new SearchCategoryAdapter(getActivity(), new ArrayList<>());
   recyclerView.setLayoutManager(new LinearLayoutManager(getActivity()));
   recyclerView.setAdapter(mSearchCategoryAdapter);

   // request sent to obtain search result.
   fetchSearchedTweets();

   return view;
}

 

The LoklakApi interface is implemented using the created Rest client and then getSearchedTweets method is invoked which takes in search query, category of tweets and maximum number of results in the mentioned order. If the network request is successful then setSearchResultView is invoked else setNetworkErrorView.

private void fetchSearchedTweets() {
   LoklakApi loklakApi = RestClient.createApi(LoklakApi.class);
   loklakApi.getSearchedTweets(mSearchQuery, mTweetSearchCategory, 30)
           .subscribeOn(Schedulers.io()) // network request sent in a background thread
           .observeOn(AndroidSchedulers.mainThread())
           .subscribe(this::setSearchResultView, this::setNetworkErrorView);
}

 

setSearchResultView displays the obtained result if any in RecyclerView else shows a message that there is no result for the search query.

private void setSearchResultView(Search search) {
   List<Status> statusList = search.getStatuses();
   networkErrorTextView.setVisibility(View.GONE);
   if (statusList.size() == 0) { // request successful but no results
       recyclerView.setVisibility(View.GONE);

       Resources res = getResources();
       String noSearchResultMessage = res.getString(R.string.no_search_match, mSearchQuery); // no result matched message
       // no result message displayed
       noSearchResultFoundTextView.setVisibility(View.VISIBLE);
       noSearchResultFoundTextView.setText(noSearchResultMessage);
   } else { // there are some results, so display them in RecyclerView
       recyclerView.setVisibility(View.VISIBLE);
       mSearchCategoryAdapter.setStatuses(statusList);
   }
}

 

In case of a failed network request, a TextView is displayed asking the user to check the network connections and click on it to retry.

private void setNetworkErrorView(Throwable throwable) {
   Log.e(LOG_TAG, throwable.toString());
   recyclerView.setVisibility(View.GONE);
   networkErrorTextView.setVisibility(View.VISIBLE);
}

 

When the TextView, networkErrorTextView, is clicked a network request is sent again and the visibility of networkErrorTextView is changed to GONE, as implemented in setOnClickNetworkErrorTextViewListner

@OnClick(R.id.network_error)
public void setOnClickNetworkErrorTextViewListener() {
   networkErrorTextView.setVisibility(View.GONE);
   fetchSearchedTweets();
}

 

References:

Resources

Posting Scraped Tweets to Loklak server from Loklak Wok Android

Loklak Wok Android is a peer harvester that posts collected  messages to the Loklak Server. The suggestions to search tweets are fetched using suggest API endpoint. Using the suggestion queries, tweets are scraped. The scraped tweets are shown in a RecyclerView and simultaneously they are posted to loklak server using push API endpoint. Let’s see how this is implemented.

Adding Dependencies to the project

This feature heavily uses Retrofit2, Reactive extensions(RxJava2, RxAndroid and Retrofit RxJava adapter) and RetroLambda (for Java lambda support in Android).

In app/build.gradle:

apply plugin: 'com.android.application'
apply plugin: 'me.tatarka.retrolambda'

android {
   ...
   packagingOptions {
       exclude 'META-INF/rxjava.properties'
   }
}

dependencies {
   ...
   compile 'com.google.code.gson:gson:2.8.1'

   compile 'com.squareup.retrofit2:retrofit:2.3.0'
   compile 'com.squareup.retrofit2:converter-gson:2.3.0'
   compile 'com.squareup.retrofit2:adapter-rxjava2:2.3.0'

   compile 'io.reactivex.rxjava2:rxjava:2.0.5'
   compile 'io.reactivex.rxjava2:rxandroid:2.0.1'
}

 

In build.gradle project level:

dependencies {
   classpath 'com.android.tools.build:gradle:2.3.3'
   classpath 'me.tatarka:gradle-retrolambda:3.2.0'
}

 

Implementation

The suggest and push API endpoint is defined in LoklakApi interface

public interface LoklakApi {

   @GET("/api/suggest.json")
   Observable<SuggestData> getSuggestions(@Query("q") String query, @Query("count") int count);

   @POST("/api/push.json")
   @FormUrlEncoded
   Observable<Push> pushTweetsToLoklak(@Field("data") String data);
}

 

The POJOs (Plain Old Java Objects) for suggestions and posting tweets are obtained using jsonschema2pojo, Gson uses POJOs to convert JSON to Java objects.

The REST client is created by Retrofit2 and is implemented in RestClient class. The Gson converter and RxJava adapter for retrofit is added in the retrofit builder. create method is called to generate the API methods(retrofit implements LoklakApi Interface).

public class RestClient {

   private RestClient() {
   }

   private static void createRestClient() {
       sRetrofit = new Retrofit.Builder()
               .baseUrl(BASE_URL)
               // gson converter
               .addConverterFactory(GsonConverterFactory.create(gson))
               // retrofit adapter for rxjava
               .addCallAdapterFactory(RxJava2CallAdapterFactory.create())
               .build();
   }

   private static Retrofit getRetrofitInstance() {
       if (sRetrofit == null) {
           createRestClient();
       }
       return sRetrofit;
   }

   public static <T> T createApi(Class<T> apiInterface) {
       // create method to generate API methods
       return getRetrofitInstance().create(apiInterface);
   }

}

 

The suggestions are fetched by calling getSuggestions after LoklakApi interface is implemented. getSuggestions returns an Observable of type SuggestData, which contains the suggestions in a List. For scraping tweets only a single query needs to be passed to LiquidCore, so flatmap is used to transform the observabe and then fromIterable operator is used to emit single queries as string to LiquidCore which then scrapes tweets, as implemented in fetchSuggestions

private Observable<String> fetchSuggestions() {
   LoklakApi loklakApi = RestClient.createApi(LoklakApi.class);
   Observable<SuggestData> observable = loklakApi.getSuggestions("", 2);
   return observable.flatMap(suggestData -> {
       List<Query> queryList = suggestData.getQueries();
       List<String> queries = new ArrayList<>();
       for (Query query : queryList) {
           queries.add(query.getQuery());
       }
       return Observable.fromIterable(queries);
   });
}

 

As LiquidCore uses callbacks to create a connection between NodeJS instance and Android, to maintain a flow of observables a custom observable is created using create operator which encapsulates the callbacks inside it. For a detail understanding of how LiquidCore event handling works, please go through the example. The way it is implemented in getScrapedTweets:

private Observable<ScrapedData> getScrapedTweets(final String query) {
   final String LC_TWITTER_URI = "android.resource://org.loklak.android.wok/raw/twitter";
   URI uri = URI.create(LC_TWITTER_URI);

   return Observable.create(emitter -> { // custom observable creation
       EventListener startEventListener = (service, event, payload) -> {
               service.emit(LC_QUERY_EVENT, query);
           service.emit(LC_FETCH_TWEETS_EVENT);
       };

       EventListener getTweetsEventListener = (service, event, payload) -> {
           ScrapedData scrapedData = mGson.fromJson(payload.toString(), ScrapedData.class);
           emitter.onNext(scrapedData); // data emitted using ObservableEmitter
       };

       MicroService.ServiceStartListener serviceStartListener = (service -> {
           service.addEventListener(LC_START_EVENT, startEventListener);
           service.addEventListener(LC_GET_TWEETS_EVENT, getTweetsEventListener);
       });

       MicroService microService = new MicroService(getActivity(), uri, serviceStartListener);
       microService.start();
   });
}

 

Now that we are getting suggestions and using them to get scraped tweets, this needs to be done periodically, so that tweets are pushed continuously to the loklak server. For this interval operator is used. A List is maintained which contains the suggestion queries based on which tweets are to be scraped. Once the scraping is done, the suggestion query is removed from the list when they are displayed in RecyclerView. And if the list is empty, then only a new set of suggestions are fetched.

Observable.interval(4, TimeUnit.SECONDS)
       .flatMap(this::getSuggestionsPeriodically)
       .flatMap(query -> {
           mSuggestionQuerries.add(query); // query added to list
           return getScrapedTweets(query);
       });

 

Method reference is used to maintain the modularity, so the logic of periodically fetching suggestions is implemented in getSuggestionsPeriodically

private Observable<String> getSuggestionsPeriodically(Long time) {
   if (mSuggestionQuerries.isEmpty()) { // checks if list is empty
       mInnerCounter = 0;
       return fetchSuggestions(); // new suggestions
   } else { // wait for a previous request to complete
       mInnerCounter++;
       if (mInnerCounter > 3) { // if some strange error occurs
           mSuggestionQuerries.clear();
       }
       return Observable.never(); // no observable is passed to subsriber, subscriber waits
   }
}

 

Now, it’s time to display the fetched tweets and then push the tweets to loklak server. When periodic fetching of suggestions was implemented we used interval operator and then flatMap to transform observables i.e. chaining network requests.

Till this point the observable we were creating were Cold Observable.Cold observables only emit values when a subscription is made. As we need to display scraped tweets and then push it, i.e. one source of observables and two (multiple) subscribers. By intuition the observable should be subscribed two times, for example:

Observable observable = Observable.interval(4, TimeUnit.SECONDS)
       .flatMap(this::getSuggestionsPeriodically)
       .flatMap(query -> {
           mSuggestionQuerries.add(query);
           return getScrapedTweets(query);
       });

// first time subscription
observable
       .subscribeOn(Schedulers.io())
       .observeOn(AndroidSchedulers.mainThread())
       .subscribe(
	// display in RecyclerVIew
       );


// second time subscription
observable
       .flatMap(// trnasformations to push data to server)
       .subscribeOn(Schedulers.io())
       .observeOn(AndroidSchedulers.mainThread())
       .subscribe(
	// display in number of tweets pushed
       );

 

But the source observable is cold observable i.e. it emits objects when it is subscribed to, due to which there will be two different network calls, one for first subscription and one for second subscription. So, both the subscriptions will have different data, which is not what is desired. The expected result is that there should be a single network call, and the data obtained from that call should be displayed and pushed to loklak server.

For this, hot Observables are used. Hot observables start emitting objects the moment they are created, irrespective of whether they are subscribed or not.

A cold observable can be converted to a hot observable by using publish operator and it starts emitting objects when connect operator is used. This is implemented in displayAndPostScrapedData:

ConnectableObservable<ScrapedData> observable = Observable.interval(4, TimeUnit.SECONDS)
           .flatMap(this::getSuggestionsPeriodically)
           .flatMap(query -> {
               mSuggestionQuerries.add(query);
               return getScrapedTweets(query);
           })
           .retry(2)
           .publish();

   // first time subscription to display scraped data
   Disposable viewDisposable = observable
           .subscribeOn(Schedulers.io())
           .observeOn(AndroidSchedulers.mainThread())
           .subscribe(
                   this::displayScrapedData,
                   this::setNetworkErrorView
           );
   mCompositeDisposable.add(viewDisposable);
  
   // second time subscription for pushing data to loklak
   Disposable pushDisposable = observable
           .flatMap(this::pushScrapedData) // scraped data transformed for pushing
           .subscribeOn(Schedulers.io())
           .observeOn(AndroidSchedulers.mainThread())
           .subscribe(
                   push -> {
                       mHarvestedTweets += push.getRecords();
                       harvestedTweetsCountTextView.setText(String.valueOf(mHarvestedTweets));
                   },
                   throwable -> {}
           );
   mCompositeDisposable.add(pushDisposable);

   Disposable publishDisposable = observable.connect(); // hot observable starts emitting
   mCompositeDisposable.add(publishDisposable);
}

 

The two subscriptions are made before connect operator is invoked because the hot observable emits objects due to successful network calls and network calls can’t be done on MainThread (UI Thread). So, doing the subscription before, channels the network calls to a background thread.

The scraped data is converted to JSON from objects using Gson, the JSON is converted to string and then using push API endpoint it is posted to loklak server. This is implemented in pushScrapedData method, which is used in second subscription by using method referencing.

private Observalbe<Push> pushScrapedData(ScrapedData scrapedData) throws Exception{    
    LoklakApi loklakApi = RestClient.createApi(LoklakApi.class);
    List<Status> statuses = scrapedData.getStatuses();
    String data = mGson.toJson(statuses);
    JSONArray jsonArray = new JSONArray(data);
    JSONObject jsonObject = new JSONObject();
    jsonObject.put("statuses", jsonArray);
    return loklakApi.pushTweetsToLoklak(jsonObject.toString());
}

 

Method reference for displayScrapedData and setNetworkErrorView methods are used to display the scraped data and handle unsuccessful network requests.

Only 80 tweets are preserved in RecyclerView. If number of tweets exceeds 80, then old tweets are removed.

private void displayScrapedData(ScrapedData scrapedData) {
   String query = scrapedData.getQuery();
   List<Status> statuses = scrapedData.getStatuses();
   mSuggestionQuerries.remove(query);
   if (mHarvestedTweetAdapter.getItemCount() > 80) {
       mHarvestedTweetAdapter.clearAdapter(); // old tweets removed
   }
   mHarvestedTweetAdapter.addHarvestedTweets(statuses);
   int count = mHarvestedTweetAdapter.getItemCount() - 1;
   recyclerView.scrollToPosition(count);
}

 

In case of a network error, the visibility of RecyclerView and TextView (which shows number of tweets pushed) is changed to gone and a message is displayed that there is network error.

private void setNetworkErrorView(Throwable throwable) {
   Log.e(LOG_TAG, throwable.toString());
   // recyclerView and TextView showing count of harvested tweets are hidden
   ButterKnife.apply(networkViews, GONE);
   // network error message displayed
   networkErrorTextView.setVisibility(View.VISIBLE);
}

References

Resources

Realm database in Loklak Wok Android for Persistent view

Loklak Wok Android provides suggestions for tweet searches. The suggestions are stored in local database to provide a persistent view, resulting in a better user experience. The local database used here is Realm database instead of sqlite3 which is supported by Android SDK. The proper way to use an sqlite3 database is to first create a contract where the schema of the database is defined, then a database helper class which extends from SQLiteOpenHelper class where the schema is created i.e. tables are created and finally write ContentProvider so that you don’t have to write long SQL queries every time a database operation needs to be performed. This is just a lot of hard work to do, as this includes a lot of steps, debugging is also difficult. A solution to this can be using an ORM that provides a simple API to use sqlite3, but the currently available ORMs lack in terms of performance, they are too slow. A reliable solution to this problem is realm database, which is faster than raw sqlite3 and has really simple API for database operations. This blog explains the use of realm database for storing tweet search suggestions.

Adding Realm database to Android project

In project level build.gradle

buildscript {
   repositories {
       jcenter()
   }
   dependencies {
       classpath 'com.android.tools.build:gradle:2.3.3'
       classpath "io.realm:realm-gradle-plugin:3.3.1"

       // NOTE: Do not place your application dependencies here; they belong
       // in the individual module build.gradle files
   }
}

 

And at the top of app/build.gradle “apply plugin: ‘realm-android'”  is added.

Using Realm Database

Let’s start with a simple example. We have a Student class that has only two attributes name and age. To create the model for the database, the Student class is simply extended to RealmObject.

public class Student extends RealmObject {

   private String name;
   private int age;

   // A constructor needs to be explicitly defined, be it an empty constructor
   public Student(String name, int age) {
       this.name = name;
       this.age = age;
   }

   // getters and setters
}

 

To push data to the database, Java objects are created, a transaction is initialized, then copyToRealm method is used to push the data and finally the transaction is committed. But before all this, the database is initialized and a Realm instance is obtained.

Realm.init(context); // Database initialized
Realm realm = Realm.getDefaultInstance(); // realm instance obtained
      
Student student = new Student("Rahul Dravid", 22); // Simple java object created
realm.beginTransaction() // initialization of transaction
realm.copyToRealm(student); // pushed to database
realm.commitTransaction(); // transaction committed

 

copyToRealm takes only a single parameter, the parameter can be an object or an Iterable. Off course, the passed parameter should extend RealmObject. A List of Student can be passed as a parameter to copyToRealm to push multiple data into the database.

The above way of inserting data is synchronous. Realm also supports asynchronous transactions, you guessed it right, you don’t have to depend on AsyncTaskLoader. The same operation can be performed asynchronously as

realm.executeTransaction(new Realm.Transaction() { 
    @Override public void execute(Realm realm) {
        Student student = new Student("Rahul Dravid", 22);
        realm.copyToRealm(student);
    }
});

 

Now, querying the database is as easy as inserting.

RealmResults<Student> studentList = realm.where(Student.class).findAll();

 

No, transaction is required as we are not manipulating the database, as data is just read from the database. RealmResults extend Java List, so List methods which don’t manipulate the List can be used on studentList e.g. get(int index) to obtain object at the index.

The result can also be a filtered one, for example filtering students who are 22 years old.

RealmResults<Student> studentList = realm.where(Student.class).equalTo(age, 22).findAll();

 

Now, removing data from database. deleteAllFromRealm method can be executed on the obtained RealmResults or to completely remove data of a model class delete(Model.class) method on the realm instance is invoked. The operations should be enclosed between beginTransaction and commitTransaction if synchronous behaviour is required else for asynchronous behaviour the operation is done in execute method of an anonymous object of Realm.Transaction.

studentList.deleteAllFromRealm(); // removes the filtered result
realm.delete(Student.class); // removes all data of model class

 

Storing Tweet Search Suggestions in Loklak Wok Android for Persistent view

Loklak Wok Android uses Retrofit2 for sending network requests, for which POJO classes are already created so that it becomes easy for parsing the obtained JSON from network request. Due to this using Realm database becomes more easier, as the defined POJOs can be simply extended to RealmObject to create the model class of the data e.g. Query class extends RealmObject, one of the attribute is the suggestion query i.e. mQuery.

The database is initialized in LoklakWokApplication, the application class, this way the database is initialized only once which persists throughout the app lifecycle.

@Override
public void onCreate() {
   super.onCreate();
   Realm.init(this);
   RealmConfiguration realmConfiguration = new RealmConfiguration.Builder()
           .name(Realm.DEFAULT_REALM_NAME)
           .deleteRealmIfMigrationNeeded()
           .build();
   Realm.setDefaultConfiguration(realmConfiguration);
}

 

deleteRealmIfMigrationNeeded removes the old realm database and creates a new one if any of the Model class get changed i.e. an attribute is removed, added or simply the name of attribute is changed. This is done as we are not storing user generated data, we are just using database to provide persistent view. So, the previously kept data is not important.

The database is closed in onTerminate callback of the application

@Override
public void onTerminate() {
   Realm.getDefaultInstance().close();
   super.onTerminate();
}

 

Now that database is initialized. We fetch the previously stored data and display it in RecyclerView. If the network request is successful the queries from database are replaced by the queries fetched in network request, else the queries from database are displayed providing a persistent view. The way it is implemented in onCreateView of SuggestFragment

mRealm = Realm.getDefaultInstance();
...
// old queries obtained from database
RealmResults<Query> queryRealmResults = mRealm.where(Query.class).findAll();
List<Query> queries = mRealm.copyFromRealm(queryRealmResults);
// RecyclerView adapter created with old queries
mSuggestAdapter = new SuggestAdapter(queries, this);
tweetSearchSuggestions.setLayoutManager(new LinearLayoutManager(getActivity()));
tweetSearchSuggestions.setAdapter(mSuggestAdapter);

 

The old queries are replaced in onSuccessfulRequest

private void onSuccessfulRequest(SuggestData suggestData) {
   // suggestData contains suggestion queries
   if (suggestData != null) {
       // old queries replaced with new ones
       mSuggestAdapter.setQueries(suggestData.getQueries());
   }
   setAfterRefreshingState();
}

 

Now suggestion queries needs to be inserted into the database. Only the latest suggestions are inserted i.e. queries present when onStop lifecycle method of fragment is called, and as previous queries are not needed anymore, they are deleted. The operation is performed in a synchronous way.

@Override
public void onStop() {
   ...
   mRealm.beginTransaction();
   // old queries deleted
   mRealm.delete(Query.class);
   // new queries inserted
   mRealm.copyToRealm(mSuggestAdapter.getQueries());
   mRealm.commitTransaction();
   // fragment lifecycle called i.e. a new fragment/activity opens
   super.onStop();
}

 

Conclusion: Sqlite3 and Realm comparison

Operations Sqlite3 Realm
Table creation CREATE TABLE … extends RealmObject
Inserting data INSERT INTO … copyToRealm
Searching data SELECT … realm.where(Model.class)
Deleting data DELETE FROM … realmResults.deleteAllFromRealm() or realm.delete(Model.class)

Resources: