Implementation of Pagination in Open Event Organizer Android App

Pagination (Endless Scrolling or Infinite Scrolling) breaks down a list of content into smaller parts, loaded one at a time. It is important when the quantity of data to be loaded is huge and loading all the data at once can result in timeout.

Here, we will discuss about the implementation of pagination in the list of attendees in the Open Event Organizer App (Eventyay Organizer App).

It is an Android app used by event organizers to create and manage events on the Eventyay platform. Features include event creation, ticket management, attendee list with ticket details, scanning of participants etc.

In the Open Event Organizer App, the loading of attendees would result in timeout when the number of attendees would be large. The solution for fixing this was the implementation of pagination in the Attendees fragment.

First, the API call needs to be modified to include the page size as well as the addition of page number as a Query.

@GET("events/{id}/attendees?include=order,ticket,event&fields[event]=id&fields[ticket]=id&page[size]=20")
Observable<List<Attendee>> getAttendeesPageWise(@Path("id") long id, @Query("page[number]") long pageNumber);

Now, we need to modify the logic of fetching the list of attendees to include the page number. Whenever one page ends, the next page should be fetched automatically and added to the list.

The page number needs to be passed as an argument in the loadAttendeesPageWise() method in AttendeesViewModel.

public void loadAttendeesPageWise(long pageNumber, boolean forceReload) {

    showScanButtonLiveData.setValue(false);

    compositeDisposable.add(
        getAttendeeSourcePageWise(pageNumber, forceReload)
            .doOnSubscribe(disposable -> progress.setValue(true))
            .doFinally(() -> progress.setValue(false))
            .toSortedList()
            .subscribe(attendees -> {
                attendeeList.addAll(attendees);
                attendeesLiveData.setValue(attendees);
                showScanButtonLiveData.setValue(!attendeeList.isEmpty());
            }, throwable -> error.setValue(ErrorUtils.getMessage(throwable).toString())));
}

Also in the getAttendeeSourcePageWise() method:

private Observable<Attendee> getAttendeeSourcePageWise(long pageNumber, boolean forceReload) {
    if (!forceReload && !attendeeList.isEmpty())
        return Observable.fromIterable(attendeeList);
    else
        return attendeeRepository.getAttendeesPageWise(eventId, pageNumber, forceReload);
}

Now, in the AttendeesFragment, a check is needed to increase the current page number and load attendees for the next page when the user reaches the end of the list. 

if (!recyclerView.canScrollVertically(1)) {

    if (recyclerView.getAdapter().getItemCount() > currentPage * ITEMS_PER_PAGE) {
        currentPage++;
    } else {
        currentPage++;                       
        attendeesViewModel.loadAttendeesPageWise(currentPage, true);
    }
}

When a new page is fetched, we need to update the existing list and add the elements from the new page.

@Override
public void showResults(List<Attendee> attendees) {
    attendeeList.addAll(attendees);
    fastItemAdapter.setNewList(attendeeList);
    binding.setVariable(BR.attendees, attendeeList);
    binding.executePendingBindings();
}

Now, list of attendees would be fetched pagewise, thus improving the performance and preventing timeouts.

Resources:

Further reading:

Open Event Organizer App: Project repo, Play Store, F-Droid

Continue Reading

Implementing pagination with Retrofit in Eventyay Attendee

Pagination (Paging) is a common and powerful technique in Android Development when making HTTP requests or fetching data from the database. Eventyay Attendee has found many situations where data binding comes in as a great solution for our network calls with Retrofit. Let’s take a look at this technique.

  • Problems without Pagination in Android Development
  • Implementing Pagination with Kotlin with Retrofit
  • Results and GIF
  • Conclusions

PROBLEMS WITHOUT DATABINDING IN ANDROID DEVELOPMENT

Making HTTP requests to fetch data from the API is a basic work in any kind of application. With the mobile application, network data usage management is an important factor that affects the loading performance of the app. Without paging, all of the data are fetched even though most of them are not displayed on the screen. Pagination is a technique to load all the data in pages of limited items, which is much more efficient

IMPLEMENTING DATABINDING IN FRAGMENT VIEW

Step 1:  Set up dependency in build.gradle

// Paging
implementation "androidx.paging:paging-runtime:$paging_version"
implementation "androidx.paging:paging-rxjava2:$paging_version"

Step 2:  Set up retrofit to fetch events from the API

@GET("events?include=event-sub-topic,event-topic,event-type")
fun searchEventsPaged(
   @Query("sort") sort: String,
   @Query("filter") eventName: String,
   @Query("page[number]") page: Int,
   @Query("page[size]") pageSize: Int = 5
): Single<List<Event>>

Step 3: Set up the DataSource

DataSource is a base class for loading data in the paging library from Android. In Eventyay, we use PageKeyedDataSource. It will fetch the data based on the number of pages and items per page with our default parameters. With PageKeyedDataSource, three main functions loadInitial(), loadBefore(), loadAfter() are used to to load each chunks of data.

class EventsDataSource(
   private val eventService: EventService,
   private val compositeDisposable: CompositeDisposable,
   private val query: String?,
   private val mutableProgress: MutableLiveData<Boolean>

) : PageKeyedDataSource<Int, Event>() {
   override fun loadInitial(
       params: LoadInitialParams<Int>,
       callback: LoadInitialCallback<Int, Event>
   ) {
       createObservable(1, 2, callback, null)
   }

   override fun loadAfter(params: LoadParams<Int>, callback: LoadCallback<Int, Event>) {
       val page = params.key
       createObservable(page, page + 1, null, callback)
   }

   override fun loadBefore(params: LoadParams<Int>, callback: LoadCallback<Int, Event>) {
       val page = params.key
       createObservable(page, page - 1, null, callback)
   }

   private fun createObservable(
       requestedPage: Int,
       adjacentPage: Int,
       initialCallback: LoadInitialCallback<Int, Event>?,
       callback: LoadCallback<Int, Event>?
   ) {
       compositeDisposable +=
           eventService.getEventsByLocationPaged(query, requestedPage)
               .withDefaultSchedulers()
               .subscribe({ response ->
                   if (response.isEmpty()) mutableProgress.value = false
                   initialCallback?.onResult(response, null, adjacentPage)
                   callback?.onResult(response, adjacentPage)
               }, { error ->
                   Timber.e(error, "Fail on fetching page of events")
               }
           )
   }
}

Step 4: Set up the Data Source Factory

DataSourceFactory is the class responsible for creating DataSource object so that we can create PagedList (A type of List used for paging) for events.

class EventsDataSourceFactory(
   private val compositeDisposable: CompositeDisposable,
   private val eventService: EventService,
   private val query: String?,
   private val mutableProgress: MutableLiveData<Boolean>
) : DataSource.Factory<Int, Event>() {
   override fun create(): DataSource<Int, Event> {
       return EventsDataSource(eventService, compositeDisposable, query, mutableProgress)
   }
}

Step 5: Adapt the current change to the ViewModel. 

Previously, events fetched in List<Event> Object are now should be turned into PagedList<Event>.

sourceFactory = EventsDataSourceFactory(
   compositeDisposable,
   eventService,
   mutableSavedLocation.value,
   mutableProgress
)
val eventPagedList = RxPagedListBuilder(sourceFactory, config)
   .setFetchScheduler(Schedulers.io())
   .buildObservable()
   .cache()

compositeDisposable += eventPagedList
   .subscribeOn(Schedulers.io())
   .observeOn(AndroidSchedulers.mainThread())
   .distinctUntilChanged()
   .doOnSubscribe {
       mutableProgress.value = true
   }.subscribe({
       val currentPagedEvents = mutablePagedEvents.value
       if (currentPagedEvents == null) {
           mutablePagedEvents.value = it
       } else {
           currentPagedEvents.addAll(it)
           mutablePagedEvents.value = currentPagedEvents
       }
   }, {
       Timber.e(it, "Error fetching events")
       mutableMessage.value = resource.getString(R.string.error_fetching_events_message)
   })

Step 6: Turn ListAdapter into PagedListAdapter

PageListAdapter is basically the same ListAdapter to update the UI of the events item but specifically used for Pagination. In here, List objects can also be null.

class EventsListAdapter : PagedListAdapter<Event, EventViewHolder>(EventsDiffCallback()) {

   var onEventClick: EventClickListener? = null
   var onFavFabClick: FavoriteFabClickListener? = null
   var onHashtagClick: EventHashTagClickListener? = null

   override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): EventViewHolder {
       val binding = ItemCardEventsBinding.inflate(LayoutInflater.from(parent.context), parent, false)
       return EventViewHolder(binding)
   }

   override fun onBindViewHolder(holder: EventViewHolder, position: Int) {
       val event = getItem(position)
       if (event != null)
           holder.apply {
               bind(event, position)
               eventClickListener = onEventClick
               favFabClickListener = onFavFabClick
               hashTagClickListAdapter = onHashtagClick
           }
   }

   /**
    * The function to call when the adapter has to be cleared of items
    */
   fun clear() {
       this.submitList(null)
   }

AND HERE ARE THE RESULTS…

CONCLUSION

Databinding is the way to go when working with a complex UI in Android Development. This helps reducing boilerplate code and to increase the readability of the code and the performance of the UI. One problem with data-binding is that sometimes, it is pretty hard to debug with unhelpful log messages. Hopefully, you can empower your UI in your project now with data-binding. 

Pagination is the way to go for fetching items from the API and making infinite scrolling. This helps reduce network usage and improve the performance of Android applications. And that’s it. I hope you can make your application more powerful with pagination. 

RESOURCES

Open Event Codebase: https://github.com/fossasia/open-event-attendee-android/pull/2012

Documentation: https://developer.android.com/topic/libraries/architecture/paging/ 

Google Codelab: https://codelabs.developers.google.com/codelabs/android-paging/#0

Continue Reading

Implementing Pagination for listing Badges

Badgeyay project is divided into two parts i.e front-end with Ember JS and back-end with REST-API programmed in Python.

Badgeyay comes with many features for customising the process of generation of Badges. Badgeyay also keeps record of all the badges generated by user and also gives option to directly download the previously generated badges.
All the badges appear on single page which creates problem when a user has generated lot of badges and all the badges listed on single page.

To resolve this issue and make Badgeyay more user friendly I have implemented pagination in listing badges so that if there are more number of badges, user can see the badges listed in multiple pages in my Pull Request.

To implement this, I have used actions and made the changes accordingly.

Let’s get started and understand it step by step.

Step 1:

We will use semantic icons in handlebars for changing pages.

Step 2:

I will add action for changing page to next and previous page.

// components/user-component/my-badges.js 

import Component from '@ember/component';
import Ember from 'ember';
const { inject } = Ember;
export default Component.extend({
  queryParams : ['page'],
  page        : 1,
  notify      : inject.service('notify'),
  actions     : {
    nextPage() {  // Action for Next Page
      let filter = {};
      if (this.page > 1) {
        filter.page = this.page + 1;
        this.get('store').query('my-badges', filter)
          .then(records => {
            if (records.length > 0) {
              this.set('my-badges', records);
              this.set('page', this.page + 1);
            } else {
              this.notify.error('No More Badges found');
            }
          })
          .catch(err => {
            this.get('notify').error('Please try again!');
          });
      } else {
        this.notify.error('No More Badges Found');
      }
    },
    prevPage() {  // Action for Previous Page
      let filter = {};
      if (this.page - 1 > 0) {
        filter.page = this.page - 1;
        this.get('store').query('my-badges', filter)
          .then(records => {
            this.set('my-badges', records);
            this.set('page', this.page - 1);
          })
          .catch(err => {
            this.get('notify').error('Please try again!');
          });
      } else {
        this.notify.error('No More Badges Found');
      }
    }
  }
});

 

Step 3:

Now, We have to make the query for all badges generated by a user on opening the my-badges route in Frontend.

import Route from '@ember/routing/route';

import Ember from 'ember';

const { set } = Ember;

export default Route.extend({
  beforeModel(transition) {
    this._super(...arguments);
  },

  model(params) {
    let filter = {};
    this.set('params', params);
    filter.state = 'all';
    filter.page = params.page;
    return this.get('store').query('my-badges', filter);
  }
});

 

I have implemented the pagination feature for listing all badges by the user.

 

Step 4:

Now run the server to see the implemented changes by following command.

My Badges Route with Pagination Feature:

Now we are done with implementing the pagination feature for listing all badges by the user.

Resources:

  1. Ember Docs –  Link
  2. Badgeyay Repository – Link
  3. Issue Link – Link
  4. Semantic UI –  LInk

 

Continue Reading

Enhancing pagination in Badgeyay

A Badge generator like Badgeyay must be able to generate, store and export the user data as and when needed. This blog post covers the enhancement of pagination in the frontend of badgeyay project. There are small “next” and “previous” links to toggle between pages..

Enhancing the current way of links

The problem with the pagination links was that in case of no more badges/users etc, the links would always appear on the bottom right of the table. The previous link must not appear when no previous page is there and vice versa for the next link.

Step 1 : Adding the package to package.json

Image link is the link to the user’s uploaded image on remote firebase server.

{{#if allow}}
<tfoot>
<
tr>
<
th colspan=“5”>
<
div class=“ui right floated pagination menu”>
{{#if allow_prev}}
<
a class=“icon item” {{action ‘prevPage‘}}>
<
i class=“left chevron icon”></i>
</
a>
{{/if}}
{{#if allow_next}}
<
a class=“icon item” {{action ‘nextPage‘}}>
<
i class=“right chevron icon”></i>
</
a>
{{/if}}
</
div>
</
th>
</
tr>
<
/tfoot>
{{/i
f}}

Step 2 : Initializing the variables in setupController

Once we have added the if construct to the badgeyay frontend then we need to add the variable initialization in the setupController method in EmberJS.

setupController(controller, model) {
this._super(…arguments);
set(controller,
‘reports’, model);
if (model.length < 9) {
set(controller,
‘allow_prev’, false);
set(controller,
‘allow_next’, false);
set(controller,
‘allow’, false);
}
}

Step 3 : Implementing state changed in the controllers

Now we need to handle the situation when a user clicks the links and there are more or less links to display. This is done by checking the length of the model in the controller.

if (this.page === 1) {
this.set(‘allow_prev’, false);
}
else {
this.set(‘allow_prev’, true);
}
this..set(‘allow_next’, true);

Same needs to be done for all the controllers that have pagination available.

And finally we need to pass these variables in the component template. One such example is given below.

<div class=“ui grid user-grid”>
<
div class=“row”>
<
div class=“sixteen wide column”>
{{badge-table badges=badges user=user session=session sendbadgeId=(action ‘deleteBadge’ badge) prevPage=(action ‘prevPage’) nextPage=(action ‘nextPage’) allow_prev=allow_prev allow_next=allow_next allow=allow}}
</
div>
</
div>
<
/div>

Finally, we have the pagination links working as desired..

Screenshot of changes

Resources

 

Continue Reading

Live Feeds in loklak Media wall using ‘source=twitter’

Loklak Server provides pagination to provide tweets from Loklak search.json API in divisions so as to improve response time from the server. We will be taking advantage of this pagination using parameter `source=twitter` of the search.json API on loklak media wall. Basically, using parameter ‘source=twitter’ in the API does real time scraping and provides live feeds. To improve response time, it returns feeds as specified in the count (default is 100).

In the blog, I am explaining how implemented real time pagination using ‘source = twitter’ in loklak media wall to get live feeds from twitter.

Working

First API Call on Initialization

The first API call needs to have high count (i.e. maximumRecords = 20) so as to get a higher number of feeds and provide a sufficient amount of feeds to fill up the media wall. ‘source=twitter’ must be specified so that real time feeds are scraped and provided from twitter.

http://api.loklak.org/api/search.json?q=fossasia&callback=__ng_jsonp__.__req0.finished&minified=true&source=twitter&maximumRecords=20&timezoneOffset=-330&startRecord=1

 

If feeds are received from the server, then the next API request must be sent after 10 seconds so that server gets sufficient time to scrap the data and store it in the database. This can be done by an effect which dispatches WallNextPageAction(‘’) keeping debounceTime equal to 10000 so that next request is sent 10 seconds after WallSearchCompleteSuccessAction().

@Effect()
nextWallSearchAction$
= this.actions$
.ofType(apiAction.ActionTypes.WALL_SEARCH_COMPLETE_SUCCESS)
.debounceTime(10000)
.withLatestFrom(this.store$)
.map(([action, state]) => {
return new wallPaginationAction.WallNextPageAction();
});

Consecutive Calls

To implement pagination, next consecutive API call must be made to add new live feeds to the media wall. For the new feeds, count must be kept low so that no heavy pagination takes place and feeds are added one by one to get more focus on new tweets. For this purpose, count must be kept to one.

this.searchServiceConfig.count = queryObject.count;
this.searchServiceConfig.maximumRecords = queryObject.count;return this.apiSearchService.fetchQuery(queryObject.query.queryString, this.searchServiceConfig)
.takeUntil(nextSearch$)
.map(response => {
return new wallPaginationAction.WallPaginationCompleteSuccessAction(response);
})
.catch(() => of(new wallPaginationAction.WallPaginationCompleteFailAction()));
});

 

Here, count and maximumRecords is updated from queryObject.count which varies between 1 to 5 (default being 1). This can be updated by user from the customization menu.

Next API request is as follows:

http://api.loklak.org/api/search.json?q=fossasia&callback=__ng_jsonp__.__req2.finished&minified=true&source=twitter&maximumRecords=1&timezoneOffset=-330&startRecord=1

 

Now, as done above, if some response is received from media wall, next request is sent after 10 seconds after WallPaginationCompleteSuccess() from an effect by keeping debounceTime equal to 10000.

In the similar way, new consecutive calls can be made by keeping ‘source = twitter’ and keeping count low for getting a proper focus on new feed.

Reference

Continue Reading
Iterating the Loklak Server data
Iterating the Loklak Server data

Iterating the Loklak Server data

Loklak Server is amazing for what it does, but it is more impressive how it does the tasks. Iterators are used for and how to use them, but this project has a customized iterator that iterates Twitter data objects. This iterator is Timeline.java .

Timeline implements an interface iterable (isn’t it iterator?). This interface helps in using Timeline as an iterator and add methods to modify, use or create the data objects. At present, it only iterates Twitter data objects. I am working on it to modify it to iterate data objects from all web scrapers.

The following is a simple example of how an iterator is used.

// Initializing arraylist
List<String> stringsList = Arrays.asList("foo", "bar", "baz");

// Using iterator to display contents of stringsList
System.out.print("Contents of stringsList: ");

Iterator iter = al.iterator();
while(iter.hasNext()) {
    System.out.print(iter.next() + " ");
}

 

This iterator can only iterate data the way array does. (Then why do we need it?) It does the task of iterating objects perfectly, but we can add more functionality to the iterator.

 

Timeline iterator iterates the MessageEntry objects i.e. superclass of TwitterTweet objects. According to Javadocs, “Timeline is a structure which holds tweet for the purpose of presentation, There is no tweet retrieval method here, just an iterator which returns the tweets in reverse appearing order.”

Following are some of the tasks it does:

  1. As an iterator:

This basic use of Timeline is to iterate the MessageEntry objects. It not only iterates the data objects, but also fetches them (See here).

// Declare Timeline object according to order the data object has been created
Timeline tline = new Timeline(Timeline.parseOrder("created_at"));

// Adding data objects to the timeline
tline.add(me1);
tline.add(me2);
.
.
.
// Outputing all data objects as array of JSON objects
for (MessageEntry me: tline) {
    JSONArray postArray = new JSONArray();
    for (MessageEntry post : this) {
        postArray.put(post.toJSON());
    }
}

 

  1. The order of iterating the data objects

Timeline can arrange and iterate the data objects according to the date of creation of the twitter post, number of retweets or number of favourite counts. For this there is an Enum declaration of Order in the Timeline class which is initialized during creation of Timeline object. [link]

    Timeline tline = new Timeline(Timeline.parseOrder("created_at"));

 

  1. Pagination of data objects

There is an object cursor, some methods, including getter and setters to support pagination of the data objects. It is only internally implemented, but can also be used to return a section of the result.

  1. writeToIndex method

This method can be used to write all data fetched by Timeline iterator to ElasticSearch for indexing and to dump that can be used for testing. Thus, indexing of data can concurrently be done while it is iterated. It is implemented here.

  1. Other methods

It also has methods to output all data as JSON and customized method to add data to Timeline keeping user object and Data separate, etc. There are a bit more things in this iterable class which shall be explored instead.

 

Resources:

Continue Reading
Close Menu