Mailing Attachments Using Terminal in Open Event Android

The latest version of Open Event Android App Generator, v2 lacked the feature of mailing the generated APK to the email ID that is entered at the start of the app generation process. This also included mailing the error logs in case of APK failure.

This is an important feature for app generator because the process of app generation is a time taking one. The users have to wait for the app to be generated so that they can download the generated APK. To avoid this, the generator can automatically email the APK as soon as it is generated.

I took up this issue a few days back and started working on it. I started with thinking about the ways through which it will be implemented. This required some discussions with the mentors and co-developers. We finalised on the following ways:

  • Using Sendgrid
  • Using SMTP

I will be discussing the implementation of both of them in this blog. The code for APK mailing starts with the function call Notification.send in generator.py

if completed and apk_path and not error:
   Notification.send(
       to=self.creator_email,
       subject='Your android application for %s has been generated ' % self.event_name,
       message='Hi,<br><br>'
               'Your android application for the \'%s\' event has been generated. '
               'And apk file has been attached along with this email.<br><br>'
               'Thanks,<br>'
               'Open Event App Generator' % self.event_name,
       file_attachment=apk_path,
       via_api=self.via_api
   )
else:
   Notification.send(
       to=self.creator_email,
       subject='Your android application for %s could not generated ' % self.event_name,
       message='Hi,<br><br> '
               'Your android application for the \'%s\' event could not generated. '
               'The error message has been provided below.<br><br>'
               '<code>%s</code><br><br>'
               'Thanks,<br>'
               'Open Event App Generator' % (self.event_name, str(error) if error else ''),
       file_attachment=apk_path,
       via_api=self.via_api
   )

This leads me to the class Notification.py. It has three functions:-

1. send(to, subject, message, file_attachment, via_api)
2. send_mail_via_smtp(payload):
3. send_email_via_sendgrid(payload):

As the name suggests, the first function:

send(to, subject, message, file_attachment, via_api)

mainly decides which service (out of smtp and sendgrid) should be used to send the email, on the basis of the input parameters (especially, the ‘EMAIL_SERVICE’ parameter that has to be set in config.py).
The function looks like as follows:

send(to, subject, message, file_attachment, via_api)

It is in the send() that the other two functions are called. If the email_service is smtp, it calls the Notification.send_mail_via_smtp(payload). Otherwise, the Notification.send_email_via_sendgrid(payload) is called.
The sendgrid function is pretty straightforward:

@staticmethod
def send_email_via_sendgrid(payload):

   key = current_app.config['SENDGRID_KEY']
   if not key:
       logger.info('Sendgrid key not defined')
       return
   headers = {
       "Authorization": ("Bearer " + key)
   }
   requests.post(
       "https://api.sendgrid.com/api/mail.send.json",
       data=payload,
       headers=headers
   )

It requires a personalised sendgrid key which is accessed from the config.py file. Apart from that it handles some errors by giving logs in celery tasks. The main line in the function that initiates the email is a POST request made using the python library ‘requests’. The request is made as follows:

 requests.post(
       "https://api.sendgrid.com/api/mail.send.json",
       data=payload,
       headers=headers
   )

The send_mail_via_smtp(payload): function looks for some configurations before sending the mail:

@staticmethod
def send_mail_via_smtp(payload):
   """
   Send email via SMTP
   :param config:
   :param payload:
   :return:
   """
   smtp_encryption = current_app.config['SMTP_ENCRYPTION']
   if smtp_encryption == 'tls':
       smtp_encryption = 'required'
   elif smtp_encryption == 'ssl':
       smtp_encryption = 'ssl'
   elif smtp_encryption == 'tls_optional':
       smtp_encryption = 'optional'
   else:
       smtp_encryption = 'none'
   config = {
       'host': current_app.config['SMTP_HOST'],
       'username': current_app.config['SMTP_USERNAME'],
       'password': current_app.config['SMTP_PASSWORD'],
       'encryption': smtp_encryption,
       'port': current_app.config['SMTP_PORT'],
   }
   mailer_config = {
       'transport': {
           'use': 'smtp',
           'host': config['host'],
           'username': config['username'],
           'password': config['password'],
           'tls': config['encryption'],
           'port': config['port']
       }
   }

   mailer = Mailer(mailer_config)
   mailer.start()
   message = Message(author=payload['from'], to=payload['to'])
   message.subject = payload['subject']
   message.plain = strip_tags(payload['message'])
   message.rich = payload['message']
   message.attach(payload['attachment'])
   mailer.send(message)
   mailer.stop()

It is using the Marrow Mailer Python library to email with attachments(APK). This Python library can be installed using
pip install marrow.mailer
To use Marrow Mailer you instantiate a marrow.mailer.Mailer object with the configuration, then pass Message instances to the Mailer instance’s send() method.

You can refer to the following guides for more information about sending emails through command line:
https://github.com/marrow/mailer is the official repo of Marrow Mailer repository.
https://pypi.python.org/pypi/marrow.mailer
More detailled information on sending emails using Sendgrid can be found here https://www.daveperrett.com/articles/2013/03/19/setting-up-sendmail-with-sendgrid-on-ubuntu/

Continue ReadingMailing Attachments Using Terminal in Open Event Android

Using Flask SocketIO Library in the Apk Generator of the Open Event Android App

Recently Flask SocketIO library was used in the apk generator of the Open Event Android App as it gave access to the low latency bi-directional communications between the client and the server side. The client side of the apk generator was written in Javascript which helped us to use a SocketIO official client library to establish a permanent connection to the server.

The main purpose of using the library was to display logs to the user when the app generation process goes on. This gives the user an additional help to check what is the mistake in the file uploaded by them in case an error occurs. Here the library established a connection between the server and the client so that during the app generation process the server would send real time logs to the client so that they can be viewed by the user displayed in the frontend.

To use this library we first need to download it using pip command:

pip install flask-socketio

This library depends on the asynchronous services which can be selected amongst the following listed below:

  1. Eventlet
  2. Gevent
  3. Flask development server based on Werkzeug

Amongst the three listed, eventlet is the best performant option as it has support for long polling and WebSocket transports.

The next thing was importing this library in the flask application i.e the apk generator of the app as follows:

from flask_socketio import SocketIO

current_app = create_app()
socketio = SocketIO(current_app)

if __name__ == '__main__':
    socketio.run(current_app)

The main use of the above statement (socket.run(current_app)) is that with this the startup of the web server is encapsulated. When the application is run in debug mode it is preferred to use the Werkzeug development server. However in production mode the use of eventlet web server or gevent web server is recommended.

We wanted to show the status messages currently shown to the user in the form of logs. For this firstly the generator.py file was looked upon which had the status messages and these were sent to the client side of the generator by establishing a connection between them using this library. The code written on the server side to send messages to the client side was as follows:

def handle_message(message):
    if message not None:
        namespace = “/” + self.identifier;
        send(message, namespace = namespace)

As we can see from the above code the message had to be sent to that particular namespace. The main idea of using namespace was that if there were more than one users using the generator at the same time, it would mean the send() method would send the messages to all the clients connected which would lead to the mixing up of status messages and hence it was important to use namespace so that a message was sent to that particular client.

Once the server sent the messages to the client we needed to add functionality to the our client side to receive the messages from them and also update the logs area with suitable content received. For that first the socket connection was established once the generate button was clicked which generated the identifier for us which had to be used as the namespace for that process.

socket = io.connect(location.protocol + "//" + location.host + "/" + identifier, {reconnection: false});

This piece of code established the connection of the socket and helps the client side to receive and send messages from and to the server end.

Next thing was receiving messages from the server. For this the following code snippet was added:

socket.on('message', function(message) {
    $buildLog.append(message);
});

This piece of code receives the messages from the server for unnamed events. Once the connection is established and the messages are received, the logs are updated with each message being appended so as to show the user the real time information about their build status of their app.

This was how the idea of logs being shown in the apk generator was incorporated by making the required changes in the server and client side code using the Flask SocketIO library.

Related Links:

Continue ReadingUsing Flask SocketIO Library in the Apk Generator of the Open Event Android App

Create an AutocompleteTextView dropdown for the email input in the Open Event Orga Android App

In the first version of the Open Event Organizer App, the event organizer was required to enter his full email each time he logged out of his account and therefore it was hindering the user experience. AutoCompleteTextView with shared preferences is a solution to this problem. This feature provides an editable text view that shows completion suggestions automatically while the user is typing. The list of suggestions is displayed in a drop down menu. The user can choose an item to replace the content of the edit box with. It is extremely useful in enhancing user experience.

The solution we implemented was to create an autocomplete textview for the email input, store the email address of the user on a successful login in the shared preference in a set of strings to prevent duplicacy and display it in the dropdown on subsequent login attempts.

Implementation

Change your TextInputLayout structure to accommodate the autocompletetextview. Remember to create a separate autocompletetextview object with the specific id of the view.

<android.support.v7.widget.AppCompatAutoCompleteTextView
       android:id="@+id/email_dropdown"
       android:layout_width="match_parent"
       android:layout_height="wrap_content"
       android:hint="@string/email"
       android:inputType="textEmailAddress" />

 

Create Utility methods to get/store the emails in the shared preferences. The set data structure has been used here so that there is no duplicacy while storing the emails in the shared preferences.

public Set<String> getStringSet(String key, Set<String> defaultValue) {
   return sharedPreferences.getStringSet(key, defaultValue);
}

public void saveStringSet(String key, Set<String> value) {
   SharedPreferences.Editor editor = sharedPreferences.edit();
   editor.putStringSet(key, value);
   editor.apply();
}

public void addStringSetElement(String key, String value) {
   Set<String> set = getStringSet(key, new HashSet<>());
   set.add(value);
   saveStringSet(key, set);
}

 

Create helper methods to add an email and retrieve the list of emails from the shared preferences to provide it to the views.

private void saveEmail(String email) {
   utilModel.addStringSetElement(Constants.SHARED_PREFS_SAVED_EMAIL, email);
}

private Set<String> getEmailList() {
   return utilModel.getStringSet(Constants.SHARED_PREFS_SAVED_EMAIL, null);
}

 

Create an autocompleteTextView object in your activity with the help of the R id from the layout and set the adapter with the set of strings retrieved from the shared preferences. You could create a custom adapter for this case too, but as far as the Open Event Orga App was concerned, using the array adapter made sense.

autoCompleteEmail.setAdapter(new ArrayAdapter<>(this, android.R.layout.simple_list_item_1,
   new ArrayList<String>(emails)));

 

Conclusion

It is important that the user is served with the best possible experience of the application and the autocomplete text view for the email serves just that.

Resources

Continue ReadingCreate an AutocompleteTextView dropdown for the email input in the Open Event Orga Android App

Addition of Bookmarks to the Homescreen in the Open Event Android App

In the Open Event Android app we had already built the new homescreen but the users only had access to bookmarks in a separate page which could be accessed from the navbar.If the bookmarks section were to be incorporated in the homescreen itself, it would definitely improve its access to the user. In this blog post, I’ll be talking about how this was done in the app.

These 2 images show the homescreen and the bookmarks section respectively.

No Bookmark View
Bookmark View

 

 

 

 

 

 

 

 

 

This was the proposed homescreen page for the app. This would provide easy access to important stuff to the user such as event venue,date,description etc. Also the same homescreen would also have the bookmarks showing at the top if there are any.

The list of bookmarks in the first iteration of design was modeled to be a horizontal list of cards.

Bookmarks Merging Process

These are some variables for reference.

private SessionsListAdapter sessionsListAdapter;
 private RealmResults<Session> bookmarksResult;
 private List<Session> mSessions = new ArrayList<>();

The code snippet below highlights the initial setup of the bookmarks recycler view for the horizontal List of cards. All of this is being done in the onCreateView callback of the AboutFragment.java file which is the fragment file for the homescreen.

bookmarksRecyclerView.setVisibility(View.VISIBLE);
 sessionsListAdapter = new SessionsListAdapter(getContext(), mSessions, bookmarkedSessionList);
 sessionsListAdapter.setBookmarkView(true);
 bookmarksRecyclerView.setAdapter(sessionsListAdapter);
 bookmarksRecyclerView.setLayoutManager(new LinearLayoutManager(getContext(),LinearLayoutManager.HORIZONTAL,false));

The SessionListAdapter is an adapter that was built to handle multiple types of displays of the same viewholder i.e SessionViewHolder . This SessionListAdapter is given a static variable as an argument which is just notifies the adapter to switch to the bookmarks mode for the adapter.

private void loadData() {
    bookmarksResult = realmRepo.getBookMarkedSessions();
    bookmarksResult.removeAllChangeListeners();
    bookmarksResult.addChangeListener((bookmarked, orderedCollectionChangeSet) -> {
        mSessions.clear();
        mSessions.addAll(bookmarked);
 
        sessionsListAdapter.notifyDataSetChanged();
 
        handleVisibility();
    });
 }

This function loadData() is responsible for extracting the sessions that are bookmarked from the local Realm database. We the update the BookmarkAdapter on the homescreen with the list of the bookmarks obtained. Here we see that a ChangeListener is being attached to our RealmResults. This is being done so that we do our adapter notify only after the data of the bookmarked sessions has been processed from a background thread.

if(bookmarksResult != null)
    bookmarksResult.removeAllChangeListeners();

And it is good practice to remove any ChangeListeners that we attach during the fragment life cycle in the onStop() method to avoid memory leaks.

So now we have successfully added bookmarks to the homescreen.

Resources

Continue ReadingAddition of Bookmarks to the Homescreen in the Open Event Android App

Better Bookmark Display Viewholder in Home Screen of the Open Event Android App

Earlier in the Open Event Android app we had built the homescreen with the bookmarks showing up at the top as a horizontal list of cards but it wasn’t very user-friendly in terms of UI. Imagine that a user bookmarks over 20-30 sessions, in order to access them he/she might have to scroll horizontally a lot in order to access his/her bookmarked session. So this kind of UI was deemed counter-intuitive. A new UI was proposed involving the viewholder used in the schedule page i.e DayScheduleViewHolder, where the list would be vertical instead of horizontal. An added bonus was that this viewholder conveyed the same amount of information on lesser white space than the earlier viewholder i.e SessionViewHolder.

Old Design
New Design

 

 

 

 

 

 

 

 

 

Above are two images, one in the initial design, the second in the new design. In the earlier design the number of bookmarks visible to the user at a time was at most 1 or 2 but now with the UI upgrade a user can easily see up-to 5-6 bookmarks at a time. Additionally there is more relevant content visible to the user at the same time.

Additionally this form of design also adheres to Google’s Material Design guidelines.

Code Comparison of the two Iterations

Initial Design
sessionsListAdapter = new SessionsListAdapter(getContext(), mSessions, bookmarkedSessionList);
 sessionsListAdapter.setBookmarkView(true);
 bookmarksRecyclerView.setAdapter(sessionsListAdapter);
 bookmarksRecyclerView.setLayoutManager(new LinearLayoutManager(getContext(),LinearLayoutManager.HORIZONTAL,false));

Here we are using the SessionListAdapter for the bookmarks. This was previously being used to display the list of sessions inside the track and location pages. It is again being used here to display the horizontal list of bookmarks.To do this we are using the function setBookmarkView(). Here mSessions consists the list of bookmarks that would appear in the homescreen.

Current Design
bookmarksRecyclerView.setVisibility(View.VISIBLE);
 bookMarksListAdapter = new DayScheduleAdapter(mSessions,getContext());
 bookmarksRecyclerView.setAdapter(bookMarksListAdapter);
 bookmarksRecyclerView.setLayoutManager(new LinearLayoutManager(getContext()));

Now we are using the DayScheduleAdapter which is the same adapter used in the schedule page of the app. Now we use a vertical layout instead of a horizontal layout  but with a new viewholder design. I will be talking about the last line in the code snippet shortly.

ViewHolder Design (Current Design)

<RelativeLayout android:id="@+id/content_frame">
    <LinearLayout android:id="@+id/ll_sessionDetails">
        <TextView android:id="@+id/slot_title"/>
        <RelativeLayout>
            <TextView android:id="@+id/slot_start_time”/>
            <TextView android:id="@+id/slot_underscore"/>
            <TextView android:id="@+id/slot_end_time"/>

           <TextView android:id="@+id/slot_comma"/>
            <TextView android:id="@+id/slot_location”/>
            <Button android:id="@+id/slot_track"/>
            <ImageButton android:id="@+id/slot_bookmark"/>
        </RelativeLayout>

    </LinearLayout>
    <View android:id=”@+id/divider”/>
 </RelativeLayout>

This layout file is descriptive enough to highlight each element’s location.

In this viewholder we can also access to the track page from the track tag and can also remove bookmarks instantly.

The official widget to make a scroll layout in android is ScrollView. Basically, adding a RecyclerView inside ScrollView can be difficult . The problem was that the scrolling became laggy and weird.  Fortunately, with the appearance of Material Design , NestedScrollView was released and this becomes much easier.

bookmarksRecyclerView.setNestedScrollingEnabled(false);

With this small snippet of code we are able to insert a RecyclerView inside the NestedScrollView without any scroll lag.
So now we have successfully updated the UI/UX for the homescreen  to meet the requirements as given by the Material Design guidelines.

Resources

Continue ReadingBetter Bookmark Display Viewholder in Home Screen of the Open Event Android App

Building a showcase site to display sample events and auto deploying them on each PR merge in Open Event Webapp

Open Event Webapp generates static websites of the event fed to it in the form of JSON data. Earlier, we used to automatically deploy the FOSSASIA Summit event website on the Github pages of the repository on every merge of a pull request.  The presence of it helped in showcasing and testing purposes. But a single event was not enough. Events are diverse, taking place in a large number of rooms, h a variety of sessions and extending over several days. So, having multiple events on the showcase site and continuously showcasing and testing them was a really great idea.

Here is the feature request for it. I will be explaining the major portions of it. The whole code can be found here

First of all, we need to build the main index page which we will showcase our sample events. It will contain the small images of the event and on clicking them, the selected event will be opened. For displaying features and capability of the generator,  we have two separate sections of events: one having the single session page and the other having the expandable sessions page. There will be links to the other components of the Open Event Ecosystem like Android App generator,  Web App generatorOrganizer App, and Eventyay.

The contents of that page are kept in the overviewSite folder. It contains the event background images of the events and the main index file. The basic structure of the file is shown below. The whole file can be viewed here

<div class="container-fluid bg-3 text-center">
 <div class="row margin">
   <div class="col-md-6 col-sm-12 col-xs-12">
     <div class="row">
       <h2 class="margin"> Apps with expandable sessions page </h2>
       <div class="col-md-6 col-sm-6 col-xs-12 margin">
         <p><strong>Open Tech Summit 2017</strong></p>
         <a href='./OpenTechSummit/index.html' target='_blank'>
           <img src="./otssmall.jpg" class=""  alt="Image">
         </a>
       </div>
      </div>
    </div>
   <div class="col-md-6 col-sm-12 col-xs-12">
     <div class="row">
       <h2 class="margin"> Apps with single sessions page </h2>
       <div class="col-md-6 col-sm-6 col-xs-12 margin">
         <p><strong>Mozilla All Hands 2017</strong></p>
         <a href='./MozillaAllHands2017/index.html' target='_blank'>
           <img src="./mozilla_banner.jpg" class=""  alt="Image">
         </a>
       </div>
     </div>
   </div>
 </div>
</div>

But, this is just the front end of the page. We haven’t generated the sample events yet and neither have we made any changes to make them auto deploy on each PR merge. The test script of the app which contains unit, acceptance and selenium tests runs on each PR made against the repo when Travis CI build is triggered. So, it makes sense to write code for generating the event there itself. When the test script has finished executing, all of the events would have been generated and present inside a single folder named a@a.com (We needed something short and easy to remember and the name doesn’t really matter). We then copy the contents of the overviewSite folder into the above folder. It already contains the folder of different sample events.

Here is the related code. The full test script file can be found here

describe('generate', function() {
 describe('.create different event sites and copy assets of overview site', function() {
   // Sample events are generated inside the a@a.com folder
   it('should generate the Mozilla All Hands 2017', function(done) {
     var data = {};
     // API endpoint configuration of the All Hands 2017 event and the session style set to single pages
     data.body = {
       "email": "a@a.com",
       "name": "Open Event",
       "apiendpoint":    "https://raw.githubusercontent.com/fossasia/open-event/master/sample/MozillaAllHands17",
       "sessionMode": "single",
       "datasource": "eventapi",
       "assetmode" : "download"
     };
     generator.createDistDir(data, 'Socket', function(appFolder) {
       assert.equal(appFolder, "a@a.com/MozillaAllHands2017");
       done();
     });
   });
   // For copying the static files required for the showcase site
   it('should copy all the static files', function(done) {
     var staticPath = __dirname + '/../src/backend/overviewSite/';
     function copyStatic(fileName) {
       // Copy the static files in the overviewSite folder to the a@a.com folder
     }
   });
 });
});

Everything is almost done now. We then just make some changes in the deploy script to publish the whole folder containing the different event samples instead of a particular sample and everything works fine.

We navigate to the a@a.com folder inside the dist directory and initialize a git repository there. We then set the username and email and fetch the contents of the gh-pages branch of the official repo (using the Github token. It is defined in the Travis Settings as a private environment variable) and reset it. We then add all the files present in the current directory to the staging area, commit them and push it to the upstream repository. Here is the relevant code. The whole github deploy script file can be viewed here

eval cd dist/a@a.com/
git init
git config --global user.name "Travis CI"
git config --global user.email "noreply+travis@fossasia.org"
git remote add upstream "https://$GH_TOKEN@github.com/"${TRAVIS_REPO_SLUG}".git"
git fetch upstream
git reset upstream/gh-pages
touch .
git add -A .
git commit -m "rebuild pages at ${rev}"
git push -q upstream HEAD:gh-pages

Here is the screenshot showing the successful deployment of the sample events. You can view the whole log here. A screenshot of it is below. You can view the higher quality image by clicking on it.

29e53dc0-8841-4329-8a37-8d2e9f08756c.png

After all the work, this is how the showcase page looks like. Samples are re-generated and the site is auto-deployed on every Pull Request merged in the repository.

66804d19-6862-494f-80b6-e1749493e780.png

On clicking on any of the listed events, we jump to the index page of that particular event. Like, If we click on the Facebook Developer Conference, a new tab is opened and we are taken to the front page of that event!

6a47f623-82c6-49e2-af1c-4d152aaaf9b7.png

References

Continue ReadingBuilding a showcase site to display sample events and auto deploying them on each PR merge in Open Event Webapp

Shrinking Model Classes Boilerplate in Open Event Android Projects Using Jackson and Lombok

JSON is the de facto standard format used for REST API communication, and for consuming any of such API on Android apps like Open Event Android Client and Organiser App, we need Plain Old Java Objects, or POJOs to map the JSON attributes to class properties. These are called models, for they model the API response or request. Basic structure of these models contain

  • Private properties representing JSON attributes
  • Getters and Setters for these properties used to change the object or access its data
  • A toString() method which converts object state to a string, useful for logging and debugging purposes
  • An equals and hashcode method if we want to compare two objects

These can be easily and automatically be generated by any modern IDE, but add unnecessarily to the code base for a relatively simple model class, and are also difficult to maintain. If you add, remove, or rename a method, you have to change its getters/setters, toString and other standard data class methods.

There are a couple of ways to handle it:

  • Google’s Auto Value: Creates Immutable class builders and creators, with all standard methods. But, as it generates a new class for the model, you need to add a library to make it work with JSON parsers and Retrofit. Secondly, there is no way to change the object attributes as they are immutable. It is generally a good practice to make your models immutable, but if you are manipulating their data in your application and saving it in your database, it won’t be possible except than to create a copy of that object. Secondly, not all database storage libraries support it
  • Kotlin’s Data Classes: Kotlin has a nice way to made models using data classes, but it has certain limitations too. Only parameters in primary constructor will be included in the data methods generated, and for creating a no argument constructor (required for certain database libraries and annotation processors), you either need to assign default value to each property or call the primary constructor filling in all the default values, which is a boilerplate of its own. Secondly, sometimes 3rd party libraries are needed to work correctly with data classes on some JSON parsing frameworks, and you probably don’t want to just include Kotlin in your project just for data classes
  • Lombok: We’ll be talking about it later in this blog
  • Immutables, Xtend Lang, etc

This is one kind of boilerplate, and other kind is JSON property names. As we know Java uses camelcase notation for properties, and JSON attributes are mostly:

  • Snake Cased: property_name
  • Kebab Cased: property-name

Whether you are using GSON, Jackson or any other JSON parsing framework, it works great for non ambiguous property names which match as same in both JSON and Java, but requires special naming attributes for translating JSON attributes to camelcase and vice versa. Won’t you want your parser to intelligently convert property_name to propertyName without having you write the tedious mapping which is not only a boilerplate, but also error prone in case your API changes and you forget to update the annotations, or make a spelling mistake, as they are just non type-safe strings.

These boilerplates cause serious regressions during development for what should be a simple Java model for a simple API response. These both kinds of boilerplate are also related to each other as all JSON parsers look for getters and setters for private fields, so there are two layers of potential errors in modeling JSON to Java Models. This should be a lot simpler than it is. So, in this blog, we’ll see how we can configure our project to be 0 boilerplate tolerant and error free. We reduced approximately 70% boilerplate using this configuration in our projects. For example, the Event model class we had (our biggest) reduced from 590 lines to just 74!

We will use a simpler class for our example here:

public class CallForPapers {

    private String announcement;
    @JsonProperty("starts-at")
    private String startsAt;
    private String privacy;
    @JsonProperty("ends-at")
    private String endsAt;

    // Getters and Setters

    @Override
    public String toString() {
        return "CallForPapers{" +
                "announcement='" + announcement + '\'' +
                ", startsAt='" + startsAt + '\'' +
                ", privacy='" + privacy + '\'' +
                ", endsAt='" + endsAt + '\'' +
                '}';
    }
}

 

Note that getters and setters have been omitted for brevity. The actual class is 57 lines long

As you can see, we are using @JsonProperty annotation to properly map the starts-at attribute to startsAt property and similarly on endsAt. First, we’ll remove this boilerplate from our code. Note that this seems a bit overkill for 2 attributes, but imagine the time you’ll save by not having to maintain 100s of attributes for the whole project.

Jackson is smart enough to map different naming styles to one another in both serializing and deserializing. This is done by using Naming Strategy class in Jackson. There is an option to globally configure it, but I found that it did not work for our case, so we had to apply it to each model. It can be simply done by adding another annotation on the top of your class declaration and removing the JsonProperty attribute from your fields

@JsonNaming(PropertyNamingStrategy.KebabCaseStrategy.class)
public class CallForPapers {

    private String announcement;
    private String startsAt;
    private String privacy;
    private String endsAt;

    // Getters and Setters

    @Override
    public String toString() {
        return "CallForPapers{" +
                "announcement='" + announcement + '\'' +
                ", startsAt='" + startsAt + '\'' +
                ", privacy='" + privacy + '\'' +
                ", endsAt='" + endsAt + '\'' +
                '}';
    }
}

 

Our class looks like this now. But be careful to properly name your getters and setters because now, Jackson will map attributes by method names, so if you name the setter for startsAt -> setStartsAt it will automatically understand that the attribute to be mapped is “starts-at”. But, if the method name is something else, then it won’t be able to correctly map the fields. If your properties are not private, then Jackson may instead use them to map fields, so be sure to name your public properties in a correct manner.

Note: If your API does not use kebab case, there are plenty of other options or naming strategies present in Jackson, one example will be

  • PropertyNamingStrategy.SnakeCaseStrategy for attributes like “starts_at”

 Needless to say, this will only work if your API uses a uniform naming strategy

Now we have removed quite a burden from the development lifecycle, but 70% of class is still getters, setters, toString and other data methods. Now, we’ll configure lombok to automatically generate these for us. First, we’ll need to add lombok in our project by adding provided dependency in build.gradle and sync the project

provided 'org.projectlombok:lombok:1.16.18'

 

And now you’d want to install Lombok plugin in Android Studio by going to Files > Settings > Plugins and searching and installing Lombok Plugin and restarting the IDE

 

After you have restarted the IDE, navigate to your model and add @Data annotation at the top of your class and remove all getters/setters, toString, equals, hashcode and if the plugin was installed correctly and lombok was installed from the gradle dependencies, these will be automatically generated for you at build time without any problem. A way for you to see the generated methods is to the structure perspective in the Project Window.

 

There are many more fun tools in lombok and more fine grained control options are provided. Our class looks like this now

@Data
@JsonNaming(PropertyNamingStrategy.KebabCaseStrategy.class)
public class CallForPapers {
    private String announcement;
    private String startsAt;
    private String privacy;
    private String endsAt;
}

 

Reduced to 16 lines (including imports and package). Now, there are some corner cases that you want to iron out for the integration between lombok and Jackson to work correctly.

Lombok uses property names for generating its getters and setters. But there’s a different convention for handling booleans. For the sake of simplicity, we’ll only talk about primitive boolean. You can check out the links below to learn more about class type. The primitive boolean property of the standard Java format, for example hasSessions will generate hasSessions and getter and setHasSessions. Jackson is smart but it expects a getter named getHasSessions creating problems in serialization. Similarly, for a property name isComplete, generate getter and setter will be isComplete and setComplete, creating a problem in deserialization too. Actually, there are ways how Jackson can get boolean values mapped correctly with these getters/setters, but that method needs to rename property itself, changing the getters and setters generated by Lombok. There is actually a way to tell Lombok to not generate this format of getter/setter. You’d need to create a file named lombok.config in your project directory app/ and write this in it

lombok.anyConstructor.suppressConstructorProperties = true
lombok.addGeneratedAnnotation = false
lombok.getter.noIsPrefix = true

 

There are some other settings in it that make it configured for Android specific project

There are some known issues in Android related to Lombok. As lombok itself is an annotation processor, and there is no order for annotation processors to run, it may create problems with other annotation processors. Dagger had issues with it until they fixed it in their later versions. So you might need to check out if any of your libraries depend upon the lombok generated code like getters and setters. Certain database libraries use that and Android Data Binding does too. Currently, there is no solution to the problem as they will throw an error about not finding a getter/setter because they ran before lombok. A possible workaround is to make properties public so that instead of using getters and setters, these libraries use them instead. This is not a good practice, but as this is a data class and you are already creating getters and setters for all fields, this is not a security vulnerability.

There are tons of options for both Jackson and Lombok with a lot of features to help the development process, so be sure to check out these links:

Continue ReadingShrinking Model Classes Boilerplate in Open Event Android Projects Using Jackson and Lombok

Save Server Response to File Using Python in Open Event Android App Generator

The Open Event Android project helps event organizers to generate Apps (apk format) using Open Event App generator for their events/conferences by providing API endpoint or zip generated using Open Event server.

The Open Event Android project has an assets folder which contains sample JSON files for the event, which are used to load the data when there is no Internet connection. When an organizer generates an app with the help of an API endpoint, then all the assets, e.g. images, are removed and because of it generated don’t have sample JSON files. So if the device is offline then it will not load the data from the assets (issue #1606). The app should contain sample event JSON files and should be able to load the data without the Internet connection.

One solution to this problem is to fetch all the event data (JSON files) from the server and save it to the assets folder while generating the app using Open Event App generator. So in this blog post, I explain how to make a simple request using Python and save the response in a file.

Making a simple request

To fetch the data we need to make a request to the server and the server will return the response for that request. Here’s step to make a simple request,

Import Requests module using,

import requests

The requests module has get, post, put, delete, head, options to make a request. Example:

requests.get(self.api_link + '/' + end_point)

Here all method returns requests.Response object. We can get all the information we need from this object.

So we can make response object like this,

response = requests.get(self.api_link + '/' + end_point)

Saving response in a file

For saving response in a file we need to open/create a file for writing.

Create or open file

The open() method is used for opening file. It takes two arguments one is file name and second in access mode which determines the mode in which the file has to be opened, i.e., read, write, append, etc.

file = open(path/to/the/file, "w+")

Here “w+” overwrites the existing file if the file exists. If the file does not exist, it creates a new file for reading and writing.

Write and close file

Now write the content of the response using the write() method of file object then close the file object using the close()  method. The close() method flushes any unwritten information and closes the file object.

file.write(response.text)
file.close()

Here response.text gives the content of the response in Unicode format.

Conclusion

The requests.Response object contains much more data like encoding, status_code, headers etc. For more info about request and response refer additional resources given below.

Additional Resources

Continue ReadingSave Server Response to File Using Python in Open Event Android App Generator

Creating A Sample Event Web App Using the Open Event Framework

I have been creating sample events for the Open Event Framework by creating JSON files for them. In this blog, I will walk you through 4 easy steps to create a sample event for FOSSASIA Open Event. These are the steps that I followed while I contributed in making sample web apps:

Let’s start by taking the example of a sample event. I recently created a sample for FBF8’2017. F8 is conference held by Facebook annually.

I started by visiting the official F8 site. I personally used Google Chrome for it as I was almost positive that the official site will be having an API which will make it easier for me to extract JSON for the event. Also, Google Chrome developer tools make it easier to access the API. I have previously explained how to access JSON through APIs in one of my previous blog posts that can be found here.

So in the F8 site, I looked for the page that will lead me to the full schedule of the conference:

After clicking on the “SEE FULL SCHEDULE” button, this is where it leads us, with the developer tools ON.

That weird named file on that appears in under the XHR tab contains our JSONs!

[{1492502400: [{day: 1, sessionTitle: "Registration", sessionSlug: "day-1-registration",…},…],…}, {,…}]

0:{1492502400: [{day: 1, sessionTitle: "Registration", sessionSlug: "day-1-registration",…},…],…}

1:{,…}

This is how the file looks on clicking it at first. You must be seeing small clickable arrows on the sides which expand the JSON to reveal all the contents. But okay, we have crossed our first step: Obtaining JSON from the website’s API!

Step 2: Tailoring the JSON according to the Open Event format needs to be done now. To view the latest API format, visit here. Now this step is obviously most time-consuming and tedious. We need to modify the API we obtained from the F8 site to meet the Open Event requirements.

I mainly used Sublime text editor to do this. It has this great feature to find and replace text in text files.

Taking an example of a session object. This is what we will get from the API:-

allDay:false

createdAt:"2017–01–28T01:24:12.770Z"

day:1

displayTime:"3:00pm"

endTime:{__type: "Date", iso: "2017–04–18T15:50:00.000Z"}

iso:"2017–04–18T15:50:00.000Z"

__type:"Date"

hasDetails:true

objectId:"62yCeJMvAf"

ogImage:{__type: "File", name: "296643cca9aad196bbd3534324c52049_Making Great Camera Effects.png",…}

name:"296643cca9aad196bbd3534324c52049_Making Great Camera Effects.png"

url:"https://f82017-parse.s3.amazonaws.com/296643cca9aad196bbd3534324c52049_Making%20Great%20Camera%20Effects.png"

__type:"File"

onMySchedule:false

sessionDescription:"With AI that transforms images, Facebook's Camera Effects Platform lets you create animated masks, interactive overlays, frames, and other effects that people can apply to their photos and videos in the Facebook camera. Learn best technical and design practices for creating effects using this innovative technology, and see examples of successful effects."

sessionLocation:"G"

sessionSlug:"making-great-camera-effects"

sessionTitle:"Making Great Camera Effects"

sortTime:"1492527600"

speakers:[{speakerName: "Volodymyr Giginiak", createdAt: "2017–01–28T01:27:49.292Z",…},…]

0:{speakerName: "Volodymyr Giginiak", createdAt: "2017–01–28T01:27:49.292Z",…}

1:{speakerName: "Stef Smet", createdAt: "2017–01–28T01:30:16.022Z",…}

startTime:{__type: "Date", iso: "2017–04–18T15:00:00.000Z"}

iso:"2017–04–18T15:00:00.000Z"

__type:"Date"

tags:["Camera"]

0:"Camera"

updatedAt:"2017–04–18T19:19:16.501Z"

Now, this is where I used the find & replace feature in Sublime text to make it look like the open-event standard JSON. For instance, replacing “sessionLocation” with “microlocation”, “sessionDescription” with “long_abstract” and so on. Yes, there were a few manual changes as well that I had to make.

This is how the final JSON session object looked like after completing step 2:

{
     "short_abstract": "",
     "id": 26,
     "subtitle": "making-great-camera-effects",
     "title": "Making Great Camera Effects",
     "long_abstract": "With AI that transforms images, Facebook's Camera Effects Platform lets you create animated masks, interactive overlays, frames, and other effects that people can apply to their photos and videos in the Facebook camera. Learn best technical and design practices for creating effects using this innovative technology, and see examples of successful effects.",
     "end_time": "2017-04-18T15:50:00-07:00",
     "microlocation": {
        "id": 5,
        "name": "Hall G"
     },
     "start_time": "2017-04-18T15:00:00-07:00",
     "comments": "",
     "language": "",
     "slides": "",
     "audio": "",
     "video": "",
     "signup_url": "",
     "state": "accepted",
     "session_type": {
        "id": 6,
        "name": "50 minutes session",
        "length": "00.50"
     },
     "track": {
        "id": 2,
        "name": "Camera",
        "color": "#FF3046"
     },
     "speakers": [{
           "city": "",
           "heard_from": "",
           "icon": "",
           "long_biography": "",
           "small": "",
           "photo": "",
           "thumbnail": "",
           "short_biography": "",
           "speaking_experience": "",
           "sponsorship_required": "",
           "organisation": "Facebook",
           "id": 40,
           "name": "Volodymyr Giginiak"
        }
     ]
  }

Apart from this, there were places where I had to dig for information about speakers, their social website links, their images and so on because the open-event JSON format expects all this.

Step 3: JSON Validation

It is very likely that one of your JSON files will contain invalid JSON strings. So it is important to validate them. I used this online tool and I liked it very much. Once the JSON is validated, everything is set. All you need to now do is-move to step 4!

Step 4: Web app generation

Compress your JSON archive in .zip format and then go to the webapp generator. Enter your email ID and upload the zip file. Now, you can deploy, download and preview your personalised web-app, generated by FOSSASIA open-event web app generator.

The official web-app generator repo is here.

If you want to contribute to FOSSASIA by creating a sample, you can create an issue in the official open-event repo, along with a PR which contains the JSON sample files, the zipped file and a link to the web-app deployment on GitHub pages.

Continue ReadingCreating A Sample Event Web App Using the Open Event Framework

Implementing Session Bookmark Feature in Open Event Webapp

The event websites generated by Open Event Webapp can be large in size with a massive number of sessions in it. As such, it becomes a little difficult for the user to keep track of the sessions he is interested in without any bookmark facility.  With the help of it, he/she can easily mark a session as his favorite and then see them all at once at the click of the button. This feature is available and synchronized across all the pages (track, rooms, and schedule) of the event. That is, if you bookmark a session on the tracks page, it would automatically appear on the other pages too. Likewise, if you unmark a session on any of the page, it would be reflected on rest of the pages.

The event websites generated by the webapp are static in nature. It means that there is no backend server and database where the data about the bookmarked sessions and the user can be stored. So how are we storing information about the marked sessions and persisting it across different browser sessions? There must be a place where we are storing the data! Yes. It’s the LocalStorage Object. HTML5 introduced a localStorage object for storing information which persists even when the site or the browser is closed and opened again. Also, the max size of the localStorage object is quite sufficient (around 5 – 10 MB) for storing textual data and should not be a bottleneck in the majority of the cases.

One very important thing to remember about the localStorage object is that it only supports strings in key and value pairs. Hence, we can’t store any other data type like Boolean, Number or Date directly in it. We would have to convert the concerned object into a string using the JSON.stringify() method and then store it.

Okay, let’s see how we implement the bookmark facility of sessions. We initialize the localStorage object. We create a key in it which is the name of the event and corresponding to its value, we store an object which will contain the information about the bookmarked sessions of that particular event. It’s key will be the id of the session and the value (0 or 1) will define whether it is bookmarked or not. Initially, when no sessions have been bookmarked, it will be empty.

function initPage() {
 //Check whether the localStorage object is already instantiated

 if(localStorage.hasOwnProperty(eventName) === false) {
   localStorage[eventName] = '{}'; // This object will store the bookmarked sessions info
 }
}

Now, we need to actually provide a button to the user for marking the sessions and then link it to the localStorage object. This is the session element with the star-shaped bookmark button

b77bf0d9-7303-4c03-ae29-bd8a4725002b.png

Here is the related code

<a class="bookmark" id="b{{session_id}}">
 <i class="fa fa-star" aria-hidden="true">
 </i>
</a>

Then we need to handle the click event which occurs when the user clicks this button. We will extract the id of the session and then store this info in the event object inside the localStorage object.

$('.bookmark').click(function (event) {
   // Get the event object from the localStorage
   var temp = JSON.parse(localStorage[eventName]);

   // Get the current color of the bookmark icon. If it's black, it means that the session is already bookmarked.
   var curColor = $(this).css("color");

   // Extract the id of the session
   var id = $(this).attr('id').substring(1);

   if (curColor == "rgb(0, 0, 0)") {
     // Unmark the session and revert the font-icon color to default
     temp[id] = 0;
     $(this).css("color", "");
   } else {
     // Bookmark the session and color the font icon black
     temp[id] = 1;
     $(this).css("color", "black");
   }

   localStorage[eventName] = JSON.stringify(temp);
   event.stopPropagation();
 });

So suppose, our event name is FOSSASIA Summit and we bookmarked sessions with the id of 2898 and 3100 respectively, then the structure of the localStorage object will be something like this:-

{'FOSSASIA Summit': { '2898': '1', '3100': '1'}};

To finally show all the bookmarked sessions at once, we defined two modes of the page: Starred and All. In the Starred mode, only the bookmarked sessions are shown. We simply select the event object inside the localStorage and iterate over all the session id’s and only show those which have their value set to 1. In the latter mode, all the sessions are displayed.

Starred Mode (Showing only the bookmarked sessions)

ca1f5fa8-29a2-4b96-8eb1-b22d8b07bda8.png

All Mode (Every session is displayed, both marked and unmarked)

cd23f5f5-4bd1-4d1e-aa96-f4aa28cb5e07.png

References:

Continue ReadingImplementing Session Bookmark Feature in Open Event Webapp