Enhancing network requests by chaining or zipping with RxJava

In Eventyay Attendee, making HTTP requests to fetch data from the API is one of the most basic technique used. RxJava comes in as a great method to help us making asynchronous requests and optimize the code a lot. This blog post will deliver some advanced RxJava used in Eventyay Attendee.

  • Why using RxJava?
  • Advanced RxJava Technique – Chaining network calls with RxJava
  • Advanced RxJava Technique – Merging network calls with RxJava
  • Conclusions
  • Resources

WHY USING RXJAVA?

There are many reasons why RxJava is a great API in Android Development. RxJava is an elegant solution to control data flow in programming, where developers can cache data, get data, update the UI after getting the data, handle asynchronous tasks. RxJava also works really well with MVVM architectural pattern.

CHAINING NETWORK CALLS WITH RXJAVA

Chaininig RxJava is a technique using flatMap() operator of Rxjava. It will use the result from one network call in order to make the next network call. 

In Eventyay Attendee, this technique is used when we want to update the user profile image. First, we need to upload the new profile image to the server in order to get the image URL, and then we use that URL to update the user profile

compositeDisposable += authService.uploadImage(UploadImage(encodedImage)).flatMap {
  authService.updateUser(user.copy(avatarUrl = it.url))
}.withDefaultSchedulers()
  .doOnSubscribe {
      mutableProgress.value = true
  }
  .doFinally {
      mutableProgress.value = false
  }
  .subscribe({
      mutableMessage.value = resource.getString(R.string.user_update_success_message)
      Timber.d(“User updated”)
  }) {
      mutableMessage.value = resource.getString(R.string.user_update_error_message)
      Timber.e(it, “Error updating user!”)
  }

In conclusion, zipping RxJava helps to make HTTP requests more continuous and reduce unnecessary codes. 

ZIPPING NETWORK CALLS WITH RXJAVA

Zipping RxJava is a technique using zip() operator of Rxjava. It will wait for items from two or more Observables to arrive and then merge them together for emitting. This technique would be useful when two observables emit the same type of data.

In Eventyay Attendee, this technique is used when fetching similar events by merging events in the same location and merging events in the same event type.

var similarEventsFlowable = eventService.getEventsByLocationPaged(location, requestedPage, 3)
if (topicId != -1L) {
  similarEventsFlowable = similarEventsFlowable
      .zipWith(eventService.getSimilarEventsPaged(topicId, requestedPage, 3),
          BiFunction { firstList: List<Event>, secondList: List<Event> ->
              val similarList = mutableSetOf<Event>()
              similarList.addAll(firstList + secondList)
              similarList.toList()
          })
}

compositeDisposable += similarEventsFlowable
  .take(1)
  .withDefaultSchedulers()
  .subscribe({ response ->
      …
  }, { error ->
      …
  })

In conclusion, zipping RxJava helps running all the tasks in parallel and return all of the results in a single callback.

CONCLUSION

Even though RxJava is pretty hard to understand and master, it is a really powerful tool in Android Development and MVVM models. These techniques above are really simple to implement and they could improve the app by r

RESOURCES

Eventyay Attendee Source Code: 

https://github.com/fossasia/open-event-attendee-android/pull/2010

https://github.com/fossasia/open-event-attendee-android/pull/2117

RxJava Documentation: http://reactivex.io/documentation

Continue Reading

Building PSLab Desktop Apps using JavaScript, Python  and Electron

So before we get started let me quickly show you what we accomplished over a period of 2 and a half months building the Pocket Science Lab desktop app using the very technology I am going to talk about.

The PSLab desktop app was originally written using pyQt. The UI of the python stack looked really outdated and maintaining it was becoming a problem due to poor developer support. So, after a lot of discussion and experimentation, we finally decided to re-implement the whole app in electronJS. There are three major benefits of writing an app with Electron:

  • You can re-use your HTML, CSS knowledge to write the UI part in no time.
  • You can re-use your JavaScript knowledge to write the business logic of the app.
  • It works…

Why?

Before we even dive down to why use X and why use Y, let us try to answer a very basic question, The reason I had to invest a massive amount of time trying to experiment with different combinations and config parameters is because most of the blogs on the internet just wrote about “How to get started”, nobody talked about how to complete the build. And that is what I am going to cover. Anyone can figure out how to start out but  the real grind only begins when you get to the details of the app. But I hope to hit the escape velocity with my take on the subject.

The Tech Stack — Why we chose Electron for PSLab Desktop app

Now let’s talk about why we need React or Python in an Electron App.

1. React

It is perfectly fine to use everything minimal, plain old HTML, CSS and JS will do just fine for your UI, but for how long? React opens up so many possibilities and makes the code logic flow naturally through a very well designed UI language. The state management system is also brilliant and the most crucial thing that will definitely make your life easier is the life cycle methods offered by React class components. Making use of react also lets you take advantage of well known React oriented libraries like Material UI React, and Styled-Components. These things are game changers and can transform your UI to an absolute gem that your users will enjoy.

Oh, by the way, React UI is blazing fast…

But that is not the best part, we will not configure React on our own, but rather let the crowd favorite CRA do it for us. CRA (Create-React-App) will not only abstract out all the webpack config files, but will also make sure your react dependencies stay updated. This will help in keeping your code up to date in the long run.

2. Python

The task that is being performed by Python in our case was inevitable and could not be performed by JavaScript because PSLab only had Python libraries. We did not have the time to write everything again for JS so we figured a way to make use of Python in the electron app. If you can do everything with JS and have the necessary libraries to do so, then you don’t need this. For example, if you need to predict some value using a model you trained via TensorFlow, then this would be one such use case.

Bear in mind that this could be extended to any language as long as you get your hands on a library that can deal with the communication.

3. Electron

In short, it gives you the platform to make use of the above two. It becomes more interesting when we go into the details of how electron works. But for now, let us just say, it will provide you the space on which react will render the UI, it will provide the means for your UI to talk to the Python code, it will also provide APIs to talk to native system features like filesystem, notifications etc.

Resources:How to build an Electron app using create-react-app. No webpack configuration or “ejecting” necessary. Christian Sepulveda Christian Sepulveda: https://medium.com/free-code-camp/building-an-electron-application-with-create-react-app-97945861647c

Continue Reading

Two flavors of PSLab Android App to support Google Maps (in Play Store flavor) and Open Street maps (in Fdroid flavor)

What are the flavors of an App? And why are they needed in PSLab Android App?

While working on the PSLab Android Project, I ran into the need to create different variants of the app with different dependencies. In this blog, I have tried to explain the process of creating various flavors of the app in the easiest way possible. 

Android Allows Developers to create different variants of the same app with the same code base but having some functionalities different across the variants. These functionalities may include some special/pro features, some different dependencies, etc. Such variants are called flavors of the App. Most common flavors are Paid and Free version of the app.

In the PSLab Android Application, we needed to generate flavors, when we required to use Google Maps in the App. The app is also published on the Fdroid, which doesn’t allow dependencies of Google Maps. Hence 2 flavors of the app have been created, 

  1. Play Store Flavor (With Google Maps)
  2. F-Droid Flavor (With Open Street Maps)

Declaring Flavors in the build.gradle File

In order to create flavors of the app, first, we need to declare flavors in the Gradle file. In PSLab Android app we are creating 2 flavors, which are declared in the build.gradle file as under

flavorDimensions 'default'
productFlavors {
   fdroid {
       dimension = 'default'
   }
   playstore {
       dimension = 'default'
   }
}

flavorDimensions is used to package flavors if there are many flavors for an App. Since we have only two flavors fdroid and playstore, hence we are using single dimension default for both the flavors. Once this has been added to the build.gradle file we need to sync the gradle. 

After the Sync is complete, if we open the Build Variants tab from the left corner of the Android Studio, it would look something like this: 

(Figure 1: Build Variant Window of Android Studio)

As can be seen in the screenshot above, once the gradle is successfully synced, Android Studio automatically creates debug and release build variants for each flavor and we can easily toggle between variants and build/ run / make apk for each variant. Congratulations! We have successfully finished the first step towards creating flavors of an app.

Directory Structure after creating Flavors

Apart from creating the build variants of different flavors, Android studio also creates src/<flavor name> folders for us. Now if we want to add new activities and classes to these flavors we can create java, res, values folders inside this folder. We can define separate Manifest file as well for each flavor individually. The directory structure of the PSLab Android project after creating required packages inside the automatically generated src/fdroid and src/playstore folders looks like below,  

(Figure 2: Directory structure after creating flavors)

Defining Flavor specific dependencies

We can have some dependencies for one app flavor and some for others. For example, in PSLab Android app, we need Google Maps dependencies only in playstore flavor and Open Street Maps dependencies only in fdroid flavor. We can easily define flavor specific dependencies by adding flavor name before Implementation command in gradle file. Like below,

// Map libraries
fdroidImplementation "org.osmdroid:osmdroid-android:$rootProject.osmVersion"
fdroidImplementation "org.osmdroid:osmdroid-mapsforge:$rootProject.mapsforgeVersion"
fdroidImplementation "org.osmdroid:osmdroid-geopackage:$rootProject.geoPackageVersion"
playstoreImplementation "com.google.android.gms:play-services-maps:$rootProject.googleMapsVersion"

Same Activity/Class with different Flavor 

Now the main purpose of creating flavors is to have some different functionalities between the flavors. For that we need the base app to call different class/activity from the src/<flavor name> folder depending on the selected flavor. We will discuss this in reference to PSLab Android app. 

So, for PSLab android app we want app to open Google Maps in Play Store flavor and Open Street Maps in froid flavor. For this we need to create a duplicate Activity. Which means we will have two separate implementation of same Activity MapsActivity.java , one in the F-Droid source folder and one in playstore source folder. So MapsActivity.java will only be declared once in the src/main/AndroidManifest.xml file, but there will be two different classes for this activity in each flavor folder. Now when the main app will call MapsActivity.class from any intent depending on the selected build variant either playstore version of MapsActivity will be launched or the F-Droid version. So after creating two instances of the MapsActivity.java the directory structure would look something like given in the screenshot below,

(Figure 3: Directory structure after creating MapsActivity)

As can be seen in the directory structure, now both Play Store and F-Droid folders have their own instances of MapsActivity.java , and now we can easily implement code for Open Street Maps and Google Maps in the respective MapsActivity.java and we have two versions of the app working flawlessly. 

References

Tags: GSoC ‘19, PSLab, Android, Flavors, GoogleMaps, OpenStreetMaps, Build Variants

Continue Reading

Implementing Stripe payment in Eventyay Attendee

In Eventyay Attendee, getting tickets for events has always been a core function that we focus on. When searching for events based on location, autosuggestion based on user input really comes out as a great feature to increase the user experience. Let’s take a look at the implementation

  • Why using Stripe?
  • Implementing Stripe Payment in Eventyay Attendee
  • Conclusion
  • Resources

WHY USING STRIPE?

There are many great APIs to be taken into consideration for making payments but we choose Stripe as one of our payment gateways because of simple implementations, detailed documentation, a good number of supported card type and good security support

IMPLEMENTING STRIPE PAYMENT IN EVENTYAY ATTENDEE

Step 1: Setup dependency in the build.gradle

// Stripe
implementation ‘com.stripe:stripe-android:10.3.0’

Step 2: Set up UI to take card information

The information needed for making payments are Card Number, CVC, Expiration Date, which can be made with simple UI (EditText, Spinner,…). Stripe support getting information with CardInputWidget but we made a custom UI for that. Here is the UI we created.

Step 3: Create a card and validate information

Stripe has an object called Card, which takes card number, expiration date and CVC number as parameter to detect the card type and validate the card information with function .validateCard()

PAYMENT_MODE_STRIPE -> {
  card = Card.create(rootView.cardNumber.text.toString(), attendeeViewModel.monthSelectedPosition,
      rootView.year.selectedItem.toString().toInt(), rootView.cvc.text.toString())

  if (!card.validateCard()) {
      rootView.snackbar(getString(R.string.invalid_card_data_message))
      false
  } else {
      true
  }
}

Step 4: Send the token to the server

If card information is valid, we can create a token from the Card and then send it to the server. The token will act as the identifier of the card in order for the server to charge the payment and create tickets for the user. 

private fun sendToken(card: Card) {
  Stripe(requireContext())
      .createToken(card, BuildConfig.STRIPE_API_KEY, object : TokenCallback {
          override fun onSuccess(token: Token) {
              val charge = Charge(attendeeViewModel.getId().toInt(), token.id, null)
              attendeeViewModel.chargeOrder(charge)
          }
          override fun onError(error: Exception) {
              rootView.snackbar(error.localizedMessage.toString())
          }
      })
}

Step 5: So the rest is already handled by the server. Android application will then just receive the response from the server to see if the order is charged successfully or not.

CONCLUSION

With Stripe, user can easily make payments to get tickets for events. Stripe is a great payment gateway as it is really easy to implement in Android. Hopefully, this blog post will help you create a great shopping cart app or any kind of application that requires fast, simple and easy payments.

RESOURCES

Eventyay Attendee Pull Request on Stripe: https://github.com/fossasia/open-event-attendee-android/pull/1863

Documentation from Stripe for Android: https://stripe.com/docs/mobile/android


Continue Reading

Integrating Redux In Settings

Settings page in SUSI.AI earlier implemented using Flux needed to be migrated to Redux, Redux integration eases handling of global state management. With Redux being integrated with Settings, the code could be split into smaller manageable components. Earlier implementation involved passing data to other Settings Tab and using Settings component as a state source. This resulted in Settings component getting more lines of code(~1300 LOC). After using Redux, only ~280 LOC is present in settings component file, and separate files for each settings tab view. 

Code Integration

Once user login, the App component fetches User Settings in ComponentDidMount.

componentDidMount = () => {
   const { accessToken, actions } = this.props;

   window.addEventListener('offline', this.onUserOffline);
   window.addEventListener('online', this.onUserOnline);

   actions.getApiKeys();
   if (accessToken) {
     actions.getAdmin();
     actions.getUserSettings().catch(e => {
       console.log(e);
     });
   }
 };

src/App.js

The action getUserSettings fetch users settings from apis.getUserSettings and populates the store settings using SETTINGS_GET_USER_SETTINGS reducer function.

Default States:

The store consists of all the settings required for persisting data.

 const defaultState = {
 theme: 'light',
 server: 'https://api.susi.ai',
 enterAsSend: true,
 micInput: true,
 speechOutput: true,
 speechOutputAlways: false,
 speechRate: 1,
 speechPitch: 1,
 ttsLanguage: 'en-US',
 userName: '',
 prefLanguage: 'en-US',
 timeZone: 'UTC-02',
 customThemeValue: {
   header: '#4285f4',
   pane: '#f3f2f4',
   body: '#fff',
   composer: '#f3f2f4',
   textarea: '#fff',
   button: '#4285f4',
 },
 localStorage: true,
 countryCode: 'US',
 countryDialCode: '+1',
 phoneNo: '',
 checked: false,
 serverUrl: 'https://api.susi.ai',
 backgroundImage: '',
 messageBackgroundImage: '',
 avatarType: 'default',
 devices: {},
};

reducers/settings.js

Reducer action:

SETTINGS_GET_USER_SETTINGS sets the settings store from API payload. The customThemeValue is an object, which is populated once the settings are fetched and the string is split into array.

[actionTypes.SETTINGS_GET_USER_SETTINGS](state, { error, payload }) {
     const { settings, devices = {} } = payload;
     if (error || !settings) {
       return state;
     }

     const {
       theme = defaultState.theme,
       server,
       .
       avatarType,
     } = settings;
     let { customThemeValue } = settings;
     const themeArray = customThemeValue
       ? customThemeValue.split(',').map(value => `#${value}`)
       : defaultState.customThemeValue;
     return {
       ...state,
       devices,
       server,
       serverUrl,
       theme,
       enterAsSend: enterAsSend === 'true',
       micInput: micInput === 'true',
       speechOutput: speechOutput === 'true',
       speechOutputAlways: speechOutputAlways === 'true',
       speechRate: Number(speechRate),
       speechPitch: Number(speechPitch),
       ttsLanguage,
       userName,
       prefLanguage,
       timeZone,
       countryCode,
       countryDialCode,
       phoneNo,
       checked: checked === 'true',
       backgroundImage,
       messageBackgroundImage,
       avatarType,
       customThemeValue: {
         header: themeArray[0],
         pane: themeArray[1],
         body: themeArray[2],
         composer: themeArray[3],
         textarea: themeArray[4],
         button: themeArray[5],
       },
     };
   },

reducers/settings.js

Updating Settings

Reduce function:

The payload contains the updated state, the reducer returns a new state with payload appended.

setUserSettings: createAction(
   actionTypes.SETTINGS_SET_USER_SETTINGS,
   returnArgumentsFn,
 )

reducers/settings.js

Action:

The setUserSettings action updates the settings store, once the user presses the Save Button in the Settings Tab.

function mapStateToProps(store) {
 return {
   enterAsSend: store.settings.enterAsSend,
 };
}

actions/settings.js

Redux integration in the component

Redux is used in ChatAppTab as: The local state of ChatAppTab is initialized with redux store state. Using mapStateToProps we can access enterAsSend object from the redux store.

The connect function connects the React component with Redux store and injects props and actions into our component using HOC pattern. It injects data from the store and functions for dispatching actions to store.

Injecting props in local components:

function mapStateToProps(store) {
 return {
   enterAsSend: store.settings.enterAsSend,
 };
}

components/Settings/ChatAppTab.react.js

Once a user makes a change in the tab and saves it, we need to dispatch an action to modify our prop enterAsSend inside the redux store. 

function mapDispatchToProps(dispatch) {
 return {
   actions: bindActionCreators({ ...settingActions, ...uiActions }, dispatch),
 };
}

components/Settings/ChatAppTab.react.j

handleSubmit = () => {
   const { actions } = this.props;
   const { enterAsSend } = this.state;
   this.setState({ loading: true });
   setUserSettings({ enterAsSend })
     .then(data => {
       if (data.accepted) {
         actions.openSnackBar({
           snackBarMessage: 'Settings updated',
         });
         actions.setUserSettings({ enterAsSend });
         this.setState({ loading: false });
       } else {
         actions.openSnackBar({
           snackBarMessage: 'Failed to save Settings',
         });
         this.setState({ loading: false });
       }
     })
     .catch(error => {
       actions.openSnackBar({
         snackBarMessage: 'Failed to save Settings',
       });
     });
 };

components/Settings/ChatAppTab.react.js

handleSubmit is invoked when user presses the Save Button, action.setUserSettings is dispatched once API returns accepted: true response.

To conclude, the removal of flux and integration of redux gave the advantage of making the code more modular and separate the logic into smaller files. In the future, more settings can be easily added using the same data flow pattern.

Resources

Tags

SUSI.AI, FOSSASIA, GSoC19, Redux, SUSI Chat, Settings

Continue Reading

The Robotic Arm Controller Feature in PSLab Android Application.

Recently while working on PSLab Android Project, a new feature was implemented to add a controller for a Robotic Arm in the Android Application. In this blog, I will explain what this new feature is, how it has been implemented and what are some of the functionalities of it.

What is Robotic Arm?

Robotic Arm, as the name suggests is a small arm-like structure, which moves with the help of 4 servo motors connected to it. The image of the robotic arm is as under, 

(Figure 1 : Robotic Arm)

What is the Robotic Arm Controller Feature in Android App?

As mentioned earlier, the robotic arm uses 4 servos for the movements. The aim of the controller feature in the Android app is to allow users to adjust the rotation of each servo with a very intuitive UI, so they can move the robotic arm as they wish. Further, in this blog, I will discuss UI, implementation and some cool features of the robotic Arm Controller.

User Interface


(figure 2: UI of the robotic arm controller)

In the screenshot above, there are 4 circular controllers for each respective servo. The user can use the knob or enter the values using the keyboard by tapping on the value at the center of the knobs. Each value indicates the value in degrees the user wishes to move that servo.

Timeline

Below the 4 servo controllers is a black timeline. There are 60 boxes in each of the 4 timelines. These 60 boxes indicates the seconds. So basically if the user wants the robotic arm to perform some set of actions sequentially, using these timelines, users can set the rotation for each servo at each second, then user can use the play button on the red control panel to play this timeline and the app will send the degree value at each second to the respective servo. A filled timeline would look something like below, 


(Figure 3: Timeline)

As can be seen, there are different values for each servo at each second, so when the user starts the timeline, values at each second will be sent to the respective servos. servo4() function of the ScienceLab class is called to set values for all for servos at each second.

How to add the values to the timeline?

There is a small handle on the top right corner of each servo controller, user can long-press the handle and drag and drop the values to desired seconds for respective servos.  This functionality can be found in this video between 0:33 to 0:43

This feature uses Android’s drag and drops listener. The code for this drag and drop function is as under,

private View.OnDragListener servo1DragListener = new View.OnDragListener() {
        @Override
        public boolean onDrag(View v, DragEvent event) {
            if (event.getAction() == DragEvent.ACTION_DRAG_ENTERED) {
                View view = (View) event.getLocalState();
                TextView text = view.findViewById(R.id.degreeText);
                if (view.getId() == R.id.servo_1) {
                    ((TextView) v.findViewById(R.id.timeline_box_degree_text)).setText(text.getText());
                }
            }
            return true;
        }
    };

Such drag listeners are created for each servo controller.

Save Timeline Feature

Suppose user has set a whole 60 seconds timeline for some action, and the user wants to use that set of values again and again, it doesn’t make sense if the user has to set the values every time. So there is a feature to save the timeline as well. Whenever user has created some timeline, user can just click on the save button on the control panel and the timeline will be saved both as a CSV file and as a realm object. So the timeline will be visible in the DataLoggerActivity as well. Whenever user opens a logged timeline, values for the respective servo timeline will be set automatically.

The code to save the timeline is as below.  For each second a new ServoData object is created with degree values for all 4 servos at that second.

private void saveTimeline() {
        long block = System.currentTimeMillis();
        servoCSVLogger.prepareLogFile();
        servoCSVLogger.writeMetaData(getResources().getString(R.string.robotic_arm));
        String data = "Timestamp,DateTime,Servo1,Servo2,Servo3,Servo4,Latitude,Longitude\n";
        long timestamp;
        recordSensorDataBlockID(new SensorDataBlock(block, getString(R.string.robotic_arm)));

So first we create a block variable using the current timestamp, this block will be used for all the realm object stored for this timeline. We also prepare a CSV file and CSV header for the log. 

String degree1, degree2, degree3, degree4;
for (int i = 0; i < 60; i++) {
     timestamp = System.currentTimeMillis();
     degree1 = degree2 = degree3 = degree4 = "0";
     if (((TextView) servo1TimeLine.getChildAt(i).findViewById(R.id.timeline_box_degree_text)).getText().length() > 0) {
         degree1 = ((TextView) servo1TimeLine.getChildAt(i).findViewById(R.id.timeline_box_degree_text)).getText().toString();
     }

Now, we run a for loop for each 60 second timeline and store the set value of each servo in degree1, degree2, degree3 and degree4 variables. In the above code snippet only degree1 is shown, but the same thing is done for other values as well. Once we have 4 degree values for each servo at for a second, we store it as an ServoData Object in realm and also write it to the CSV file using the following lines of code,

recordSensorData(new ServoData(timestamp, block, degree1, degree2, degree3, degree4, lat, lon);
servoCSVLogger.writeCSVFile(data);

Here the data variable is a string with comma-separated values of the degree values of each servo.

Once the user saves the timeline the generated CSV looks something like below,


(Figure 4: Saved timeline CSV)

To set the timeline from the saved logged data, a small function just iterates over all the ServoData objects stored in the realm and set the value for the respective servo in the timeline. The function is as under,

private void setReceivedData() {
        ArrayList servoDataList = new ArrayList(recordedServoData);
        for (int i = 0; i < servoDataList.size(); i++) {
            ServoData servoData = (ServoData) servoDataList.get(i);
            ((TextView) servo1TimeLine.getChildAt(i).findViewById(R.id.timeline_box_degree_text)).setText(servoData.getDegree1());
            ((TextView) servo2TimeLine.getChildAt(i).findViewById(R.id.timeline_box_degree_text)).setText(servoData.getDegree2());
            ((TextView) servo3TimeLine.getChildAt(i).findViewById(R.id.timeline_box_degree_text)).setText(servoData.getDegree3());
            ((TextView) servo4TimeLine.getChildAt(i).findViewById(R.id.timeline_box_degree_text)).setText(servoData.getDegree4());
        }
    }

Conclusion

This feature enables the user to control the robotic arm with a very intuitive user interface. And the save timeline feature allows the user to use/share already stored timeline for some actions with other users. 

A small video to explain the save timeline functionality can be seen below,

References

Tags : GSoC ‘19, PSLab, Android, Robotic Arm

Continue Reading

Enhancing User Security With Server-Side Validation

Server-side validation of SUSI.AI is required for increasing the security of user against malicious users who can bypass client-side validations easily and submit dangerous input to SUSI.AI server. SUSI.AI uses both client-side and server-side validation for various authentication purposes.

Client-side validates user input on the browser and provides a better user interaction. If the form is disabled on incorrect fields, it would save a round trip to the server, so the network traffic which will help your server perform better. Using client-side validation, an error message can be immediately shown before the user moves to the next field.

Adding Recaptcha while on new registration, change password, and more than one login attempts help to reduce abusive traffic, spam, brute force attack. For making user accounts safe from getting hacked on SUSI.AI, a strong password is supposed to be at least 8 characters, at least one special character, one number and text characters are must. The username must have 5-51 characters. So as to prevent SUSI.AI admin panel from crashing.

Code Integration

Client side reCaptcha validation:

onCaptchaSuccess = captchaResponse => {
  if (captchaResponse) {
    this.setState({
      showCaptchaErrorMessage: false,
      captchaResponse,
    });
  }
};

Auth/Login/Login.react.js

Once the user verifies captcha on client-side, a callback onCaptchaSuccess fires which receives a parameter captchaResponse. This captchaResponse is sent over to server-side with key g-recaptcha-response

Validation for User Login, ReCaptcha needs to be shown if the user has tried to log in for more than 1 time. For this, sessionStorage needs to be maintained. sessionStorage has an expiration time, which expires once the user closes the tab.

In the constructor of Login, the attempt is initialized with sessionStorage using key loginAttempts. Now, we have the correct value of loginAttempts and can update it further if the user tries to make more attempt.

constructor(props) {
    super(props);
    this.state = {
      ...
      showCaptchaErrorMessage: false,
      attempts: sessionStorage.getItem('loginAttempts') || 0,
      captchaResponse: '',
    };
  }

Auth/Login/Login.react.js

If the user closes the login dialog, we need to store the current attempts state on componentWillUnmount

componentWillUnmount() {
  sessionStorage.setItem('loginAttempts', this.state.attempts);
}

Auth/Login/Login.react.js

If attempts are less than 1, render the reCaptcha component and send g-recaptcha-response parameter to api.susi.ai/aaa/login.json API endpoint.

Server-side reCaptcha Validation for user login

The checkOneOrMoreInvalidLogins function in LoginService checks if the user has attempted to login in more than once. Access the accounting object and get identity and store it in accounting.

private boolean checkOneOrMoreInvalidLogins(Query post, Authorization authorization, JsonObjectWithDefault permissions) throws APIException {
	Accounting accounting = DAO.getAccounting(authorization.getIdentity());
	JSONObject invalidLogins = accouting.getRequests().getRequests(this.getClass().getCanonicalName());
	long period = permissions.getLong("periodSeconds", 600) * 1000; // get time period in which wrong logins are counted (e.g. the last 10 minutes)
	int counter = 0;
	for(String key : invalidLogins.keySet()){
		if(Long.parseLong(key, 10) > System.currentTimeMillis() - period) counter++;
	}
	if(counter > 0){
		return true;
	}
	return false;
}

aaa/LoginService.java

If user has attempted to login in more than once, get from g-recaptcha-response API parameter. Check if reCaptcha is verified using VerifyRecaptcha utility class.

if(checkOneOrMoreInvalidLogins(post, authorization, permissions) ){
	String gRecaptchaResponse = post.get("g-recaptcha-response", null);
	boolean isRecaptchaVerified = VerifyRecaptcha.verify(gRecaptchaResponse);
	if(!isRecaptchaVerified){
		result.put("message", "Please verify recaptcha");
		result.put("accepted", false);
		return new ServiceResponse(result);
	}
}

aaa/LoginService.java

The VerifyRecaptcha class provides with a verify method, which can be called to check if user response if correct or not. The secretKey is key given for communication between server-side and google.

URL is created and using InputStream, content(res) is directly read from a URL using openStream() function provided. The bufferedReader reads the contents line by line(using cp), and rd.read() returns a string. The result is appended to stringBuilder sb until -1 is read.

res.close() closes the stream and releases the resources for URL reading. JSONObject is created using the string created from URL, and value mapped to success is used to check if ReCaptcha is verified. If the response is correct, it returns true.

public class VerifyRecaptcha {
	public static boolean verify(String response) {
		try {
			String url = "https://www.google.com/recaptcha/api/siteverify?"
					+ "secret=6LfPZGAUAAAAAAULjaq7Rt9-7IGJoLYoz2Di6yVV&response=" + response;
			InputStream res = new URL(url).openStream();
			BufferedReader rd = new BufferedReader(new InputStreamReader(res, Charset.forName("UTF-8")));

			StringBuilder sb = new StringBuilder();
			int cp;
			while ((cp = rd.read()) != -1) {
				sb.append((char) cp);
			}
			String jsonText = sb.toString();
			res.close();

			JSONObject json = new JSONObject(jsonText);
			return json.getBoolean("success");
		} catch (Exception e) {
			e.printStackTrace();
		}
		return false;
	}
}

tools/VerifyRecaptcha.java

User name validation on server side

Changing regex in config: Regular express for username to have 5-51 characters. Setting regex and tooltip in configs:

users.username.regex=^(.{5,51})$

users.username.regex.tooltip=Enter atleast 5 character, upto 51 character

users.username.regex=^(.{5,51})$
users.username.regex.tooltip=Enter atleast 5 character, upto 51 character

conf/config.properties

If a user tries to add userName more than 51 characters or less than 5 characters, 

The regex is compiled using Pattern.compile and then matched against the value provided by the user in API parameter. If the pattern matches, it means the user provided the correct userName.

if(possibleKeys[i].equals("userName") && value != null){
    String usernamePattern = DAO.getConfig("users.username.regex", "^(.{5,51})");
    String usernamePatternTooltip = DAO.getConfig("users.username.regex.tooltip",
    "Enter atleast 5 character, upto 51 character");
    Pattern pattern = Pattern.compile(usernamePattern);
    if(!pattern.matcher(value).matches()) {
        throw new APIException(400, usernamePatternTooltip);
    }
}

aaa/ChangeUserSettings.java

With server-side validation incorporated, the SUSI.AI users accounts are much safer than only with client-side validation. Validation over both client-side and server-side complement each, server-side validation being much more robust.

Resources

Tags

SUSI.AI, FOSSASIA, GSoC19, SUSI.AI Server

Continue Reading

Implementing pagination with Retrofit in Eventyay Attendee

Pagination (Paging) is a common and powerful technique in Android Development when making HTTP requests or fetching data from the database. Eventyay Attendee has found many situations where data binding comes in as a great solution for our network calls with Retrofit. Let’s take a look at this technique.

  • Problems without Pagination in Android Development
  • Implementing Pagination with Kotlin with Retrofit
  • Results and GIF
  • Conclusions

PROBLEMS WITHOUT DATABINDING IN ANDROID DEVELOPMENT

Making HTTP requests to fetch data from the API is a basic work in any kind of application. With the mobile application, network data usage management is an important factor that affects the loading performance of the app. Without paging, all of the data are fetched even though most of them are not displayed on the screen. Pagination is a technique to load all the data in pages of limited items, which is much more efficient

IMPLEMENTING DATABINDING IN FRAGMENT VIEW

Step 1:  Set up dependency in build.gradle

// Paging
implementation “androidx.paging:paging-runtime:$paging_version”
implementation “androidx.paging:paging-rxjava2:$paging_version”

Step 2:  Set up retrofit to fetch events from the API

@GET(“events?include=event-sub-topic,event-topic,event-type”)
fun searchEventsPaged(
  @Query(“sort”) sort: String,
  @Query(“filter”) eventName: String,
  @Query(“page[number]”) page: Int,
  @Query(“page[size]”) pageSize: Int = 5
): Single<List<Event>>

Step 3: Set up the DataSource

DataSource is a base class for loading data in the paging library from Android. In Eventyay, we use PageKeyedDataSource. It will fetch the data based on the number of pages and items per page with our default parameters. With PageKeyedDataSource, three main functions loadInitial(), loadBefore(), loadAfter() are used to to load each chunks of data.

class EventsDataSource(
  private val eventService: EventService,
  private val compositeDisposable: CompositeDisposable,
  private val query: String?,
  private val mutableProgress: MutableLiveData<Boolean>

) : PageKeyedDataSource<Int, Event>() {
  override fun loadInitial(
      params: LoadInitialParams<Int>,
      callback: LoadInitialCallback<Int, Event>
  ) {
      createObservable(1, 2, callback, null)
  }

  override fun loadAfter(params: LoadParams<Int>, callback: LoadCallback<Int, Event>) {
      val page = params.key
      createObservable(page, page + 1, null, callback)
  }

  override fun loadBefore(params: LoadParams<Int>, callback: LoadCallback<Int, Event>) {
      val page = params.key
      createObservable(page, page – 1, null, callback)
  }

  private fun createObservable(
      requestedPage: Int,
      adjacentPage: Int,
      initialCallback: LoadInitialCallback<Int, Event>?,
      callback: LoadCallback<Int, Event>?
  ) {
      compositeDisposable +=
          eventService.getEventsByLocationPaged(query, requestedPage)
              .withDefaultSchedulers()
              .subscribe({ response ->
                  if (response.isEmpty()) mutableProgress.value = false
                  initialCallback?.onResult(response, null, adjacentPage)
                  callback?.onResult(response, adjacentPage)
              }, { error ->
                  Timber.e(error, “Fail on fetching page of events”)
              }
          )
  }
}

Step 4: Set up the Data Source Factory

DataSourceFactory is the class responsible for creating DataSource object so that we can create PagedList (A type of List used for paging) for events.

class EventsDataSourceFactory(
  private val compositeDisposable: CompositeDisposable,
  private val eventService: EventService,
  private val query: String?,
  private val mutableProgress: MutableLiveData<Boolean>
) : DataSource.Factory<Int, Event>() {
  override fun create(): DataSource<Int, Event> {
      return EventsDataSource(eventService, compositeDisposable, query, mutableProgress)
  }
}

Step 5: Adapt the current change to the ViewModel. 

Previously, events fetched in List<Event> Object are now should be turned into PagedList<Event>.

sourceFactory = EventsDataSourceFactory(
  compositeDisposable,
  eventService,
  mutableSavedLocation.value,
  mutableProgress
)
val eventPagedList = RxPagedListBuilder(sourceFactory, config)
  .setFetchScheduler(Schedulers.io())
  .buildObservable()
  .cache()

compositeDisposable += eventPagedList
  .subscribeOn(Schedulers.io())
  .observeOn(AndroidSchedulers.mainThread())
  .distinctUntilChanged()
  .doOnSubscribe {
      mutableProgress.value = true
  }.subscribe({
      val currentPagedEvents = mutablePagedEvents.value
      if (currentPagedEvents == null) {
          mutablePagedEvents.value = it
      } else {
          currentPagedEvents.addAll(it)
          mutablePagedEvents.value = currentPagedEvents
      }
  }, {
      Timber.e(it, “Error fetching events”)
      mutableMessage.value = resource.getString(R.string.error_fetching_events_message)
  })

Step 6: Turn ListAdapter into PagedListAdapter

PageListAdapter is basically the same ListAdapter to update the UI of the events item but specifically used for Pagination. In here, List objects can also be null.

class EventsListAdapter : PagedListAdapter<Event, EventViewHolder>(EventsDiffCallback()) {

  var onEventClick: EventClickListener? = null
  var onFavFabClick: FavoriteFabClickListener? = null
  var onHashtagClick: EventHashTagClickListener? = null

  override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): EventViewHolder {
      val binding = ItemCardEventsBinding.inflate(LayoutInflater.from(parent.context), parent, false)
      return EventViewHolder(binding)
  }

  override fun onBindViewHolder(holder: EventViewHolder, position: Int) {
      val event = getItem(position)
      if (event != null)
          holder.apply {
              bind(event, position)
              eventClickListener = onEventClick
              favFabClickListener = onFavFabClick
              hashTagClickListAdapter = onHashtagClick
          }
  }

  /**
    * The function to call when the adapter has to be cleared of items
    */
  fun clear() {
      this.submitList(null)
  }

AND HERE ARE THE RESULTS…

CONCLUSION

Databinding is the way to go when working with a complex UI in Android Development. This helps reducing boilerplate code and to increase the readability of the code and the performance of the UI. One problem with data-binding is that sometimes, it is pretty hard to debug with unhelpful log messages. Hopefully, you can empower your UI in your project now with data-binding. 

Pagination is the way to go for fetching items from the API and making infinite scrolling. This helps reduce network usage and improve the performance of Android applications. And that’s it. I hope you can make your application more powerful with pagination. 

RESOURCES

Open Event Codebase: https://github.com/fossasia/open-event-attendee-android/pull/2012

Documentation: https://developer.android.com/topic/libraries/architecture/paging/ 

Google Codelab: https://codelabs.developers.google.com/codelabs/android-paging/#0

Continue Reading

How to work with MPAndroidChart? – Neurolab Memory graph program mode

Memory graph mode

Overview

In Android App development, implementation of Android Charts play an integral part in developing database oriented apps, be it casual or professional apps.

Data Analysis plays an integral part in various sectors and industries in today’s world. In sports, e.g- cricket, we see the use of charts like bar charts, line charts in various stages of the game to present the game and score analysis.

In healthcare, we see the use of live real time graphs to show human health analysis like the heart rate, brain waves, etc. In the IT industry, professionals use graphical data analysis for their presentations in meetings and conferences.

I have been working with FOSSASIA on the Neurolab Android App, where I had to work on this unique program mode called “Memory Graph” wherein I had to work extensively with MPAndroidChart library. While working, I understood the various use cases of charts and graphs on the Android platform. In this blog, I want to showcase the building of one part of the app with a proper tutorial.

Tutorial

In the Android SDK, we, the developers have been given the benefit of a tool known as GraphView. But this view being a very default and basic tool in the toolkit, has limitations in terms of customization, needs a lot of boilerplate code to set up the base to work upon.

Also plotting real time graphs becomes a challenge while using the GraphView.

In this tutorial, we will be focusing on the Memory Graph program mode of the app wherein we will be using an external library – MPAndroidChart to help us achieve our aim.

                                                        Memory Graph

      1. Firstly, open the app level build.gradle and implement the library dependency in there.

                 implementation ‘com.github.PhilJay:MPAndroidChart:v3.1.0’

         Note – The library version may change depending upon the time you are reading     this blog. Refer here for the latest library updates.

XML layout

1. The layout for our chart screen should be simple without any other components on the screen so that it is perfectly comfortable for the users to understand the graph and not get overwhelmed. Here, we are going to use a RelativeLayout as the parent to the Chart. For our Memory graph program mode, I worked with the LineChart view from the MPAndroidChart library. The xml code for the layout:

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <com.github.mikephil.charting.charts.LineChart
        android:id="@+id/brain_chart"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />
</RelativeLayout>

Layout binding and Java code

1. In the onCreate method of our MemoryGraph Activity, we set the layout parameters as flags for our chart layout to use of the screen window. We find the LineChart from the layout by its id and bind it up using the private variable which is going to be used to set it up programmatically. Here is the code for the onCreate method:

private LineChart graph;
    private Thread thread;

    @Override
    public void onCreate(@Nullable Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.fragment_memory_graph);
        getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN,
                WindowManager.LayoutParams.FLAG_FULLSCREEN);
        graph = findViewById(R.id.mem_graph);
    }

2. We are going to create a method named initGraph to set up the graph and make it ready to plot the data in real-time. Here is the code for the method:

graph.setOnChartValueSelectedListener(this);                // enable description text
        graph.getDescription().setEnabled(true);

        // enable touch gestures
        graph.setTouchEnabled(true);

        // enable scaling and dragging
        graph.setDragEnabled(true);
        graph.setScaleEnabled(true);
        graph.setDrawGridBackground(false);

        // if disabled, scaling can be done on x- and y-axis separately
        graph.setPinchZoom(true);

        // set an alternative background color
        graph.setBackgroundColor(getResources().getColor(R.color.memory_graph_background));

        LineData data = new LineData();
        data.setValueTextColor(Color.WHITE);

        // add empty data
        graph.setData(data);

        graph.getDescription().setText(getResources().getString(R.string.axis_desc_time));
        graph.getDescription().setTextColor(Color.WHITE);

        // get the legend (only possible after setting data)
        Legend l = memGraph.getLegend();

        // modify the legend ...
        l.setForm(Legend.LegendForm.LINE);
        l.setTextColor(Color.WHITE);

        XAxis xl = graph.getXAxis();
        xl.setTextColor(Color.WHITE);
        xl.setDrawGridLines(false);
        xl.setAvoidFirstLastClipping(true);
        xl.setEnabled(true);

        YAxis leftAxis = graph.getAxisLeft();
        leftAxis.setTextColor(Color.WHITE);
        leftAxis.setAxisMaximum(100f);
        leftAxis.setAxisMinimum(0f);
        leftAxis.setDrawGridLines(true);
        leftAxis.setGridColor(Color.WHITE);

        YAxis rightAxis = graph.getAxisRight();
        rightAxis.setEnabled(false);

Now, in here to use the setOnChartValueSelectedListener callback we need to implement the OnChartValueSelectedListener interface in our Activity. This will be required if the user clicks on a plot point. We need to set up the chart to respond to that user click on a particular data value on the chart.

3. Next, we are going to create a set of data for our graph to work with. As this set we create will be worked on by LineChart, hence our set will be of type LineDataSet. Using this LineDataSet, we will be able to set the axes labels, description, axes dependency (left or right), graph background, graph line colors and other necessary details. All parameters which can be tweaked through the `LineDataSet` for a LineGraph can be found here.

Here is the code for creating the LineDataSet :

private LineDataSet createSet() {

        LineDataSet set = new LineDataSet(null, "Brain waves");
        set.setAxisDependency(YAxis.AxisDependency.LEFT);
        set.setColor(Color.GREEN);
        set.setCircleColor(Color.WHITE);
        set.setLineWidth(2f);
        set.setCircleRadius(4f);
        set.setFillAlpha(65);
        set.setFillColor(Color.GREEN);
        set.setHighLightColor(Color.rgb(244, 117, 117));
        set.setValueTextColor(Color.WHITE);
        set.setValueTextSize(9f);
        set.setDrawValues(false);
        return set;
    }

4. Now, that we have set up the data for our LineChart to be used, we would be moving on to plotting the data. Since, everybody would not be able to have datasets containing brain-wave data, we would be taking the help of the ‘random’ function provided in Java. We create a function called `addEntry()`, wherein we add a new ‘Entry’ to the data set. Here, as we are working with a single dataset (the set of random values), the index for our dataset in the LineChart, will be zero.

private void addEntry() {

        LineData data = graph.getData();

        if (data != null) {

            ILineDataSet set = data.getDataSetByIndex(0);

            if (set == null) {
                set = createSet();
                data.addDataSet(set);
            }

            data.addEntry(new Entry(set.getEntryCount(), (float) (Math.random() * 40) + 30f), 0);
            data.notifyDataChanged();

            // let the graph know it's data has changed
            graph.notifyDataSetChanged();

            // limit the number of visible entries
            graph.setVisibleXRangeMaximum(120);

            // move to the latest entry
            graph.moveViewToX(data.getEntryCount());

        }
    }

5. We then create data feeding function called ‘feedMultiple’, for the chart. It contains a new runnable, which calls the ‘addEntry’ function. We then execute that runnable in a UI thread from within a for-loop, thus creating continuous values to plot. Here is the code for the ‘feedMultiple’ function;

private void feedMultiple() {

        if (thread != null)
            thread.interrupt();

        final Runnable runnable = () -> addEntry();

        thread = new Thread(() -> {
            for (int i = 0; i < 1000; i++) {

                // Don't generate garbage runnables inside the loop.
                runOnUiThread(runnable);

                try {
                    Thread.sleep(25);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });

        thread.start();
    }

6. Now we can call our ‘feedMultiple’ method from the onCreate function of our MemoryGraph Activity java file and we are good to go!

The full code for Memory Graph Mode can be found here: https://github.com/fossasia/neurolab-android/blob/development/app/src/main/java/io/neurolab/fragments/MemoryGraphFragment.java

Hope this blog helps you strengthen your Android development skills.

Resources

  1. Author – Amar, Article – Working with MPAndroidChart, Source – Code your world, Date – July 2016, Website – https://coderzduniya.blogspot.com/2016/07/working-with-mpandroidchart-how-to.html

2. Author – PhilJay, Source – Github, Library – MPAndroidChart Library, Website – https://github.com/PhilJay/MPAndroidChart/

Tags: FOSSASIA, GSOC19, Neurolab, Android, Graphs, Open-source

Continue Reading

Creating Common Loading Component in SUSI.AI

A circular loading component appears whenever an asynchronous event takes time on the front-end, to show the user that content is yet to be fetched and processed.

Creating Common Loading component eased the process of handling circular loading alignment. Using common loader decreased code repetition. There were major cases for loader component, mobile view, and desktop view.

Why Styled-components for SUSI.AI?

  • Painless maintenance
  • On-demand CSS injection: The components being rendered in the page are kept on track and only those component’s styles are injected into the DOM. If we combine react-loadable or some other code splitting, the app becomes really performant.
  • Styled components generate their own class name like sc-g3h4h6 and add it as an attribute to DOM, which avoids class name overlaps and misalignments.
  • Styled components are easier to detect as they can be detected by linter and husky. Whereas, static CSS files are not detected, which leads to dead code in the codebase and make them hard to manage. 
  • Simple dynamic styling: adapting the styling of a component based on its props, without having to manually manage dozens of classes.
  • Automatic vendor prefixing
  • Reuse, Reduce code written for CSS
  • Supports media queries, which makes creating a responsive PWA easier

 Let’s have a look at it’s implemented.

Code Integration

const Container = styled.div`
 display: flex;
 align-items: center;
 justify-content: center;
 ${props =>
   props.height
     ? css`
         height: ${props => props.height + 'rem'};
         @media (max-width: 512px) {
           height: ${props =>
             props.height > 20
               ? props.height - 15 + 'rem'
               : props.height + 'rem'};
         }
       `
     : css`
         margin: 19rem 0;
         height: 100%;
         @media (max-width: 512px) {
           margin: 14rem 0;
         }
       `}
`;

const CircularLoader = ({ height = 'auto', color = 'primary', size = 64 }) => {
 return (
   <Container height={height}>
     <CircularProgress color={color} size={size} />
   </Container>
 );
};

components/shared/CircularLoader.js

The CircularLoader component by default should acquire size 64 width and size, and color primary i.e. SUSI.AI blue color.

The container created using styled component, by default should be horizontally and vertically centered. This is used to handle the case where the content takes the whole width and height(100%) of page. 

If we want the container to be of specific height(lesser than the page in which the Loader is being rendered), we pass in height props.

For handling the mobile views, in case of height, 100%, margin-top and margin-bottom are reduced by 5rem.

In case of height passed in through component, check if the height is greater(than 20) so that the mobile view user can see the footer as well, reduce height by 15rem. If the height is lesser than 20rem, the footer is in the viewport.

Before PR, let’s have a look at how the Loader was rendered inside component:

{loading ? (
  <LoadingContainer>
    <CircularProgress size={64} />
  </LoadingContainer>
 ) : (
   ...
 )}

Using Common shared loader,

{loading ? <CircularLoader height={27} /> : ...

Settings/Settings.react.js

Adding styled component across the SUSI.AI web app proved to be of great help, the mobile views were easily made using media queries. Components positioning logic could be changed based on props, and most importantly, styled-components could be reused and were much easier to manage and maintain than static CSS classes or inline styles.

However, for other cases like when single or less number of styles were applied, maintaining everything using styled component proved to be additional overhead. A combination of inline styles and styled-components would be apt depending on the nature of style. 

Resources

Tags

SUSI.AI, FOSSASIA, GSoC19, styled-components, SUSI.AI Chat

Continue Reading
Close Menu