Adding swap space to your DigitalOcean droplet, if you run out of RAM

The Open Event Android App generator runs on a DigitalOcean. The deployment runs on a USD 10 box, that has 1 GB of RAM, but for testing I often use a USD 5 box, that has only 512mb of RAM.

When trying to build an android app using gradle and Java 8, there could be an issue where you run out of RAM (especially if it’s 512 only).

What we can do to remedy this problem is creating a swapfile. On an SSD based system, Swap spaces work almost as fast as RAM, because SSDs have very high R/W speeds.

Check hard disk space availability using

df -h

There should be an output like this

Filesystem      Size  Used Avail Use% Mounted on
udev            238M     0  238M   0% /dev
tmpfs            49M  624K   49M   2% /run
/dev/vda1        20G  1.1G   18G   6% /
tmpfs           245M     0  245M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           245M     0  245M   0% /sys/fs/cgroup
tmpfs            49M     0   49M   0% /run/user/1001

The steps to create a swap file and allocating it as swap are

sudo fallocate -l 1G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

We can verify using

sudo swapon --show
NAME      TYPE  SIZE USED PRIO
/swapfile file 1024M   0B   -1

And now if we see RAM usage using free -h , we’ll see

              total        used        free      shared  buff/cache   available
Mem:           488M         37M         96M        652K        354M        425M
Swap:          1.0G          0B        1.0G

Do not use this as a permanent measure for any SSD based filesystem. It can corrupt your SSD if used as swap for long. We use this only for short periods of time to help us build android apks on low ram systems.

Doing a table join in Android without using rawQuery

The Open Event Android App, downloads data from the API (about events, sessions speakers etc), and saves them locally in an SQLite database, so that the app can work even without internet connection.

Since there are multiple entities like Sessions, Speakers, Events etc, and each Session has ids of speakers, and id of it’s venue etc, we often need to use JOIN queries to join data from two tables.

 

Android has some really nice SQLite helper classes and methods. And the ones I like the most are the SQLiteDatabase.query, SQLiteDatabase.update, SQLiteDatabase.insert ones, because they take away quite a bit of pain for typing out SQL commands by hand.

But unfortunately, if you have to use a JOIN, then usually you have to go and use the SQLiteDatabase.rawQuery method and end up having to type your commands by hand.

But but but, if the two tables you are joining do not have any common column names (actually it is good design to have them so – by having all column names prefixed by tablename_ maybe), then you can hack the usual SQLiteDatabase.query() method to get a JOINed query.

Now ideally, to get the Session where speaker_id was 1, a nice looking SQL query should be like this –

SELECT * FROM speaker INNER JOIN session
ON speaker_id = session_speaker_id
WHERE speaker_id = 1

Which, in android, can be done like this –

String rawQuery = "SELECT * FROM " + SpeakerTable.TABLE_NAME + " INNER JOIN " + SessionTable.TABLE_NAME
        + " ON " + SessionTable.EXP_ID + " = " + SpeakerTable.ID
        + " WHERE " + SessionTable.ID + " = " +  id;
Cursor c = db.rawQuery(
        rawQuery,
        null
);

But of course, because of SQLite’s backward compatible support of the primitive way of querying, we turn that command into

SELECT *
FROM session, speaker
WHERE speaker_id = session_speaker_id AND speaker_id = 1

Now this we can write by hacking the terminology used by the #query() method –

Cursor c = db.query(
        SessionTable.TABLE_NAME + " , " + SpeakerTable.TABLE_NAME,
        Utils.concat(SessionTable.PROJECTION, SpeakerTable.PROJECTION),
        SessionTable.EXP_ID + " = " + SpeakerTable.ID + " AND " + SpeakerTable.ID + " = " +  id,
        null,
        null,
        null,
        null
);

To explain a bit, the first argument String tableName can take table1, table2 as well safely, The second argument takes a String array of column names, I concatenated the two projections of the two classes. and finally, put by WHERE clause into the String selection argument.

You can see the code for all database operations in the android app here  https://github.com/fossasia/open-event-android/blob/master/android/app/src/main/java/org/fossasia/openevent/dbutils/DatabaseOperations.java

Getting code coverage in a Nodejs project using Travis and CodeCov

We had set up unit tests on the webapp generator using mocha and chai, as I had blogged before.

But we also need to get coverage reports for each code commit and the overall state of the repo.

Since it is hosted on Github, Travis comes to our rescue. As you can see from our .travis.yml file, we already had Travis running to check for builds, and deploying to heroku.

Now to enable Codecov, simply go to http://codecov.io and enable your repository (You have to login with Github so see your Github repos) .

Once you do it, your dashboard should be visible like this https://codecov.io/github/fossasia/open-event-webapp

We use istanbul to get codecoverage. To try it out just use

istanbul cover _mocha

On the root of your project (where the /test/ folder is ) . That should generate a folder called coverage or lcov. Codecov can read lcov reports. They have provided a bash file which can be run to automatically upload coverage reports. You can run it like this –

bash <(curl -s https://codecov.io/bash)

Now go back to your codecov dashboard, and your coverage report should show up.

Screenshot from 2016-08-29 21-23-00

If all is well, we can integrate this with travis so that it happens on every code push. Add this to your travis.yml file.

script:
  - istanbul cover _mocha
after_success:
- bash <(curl -s https://codecov.io/bash)

This will ensure that on each push, we run coverage first. And if it is successful, we push the result to codecov.

We can see coverage file by file like this

Screenshot from 2016-08-29 21-23-35

And we can see coverage line by line in a file like this

Screenshot from 2016-08-29 21-26-55

 

Motion in android

So earlier this year I attended a talk where the speaker wanted to introduce us to meaningful motion in android apps and he convinced us to use this in our apps as well. Motion came in with Material design, actually not really came but became popular with Material design and since google has added the same kind of motions to their apps as well, developers have started using it.

I love motion, not only does it boost engagement but it’s instantly noticeable. Think of the apps you use that feature motion design and how pleasing, satisfying, fluent and natural they feel to experience. Eg. Zomato, Play music etc.

Now think of some apps that don’t use any kind of motions and you’ll realise they look a bit boring and you as users will always prefer apps with some kind of motion.

Touch

So firstly let’s discover the feedback on touch. It helps to communicate to the user in a visual form that some interaction has been made. But also keep in mind that this animation should be enough for them to gain clarity and encourage further explorations and not distract them.

For adding backgrounds you can use the following :

  • ?android:attr/selectableItemBackground — Show a ripple effect within the bounds of the view.
  • ?android:attr/selectableItemBackgroundBorderless — Show a ripple effect extending the bounds of the view.

View Property Animator

Introduced in API 12, this allows us to perform animated operations (in parallel) on a number of view properties using a single Animator instance

Some of the parameters that can be added to a view are as follows :

  • alpha() -Set the alpha value to be animated to
  • scaleX() & scaleY()— Scales the view on it’s X and / or Y axis
  • translationZ() — Translates the view on its Z axis
  • setDuration()— Sets the duration of the animation
  • setStartDelay() — Sets the delay on the animation
  • setInterpolator() — Sets the animation interpolator
  • setListener() — Set a listener for when the animation starts, ends, repeats or is cancelled.

Now let’s write some code on how to do this on a button for example:

mButton.animate().alpha(1f)
.scaleX(1f)        
.scaleY(1f)        
.translationZ(10f)        
.setInterpolator(new FastOutSlowInInterpolator())        .setStartDelay(200)        
.setListener(new Animator.AnimatorListener() {            
@Override            
public void onAnimationStart(Animator animation) { }             
@Override            
public void onAnimationEnd(Animator animation) { }             
@Override            
public void onAnimationCancel(Animator animation) { }             
@Override            
public void onAnimationRepeat(Animator animation) { }        
})        
.start();

Note : Use ViewCompat class to implement the ViewPropertyAnimator from Android API version 4 and up

Object Animator

Similar to the ViewPropertyAnimator, the ObjectAnimator allows us to perform animations on various properties of the target view (both in code and XML resource files). However, there a couple of differences:

  • The ObjectAnimator only allows animations on a single property per instance e.g.Scale X followed by Scale Y.
  • However, it allows animations on a custom Property e.g. A view’s foreground colour.

Her we need to set the evaluator, set the delay and call start().

private void animateForegroundColor(@ColorInt final int targetColor) {   
    ObjectAnimator animator = ObjectAnimator.ofInt(YOUR_VIEW, FOREGROUND_COLOR, Color.TRANSPARENT, targetColor);                    animator.setEvaluator(new ArgbEvaluator()); animator.setStartDelay(DELAY_COLOR_CHANGE); animator.start();}

Interpolators

An Interpolator can be used to define the rate of change for an animation, meaning the speed, acceleration and behaviour during animating can be altered. Some of them are :

These are some of the basics. there are a lot of other things like

  • Window transitions(Explode, fade, slide etc.)
  • Shared element Transitions
  • Other custom transitions
  • Animated Vector drawables

you can play around with these for a better understanding but be sure to actually try everything on a device/emulator since you’ll get to actually see the changes in the UI and in turn understand better.

File upload progress in a Node app using Socket.io

If you look at the webapp generator, you’ll see that there is an option to upload a zip file containing event data. We wanted to give visual cue to the user when he is uploading to see how much file has uploaded.

We are uploading the file, and giving the generate start command via socket.io events instead of POST requests here.

To observe file upload progress on socket (when sending file using a Buffer), there is an awesome node module available called socketio-upload-progress.

In our webapp you can see we implemented it on the frontend here in the form.js and here in the backend in app.js

Basically on the backend you should add the socketio-file-upload module as a middleware to express

var siofu = require("socketio-file-upload");
var app = express()
    .use(siofu.router)
    .listen(8000);

After a socket is opened, set up the upload directory and start listening for uploads

io.on("connection", function(socket){
    var uploader = new siofu();
    uploader.dir = "/path/to/save/uploads";
    uploader.listen(socket);
});

On the frontend, we’ll listen for an input change on an file input type element whose id is siofu_upload

var socket = io.connect();
var uploader = new SocketIOFileUpload(socket);
uploader.listenOnInput(document.getElementById("siofu_input"));

One thing to note here is that, if you observe percentage of upload on frontend, it’ll give you false values. The correct values of how much data is actually transferred can be found in the backend. So observe progress in backend, and send percentage to frontend using the same socket.

  uploader.on('progress', function(event) {
    console.log(event.file.bytesLoaded / event.file.size)
    socket.emit('upload.progress', {
      percentage:(event.file.bytesLoaded / event.file.size) * 100
    })
});

 

Lambda expressions in Android

What are Lambda expressions

Lambda Expressions are one of the most important features added to Java 8. Prior to Lambda Expressions, implementing functional interfaces i.e interfaces with only one abstract method has been done using syntax that has a lot of boilerplate code in it.
In cases like this, what we are trying to do is pass a functionality as an argument to a method, such as what happens when a button is clicked.

Lambda expressions enables you to do just that, in a way that is much more compact and clear.

Syntax of Lambda Expressions

A lambda expression consist of the following:

  • A comma separated list of formal parameters enclosed in parentheses. The data types of the parameters in a lambda expression can be omitted. Also the parenthesis can be omitted if there is only one parameter. For example:
TextView tView = (TextView) findViewById(R.id.tView);
tView.setOnLongClickListener(v -> System.out.println("Testing Long Click"));
  • The arrow token ->
  • A body which contains a single expression or a statement block. If a single expression is specified, the java runtime evaluates the expression and then return its value. To specify a statement block, enclose statements in curly braces "{}"

Lambda Expressions in Android

To use Lambda Expressions and other Java 8 features in Android, you need to use the Jack tool-chain. Open your module level build.gradle file and add the following:

android {
  ...
  defaultConfig {
    ...
    jackOptions {
      enabled true
    }
  }
  compileOptions {
    sourceCompatibility JavaVersion.VERSION_1_8
    targetCompatibility JavaVersion.VERSION_1_8
  }
}

Sync your build.gradle file and if you are having any issue with build tools, you may need to update buildToolsVersion in your build.gradle file to "24rc4" or just download the latest Android SDK Build-tools from the SDK Manager, under the Tools (Preview channel).

Example

Adding a click listener to a button

without lambda expression

Button button = (Button)findViewById(R.id.button);
button.setOnClickListener(button.setOnClickListener(new View.OnClickListener() {
    @Override
    public void onClick(View v) {
        Toast.makeText(this, "Button clicked", Toast.LENGTH_LONG).show();
    }
}););

with lambda expressions It is as simple as:

Button button = (Button)findViewById(R.id.button);
button.setOnClickListener(v -> Toast.makeText(this, "Button clicked", Toast.LENGTH_LONG).show(););

As we can see above, using lambda expressions makes implementing a functional interface clearer and compact. Standard functional interfaces can be found in the java.util.function package [included in Java 8]. These interfaces can be used as target types for lambda expressions and method references.

Another way to have Java 8 features in your Android app is using the RetroLambda plugin.

Using ftp-deploy in node.js to publish websites over FTP

In the Open Event Webapp Generator, we recently added the functionality for organisers to submit their ftp credentials and when the website is generated, it’ll automatically upload the website to the chosen ftp server (allowing creation of subdirectory internally, if the organiser so wants).

To achieve we used the very useful nodejs module ftp-deploy which is a wrapper on the popular jsftp library

The code dealing with ftp deployment in our webapp generator can be found here  –

https://github.com/fossasia/open-event-webapp/blob/development/src/backend/ftpdeploy.js

As can be seen, deploying using ftp-deploy is pretty straightforward. Primarily we need a config object

 


  var config = {
    username: ftpDetails.user, //prompted on commandline if not given
    password: ftpDetails.pass, // optional, prompted if none given
    host: ftpDetails.host,
    port: 21,
    localRoot: path.join(__dirname, '/../../dist', appFolder), //local folder containing website
    remoteRoot: ftpDetails.path, //path on ftp server to host website
    exclude: ['.git', '.idea', 'tmp/*'],
    continueOnError: true
};

You can set up some event listeners for events like uploaded uploading and upload-error

Responsive Image Overlay

Image overlay is a very common concept in front-end development. It is easy to implement but difficult when we deal it with different screen sizes, where we need to cover the image with the overlay each time the screen size is changed. I have gone through various blog posts when I need to implement the same for Open-event webapp and researched a solution that works for all screen sizes without any media query.

1234

How to add an overlay to an image ?

If we need four images in a single row nearly 300*300px.  The code below shows the markup.

image-holder : The parent class to take the image and overlay inside it.

background-image: This class takes image source.

responsive-overlay: This is the key point to make it responsive. Responsive-overlay contains a class hover-state to add overlay absolutely and a class social-links.

social-links: It adds content to hover-state.

 

<div class="image-holder">
  <img class="background-image" alt="" src="">
   <div class="responsive-overlay">
     <div class="hover-state text-center preserve3d">
       <div class="social-links vertical-align">

       </div>
     </div>
   </div>
 </div>

The styling is written with SASS in .scss file as shown below.

//overlayimage and backgroundshade can be set in config.scss

 .image-holder {
   position: relative;
   overflow: hidden;
   margin-bottom: 12px;

   .background-image {
     height: 300px;
     width: 300px;
     display: block;
     margin: 0 auto;
     background-color: $background-shade;
    }
 
   .responsive-overlay {
     @include responsiveoverlay;

    .preserve3d {
       height: 300px;
      }

    .hover-state {
     @include hoverstate;
     height: 300px;
     width: 300px;
    }

  @mixin responsiveoverlay {
     height: 100%;
     position: absolute;
     top: 0;
     width: 100%;
}

   @mixin hoverstate {
     background: $overlayimage;
     display: block;
     height: 300px;
     left: 0;
     margin: 0 auto;
     opacity: 0;
     position: relative;
     top: 0;
     -moz-transition: all 0.3s ease-out;
     -webkit-transition: all 0.3s ease-out;
     transition: all 0.3s ease-out;
     width: 300px;
     z-index: 2;
   }

This code will work for responsiveness as well. The main catch here is the responsive-overlay class which is made 100% in width but set to position absolute. The images which are 300 * 300 px in size will take an overlay of the same size because of hover-state class. Instead, if we adjust sizes of images in small screens the above code will adjust overlay on the image automatically.

Like, on tablets we can have an overlay like this.

345

And on mobile screen output is like that :

23

Conclusion

Responsiveness is easy if we follow correct concepts. Here, the concepts of absolute and relative positioning in CSS have done the magic. Now we can play by adding different contents and effect on hover following the same basics.

Doing asynchronous tasks serially using ‘async’ in node.js

In the open-event-webapp generator we need to perform a lot of asynchronous tasks in the background like –

  • Downloading images and audio assets
  • Downloading the jsons from the endpoints
  • Generating the html from handelbar templates
  • and so on . .

Sometimes tasks depend on previous tasks, and in such cases we need to perform them serially. Also there are tasks like image downloads, that would be better if done parallelly.

To achieve both these purposes, there is an awesome node.js library called async that helps achieve this.

To perform asynchronous tasks serially (one task, then another task), we can use something like this –

 

async.series([
    (done) => {
       someAsyncFunction(function () { done () })
    },
    //(done) => {..}, (done) => {..} more tasks here
    (done) => {
       someAsyncFunction(function () { done () })
    });
      
    }
]);

Basically async takes an array of functions. Each function contains a callback that you need to call when the internal task is finished. The 2nd task starts, only after the done() callback of first task is executed.

An example of it’s usage can be seen in the open-event-webapp project here

GET and POST requests

If you wonder how to get or update page resource, you have to read this article.

It’s trivial if you have basic knowledge about HTTP protocol. I’d like to get you little involved to this subject.

So GET and POST are most useful methods in HTTP protocol.

What is HTTP?

Hypertext transfer protocol – allow us to communicate between client and server side. In Open Event project we use web browser as client and for now we use Heroku for server side.

Difference between GET and POST methods

GET – it allows to get data from specified resources

POST – it allows to submit new data to specified resources for example by html form.

GET samples:

For example we use it to get details about event

curl http://open-event-dev.herokuapp.com/api/v2/events/95

Response from server:

Of course you can use this for another needs, If you are a poker player I suppose that you’d like to know how many percentage you have on hand.

curl http://www.propokertools.com/simulations/show?g=he&s=generic&b&d&h1=AA&h2=KK&h3&h4&h5&h6&_

POST samples:

curl -X POST https://example.com/resource.cgi

You can often find this action in a contact page or in a login page.

How does request look in python?

We use Requests library to communication between client and server side. It’s very readable for developers. You can find great documentation  and a lot of code samples on their website. It’s very important to see how it works.

>>> r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
>>> r.status_code
200

I know that samples are very important, but take a look how Requests library fulfils our requirements in 100%. We have decided to use it because we would like to communicate between android app generator and orga server application. We have needed to send request with params(email, app_name, and api of event url) by post method to android generator resource. It executes the process of sending an email – a package of android application to a provided email address.

data = {
    "email": login.current_user.email,
    "app_name": self.app_name,
    "endpoint": request.url_root + "api/v2/events/" + str(self.event.id)
}
r = requests.post(self.app_link, json=data)