Auto Updating SUSI Android APK and App Preview on appetize.io

This blog will cover the way in which the SUSI Android APK is build automatically after each commit and pushed to “apk” branch in the github repo. Other thing which will be covered is that how the app preview on appetize.io can be updated after each commit. This is basically for the testers who wish to test the SUSI Android App. There are four ways to test the SUSI Android App. One is to simply download the alpha version of the app from the Google PlayStore. Here is the link to the app. Join the alpha testing and report bugs on the github issue tracker of the repo. Other way is to build the app from Android Studio but you may need to set the complete project. If you are looking to contribute in the project, this is the advised way to test the app. The other two ways are explained below.

Auto Building of APK and pushing to “apk” branch

We have written a script which does following steps whenever a PR is merged:

  1. Checks if the commit is of a PR or a commit to repo
  2. If not of PR, configures a user whose github account will be used to push the APKs.
  3. Clones the repo, generates the debug and release APK.
  4. Deletes everything in the apk branch.
  5. Commits and Pushes new changes to apk branch.

This script is written for people or testers who do not have android studio installed in their computer and want to test the app. So, they can directly download the apk from the apk branch and install it in their phone. The APK is always updated after each commit. So, whenever a tester downloads the APK from apk branch, he will always get the latest app.

if [[ $CIRCLE_BRANCH != pull* ]]
then
    git config --global user.name "USERNAME"
    git config --global user.email "EMAIL"

    git clone --quiet --branch=apk https://USERNAME:$GITHUB_API_KEY@github.com/fossasia/susi_android apk > /dev/null
    ls
    cp -r ${HOME}/${CIRCLE_PROJECT_REPONAME}/app/build/outputs/apk/app-debug.apk apk/susi-debug.apk
    cp -r ${HOME}/${CIRCLE_PROJECT_REPONAME}/app/build/outputs/apk/app-release-unsigned.apk apk/susi-release.apk
    cd apk

    git checkout --orphan workaround
    git add -A

    git commit -am "[Circle CI] Update Susi Apk"

    git branch -D apk
    git branch -m apk

    git push origin apk --force --quiet > /dev/null
fi

Auto Updating of App Preview on appetize.io

The APKs generated in the above step can now be used to set up the preview of the app on the appetize.io. Appetize.io is an online simulator to run mobile apps ( IOS and Android). Appetize.io provides a nice virtual mobile frame to run native apps with various options like screen size, mobile, OS version, etc. Appetize.io provides some API to update/publish the app. In SUSI, we once uploaded the app on appetize.io and now we are using the API provided by them to update the APK everytime a commit is pushed in the repository.

API information (Derived from official docs of appetize.io):

You may upload a new version of an existing app, or update app settings.

Send an HTTP POST request to

https://APITOKEN@api.appetize.io/v1/apps/PUBLICKEY

Replace APITOKEN with your API token and PUBLICKEY with the public key of the app you’re updating. Your API token must be permissioned to the same account as was used to upload the app. The POST body must be a JSON object. To delete a previously set field, use a value of null.

Optional Fields

  1. url: (string) a publicly accessible link to your .zip, .tar.gz, or .apk file, used to upload a new version of your app.
  2. note: (string) a note for your own purposes, will appear on your management dashboard.

For the url parameter, we have used https://github.com/fossasia/susi_android/raw/apk/susi-debug.apk and note can be anything. We have used Update SUSI Preview.

curl https://$APPETIZE_API_TOKEN@api.appetize.io/v1/apps/mbpprq4xj92c119j7nxdhttjm0 -H 'Content-Type: application/json' -d '{"url":"https://github.com/fossasia/susi_android/raw/apk/susi-debug.apk", "note": "Update SUSI Preview"}'

Summary

This blog covered about how to implement an automatic structure to generate APKs for testing and using that APK to build a preview on websites like appetize.io and then using the APIs provided by them to update the APK after each PR merge in the repo. Check out the resources below to learn more about the topic. So, if you are thinking of contributing to SUSI Android App, this may help you a little in testing the app. But if not, then you can also use the similar technique for your android app as well and ease the life of testers.

Resources

  1. Docs of appetize.io to learn more about the API https://appetize.io/docs
  2. Tutorial on using curl to make API requests https://curl.haxx.se/docs/httpscripting.html
  3. Tutorial on writing basic shell scripts https://ryanstutorials.net/bash-scripting-tutorial/
Continue ReadingAuto Updating SUSI Android APK and App Preview on appetize.io

Editing files and piped data with “sed” in Loklak server

What is sed ?

“sed” is used in “Loklak Server” one of the most popular projects of FOSSASIA. “sed” is acronym for “Stream Editor” used for filtering and transforming text, as mentioned in the manual page of sed. Stream can be a file or input from a pipeline or even standard input. Regular expressions are used to filter the text and transformation are carried out using sed commands, either inline or from a file. So, most of the time writing a single line does the work of text substitution, removal or to obtaining a value from a text file.

Basic Syntax of “sed”

$sed [options]... {inline commands or file having sed commands} [input_file]...

Loklak Server uses a config.properties file – contains key-value pairs – which is the basis of the server as it contains configuration values, used by the server during runtime for various operations.

Let’s go through a simple sed example that prints line containing word “https” at the beginning in the config.properties file.

$sed -n '/^https/p' config.properties

Here “-n” option suppresses automatically printing of pattern space (pattern space is where each line is put that is to be processed by sed). Without “-n” option sed will print the whole file.

Now, the regular expression part,  “/^https” matches all the lines that has “https” at the start of line and “/p” is print command to print the output in console. Finally we provide the filename i.e. config.properties. If filename is not provided then sed waits for input from standard input.

Use cases of “sed” in Loklak Server

  • Displaying proper port number in message while starting or installing Loklak Server

The default port of loklak server is port number 9000, but it can be started in any non-occupied port by using “-p” flag with bin/start.sh and bin/installation.sh like

$ bin/installation.sh -p 8888

starts installation of Loklak Server in port 8888. To display the proper localhost address so that user can open it in a browser the port number in shortlink.urlstub parameter in config.properties needs to be changed. This is carried out by the function change_shortlink_urlstub in bin/utility.sh. The function is defined as

Now let’s try to understand what the sed command is doing.

“-i” option is used for in-place editing of the specified file i.e. config.properties in conf directory.

s” is substitute command of sed. The regular expression can be divided into two parts, between “/”:

  1. \(shortlink\.urlstub=http:.*:\)\(.*\) this is used to find the match in a line.
  2. \1′”$1″ is used to substitute the matched string in part 1.

The regular expressions can be split into groups so that operations can be performed on each group separately. A group is enclosed between “\(“ and “\)”. In our 1st part of regular expressions, there are two groups.

Dissecting the first group i.e. \(shortlink\.urlstub=http:.*:\):

  • shortlink\.urlstub=http:” will match the expression “shortlink.urlstub=http:”, here “\” is used as an escape sequence as “.” in regex represents any character.
  • “.*:”, “.” represents any character and “*” represents 0 or more characters of the previous character. So, it will basically match any expression. But, it ends with a “:”, which means any expression that ends with a “:”. Thus it matches the expression “//localhost:”.

So, the first group collectively matches the expression “shortlink.urlstub=http://localhost:”.

As described above second group i.e. \(.*\) will match any expression, here matches “9000”.

Now coming to the 2nd part of regular expression i.e. \1′”$1″:

  • “\1” represents the match of the first group in 1st part i.e. “shortlink.urlstub=http://localhost:”.
  • “$1” is the value of the first parameter provided to our function “change_shortlink_urlstub” which is an unused port where we want to run Loklak Server.

So 2nd part picks up the match from the first group and concatenates with the first parameter of the function. Assuming the first parameter to the function is “8888”, the expression for 2nd part becomes “shortlink.urlstub=http://localhost:8888” which replaces “shortlink.urlstub=http://localhost:9000”.

So a correct localhost address is displayed in the console while starting or installing Loklak Server.

  • Extracting value of a key from config.properties

“grep” and “sed” are used to extract the values of key from config.properties in bash scripts e.g. extracting default port, value of “port.http” in bin/start.sh, value of “shortlink.urlstub” in bin/start.sh and bin/installation.sh.

$ grep -iw 'port.http' conf/config.properties | sed 's/^[^=]*=//'

Here grep is used to filter a line and pass the filtered line to sed by piping the output. “i” flag is for ignoring case sensitivity and “w” flag is used for matching of word only. The output of

$ grep -iw 'port.http' conf/config.properties
port.http=9000

The aim is to get “9000”, the value of “port.http”. The approach used here is to substitute “port.http=” in the output of grep command above with a string of zero characters, that way only “9000” remains.

Let’s deconstruct the “sed” part, s/^[^=]*=//:

s” command of sed is used for substitution.  Here 1st part is “^[^=]*=” and 2nd part is nothing, as no characters are enclosed within “//”.

  • “^” means to match at the start.
  • [^characters] represents not to consider a set of characters. For matching a set of characters “^” is not used inside square brackets, [characters]. Here [^=] means not to include equal to – “=” symbol – and “*” after that makes it to match characters that are not “=”.

So “^[^=]*=” matches a sequence of characters that doesn’t start with “=” and followed by “=”. Thus the matched expression in this case is “http.port=” (“h” is not “=” and it ends with “=”), which is substituted by a string with zero characters which leaves “9000”.  Finally, “9000” is assigned to the required variable is bash script.

Continue ReadingEditing files and piped data with “sed” in Loklak server