DevOps – blog.fossasia.org

Use Travis to extract testing APKs for PSLab

Post author:CloudyPadmal
Post published:August 21, 2018
Post category:Community Pocket Science Lab Tutorial
Post comments:0 Comments

Travis CI is a popular continuous integration tool built to test software and deployments done at GitHub repositories. They provide free plans to open source projects. PSLab Android is using Travis to ensure that all the pull requests made and the merges are build-bug frees. Travis can do this pretty well, but that is not all it can do. It’s a powerful tool and we can think of it as a virtual machine (with high specs) installed in a server for us to utilize efficiently.

There was a need in PSLab development that, if we want to test the functionalities of code at a branch. To fulfil this need, one has to download the branch to his local machine and use gradle to build a new apk. This is a tedious task, not to mention reviewers, even developers wouldn’t like to do this.

If the apk files were generated and in a branch, we can simply download them using a console command.

$ wget https://raw.github.com/fossasia/<repository>/<branch>/<file with extension>

With the help of Travis integration, we can create a simple script to generate these apks for us. This is done as follows;

Once the Travis build is complete for a triggering event such as a pull request, it will run it’s “after_success” block. We are going to execute a script at this point. Simply add the following code snippet.

after_success:
  - 'if [ "$TRAVIS_PULL_REQUEST" == "false" ]; then bash <script-name>.sh; fi'

This will run the script you have mentioned using bash. Here we will have the following code snippets in the specified bash script.

First of all we have to define the branch we want to build. This can be done using a variable assignment.

export DEVELOPMENT_BRANCH=${DEVELOPMENT_BRANCH:-development}

Once the build is complete, there will be new folders in the virtual machine. One of them is the app folder. Inside this folder contains the build folder where all the apk files are generated. So the next step is to copy these apk files to a place of our preference. I am using a folder named apk to make it much sense.

cd apk
\cp -r ../app/build/outputs/apk/*/**.apk .
\cp -r ../app/build/outputs/apk/debug/output.json debug-output.json
\cp -r ../app/build/outputs/apk/release/output.json release-output.json
\cp -r ../README.md .

Usually, the build folder has following apk files

app-debug.apk
app-release-unsigned.apk
app-release.apk

Release apks are usually signed with a key and it would cause issues while installation. So we have to filter out the debug apk that we usually use for debugging and get it as the output apk. This involves simple file handling operations in Linux and a bit of git.

First of all, rename the apk file so that it will be different from other files.

# Rename apks with dev prefixes
mv app-debug.apk app-dev-debug.apk

Then add and commit them to a specific branch where we want the output from.

git add -A
git commit -am "Travis build pushed [development]"
git push origin apk --force --quiet> /dev/null

Once it is all done, you will have a branch created and updated with the apk files you have defined.

Figure 1: UI of pslab-android apk branch

Reference:

https://travis-ci.org/

Deploying a Postgres-Based Open Event Server to Kubernetes

Post author:schedutron
Post published:August 9, 2018
Post category:Event Management FOSSASIA GSoC Open Event
Post comments:0 Comments

In this post, I will walk you through deploying the Open Event Server on Kubernetes, hosted on Google Cloud Platform’s Compute Engine. You’ll be needing a Google account for this, so create one if you don’t have one.

First, I cd into the root of our project’s Git repository. Now I need to create a Dockerfile. I will use Docker to package our project into a nice image which can be then be “pushed” to Google Cloud Platform. A Dockerfile is essentially a text doc which simply contains the commands required to assemble an image. For more details on how to write one for your project specifically, check out Docker docs. For Open Event Server, the Dockerfile looks like the following:

FROM python:3-slim

ENV INSTALL_PATH /open_event
RUN mkdir -p $INSTALL_PATH

WORKDIR $INSTALL_PATH

# apt-get update and update some packages
RUN apt-get update && apt-get install -y wget git ca-certificates curl && update-ca-certificates && apt-get clean -y


# install deps
RUN apt-get install -y --no-install-recommends build-essential python-dev libpq-dev libevent-dev libmagic-dev && apt-get clean -y

# copy just requirements
COPY requirements.txt requirements.txt
COPY requirements requirements

# install requirements
RUN pip install --no-cache-dir -r requirements.txt 
RUN pip install eventlet

# copy remaining files
COPY . .

CMD bash scripts/docker_run.sh

These commands simply install the dependencies and set up the environment for our project. The final CMD command is for running our project, which, in our case, is a server.

After our Dockerfile is configured, I go to Google Cloud Platform’s console and create a new project:

Once I enter the product name and other details, I enable billing in order to use Google’s cloud resources. A credit card is required to set up a billing account, but Google doesn’t charge any money for that. Also, one of the perks of being a part of FOSSASIA was that I had about $3000 in Google Cloud credits! Once billing is enabled, I then enable the Container Engine API. It is required to support Kubernetes on Google Compute Engine. Next step is to install Google Cloud SDK. Once that is done, I run the following command to install Kubernetes CLI tool:

gcloud components install kubectl

Then I configure the Google Cloud Project Zone via the following command:

gcloud config set compute/zone us-west1-a

Now I will create a disk (for storing our code and data) as well as a temporary instance for formatting that disk:

gcloud compute disks create pg-data-disk --size 1GB
gcloud compute instances create pg-disk-formatter
gcloud compute instances attach-disk pg-disk-formatter --disk pg-data-disk

Once the disk is attached to our instance, I SSH into it and list the available disks:

gcloud compute ssh "pg-disk-formatter"

Now, ls the available disks:

ls /dev/disk/by-id

This will list multiple disks (as shown in the Terminal window below), but the one I want to format is “google-persistent-disk-1”.

Now I format that disk via the following command:

sudo mkfs.ext4 -F -E lazy_itable_init=0,lazy_journal_init=0,discard /dev/disk/by-id/google-persistent-disk-1

Finally, after the formatting is done, I exit the SSH session and detach the disk from the instance:

gcloud compute instances detach-disk pg-disk-formatter --disk pg-data-disk

Now I create a Kubernetes cluster and get its credentials (for later use) via gcloud CLI:

gcloud container clusters create opev-cluster
gcloud container clusters get-credentials opev-cluster

I can additionally bind our deployment to a domain name! If you already have a domain name, you can use the IP reserved below as an A record of your domain’s DNS Zone. Otherwise, get a free domain at freenom.com and do the same with that domain. This static external IP address is reserved in the following manner:

gcloud compute addresses create testip --region us-west1

It’ll be useful if I note down this IP for further reference. I now add this IP (and our domain name if I used one) in Kubernetes configuration files (for Open Event Server, these were specific files for services like nginx and PostgreSQL):

#
# Deployment of the API Server and celery worker
#
kind: Deployment
apiVersion: apps/v1beta1
metadata:
name: api-server
namespace: web
spec:
replicas: 1
template:
metadata:
labels:
app: api-server
spec:
volumes:
- name: data-store
emptyDir: {}
containers:
- name: api-server
image: eventyay/nextgen-api-server:latest
command: ["/bin/sh","-c"]
args: ["./kubernetes/run.sh"]
livenessProbe:
httpGet:
path: /health-check/
port: 8080
initialDelaySeconds: 30
timeoutSeconds: 3
ports:
- containerPort: 8080
protocol: TCP
envFrom:
- configMapRef:
name: api-server
env:
- name: C_FORCE_ROOT
value: 'true'
- name: PYTHONUNBUFFERED
value: 'TRUE'
- name: DEPLOYMENT
value: 'api'
- name: FORCE_SSL
value: 'yes'
volumeMounts:
- mountPath: /opev/open_event/static/uploads/
name: data-store
- name: celery
image: eventyay/nextgen-api-server:latest
command: ["/bin/sh","-c"]
args: ["./kubernetes/run.sh"]
envFrom:
- configMapRef:
name: api-server
env:
- name: C_FORCE_ROOT
value: 'true'
- name: DEPLOYMENT
value: 'celery'
volumeMounts:
- mountPath: /opev/open_event/static/uploads/
name: data-store
restartPolicy: Always

These files will look similar for your specific project as well. Finally, I run the deployment script provided for Open Event Server to deploy with defined configuration:

./kubernetes/deploy.sh create all

This is the specific deployment script for this project; for your app, this will be a bunch of simple kubectl create * commands specific to your app’s context. Once this script completes, our app is deployed! After a few minutes, I can see it live at the domain I provided above!

I can also see our project’s statistics and set many other attributes at Google Cloud Platform’s console:

And that’s it for this post! For more details on deploying Docker-based containers on Kubernetes clusters hosted by Google, visit Google Cloud Platform’s documentation.

Resources

Open Event Server developers, FOSSASIA, GCE Kubernetes
Docs for Kubernetes with GCE, Google, GCE Docs

Too big Crowd for only One Photo / One of Many Group Photos by Michael Cannon

FOSSASIA Summit 2018 Singapore – Call for Speakers

Post author:Mario Behling
Post published:October 23, 2017
Post category:Event FOSSASIA Open Event
Post comments:0 Comments

The FOSSASIA Open Tech Summit is Asia’s leading Open Technology conference for developers, companies, and IT professionals. The event will take place from Thursday, 22nd – Sunday, 25th March at the Lifelong Learning Institute in Singapore.

During four days developers, technologists, scientists, and entrepreneurs convene to collaborate, share information and learn about the latest in open technologies, including Artificial Intelligence software, DevOps, Cloud Computing, Linux, Science, Hardware and more. The theme of this year’s event is “Towards the Open Conversational Web“.

For our feature event we are looking for speaker submissions about Open Source for the following areas:

Artificial Intelligence, Algorithms, Search Engines, Cognitive Experts
Open Design, Hardware, Imaging
Science, Tech and Education
Kernel and Platform
Database
Cloud, Container, DevOps
Internet Society and Community
Open Event Solutions
Security and Privacy
Open Source in Business
Blockchain

There will be special events celebrating the 20th anniversary of the Open Source Initiative and its impact in Open Source business. An exhibition space is available for company and project stands.

Submission Guidelines

Please propose your session as early as possible and include a description of your session proposal that is as complete as possible. The description is of particular importance for the selection. Once accepted, speakers will receive a code for a speakers ticket. Speakers will receive a free speakers ticket and two standard tickets for their partner or friends. Sessions are accepted on an ongoing basis.

Submission Link: 2018.fossasia.org/speaker-registration

Dates & Deadlines

Please send us your proposal as soon as possible via the FOSSASIA Summit speaker registration.

Deadline for submissions: December 27th, 2017

Late submissions: Later submissions are possible, but early submissions have priority

Notification of acceptance: On an ongoing basis

Schedule Announced: January 20, 2018

FOSSASIA Open Tech Summit: March 22nd – 25th, 2018

Sessions and Tracks

Talks and Workshops

Talk slots are 20 minutes long plus 5-10 minutes for questions and answers. The idea is, that participants will use the sessions to get an idea of the work of others and are able to follow up in more detail in break-out areas, where they discuss more and start to work together. Speakers can also sign up for either a 1-hour long or a 2-hours workshop sessions. Longer sessions are possible in principle. Please tell us the proposed length of your session at the time of submission.

Lightning talks

You have some interesting ideas but do not want to submit a full talk? We suggest you go for a lightning talk which is a 5 minutes slot to present your idea or project. You are welcome to continue the discussion in breakout areas. There are tables and chairs to serve your get-togethers.

Stands and assemblies

We offer spaces in our exhibition area for companies, projects, installations, team gatherings and other fun activities. We are curious to know what you would like to make, bring or show. Please add details in the submission form.

Developer Rooms/Track Hosts

Get in touch early if you plan to organize a developer room at the event. FOSSASIA is also looking for team members who are interested to co-host and moderate tracks. Please sign up to become a host here.

Publication

Audio and video recordings of the lectures will be published in various formats under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. This license allows commercial use by media institutions as part of their reporting. If you do not wish for material from your lecture to be published or streamed, please let us know in your submission.

Sponsorship & Contact

If you would like to sponsor FOSSASIA or have any questions, please contact us via office@fossasia.org.

Links

Speaker Registration and Proposal Submission:
2018.fossasia.org/speaker-registration

FOSSASIA Summit: 2018.fossasia.org

FOSSASIA Summit 2017: Event Wrap-Up

FOSSASIA Photos: flickr.com/photos/fossasia/

FOSSASIA Videos: Youtube FOSSASIA

FOSSASIA on Twitter: twitter.com/fossasia

Submission Guidelines

Dates & Deadlines

Sessions and Tracks

Talks and Workshops

Lightning talks

Stands and assemblies

Developer Rooms/Track Hosts

Publication

Sponsorship & Contact

Suggested Topics

Links