How to teach SUSI skills calling an External API

SUSI is an intelligent  personal assistant. SUSI can learn skills to understand and respond to user queries better. A skill is taught using rules. Writing rules is an easy task and one doesn’t need any programming background too. Anyone can start contributing. Check out these tutorials and do watch this video to get started and start teaching susi.

SUSI can be taught to call external API’s to answer user queries.

While writing skills we first mention string patterns to match the user’s query and then tell SUSI what to do with the matched pattern. The pattern matching is similar to regular expressions and we can also retrieve the matched parameters using $<parameter number>$ notation.

Example :

 My name is *
 Hi $1$!

When the user inputs “My name is Uday” , it is matched with “My name is *” and “Uday” is stored in $1$. So the output given is “Hi Uday!”.

SUSI can call an external API to reply to user query. An API endpoint or url when called must return a JSON or JSONP response for SUSI to be able to parse the response and retrieve the answer.

Rule Format for a skill calling an external API

The rule format for calling an external API is :

<regular expression for pattern matching>
!console: <return answer using $object$ or $required_key$>
{
“url”: “<API endpoint or url>”,
“path”: “$.<key in the API response to find the answer>”,

}
eol
  • Url is the API endpoint to be called which returns a JSON or JSONP response.
    The parameters to the url if any can be added using $$ notation.
  • Path is used to help susi know where to look for the answer in the returned response.
    If the path points to a root element, then the answer is stored in $object$, otherwise we can query $key$ to get the answer which is a value to the key under the path.
  • eol or end of line indicates the end of the rule.

Understanding the Path Attribute

Let us understand the Path attribute better through some test cases.

In each of the test cases we discuss what the path should be and how to retrieve the answer for a given required answer from the json response of an API.

  1. API response in json :

    { 
       “Key1” : “Value1”
    }

Required answer : Value1
Path : “$.Key1    =>   Retrieve Answer:  $object$

 

  1. API response in json :

    { 
      “Key1” : [{“Key11” : “Value11”}]
    }

Required answer : Value11
Path : $.Key1[0]   =>  Retrieve Answer: $Key11$
Path : $.Key1[0].Key11   => Retrieve Answer: $object$

 

  1. API response in json :

    { 
      “Key1” : {“Key11” : “Value11”}
    }


Required answer : Value11
Path : $.Key1  => Retrieve Answer:  $Key11$
Path : $.Key1.Key11  => Retrieve Answer: $object$

 

  1. API response in json :
{ 
  “Key1” : {
               “Key11” : “Value11”,
               “Key12” : “Value12”
           }
}

Required answer : Value11 , Value12
Path : $.Key1  => Retrieve Answer:  $Key11$ , $Key12$

Where to write these rules?

Now, since we know how to write rules let’s see where to write them.

We use etherpads to write and test rules and once we finish testing our rule we can push those rules to the repo.

Steps to open, write and test rules:

  1. Open a new etherpad with a desired name <etherpad name> at http://dream.susi.ai/
  2. Write your skills code in the etherpad following the code format explained above.
  3. Now, to test your skill let’s chat with susi. Start a conversation with susi at http://susi.ai/chat to test your skills.
  4. Load your skills by typing dream <etherpad name> and wait for a response saying dreaming enabled for <etherpad name>
  5. Test your skill and follow step 4 every time you make changes to the code in your etherpad.
  6. After you are done testing, type stop dreaming and if you are satisfied with your skill do send a PR to help susi learn.

Examples

Let us try an example to understand this better.

1. Plot of a TV Show

Tvmaze is an open  TV API that provides information about tv shows. Let us write a rule to know the plot of a tv show. We can find many such APIs. Check out this link listing few of them.

  1.  Open an etherpad at http://dream.susi.ai/ named tvshowplot. 

  2.   Enter the code to query plot of a TV show in the etherpad at                           http://dream.susi.ai/p/tvshowplot

  * plot of *|* summary of *
  !console:$object$
  {
      "url":"http://api.tvmaze.com/singlesearch/shows?q=$2$",
      "path":"$.summary"
  }
  eol
  1. Now lets test our skill by starting a conversation with susi at http://susi.ai/chat.
  • User Query: dream tvshowplot
    Response:  dreaming enabled  for tvshowplot
  • User Query: what is the plot of legion
    Response: Legion introduces the story of David Haller: Since he was a teenager, David has struggled with mental illness. Diagnosed as schizophrenic, David has been in and out of psychiatric hospitals for years. But after a strange encounter with a fellow patient, he’s confronted with the possibility that the voices he hears and the visions he sees might be real. He’s based on the Marvel comics character Legion, the son of X-Men founder Charles Xavier (played by Patrick Stewart and James McAvoy in the films), first introduced in 1985.

Intermediate Processing:

Pattern Matching : $1$ = “what is the” ; $2$ = “legion”

Url : http://api.tvmaze.com/singlesearch/shows?q=legion

API response:

{
   "id": 6393,
   "url": "http:\/\/www.tvmaze.com\/shows\/6393\/legion",
   "name": "Legion",
   "type": "Scripted",
   "language": "English",
   "genres": [
     "Drama",
     "Action",
     "Science-Fiction"
   ], 
   "summary": "<p><strong>Legion<\/strong> introduces the story of David Haller: Since he was a teenager, David has struggled with mental illness. Diagnosed as schizophrenic, David has been in and out of psychiatric hospitals for years. But after a strange encounter with a fellow patient, he's confronted with the possibility that the voices he hears and the visions he sees might be real. He's based on the Marvel comics character Legion, the son of X-Men founder Charles Xavier (played by Patrick Stewart and James McAvoy in the films), first introduced in 1985.<\/p>",
   "updated": 1491955072,  
 }

Note: The API response has been trimmed to show only the relevant content.

Path : $.summary

Retrieving Answer: so our required answer in the api response is under the key summary and is retrieved using $object$ since it is a root element.

 

Screenshots:

2. Cooking Recipes

Let us try it out with another API.
Recipepuppy is an cooking recipe API where users can query various recipes.

  1.  Open a etherpad at http://dream.susi.ai/ named recipe. 

  2.   Enter the code to query a recipe in the etherpad at  http://dream.susi.ai/p/recipe
#Gives recipes and links to cook a dish
* cook *
!console:<p>To cook  <strong>$title$</strong> : <br>The ingridients required are: $ingredients$. <br> For instruction to prepare the dish $href$ </p>
{
  "url":"http://www.recipepuppy.com/api/?q=$2$",
  "path":"$.results"
}
eol
  1. Now lets test our skill by starting a conversation with susi at http://susi.ai/chat.
  • User Query: dream recipe
    Response:  dreaming enabled  for recipe
  • User Query: how to cook chicken biryani
    Response: To cook Chicken Biryani Recipe :
    The ingridients required are: chicken, seeds, chicken broth, rice, butter, peas, garlic, red onions, cardamom, curry paste, olive oil, tomato, coriander, cumin, brown sugar, tumeric.
    For instruction to prepare the dish Click Here!

Intermediate Processing:

Pattern Matching : $1$ = “how to” ; $2$ = “chicken biryani”

Url : http://www.recipepuppy.com/api/?q=chicken biryani

API response:

{
   "title": "Recipe Puppy",
   "version": 0.1,
   "href": "http:\/\/www.recipepuppy.com\/",
   "results": [
     {
       "title": "Chicken Biryani Recipe",
       "href": "http:\/\/www.grouprecipes.com\/53040\/chicken-biryani.html",
       "ingredients": "chicken, seeds, chicken broth, rice, butter, peas, garlic, red onions, cardamom, curry paste, olive oil, tomato, coriander, cumin, brown sugar, tumeric",
       "thumbnail": "http:\/\/img.recipepuppy.com\/413822.jpg"
     },
 ]
 }

Note: The API response has been trimmed to show only the relevant content.

Path : $.results[0]

Retrieving Answer: so our required answer in the api response is under the key results and since it’s an array we are using the first element of the array and since the element is a dictionary too we use its keys correspondingly to answer. The $href$ is rendered as “Click Here” hyperlinked to the actual url.

 

Screenshots:

 

We have successfully taught susi a skill which tells users about the plot of a tv show and a skill to query recipes.
Cheers!
Following similar procedure, we can make use of other APIs and teach susi several new skills.

 

Deploying Susi Server on Google Cloud with Kubernetes

Susi (acronym for Scientific User Support Intelligence) is an advanced AI made by people at FOSSASIA. It is an AI made by the people and for the people.
Susi is an Open Source Project under LGPL Licence.

SUSI.AI already has many Skills and anyone can add new skills through simple console rules.

If you want to participate in the development of the SUSI server you can start by learning to deploy it on a cloud system like Google Cloud.

This way whenever you make a change to Susi Server, you can test it out on various Susi Apps instantly.

Google Cloud with Kubernetes provide this ability. Let’s dig deep into what is Google Cloud Platform and Kubernetes.

What is Google Cloud Platform ?

Google Cloud Platform lets you build and host applications and websites, store data, and analyze data on Google’s scalable infrastructure.
Google Cloud Platform (at the time of writing this article) also provides free credits worth $300 for 1 year for testing out the Platform and test your applications.

What is Kubernetes ?

Kubernetes is an open-source system for automatic deployment, management and scaling of containerized applications. It makes it easy to roll out updates to your application with simple commands from your development machine and scale horizontally easily by adding more clusters as demand increase.

Deploying Susi Server on Kubernetes

Deploying Susi Server on Kubernetes is a fairly easy task. Follow up the steps to get it running.

Create a Google Cloud Account

Sign up for a Google Cloud Account (https://cloud.google.com/free-trial/) and get 300$ credits for initial use.

Create a New Project

After successful sign up, create a new project on Google Cloud Console.
Let’s name it Susi-Kubernetes . 

You will be provided a ProjectID. Remember it for further reference.

Install Google Cloud SDK and kubectl

Go to https://cloud.google.com/sdk/ and see instructions to setup Google Cloud SDK on your respective OS.

After Google Cloud SDK install, run

gcloud components install kubectl

This will install kubectl for interacting with Kubernetes.

Login and setup project

  1. Login to your Google Cloud Account using
$ gcloud auth login

2. List all the projects using

$ gcloud config list project
[core]
project = <PROJECT_ID>

3. Select your project

$ gcloud config set project <PROJECT_ID>

4. Install JDK8 for susi_server setup and set it as default.

5. Clone your fork of the Susi Server Repository

$ git clone https://github.com/<your_username>/susi_server.git
$ cd susi_server/

6. Build project and run Susi Server locally

$ ./gradlew build
$ bin/start.sh

Susi server must have been started started and web interface is accessible on http://localhost:4000

Install Docker and build Docker image for Susi

  1. Install Docker.
    Debian and derivatives:  sudo apt install docker
    Arch Linux:   sudo pacman -S docker 
  2. Build Docker Image for Susi
    $ docker build -t gcr.io/<Project_id>/susi:v1 .
  3. Push Image to Google Container Registry private to your project.
$ gcloud docker -- push gcr.io/<Project_id>/susi:v1

Create Cluster and Deploy your Susi Server there

  1. Create Cluster. You may specify different zone, number of nodes and machine type depending upon requirement.
    $ gcloud container clusters create <Cluster-Name> --num-nodes 2 --machine-type n1-standard-1 --zone us-central1-c
  2. Run your deployment. You may specify any name for deployment.
    $ kubectl run <deployment_name> --image=gcr.io/<Project_id>/susi:v1 --port=80
    $ kubectl get deployments
    $ kubectl expose deployment susi --type=LoadBalancer
  3. Check your deployment and get Public IP for Access.
    $ kubectl get services
    NAME         CLUSTER-IP     EXTERNAL-IP     PORT(S)       AGE
    kubernetes   10.3.240.1     <none>          443/TCP        1d
    susi         10.3.241.145   <PUBLIC_IP>     80:31155/TCP   1d
  4. Go to provided public IP to check, if Susi Server is running.

Congratulations, you successfully setup Susi Server on Google Cloud with Kubernetes.

Updating the deployment

Next step is to update deployment when you wish to roll out changes. To do so.

Build Docker Image and Push it to Google Container Registry

$ docker build -t gcr.io/<Project_Id>/susi:v2 .
$ gcloud docker -- push gcr.io/<Project_Id>/susi:v2

Update Deployment Image with Kubernetes

$ kubectl set image deployment/<Deployment_Name> \
  <Deployment_Name>=gcr.io/<Project_id>/susi:v2
deployment "<Deployment_Name>" image updated

Go to public ip to see the changes.

That’s it. Now, you have fully running Susi Server on your own Google Cloud Cluster using Kubernetes.

Susi AI Skill Development

What is Susi?

Susi is an open source intelligent personal assistant which has the capability to learn and respond better to queries. It is also capable of making to-do lists, setting alarms, providing weather and traffic info all in real time. Susi responds based on skills.

What is a skill? How do we teach a skill?

A skill is a piece of code which performs a set of actions in order to respond to the user’s query. These skills are based on pattern matching which help them mapping the user’s query to a specific skill and responding accordingly. Teaching a skill to Susi is surprisingly very easy to implement. One can take a look at the Susi Skill Development Tutorial and a video workshop by Michael Christen.

I will try to give a basic idea on how to create a skill, it’s basic structure and some of the skills I developed in the first week.

Prepare to create a skill:

  • Head over to http://dream.susi.ai
  • Create a etherpad with some relevant name
  • Delete all text currently present in there
  • Start writing your skill

Adding to this, for testing a skill one can head over to Susi Web Chat Interface.

Basic Structure for calling an API:

<Regular expression to be matched here>

!console:<response given to the user>
 {
 "url":"<API endpoint>",
 "path":"<Json path here>"
 }
 eol

So, let me explain this line by line.

  1. The regular expression is the one to which the user’s query is matched first.
  2. The console is meant to output the actual response the user sees as response.
  3. In place of the “url”, the API endpoint is passed in.
  4. “path” here specifies how we traverse through the response Json or Jsonp to get the object, starts with “$.”.
  5. At last, “eol” which is the end-of-line marks the end of a skill.

Let’s take an example for better understanding of this:

random gif
!console: $url$
{
    "url" : "http://api.giphy.com/v1/gifs/trending?api_key=dc6zaTOxFJmzC",
    "path" : "$.data[0].images.fixed_height"
}
eol 

 

This skill responds with a link to a random gif.

Steps involved:

  1. Match the string “random gif” with the user’s query.
  2. On successful match, make an API call to the API endpoint specified in “url”
  3. On response, extract the object at the specified path in the json under “path”
  4. Respond to the user with the “url” key’s value which would here be an URL of a GIF.

Let’s try it out on Susi Web Chat. For this, you will first have to load your skill using the dream command followed by etherpad name: dream <etherpad name>. And then you can start testing your skill.

So, we queried “random gif” and we got a response “Click Here!”. The complete URL didn’t show up because all the URLs are currently parsed and a hyperlink for each is created. So try clicking on it to find a GIF.

 

Now, let’s look at one more skill I developed during this period.

# Returns the name of the president of a country

 president of *|who is the president of *| president *
 !console:$plaintext$
 {      "url":"https://api.wolframalpha.com/v2/query?input=president+$1$&output=JSON&appid=9WA6XR-26EWTGEVTE&includepodid=Result",
   "path" : "$.queryresult.pods[0].subpods[0]"
 }
 eol

 

Let’s understand this step by step:

  1. We have here “president of *|who is the president of *| president *”, which means the user’s query matches with anyone of the following because of the use of pipe symbol “|”. The “*” here replaces a word or a list of words, which can be accessed like: “${index}$”  where index is replaced by the position of the “*” in the expression starting from 1.
  2. Now we have something new in the URL. See that  $1$  inside the URL? On runtime, that is replaced with the content of the “*” variable. So if a user puts in query like: “president of usa”, “usa” is mapped to $1$ and is replaced in the URL and appropriate API request is made.
  3. Then the path is traversed in the json response and the value of the “plaintext” key is used to respond to the user.

 

It’s now time to try it out on Susi Web Chat.

So, we got our desired response here, i.e., the name of the president of usa.

Displaying error notifications in whatsTrending? app

The issue I am solving in the whatsTrending app is to display error notifications when the date fields and the count field are not validated and when a user enters invalid data. Specifically we want to display error notifications for junk values and dates with formats other than YYYY-MM-DD and any other invalid data in the whatsTrending app’s filter option.

The whatsTrending app is a web app that shows the top trending hashtags of twitter messages in a given date range using tweets collected by the loklak search engine. Users can also limit the number of top hash tags they want to see and use filters with start and end dates.

App to know trending hashtags on twitter

What is the problem? The date fields and the count field are not validated which means junk values and date with formats other than YYYY-MM-DD do not show any error.

So how can the problem be solved? Well the format (pattern) of the date can be verified by regular expression. A regular expression describes a pattern in a given text.So the format checking problem can be described as finding the pattern YYYY-MM-DD in the input date where Y, M and D are numbers.The Regex should specify that the pattern should be present at the beginning of the text.

More detailed information about regex can be found here.

The regex for this pattern is :

/^\d{4}-\d{2}-\d{2}$/

The pattern says there should be 4 numbers followed by ‘-’ then two numbers then again ‘-’ and then again two numbers.

This can be implemented the following way :

$scope.isValidDate = function(dateString) {
        var regEx = /^\d{4}-\d{2}-\d{2}$/;
        if (dateString.match(regEx) === null) {
            return false;
        }

        dateComp = dateString.split('-');
        var i=0;
        for (i=0; i<dateComp.length; i++) { dateComp[i] = parseInt(dateComp[i]); } if (dateComp.length > 3) {
            return false;
        }

        if (dateComp[1] > 12 || dateComp[1] <= 0) { return false; } if (dateComp[2] > 31 || dateComp[2] <= 0) { return false; } if (((dateComp[1] === 4) || (dateComp[1] === 6) || (dateComp[1] === 9) || (dateComp[1] === 11)) && (dateComp[2] > 30)) {
            return false;
        }

        if (dateComp[1] ===2) {
            if (((dateComp[0] % 4 === 0) && (dateComp[0] % 100 !== 0)) || (dateComp[0] % 400 === 0)) {
                if (dateComp[2] > 29) {
                    return false;
                }
            } else {
                if (dateComp[2] > 28) {
                    return false;
                }
            }
        }

        return true;
    }

So the first part of the code checks for the above mentioned pattern in the input. If not found it returns false.If found then we split the entire date into a list containing year, month and day and the remaining part if any is removed.Each component is converted to integer.Then further validation is done on the month and day as can be seen from the code above.The range of the month and date is checked.Also leap year checking is done.

In the same way the count field is also validated. The regex for this field is much simpler. We just need to check that the input consists only of numbers and nothing else.
So the regex for this is :

 /^[0-9]+$/

This means repetition of digits in the range 0-9.We search for this pattern in the text. If found we return true else false.The function for this is as follows:

$scope.isNumber = function(numString) {
        var regEx = /^[0-9]+$/;
        return String(numString).match(regEx) != null;
    }

Next we need to call these function and see if their is any error. If there is an error we need to display it.This can be done using a modal. Bootstrap has got an inbuilt modal which can be invoked using javascript.

Showing error using modal

So at first we need to define the modal and its content (empty if necessary as in this case)using HTML.The HTML code for this can be found here.

A small yet nice tutorial on Bootstrap modal can be found here
Next we need to set the content of the modal and invoke it from our JS file on encountering an error.

$scope.displayErrorModal = function(val, type) {
        if (type === 0) {
            if (!$scope.isValidDate(val)) {
                $scope.loading = false;
                $('.modal-body').html('Please enter valid date in YYYY-MM-DD format'); 
                $('#myModal').modal('show');                 
                return false; 
            } 
         } else { 
             if (!$scope.isNumber(val)) { 
                 $scope.loading = false; 
                 $('.modal-body').html('Please enter a valid number'); 
                 $('#myModal').modal('show');                    
                 return false; 
             } 
         } 
         return true; 
}

The above function accepts a parameter val and another parameter type.The parameter type tells what validation needs to be performed, date validation or number validation and calls previous two methods accordingly and passes val which is the value to validated.If any of the validation fails then it sets the content of the modal using : $(‘.modal-body’).html(“your content”) and then invokes it using : $(‘#modalID’).modal(‘show’). This displays a nice modal on the page and the user is notified about the error.

So this is it for this post.Thanks for reading it.My next post will be on fixing the design of the boilerplate app.

Generating a documentation site from markup documents with Sphinx and Pandoc

Generating a fully fledged website from a set of markup documents is no easy feat. But thanks to the wonderful tool sphinx, it certainly makes the task easier. Sphinx does the heavy lifting of generating a website with built in javascript based search. But sometimes it’s not enough.

This week we were faced with two issues related to documentation generation on loklak_server and susi_server. First let me give you some context. Now sphinx requires an index.rst file within /docs/  which it uses to generate the first page of the site. A very obvious way to fill it which helps us avoid unnecessary duplication is to use the include directive of reStructuredText to include the README file from the root of the repository.

This leads to the following two problems:

  • Include directive can only properly include a reStructuredText, not a markdown document. Given a markdown document, it tries to parse the markdown as  reStructuredText which leads to errors.
  • Any relative links in README break when it is included in another folder.

To fix the first issue, I used pypandoc, a thin wrapper around Pandoc. Pandoc is a wonderful command line tool which allows us to convert documents from one markup format to another. From the official Pandoc website itself,

If you need to convert files from one markup format into another, pandoc is your swiss-army knife.

pypandoc requires a working installation of Pandoc, which can be downloaded and installed automatically using a single line of code.

pypandoc.download_pandoc()

This gives us a cross-platform way to download pandoc without worrying about the current platform. Now, pypandoc leaves the installer in the current working directory after download, which is fine locally, but creates a problem when run on remote systems like Travis. The installer could get committed accidently to the repository. To solve this, I had to take a look at source code for pypandoc and call an internal method, which pypandoc basically uses to set the name of the installer. I use that method to find out the name of the file and then delete it after installation is over. This is one of many benefits of open-source projects. Had pypandoc not been open source, I would not have been able to do that.

url = pypandoc.pandoc_download._get_pandoc_urls()[0][pf]
filename = url.split(‘/’)[-1]
os.remove(filename)

Here pf is the current platform which can be one of ‘win32’, ‘linux’, or ‘darwin’.

Now let’s take a look at our second issue. To solve that, I used regular expressions to capture any relative links. Capturing links were easy. All links in reStructuredText are in the same following format.

`Title <url>`__

Similarly links in markdown are in the following format

[Title](url)

Regular expressions were the perfect candidate to solve this. To detect which links was relative and need to be fixed, I checked which links start with the \docs\ directory and then all I had to do was remove the \docs prefix from those links.

A note about loklak and susi server project

Loklak is a server application which is able to collect messages from various sources, including twitter.

SUSI AI is an intelligent Open Source personal assistant. It is capable of chat and voice interaction and by using APIs to perform actions such as music playback, making to-do lists, setting alarms, streaming podcasts, playing audiobooks, and providing weather, traffic, and other real time information

This API or that Library – which one?

Last week, I was playing with a scraper program in Loklak Server project when I came across a library Boilerpipe. There were some issues in the program related to it’s implementation. It worked well. I implemented it, pulled a request but was rejected due to it’s maintenance issues. This wasn’t the first time an API(or a library) has let me down, but this added one more point to my ‘Linear Selection Algorithm’ to select one.

Once Libraries revolutionized the Software Projects and now API‘s are taking abstraction to a greater level. One can find many API’s and libraries on GitHub or on their respective websites, but they may be buggy. This may lead to waste of one’s time and work. I am not blogging to suggest which one to choose between the two, but what to check before getting them into use in development.

So let us select a bunch of these and give score +1 if it satisfies the point, 0 for Don’t care condition and -1 , a BIG NO.

Now initialize the variable score to zero and lets begin.

1. First thing first. is it easy to understand

Does this library code belongs to your knowledge domain? Can you use it without any issue? Also consider your project’s platform compatibility with the library. If you are developing a prototype or a small software(like for an event like Hackathon), you shall choose easy-to-read tutorial as higher priority and score++. But if you are working on a project, you shouldn’t shy going an extra mile and retain the value of score.

2. Does it have any documentation or examples of implementation

It shall have to be well written, well maintained documentation. If it doesn’t, I am ok with examples. Choose well according to your comfort. If none, at least code shall be easy to understand.

3. Does it fulfill all my needs?

Test and try to implement all the methods/ API calls needed for the project. Sometimes it may not have all the methods you need for your application or may be some methods are buggy. Take care of this point, a faulty library can ruin all your hard work.

4. Efficiency and performance (BONUS POINT for this one)

Really important for projects with high capacity/performance issues.

5. See for the Apps where they are implemented

If you are in a hackathon or a dev sprint, Checking for applications working on this API shall work. Just skip the rest of the steps (except the first).

6. Can you find blogs, Stack Overflow questions and tutorials?

If yes, This is a score++

7. An Active Community, a Super GO!

Yaay! An extra plus with the previous point.

8. Don’t tell me it isn’t maintained

This is important as if the library isn’t maintained, you are prone to bugs that may pop up in  future and couldn’t be solved. Also it’s performance can never be improved. If there is no option, It is better to use it’s parts in your code so that you can work on it, if needed.

Now calculate the scores, choose the fittest one and get to work.

So with the deserving library in your hand, my first blog post here ends.

Ticket Ordering or Positioning (back-end)

One of the many feature requests that we got for our open event organizer server or the eventyay website is ticket ordering. The event organizers wanted to show the tickets in a particular order in the website and wanted to control the ordering of the ticket. This was a common request by many and also an important enhancement. There were two main things to deal with when ticket ordering was concerned. Firstly, how do we store the position of the ticket in the set of tickets. Secondly, we needed to give an UI in the event creation/edit wizard to control the order or position of a ticket. In this blog, I will talk about how we store the position of the tickets in the backend and use it to show in our public page of the event.

Continue reading Ticket Ordering or Positioning (back-end)

The Open Event Ecosystem

{ Repost from my personal blog @ https://blog.codezero.xyz/open-event-ecosystem }

This post contains data collected and compiled from various sources that include (but not limited to) project readme files, my presentation from FOSSASIA 17, wikipedia, my head.

This aims to be a place for any new contributor to know the basics about the entire Open Event Project.

This could also help newcomers choose an Open Event sub-project of their liking and start contributing .

The Open Event Project offers event managers a platform to organise all kinds of events including concerts, conferences, summits and regular meet-ups. The components support organisers in all stages from event planning to publishing, marketing and ticket sales. Automated web and mobile apps help attendees to get information easily.

There are seven components of the project:

  • Open Event Universal Format – A standard JSON Schema specification for data sharing/transfer/import/export between various components of Open Event.
  • Open Event API Server – The core of the project. The API Server and database.
  • Open Event Frontend – The primary frontend for the Open Event API Server where organisers, speakers and attendees can sign-up and perform various functions.
  • Open Event Organiser App – A cross-platform mobile application for Organisers with ticketing, checking and quick management capabilities.
  • Open Event Android application generator – An android application generator that allows event organisers to generate customised android applications for their events.
  • Open Event Web application generator – A web application generator that allows event organisers to generate customised static easily-hostable web pages for their events.
  • Open Event Data scrappers – A scrapper that allows users to scrape events from popular websites such as eventbrite into the Open Event Universal Format for it to be imported into Open Event.

Open Event Universal Format

A standard JSON Schema specification for data sharing/transfer/import/export between various components of Open Event.

Repository: fossasia/open-event.

Open Event API Server

The core of the project. The API Server and database.

Repository: fossasia/open-event-orga-server.

The system exposes a RESTful API to fetch and modify data. It also provides endpoints that allow organisers to import and export event data in a standard compressed file format that includes the event data in JSON (Open Event Universal Format) and binary media files like images and audio.

The Open Event API Server comprises of:

  • Flask web framework – Flask is a microframework for python to create web applications. Flask also provided us with a Jinja2 templating engine.
  • PostgreSQL – Our database. PostgreSQL is an open-sourced Object-relational database management system (ORDBMS). We use SQLAlchemy ORM to communicate with the database and perform database operations.
  • Celery – Celery is a Distributed Task Queue. It allows us to run time consuming and/or resource intensive tasks in the background (or) on a separate worker server. We use celery to process event import/export tasks to process email send requests.
  • Redis – Redis is an in-memory data structure store. It’s generally used for caching and for message brokering b/w different services due it’s insanely fast read-write speeds (since it’s an in-memory data store). We use it for caching results of time-consuming less volatile database calls and also for brokering jobs and their statuses b/w the web server and Celery task queue.

In the near future, we plan to implement more additional components too.

  • Elasticsearch – a distributed, RESTful search and analytics engine. To start with, we’ll be using it to index our events and provide much fast search results to the user.
  • Kibana – data visualization and analytics plugin for elasticsearch. It will allow us to better understand search trends, user demographics and help us provide a better user experience.

We’re now in the process of slowly phasing out the Open Event Server’s existing UI and keep it only as an API Server since we’re moving towards an API-Centric approach with a Fresh new Open Event Frontend.

The Open Event server’s repository contains multiple branches each serving a specific purpose.

  • development – All development goes on in this branch. If you’re making a contribution, please make a pull request to development. PRs to must pass a build check and a unit-test check on Travis (https://open-event-dev.herokuapp.com – Is running off the development branch. It is hosted on Heroku.). This is probably the most unstable branch of all.
  • master – This contains shipped code. After significant features/bug-fixes are accumulated on development, we make a version update, and make a release. (https://eventyay.com – Is running off the master branch. (whichever is the latest release.) Hosted on Google Cloud Platform (Google Container Engine + Kubernetes).
  • staging – This branch is mainly for testing eventyay server configurations on a staging server without affecting the production environment. Push access is restricted to the devops team.
  • gh-pages – This contains the documentation website on http://dev.eventyay.com. The site is build automatically on each commit in the development branch through a script and using travis. It includes the md files of the Readme and /docs folder.

The Open Event Server has Unit tests to ensure each part of the server works as expected. This is also part of our workflow. So, every PR made to the repository, is tested by running the Unit tests and the code coverage is reported on the PR itself. (We use travis and codecov for continuous integration and code coverage reporting respectively).

Open Event Frontend

The primary frontend for the Open Event API Server where organisers, speakers and attendees can sign-up and perform various functions.

Repository: fossasia/open-event-frontend.

Open Event frontend is built on Ember.js – A javascript web application framework. Ember.js uses Ember data – its data persistence module to communicate with the Open Event API Server via the exposed RESTful endpoints.

The frontend is built upon components that are re-usable across various parts of the application. This reduces code duplication and increases maintainability.

The frontend also has various services to share data and functionality b/w components/routes/controllers.

Each merge into the Open Event frontend repository triggers two deployments:

Currently, both the staging and development use the development branch since the frontend is still under active development. Once we reach the release stage, staging & production deployments will start using the master branch just like the Open Event API Server.

As a part of the development workflow, to ensure proper code-styles throughout the project, and to prevent regressions, the project has Acceptance tests, Integration tests, Unit tests and lint checks on both javascript (*.js) and handlebar files (*.hbs). These tests are run for every PR made to the repository. (We use travis and codecov for continuous integration and code coverage reporting respectively).

Open Event Organizer App

A cross-platform mobile application for Organiser with ticketing, checking and quick management capabilities.

Repository: fossasia/open-event-orga-app.

The organiser application is a mobile application that can run on Android, iOS and Windows Phone. It is built using Ionic Framework – cross-platform app development framework. The app uses the RESTful APIs exposed by the Open Event API Server to get data and perform operations.

The core features of this application are

  • Scan a QR code from an attendee’s ticket to view information about the attendee and to check him/her in.
  • Check-in attendees (Attendees can be searched for using name and/or email)
  • Continuous data sync with the Open Event Organiser Server

Other planned feature include:

  • Overview of sales – The organisers can see how their event is performing with just a few taps
  • Overview of tracks and sessions – They can see what sessions are next and can use that information to (for example) get everything ready for that session.
  • Quick session re-scheduling – They can re-schedule sessions if required. This should also trigger notification to participant that have registered for that event.
  • Push notifications for certain triggers – (for example) Organisers can get notifications if any session proposal is received.

This project has only one development branch (master). Each commit to that branch triggers an apk deployment to the apk branch via Travis.

Open Event Android Application Generator

An android application generator that allows event organisers to generate customised android applications for their events.

Repository: fossasia/open-event-android.

This project consists of two components.

  • The App Generator – A web application that is hosted on a server and generates an event Android app from a zip with JSON and binary files (examples here) or through an API.
  • A generic Android app – the output of the app generator. The mobile app can be installed on any Android device for browsing information about the event. Updates can be made automatically through API endpoint connections from an online source (e.g. server), which needs to defined in the provided event zip with the JSON files. The Android app has a standard configuration file, that sets the details of the app (e.g. color scheme, logo of event, link to JSON app data).

  • Flask web framework – Flask is a microframework for python to create web applications. Flask also provided us with a Jinja2 templating engine.
  • Celery – Celery is a Distributed Task Queue. It allows us to run time consuming and/or resource intensive tasks in the background (or) on a separate worker server. The android application compile and build process is run as a celery job. This also allows multiple users to simultaneously use the generator.
  • Redis – Redis is an in-memory data structure store. It’s generally used for caching and for message brokering b/w different services due it’s insanely fast read-write speeds (since it’s an in-memory data store). We use it for brokering jobs and their statuses b/w the web server and Celery task queue.
  • Java Development Kit (JDK) – Java Development Kit provides us with a set of tools consisting of (but not limited to) a compiler, runtime environment, loader which enables us to compiler, build and run java based applications.
  • Android SDK Tools – The Android SDK Toolset provides us with Android API libraries, debugging tools, emulation capabilities and other tools that are needed to develop, build and run java based android applications.
  • Gradle Build Tool – Gradle is an open source build automation system. It allows developers to define a build process as a set of tasks that can be easily executed on any machine with predictable outputs as long as the gradle files are available.

As with other projects, this also has multiple branches each serving a different purpose

  • development – All development goes on in this branch. If you’re making a contribution, you are supposed to make a pull request to development. PRs to master must pass a build check and a unit-test (app/src/test) check on Travis.
  • master – This contains shipped code. After significant features/bugfixes are accumulated on development, we make a version update, and make a release. All tagged commits on master branch will automatically generate a release on Github with a copy of fDroid-debug and GooglePlay-debug apks.
  • apk – This branch contains two apk’s, that are automatically generated on merged pull request a) from the dev branch and b) from the master branch using the Open Event sample of the FOSSASIA Summit. This branch also assists in deploying the generator to http://droidgen.eventyay.com by triggering a travis build every time an apk is pushed to this branch. The reason this type of a round-about way was implemented is that, travis doesn’t allow android and docker builds on the same branch. So, we’re forced to use the apk branch to do the docker build for us.

Open Event Web application generator

A web application generator that allows event organisers to generate customised static easily-hostable web pages for their events.

Repository: fossasia/open-event-webapp.

The Open Event Web App project has two components :

  • An event website generator – A web application that generates the event website based on event data provided by the user either in the form of an archive as per the Open Event Universal Format or an API Endpoint to an Open Event API Server instance
  • A generic web application – This will be customised and additional pages will be generated from a set of templates by the generator based on the provided event data. The generated website bundle can be hosted on any static hosting provider (Github pages, any simple web hosting service, or any web server that can serve HTML files etc).

The generator also gives you the ability to directly upload the generated files to any server (via FTP) or to the gh-pages branch of any repository on GitHub.

  • Express.js – A web application framework for Node.js. The web application generator’s user-facing frontend runs on Express.js framework.
  • Socket.IO – A javascript library for real-time web applications. It allows a client and a server to communicate with each other simultaneously in a real-time manner. (Confusing ? If you wanna build something like a chat-room … you’d need something like Socket.IO). The web application generator uses Socket.IO to handle multiple clients and also to show the correct progress of the website generation to each of those clients in a real-time manner.
  • The web generator also uses SASS – which adds awesome features (Superpowers as their site says) to good-old CS, and Handlebars – which is a templating engine that let’s you easily generate HTML output based on a templates from a given javascript object.

The branches are similar to other projects under Open Event.

  • development – All development goes on in this branch. If you’re making a contribution, you are supposed to make a pull request to development. PRs to master must pass a build check and a unit-test (test/serverTest.js) check on Travis. Gets deployed automatically to https://opev-webgen-dev.herokuapp.com .
  • master – This contains shipped code. After significant features/bugfixes are accumulated on development, we make a version update, and make a release.

Open Event Data Scrappers

A scrapper that allows users to scrape events from popular websites such as eventbrite into the Open Event Universal Format for it to be imported into Open Event.

Repository: fossasia/event-collect.

As of now, only eventbrite is supported for data scrapping. This allows new users of Open Event to import their events from other ticketing/event management websites very easily. The scrapper accepts a query from the user, and gets the event page for that query on eventbrite, parses the HTML DOM to get the required data and compiles everything into the Open Event Universal Format so that it could be imported into Open Event.

The scrapper is written in Python and uses Beautiful Soup to parse the DOM.

Future plans include combining fossasia/query-server, Open Event server and event collect to enable automatic import of events from other websites based on user searches on eventyay.


The Open Event Ecosystem is vast and has plenty of contribution opportunities to both beginners and experts alike. Starting with contributions is as easy as picking a project, choosing an issue and submitting a PR. No strings attached. (Oh wait. there is just one string attached. Ensure you read & follow the “Best Practices at FOSSASIA Guide”)

Open Source Developer Guide and Best Practices at FOSSASIA

I would request you to please guide me as to how can I collaborate with you and also how can I contribute effectively to the organization.

At times I might get up to 20 private mails per day about “How to contribute to Open Source”. Usually I will direct developers to our mailing list or chat channels if they are asking about a specific project. But, how do contributions work at FOSSASIA? How can new developers join projects effectively? We have tried many different ways and spent a lot of time communicating with newcomers, many of them new to Git and Open Source development.

Over the years we have gotten better at helping new developers, designers and other contributors to join up. We have more than 1300 registered developers in our GitHub organization today. Not all of them can contribute every day, but we do have thousands of commits every year.

So, how are we able to scale up? I believe one reason are our Best Practices. We didn’t have a document “FOSSASIA Best Practices” yet, but in our daily work we have established excellent (mostly unwritten) best practices, that I would like to share in a concise form here now as a reference for new developers.

Happy to get your feedback and comments.

Development Best Practices at FOSSASIA

Culture and Communication

  • Please adapt your language to non-native English speakers and be super friendly. Remember we are an international community with contributors mainly from Asia and Europe. We are not used to swearing and will mostly understand words literally. At all times ensure your tone stays appropriate and friendly in the FOSSASIA community.
  • Stay modest. Sometimes newcomers already have an amazing knowledge in some areas. Still remember, it is no reason to be arrogant. Even the best developer does not know everything.
  • Be nice and welcoming. Why not add “please” in an issue or a comment “Thank you” in a pull request if someone did a good job? Show your appreciation for the good work of your co-developers.
  • If you are involved in a topic you don’t understand yet, try to learn yourself as much as possible from public channels (wikipedia, stackoverflow) but also do not hesitate to ask questions to other developers.

Communication Channels

Every project at FOSSASIA has its own channels. Many projects use Gitter, some Slack or IRC. Usually you will find information of the main communication channels of a project in the Readme.md.

While we are a community of Open Source and Free Software developers we still reserve our right to use whatever tools we deem necessary to help us to achieve our goal of developing the best possible Open Technologies – software and hardware. It is one step at a time 🙂

Private and Public Chat or Issue Tracker

Chat is a great way to connect with other contributors, but not all contributors are there all the time (consider time difference and personal schedules) and they are not always available to chat. Chat tends to be unstructured and with lots of people in a room there are often a number of threads. Therefore chat is great for help on setup issues and things where you got stuck.

Do not use chat for feature requests and detailed discussions on issues. These things are best for the issue tracker, where people from different timezones can join and where a focused conversation on one specific topic can happen.

Some people try to overcome the unstructured chats by switching to private communication. This shuts out other contributors who might have similar issues. A result I often observed is also, that contributors will bring up arguments in discussions like “I have discussed this already with xyz privately and he agrees”. Other contributors have not seen this discussion if it has not been taken place in public and we haven’t seen the arguments. We don’t know if xyz really agrees or if it was misunderstood. Private conversations are not transparent. Surely, there are cases where private chat is needed, e.g. for specific deployment questions of projects, but whenever you talk about development, please switch to public chat or open an issue.

Feature Requests and Bug Reports

  • Some newcomers are not accustomed to issue trackers and might try to inform developers on the mailing list, chat or even write private emails about bugs and features, but the right place to do this is: The issue tracker of a project.
  • Any bug or feature, please open an issue in the issue tracker right away and indicate if you want to work on it yourself.
  • Please include all relevant information when you submit an issue and wherever possible a link, information about the code that has issues and a screenshot.
  • When you file a bug report on the issue tracker, make sure you add steps to reproduce it. Especially if that bug is some weird/rare one.

Join Development

  • Before you join development, please set up the project on your local machine, run it and go through the application completely. Press on any button you can find and see where it leads to. Explore. (Don’t worry. Nothing will happen to the app or to you due to the exploring. Only thing that will happen is, you’ll be more familiar with what is where and might even get some cool ideas on how to improve various aspects of the app.).
  • If you would like to work on an issue, drop in a comment at the issue. If it is already assigned to someone, but there is no sign of any work being done, please free to drop in a comment so that the issue can be assigned to you if the previous assignee has dropped it entirely.

Commits/Pull Requests

  • All pull requests need to be associated to an issue.
  • All PRs need to be assigned to the person working on it.
  • If an issue cannot be completed in less than a day, it should be broken up into multiple issues.
  • Make pull requests from your own forks (even if you have write rights to the repository, do not create new branches, develop on your own branches).
  • State the actual change or enhancement in the commit message of PRs (do not just state “Fixes issue #123”).
  • Add the issue number into the description (this helps to speed up reviews as reviewers can simply click to get more info in the issue itself).
  • Write clear meaningful git commit messages (Do read http://chris.beams.io/posts/git-commit/).
  • Match pull requests with issues and make sure your pull requests description contains GitHub’s special keyword references that automatically close the related issue when the pull request is merged. (More info at https://github.com/blog/1506-closing-issues-via-pull-requests).
  • When you make very minor changes to a pull request of yours (like for example fixing a failing travis build or some small style corrections or minor changes requested by reviewers) make sure you squash your commits afterwards so that you don’t have an absurd number of commits for a very small fix (Learn how to squash at https://davidwalsh.name/squash-commits-git).
  • Add a screenshot if you changed anything in the UI of a project. When you’re submitting a pull request for a UI-related issue, please add a screenshot of your change or a link to a deployment where it can be tested out along with your pull request. It makes it much easier for the reviewers and helps to speed up the process. You’ll also get reviews quicker.
  • Add a link to your deployment of the project, where reviewers can check out what you have changed (especially for smaller changes this is very helpful as the reviewer might not even need to set up the system itself and test it. Again this speeds up the review process a lot).
  • Always ensure CI and tests are successful.
  • Help to resolve merge conflicts (especially if there are several PRs at the same time, merge conflicts are common. Help the reviewers and solve merge conflicts to speed up the process.).

Scope of Issues and Commits

  • Stay focused on the issue and its specifics: Sometimes it is tempting to do more changes in a pull request and simply add a nice little feature after mentioning it in the issue tracker. Please do not do this. Contributors will look at the title of issues usually to check if it is relevant for them. For example, if an issue is about changing a font, do not also change the color even if this seems like small change to you. Many projects have a design template and standard colors etc. that they want to achieve. So your changes might need to be discussed in a bigger setting even if they seem small to you. Same applies to many other areas.
  • Do only the changes in a pull request that are mentioned in the issue. Do not change anything else without ever mentioning it (remember match issues with pull requests).

Branch Policies

Most FOSSASIA Projects have:

  • a development branch (which is the working branch. Please commit to this branch.) and
  • a master branch (which is the stable branch).

Some projects also keep additional branches e.g.:

  • gh-pages for documentation purposes (often autogenerated from md-files in docs folder)
  • apk branches for Android apps (often autogenerated with travis).

Getting Started

  • Newcomers are sometimes afraid to make a pull request. Don’t be! It is the responsibility of reviewers to review them. And Git is a beautiful tool when it comes to reverting pull requests with errors.
  • In simple issues keep it simple and “simply” contribute, e.g. in an issue “change the color of the button from green to red”, there is no need to mention and ask “here is a screenshot where I changed the color to red. Should I make a PR now?”. Just make the pull request and wait for the feedback of the reviewer.
  • Take on responsibility early and help to review other contributions. Even though you do not have write access in a repository you can already help to review other commits.

Documentation

  • Please ensure you provide information about your code and the functioning of your app as part of every code contribution.
  • Add information on technologies and the code into the code itself and into documentation files e.g. into the Readme.md or /docs folder. Documentation is not a thing that should be done at the end after a few weeks or months of coding! Documentation is a continuous effort at every step of software development.
  • If you add new libraries or technologies add information about them into the Readme file.
  • If you implement a different design or change the css in a web project, please add information to the design specifications/template. If there are not any design specifications in the project yet, why not start them and add a section into the Readme.md to start with?
  • Always help to keep documentation up to date and make it easy for new contributors to join the project.

 

Thank you for helping to define many of the practices while working on the Open Event project to the developer team and for additional input and links to Niranjan Rajendran.

Links to Other Resources

FOSSASIA Summit 2017 Wrap Up

The FOSSASIA Summit 2017 was an unforgettable event thanks to everyone who helped to make it possible! We would like to thank our co-organiser the Science Centre Singapore and all sponsors, supporters, speakers and volunteers. Below are interesting numbers and facts of 2017 and information on highlights of the event.

FOSSASIA SUMMIT 2017 NUMBERS & FACTS

  • 3,145 people attended the event over 3 days including 229 speakers and 60 volunteers.
  • 41 nationalities participated in the summit: 70.8% from Singapore, followed by India, Indonesia, Germany, China, Japan, Vietnam and many others
  • There were 23.6% female attendees.
  • 60% of attendees were IT professionals.
  • 5 keynotes, 231 scheduled sessions, 22 lightning talks, and over 30 projects and companies presented their work in the exhibition.
  • Talks are already available as videos. Hundreds of photos have been uploaded to social networks. 2000+ tweets [tw] with the FOSSASIA hashtag were posted during the event.

FOSSASIA SUMMIT 2017 HIGHLIGHTS

A three-day program with nearly 20 parallel tracks made FOSSASIA Summit the biggest open tech event in the region. One very interesting fact was the entire conference was fully managed by FOSSASIA built open source event management system, EventYay. All the technical setting was also done in-house by the FOSSASIA Team. In the effort of making the event the best experience for visitors, FOSSASIA team organized a series of extracurricular activities including pre-event meet&greet, pub crawl, culture walk, social event, see you again cocktails, lucky draw etc.

Day 1 Opening Day with Keynotes

Chan Cheow Hoe, GovTech’s Chief Information Officer, emphasized how the Singapore Government’s central information technology systems and infrastructure drive the development and delivery of innovative public services for citizens and businesses.

Chan Cheow Hoe, GovTech’s CIO, photo by Nguyen Thi Tra My

Follow-up by an interesting story by Øyvind Roti who currently leads Google’s international team of Cloud Architects. He spoke about how to get involved and contribute to the Google Cloud Open Source products and related projects, including machine learning, systems, client-side libraries and data analytics tools.

Øyvind Roti, photo by Gabriel Lee

Andrey Terekhov brought Microsoft into the Open Source picture with some insights that many were not aware of. MS actually are the top contributors to Github and they are hosting many Open Source projects themselves. Andrey explained in details Microsoft’s open source strategy and developing business in Asia Pacific region, with a particular focus on scaling up open source workloads on Microsoft Azure cloud platform.

Andrey Terekhov, Open Source Sales & Marketing Lead at Microsoft, photo by Kai En Mui

The final keynote of the day was conducted by a German privacy activist – Frank Karlitschek the founder of ownCloud and later Nextcloud, an open source and fully federated and distributed network for files and communication. As the topic of the privacy and personal data on the internet are under attack by hackers and international espionage programs, Frank shared with the audience how the Internet can be used as a free and democratic medium again. 

Open Source AI Topics 

The highlight of the day was the introduction of SUSI AIFOSSAISA’s Open Source Personal Assistant. Michael Christen, founder and also core developer talked about SUSI’s current development stage as well as project’s ambition and the plan for the future. He demonstrated some amazing things you can do with SUSI such as searching for locations, finding translations in over 100 languages, asking SUSI travel information, weather etc. One of the exciting features is the auto-improvement ability: the more you interact with SUSI, the better and accurate its answers become. Michael also showed the audience how they can actually contribute and create the largest corpus of knowledge for SUSI AI Assistant.

Michael Christen about SUSI AI, OpenAI and the role of Elon Musk, photo by Michael Cannon

Liling Tan, a data scientist from Rakuten, spoke about Natural Language Processing (NLP) which is the task of the computationally understanding and production of human languages, often using a mix of hand-crafted rules and machine learning techniques. Konrad Willi Döring brought AI to next level when he presented the Brainduino Project including a brief introduction to EEG-based brain-computer interfaces as well as a look into the future of BCI technology.

Konrad Willi Döring Brainduino Project, photo by Michael Cannon

FOSSASIA’s favorite speaker, Andrew “bunnie” Huang, came back with “Let’s Make Technology more Inclusive”. Bunnie and his team examined some of the cultural and technological barriers that have stymied inclusiveness, using gender imbalance as a case study. They later on, proposed a solution called “Love to Code”, which attempts to address the issue of inclusiveness in technology. 

The day finished with a panel discussion on The Future of AI with a diverse group of  five panelists: Andrew Selle (Google Brain Team, US), Steffen Braun (KI Group), Michael Christen (SUSI AI), Harish Pillay (Internet Society), Bunnie Huang (Chibitronics PTE LTD)

It was a very interactive session between speakers and attendees, discussing the possibilities and implications of AI.

AI Panel, photo by Michael Cannon

CodeHeat Award Ceremony

From September 2016 to February 2017, FOSSASIA held a CodeHeat contest to encourage more developers to get involved and contribute to the FOSSASIA open source projects, namely Open Events Orga Server, AskSUSI project, and LokLak. 442 developers had joined the contest, over a thousand pull requests were made during over this 6 months period of CodeHeat. Three winners and two finalists from the top 10 contributors who have contributed awesome code were chosen to fly to Singapore for the FOSSASIA Summit 2017 to share what they’ve done, and meet the open source community gathered here.

CodeHeat Award Ceremony, photo by Michael Cannon

PubCrawl

A get-together at Pubcrawl has become a tradition of every FOSSASIA Summit. At the end of the first day,  speakers and participants met at Chinatown and started a fun evening strolling around various pubs, tasting local beverages and specialties. The hang-out has always been a great opportunity for speakers to carry on their unfinished conversations during the day as well as to enhance the friendship among visitors and residents.

Pub Crawl, photo by Ben Sadeghi
Andrew “bunnie” Huang, Brady Forrest and Sean “Xobs” Cross at the Pub Crawl, photo by Ben Sadeghi

Day 2 Extensive Day of Workshops and Presentations  

FOSSASIA Summit Day 2 is always the busiest day with an extensive program starting from 9 am until 6:30 pm. Dedicated tracks included Startup and Business Development – Database PGDay – Open Tech Google Track – Python – Hardware & Making DevOps  – Security and Privacy – Science – Android – Debian Mini-Debconf – Tech Kids – Open Source Software – Health Tech – Web & Mobile – Kernel & Platform – AI & Machine Learning

Open Tech – Google Open Source Track

Stephanie Taylor, the Program Manager at Google Open Source Outreach team gave an educational talk about Google Code-in program as an early opening of the Google’s Open Tech Track. This introduction was favored by local students as well as young international developers. In the following topic about Future of the Web, Anuvrat Rao introduced the latest open technology to address critical user needs on the open web.

Stephanie Taylor and GCI 2016 Students

Andrew Selle from Google Brain Team carried on the session with an overview of the open source software library TensorFlow and discussed how the open source community has shaped its development and success. Devan Mitchem introduced The Chromebook, a new, faster computer that offers thousands of apps. He also showed the audience how to integrate and experience Android apps on this machine for greater productivity and flexibility. Denis Nek wrapped up Google’s Tracks by a talk about Model–view–viewmodel (MVVM), a software architectural pattern. In this last topic, he explained why and how he could solve many common problems of android developers using this approach.

Tech Kids Track

Followed up the success of 2016’s summit, FOSSASIA 2017 extended Tech Kids Track throughout its 3-day event. Many parents brought their kids along to attend the talks and workshops. Most importantly, these young attendees showed their great interest in Open Technology. The kids’ voluntary participation in the tracks completed the aim of FOSSASIA in fostering education at a young age. With the power of open knowledge, we believe the bright future of world leaders start from today’s education.

Elda Webb and Creativity workshop, photo by Ka Ho Ying

Kids workshops covered topics such as Git for beginners, software translation with WebLate, PyGame 101 Codelab, how to developer your first mobile app, make a DIY paper spectrometer, create a promotion video with open source tools etc.

Kids and guardians learn how to work with Git, photo by Ka Ho Ying

Science Track – Mission Mars 

This fun and educational workshop was organized by Microsoft Open Source Team.  In this rescue mission, attendees learned to create a bot using an open source framework and tools. They were given access to MS code repositories and other technical resources. Workshop participants had to complete 3 missions and 2 code challenges in order to bring the Mars mission back on track. It was pretty challenging but at the same time super exciting.

Mission Mars’ Winner and Mentors

Python Track

Python Track has always attracted good audience’s response since 2015. In this year summit, the track covered very informative topics ranging from metaclasses in Python 2 and 3, computing using Cython to Go-lang (a new open source programming language), Pygame 101, the effective use of python in Science and Maths with live demos of successful experiments etc. 

PyGame 101 Codelab Workshop

A 2-hour workshop was conducted by Kushal Das giving the audience the overview of MicroPython, how to update NodeMCU devices with MicroPython firmware and using sensors with NodeMCU for their first IoT device.

MicroPython workshop using NodeMCU
Python Mentors, photo by Ka Ho Ying

Database Track – PostgreSQL Day

This was the second year FOSSASIA hosted PGDay. We were delighted to welcome amazing speakers like Dr. Michael Meskes (founder and CEO of credativ Group), Maksym Boguk (co-founder of PostgreSQL consulting), and many other  PostgreSQL developers and consultants across the globe.

It was very interesting to learn how an open source database, PostgreSQL, has rapidly extended its application into the enterprise sector, one of the examples was how PostGIS is being by agricultural producer in Australia.

PGDay at FOSSASIA Summit 2017

Day 3 More sessions and the final keynote by Daimler’s Representatives 

Day 3 Dedicated Tracks consisted of Hardware & Making – Tech Kids – Science – Android – Blockchain – Open Tech – AI & Machine Learning – Internet, Society & Politics – Web & Mobile – Security and Privacy – DevOps – Database MySQL Day – Design, Art, Community – Open Source Software.

It was wonderful to have two special guests from Daimler headquarter in Stuttgart – Jan Michael Graef (CFO of CASE) and Vlado Koljibabic (leads IT for the new CASE business and COO of the Digital and IT organization). The presence of Daimler, a traditional corporate business in the open source world was not only well received by the audience but also triggered an excitement and the curiosity of the crowd: What is the background of the growing involvement and support of Open Source by Daimler?

Daimler in the house: Danial, Vlado, Hong Phuc, Jan and Mario, photo by Michael Cannon

Daimler AG is known for one of the world’s most successful automotive companies. With its Mercedes-Benz Cars, Daimler Trucks, Mercedes-Benz Vans, Daimler Buses, and Daimler Financial Services divisions. The Group is one of the leading global suppliers of premium cars and is the world’s largest manufacturer of commercial vehicles. At FOSSASIA Summit 2017, Jan and Vlado made an introduction to CASE – these letters will shape the future of Mercedes-Benz Cars. They stand for the strategic pillars of connectivity (Connected), autonomous driving (Autonomous), flexible use (Shared & Services) and electric drive systems (Electric), which will be intelligently combined with one another by the company.

Jan Michael Graef and Vlado Koljibabic from Daimler, photo by Ka Ho Ying

In their talk Vlado and Jan outlined how Daimler recognizes the power of Open Source development and we had the chance to get insights into some very exciting ideas how Daimler is planning to shape the logistics sector with services based on Open Source technologies. The company is even considering cryptocurrency payments for services in the future and is already working on using Blockchain technologies for its automobile services for logistics companies.

Daimler is looking for outstanding developers to build some very exciting solutions based on Open Source around cars and much more. Please check out Daimler’s job opportunities here.

Web & Mobile Track – featured OpenEvent (EventYay) System

Finally, there is an Open Source event management system said Mario Behling, founder of open-event (eventyay) and the summit’s co-organiser. During the last two years, the FOSSASIA team has been working on a complete functional open source solution for event organisers. More than 5,000 commits have been made from more than 100 developers worldwide. The hosted solution of the application is available at EventYay.com and ready to be tested as an Alpha product.

The system enables organizers to manage events from concerts to conferences and meet-ups. It offers features for events with several tracks and venues. Event managers can create invitation forms for speakers, build schedules in a drag and drop interface, implement ticketing system and much more. The event information is stored in a database. The system also provides API endpoints to fetch the data, and to modify and update it. Organizers can import and export event data in a standard compressed file format that includes the event data in JSON and binary media files like images and audio.

OpenEvent Scheduler – Drag & Drop interface

The Open-event core team of 7 senior developers came together at the FOSSASIA summit to showcase the latest development, make live demos, conduct deployment workshops and discuss future applications.

Featured Open Event presentations and workshops:

    • Better Events with Open Event | Mario Behling
    • Deploy Open Event Organizer Server | Saptak Sengupta
    • Scaling Open Event Server with Kubernetes | Niranjan Rajendran
    • Open Event API | Avi Aryan
    • Open Event Web App | Aayush Arora
    • An Introduction to the Open Event Android Project and it’s capabilities| Manan Wason
    • Agile Workflow and Best Practices in the Open Event Android App Generator Project | Harshit Dwivedi

Database Track – MySQL Day

This year FOSSASIA proudly hosted MySQL Day within the database track.  12 senior developers/speakers from Oracle around the world got together at the summit. 14 scheduled talks and workshop were conducted. Beginning with Sanjay Manwani, MySQL Director from India, he talked about ‘the State of the Dolphin’, sharing an overview of the recent changes in MySQL and the direction for MySQL 8 as well as an introduction to Oracle cloud. The day continued with selective topics from MySQL optimizer features to in-depth workshops such as MySQL operations in Docker – workshop or MySQL Performance Tuning.

MySQL Team, photo by Mayank Prasad

Additionally, Ricky Setyawan organized an unconference session or a MySQL Community Meetup Space where he invited the community members to meet and to start a direct conversation with MySQL’s developers. 

See you again Cock-Tails 

After the closing session, FOSSASIA attendees were invited by Daimler to join an after-event cocktail party. People were happy for the chance to finish up their discussions while enjoying the nice view of the city from a spacious balcony with finger food, drinks and good music from the local band.  

Engineers.SG Team, photo by Ka Ho Ying
Photo by Nguyen Thi Tra My
FOSSASIA regular friends Felix Dahmen, Joerg Henning & Emin Aksehirli, photo by Guness
Music performance by a local band

Exhibition and Networking Space at FOSSASIA Summit

The biggest goal of the FOSSASIA Summit is to bring people across borders together at a physical space where they can freely share, showcase, discuss and collaborate on existing projects or new ideas. We are happy to see many open source communities across Asia at this year’s gathering. What could be better than a face-to-face discussion over coffee with people who shared the same vision and belief: ‘With open technologies, we can make the world a better place’

Google Cloud Team at FOSSASIA
Dietrich Ayala from Mozilla sharing details about A-frame with attendees
Open Hardware corner with Dan and Kiwi, FOSSASIA organizers
Sindhu Chengad explained Open Source at Microsoft
OpenSUSE Booth
Michael Meskes (right) and Engineer from Credativ Germany
Matthew Snell from xltech.io, Singapore
MySQL Team, photo by Michael Cannon
Thomas Kuiper from gandi.net, Taiwan, photo by Michael Cannon
Men gathering at Pyladies Pune table
Wan Leung Wong from TinyBoy 3D printer project, Hongkong
Fresh coffee in the house

FOSSASIA WHAT’S NEXT?

Mark your calendar for the next FOSSASIA Summit, which will take place in March 2018. We are looking forward to seeing you again in Singapore. If you are meetup organizers, community leaders, we would like to invite you to host a track at the next FOSSASIA Summit, please write to us about your experience and contribution in the open source world via [email protected]

As always thanks to Michael Cheng and Engineers.SG team for all the videos, thanks to our photographers Michael Cannon, Ka Ho Ying and the team for capturing some of the very best moment of us. You can search for more photos by typing #fossasia on loklak (or alternatively on Twitter) or Flickr. If you also want to share photos you took during the summit, please add them to the group pool.

Another Group Photo by Michael Cannon

Blog Posts by Attendees

https://www.credativ.co.uk/credativ-blog/throwback-foss-asia-2017

http://yadav-pooja.blogspot.sg/2017/03/fossasia-wonderful-experience.html

https://isantoshv.wordpress.com/2017/03/24/speaking-at-fossasia-2017

https://railsgirlssummerofcode.org/blog/2017-03-27-team-reactives-at-foss-asia

Ten ways in which FOSSASIA ’17 helped me grow

http://medwinz.blogspot.co.id/2017/04/fossasia-2017.html

https://canaan.io/2017/03/20/canaan-at-fossasia-2017-blockchain-software-for-hardware

https://urvikagola.wordpress.com/2017/03/25/speaking-at-fossasia17-seasons-of-debian-summer-of-code-winter-of-outreachy

http://rishiraj.co/2017/FOSSASIA-Summit-Singapore-and-CodeHeat/

http://blog.perfinion.com/2017/03/fossasia-2017-selinux-introduction

https://www.percona.com/foss-asia-2017

http://open.microsoft.com/2017/03/14/get-ready-fossasia-summit-2017

http://saptak013.blogspot.sg/2017/03/in-heat-of-code-mentors-pov.html

Links

FOSSASIA Summit 2017 Photos: https://www.flickr.com/groups/fossasia-2017-singapore/pool

FOSSASIA Summit 2017 Feedback form: tell us how we can make it better for you

FOSSASIA Videos: https://www.youtube.com/fossasiaorg

FOSSASIA Projects: http://labs.fossasia.org

FOSSASIA Repositories: https://github.com/fossasia

FOSSASIA on Twitter: https://twitter.com/fossasia

FOSSASIA on Facebook: https://www.facebook.com/fossasia

FOSSASIA SG Meetup: http://www.meetup.com/FOSSASIA