Mentoring in Google Code In

I worked on sTeam under FOSSASIA for my Google Summer of Code. Now on passing successfully I will be mentoring in Google Code In and help the younger minds in getting into open source development. I have put up three tasks for GCI. The tasks are an extension on my GSoC project.

The first task involves testing and writing test cases. These are not on any normal testing framework, the framework is developed on pike and specially for the project. Student will have to test the calls to COAL functions. COAL is an in house developed protocol. The task will take around a week to complete. Writing test cases is not a big task but understanding the framework, the protocol and identifying the cases is difficult. The testing framework and test cases already written can be found here.

Second task involves improving the testing framework. The testing framework has been recently developed and still requires some polishing. This task involves identifying and fixing errors in the framework and also extending the framework to add new features. New features can include a better summary of the test cases executed and more details on the errors the occurred if any.

output of testing framework
Output of Testing Framework

Third task involves extending the group command on the steam command line interface, aka, steam-shell. The group command was recently added to the shell and provides only limited functions. Student will be required to understand the use and functionalities of groups from the web interface and code these functionalities for the command line interface. This task will not require more than five days as the interface is easy to understand.

The web interface can be found at http://steam.realss.com/ , students will have to create an account and login to use the interface.

The code for steam-shell can be found at

https://github.com/societyserver/sTeam/blob/gsoc2016-societyserver-devel/tools/steam-shell.pike

Students might face troubles in installing the steam command line interface and tools, therefore we are working on solution to solve this issue. We might provide a debian package that installs the software and then the source code for the tools that need to be worked upon can be downloaded from github and executed.

GSoC 2016 Summary of work done – Improving sTeam

To understand my project you first need to understand what sTeam is. For that you can refer to the blog I wrote http://blog.fossasia.org/what-is-steam/

FOSSASIA_Logo.svggsoc2016-sun-373x373

I started off small by fixing already existing bugs. There were multiple bugs with the edit command in the command line interface of sTeam. I extended the edit command to allow opening of multiple files as tabs in a vim editor. To provide users with more options and to make working on sTeam client easier I added the feature to open new files in steam directly from inside vim. I wrote a vim script to do this and communicated with the sTeam server through this script. My first major task was to implement a TLS connection between the sTeam command line client and the server. For this I had to understand the COAL protocol, which is a home grown protocol for sTeam. I improved the tooling for sTeam by adding in commandso work with groups from the steam-shell and to allow re-login from debug.pike. After this I did some cleaning up work by removing repeated code. The code for login was being repeated in different tools so I made a separate file containing all the common code and imported this in all the tools. I wrote an extensive help command describing every command for steam-shell and giving their syntax. There was some conceptual error in the steam-shell. Rooms and gates are the same but gothrough command was allowing the users to enter a gate but not a room, this was changed to enter command supporting both gates and room, I also changed the output of look command to not show gates and rooms as separate entities.

The next two tasks were entirely new additions to the project. First I wrote a test suite to test the calls to COAL functions. Pike does not have any kind of testing framework so I had to design my own testing framework and write test cases to test the COAL function calls. This will help further development of sTeam as testing of new code becomes easier. The next addition was to write a linux command for sTeam. Steam tools were accessible only from the tools folder that got copied to a particular location on installation. Now on installation users can use the steam command from anywhere to access all the tools.

1. We have combined all the work into two branches.

The commits made by me in each branch can be seen here.

2. I wrote weekly blogs summarizing the work done during the week.

All the blogs can be found at blog.fossasia.org.

The list in reverse chronological order is as follows.

3. A list of tasks covered and all the Pull requests related to each can be seen here

Tasks

Issue

PR

Fix the edit script.

Issue-34

PR-36

Extend edit command for multiple files. Each file opens in its own tab

Issue-38

PR-40

Implementing TLS for COAL to make it COALS.

Issue-47

PR-50

Add the functionality to open files from inside vim

Issue-53

PR-61

Write a plugin to make closing of files easier by closing the logs automatically.

Issue-62

PR-65

Add the command to create groups to steam-shell.

Issue-68 Issue-97

PR-77 PR-98

Add login command to allow relogin in debug.pike

Issue-87

PR-89

Remove repeated code used for login

Issue-91

PR-92

Add a detailed help command to make sTeam easier to use for new users.

Issue-30

PR-95

Change gothrough to enter and allow them to enter rooms as well.

Issue-96

PR-99

Change the output of look command and show rooms and gates under the same section

Issue-100

PR-101

Make steam tools accessible from everywhere

Issue-126 Issue-128 Issue-130Issue-134

PR-127 PR-129 PR-131 PR-135

Write test cases to keep the software error free.

Issue-104 Issue-107 Issue-109Issue-110 Issue-111 Issue-113Issue-116 Issue-118 Issue-122 Issue-124

PR-105 PR-108 PR-112 PR-114 PR-115PR-117 PR-119 PR-123 PR-125

Documentation.

Issue-144

PR-145

4. Scrum Reports

Daily scrum reports have been posted and discussed on #steam-devel on irc.freenode.net and a backup can be found on the mailing list

5. Further Enhancements

  • The testing framework needs to be improved
  • More test cases needs to be added

 

6. Conclusion

In the end I would like to thank Google and FOSSASIA for providing me this wonderful opportunity to learn and collaborate. I would like to thank my mentors Martin and Trilok for guiding me through all the difficult times and helping me solve bugs whenever I got stuck. I would continue contributing to open source and try joining more projects under FOSSASIA to improve my skill set and to get new experience. I will also be taking active part in Google Code In and will love to be a mentor.

Testing, Documentation and Merging

As the GSoC period comes to an end the pressure, excitement and anxiety rises. I am working on the finishing steps of my project. I was successfully able to implement all the task I took up. A list of all the tasks and their implementation can be found here.

https://github.com/societyserver/sTeam/wiki/GSOC-2016-Work-Distribution#roadmap-and-work-distribution-on-steam-for-gsoc-2016

At the start of this week I had about 26 Pull Requests. Each Pull Request had independent pieces of code for a new task from the list. I had to merge all the pull requests and resolve the conflicts. My earlier tasks involved working on the same code so there were a lot of conflicts. I spend hours looking through the code and resolving conflicts. I also had to test each feature after merging any two of the branches. Finally we were able to combine all our code and come up with a branch that contains all the code implemented by me and Ajinkya Wavare.

https://github.com/societyserver/sTeam/tree/gsoc2016-societyserver-devel

https://github.com/societyserver/sTeam/tree/gsoc2016-source

These two are the branches we combined all our code in. I finished my work on linux command for sTeam by adding support for the last two tools which are export and import from git. I worked on to include a help to get new users to understand the use of the command.

https://github.com/societyserver/sTeam/pull/135

I also worked on documentation. I started with the testing suite which is implemented by me. I wrote comments to explain the work and also improved the code by removing unnecessary lines of code. After this I added the documentation for the new command in steam-shell that I had implemented. The command to work with groups from the steam-shell. One of the issue with the testing suite still stands unresolved. I have been breaking my head on it for a week now but to no results. I will attempt to solve it in the coming week.

https://github.com/societyserver/sTeam/issues/109

This error occurs for various objects in the first few runs and then the test suite runs normally error free.

Writing linux commands

sTeam includes a number of tools namely steam-shell, debug, edit, import and export from git. These tools are only accessible from the tools folder in the sTeam installation. My task is to make a steam command for linux so that all these tools can be accessible from anywhere just like any other linux application.

Problem: https://github.com/societyserver/sTeam/issues/126

The command structure decided is that we have a global sTeam command and sub commands for each of the tools. This structure can be though of as similar to the structure of the git command. The central git command has many sub commands like branch, checkout, reset, merge, etc. I started with making a pike executable and then changing the installation files to copy this executable to /usr/bin. Now this pike executable becomes accessible from anywhere in the system and acts like a linux command.

https://github.com/societyserver/sTeam/pull/127

I wrote the main function to accept the command line arguments and passed them to a function named resolveParam. In this function I am using an if-else structure to segregate the different sub commands. For shell and debug I started a child process to execute the file for these tools present in /usr/local/lib/steam/tools. The following code snippet gives an example.

void resolveParam(int argc,array(string) argv)
{
if(argv[1]=="shell")
{
array command =({"/usr/local/lib/steam/tools/steam-shell.pike"});
object shell = Process.create_process(command, (["stdin":Stdio.stdin,"stdout":Stdio.stdout,"stderr":Stdio.stderr]));
shell->wait();
}
}

I had some problem when I had to accept more arguments than just the sub command. Foe example the edit command works as ./edit <filename1> <filename2> <filename3>… Using the global command it becomes steam edit <filename1> <filename2> <filename3>… I had to pass all the rest of the arguments to the edit scripts. I had to add them in the command array. The first solution would be to loop through the arguments array and add them all, however pike has a very easy and elegant way of doing it in one sentence.

array command = ({"/usr/local/lib/steam/tools/edit.pike"})+argv[2..];

This gets our global command up and running. Rest of the time I spend on resolving merge conflicts while merging all my previous Pull Requests.

https://github.com/Siddhant085/sTeam/tree/gsoc2016

linux command
output showing the use of steam command

Testing Hero – II

I continued last weeks work on improving the testing framework and adding more test cases. While writing the test cases for create I had to write a separate test case for each kind of object. This caused a lot of repetition of code. Thus the first aim for the week was to design a mechanism to write generalized test cases so that we can have an array of object and loop through them and pass each object to the same test case.

https://github.com/societyserver/sTeam/issues/113

https://github.com/societyserver/sTeam/pull/114

Right now the structure has a central script called test.pike which imports various other scripts containing the test cases. Let us take one of these scripts, suppose move.pike. Now I wanted to write a generalized test case which performs the same action on various objects. So I created one more file containing this generalized test case and imported this into one of the testcases in move.pike. This test case in move.pike is responsible for enumerating the various kinds of objects, sending them to the generalized test case, collect the output and then send the result for the entire test to the central test.pike. Then I went ahead and implemented this model for moving various objects to non existential location and for creating various kinds of objects and the model seemed to work fairly well.

The journey was not so smooth. I had a few troubles on the way. In all the test cases I was deleting any objects that were created and used in the test. To delete any object I need to get a reference to the object. This reference keeps getting dropped for some reason and I get an error for calling the delete function on NULL as the reference no longer exists. I tried finding the cause of this and solve this bug, however I couldn’t and found a work around the errors by using if statements to check that the object references are not null before calling functions on these object references. I continued my work on generalizing the test cases and wrote the general tests for all the test cases in the move and create test suites.

In the later part of the week I started working on some merging with my team mate Ajinkya Wavare. I designed more test cases for checking the creation of groups and users. Groups could be created using the generalized test case however for users I had to add a special test case as the process of creating a user is different from creating other objects. I ended my week by writing the test case for a long standing error, i.e, call to the get_environment function.

testing hero 2
output of test

Testing Hero

Our sTeam code base had no tests written and therefore we were facing a lot of issues and were not able to merge our code easily. My next task dealt with this problem. I had to write test cases for the function calls to COAL commands. On some basic research I found out that the only testing framework available for pike is used for testing the pike interpreter itself. This includes a set of scripts. mktestsuite is one of the scripts and is responsible for generating the tests. The problem with this is that the tests should have a particular syntax and since it is used to test interpreter is assumes each line of pike code is a test. This prevented us from writing multiple line tests and also from setting up the client.

Issue:

https://github.com/societyserver/sTeam/issues/104 (Establish testing framework),

https://github.com/societyserver/sTeam/issues/107 (Add tests for create)

My first approach to the problem was to try using the scripts available to write the tests, however this didn’t turn out very well and tests written were very confusing and out of context. The lines written for setting up the client was also being counted as tests and there was no continuation that is variables defined in one line was not accessible in the next. I realized that this is not going to work out and decided to write my own testing framework. I started by writing a simple testing structure.

The framework has a central script called test.pike. This script is used to run all the test cases. This script uses the the scripts called move.pike and create.pike which are the scripts containing the actual test cases. These scripts contain various functions each of which is a test case and return 1 on passing the case and 0 on failing. test.pike, the central node, is responsible for looping through this functions and calling each of these and recording the result. The framework then outputs the total number of cases passed or failed. Again implementing this became simple as we can import pike scripts as objects and also extract the functions from them.

test
Running tests for move
test1
Running test for create (Earlier version of framework)

test.pike involved initialization for various variables. It establishes a connection to the server and initializes _Server and me, which are then passed to all the test cases. move.pike has various test cases involving moving things around. Moving user into a room, into a container or to an non existential location and moving a room inside a container. I also got my first failing test case which is moving user into a non existential location. This shouldn’t have been allowed but is not throwing any error and incorrectly writes the trail.

create.pike involved cases creating various kinds of objects and also attempting to create objects for classes that do not exist. As I went along writing these cases I also kept improving the central test.pike. I added the code for creation of a special room for testing while initializing and also clearing this room before destroying the object and exiting.

Solution:

 https://github.com/societyserver/sTeam/pull/105 (Establish testing framework),

https://github.com/societyserver/sTeam/pull/108 (Add tests for create)

Adding extensive help for sTeam

This task was something I came up with as an enhancement because of the problems I faced while using sTeam for the first time. During the first week of my using sTeam I had a tough time getting used to commands and that is when I had opened the issue to improve help. Help for commands were one liners and not very helpful so I took up the task to improve it, so that new users don’t have to face the difficulties that I faced.

Issue: https://github.com/societyserver/sTeam/issues/30

Solution: https://github.com/societyserver/sTeam/pull/95

Not a lot of technical details were involved in this task but it was time consuming. I write down a few lines explaining what each command does and also added a syntax for the commands. While doing these I also realized more improvements that could be made and added them to my task list. My mentor had explained to me how rooms and gates were the same. I discovered that the command gothrough was violating this as it allowed users to gothrough gates but not rooms. I discussed this on the irc and we came up with a solution that we should change this command to enter and allow it to operate on both rooms and gates.

This enhancement became my next task and I worked on changing this command. The function gothrough was changed to enter and the conditions required for it to work on rooms were added. This paved way for my task. The look command showed rooms and gates under different sections. Now that there were no difference between rooms and gates I combined these two sections to change the output of the look command.

gsoc look
output of look before the changes
gsoc look1
output of look after the changes

 

Issue: https://github.com/societyserver/sTeam/issues/100

Solution: https://github.com/societyserver/sTeam/pull/101

By the end of the week I had started on my next task, which was a major one. Writing testing framework and test cases for coal command calls. I will be discussing more about this in my next blog post.

What is sTeam

Whenever I tell someone that I am working for Fossasia under Google Summer of Code, I am asked to explain what my project is all about. Here I give a detailed introduction to sTeam and explain what it actually is.

kopfleiste_big

Well some of you must of heard of MUDs (Multi User Dungeon) sTeam is based on this concept. For those who don’t know what MUDs are I will provide a brief introduction. MUDs are text based games. These games create a kind of real-time virtual world. These games are based on role playing and the user acts like one of the organisms of the virtual world. He can do whatever the organisms could have done in a real world. For example if I represent a man in a virtual world, I can do whatever a man can do, things like walk, talk, eat and stuff. This way the user interacts with his virtual environment.

This whole concept is taken to create a virtual office space. Just like MUDs sTeam creates virtual office space. Now how can we imagine an office? Well it will have rooms each room will have containers with different kinds of files, there will be people moving around working or having meetings. Well all this is possible in sTeam virtually. The user is also carrying a rucksack and can copy objects into this as he moves around. This is what sTeam is.

Steam provides two interfaces web and command line. The web interface can be accessed at http://societyserver.org/ and the code is put up at https://github.com/societyserver/sTeam.

logo-200

More of similar projects are available under Fossasia. Check this site for more details http://labs.fossasia.org/ideas.html.

Re-login in debug.pike

I will first talk about what is debug.pike and then explain my task and then the solution.

Debug.pike gives the user a pike prompt with all the constants from the sTeam client. It is like a self programmable pike client for the sTeam server. All the client side variable are available using which the user can write pike code to interact with the sTeam server. Steam-shell was built on top of this and has functions that perform common actions using these variables.

My task was to build a function to allow re-login as different users. I had worked with the code for connection to the sever and the login part while working on my second task. My second task was to implement TLS and thus I was familiar with the functions available and how COAL was working.

Implementing TLS: https://github.com/societyserver/sTeam/issues/47

First I tried the code for my function on the prompt of debug.pike itself. My plan of action was to logout the current user and restart the connection. However the logout function gave me a lot of troubles. On logging out I was losing the connection to the server and was not able to establish it back even with the connect_server() function, which establishes a connection between the server and the client. Breaking up the problem I devised a solution for the problem without logging out, that is, calling the login again without logging out.

After calling the function login() when I was unable to get any results I realized that I will have to reset all the variable values that get set during the first login. I moved all the variable initializations to init and called this function after login and the problem was solved. The temporary solution was working fine. I also realized that the logout function was giving me troubles because I was not re-initializing all the variables. So now I was also able to logout and my solution was complete.

My next task was to improve the code. The initialization and the login part of the code was repeated throughout several files with minor changes. I had to bring out the common part put it in a separate file and then inherit. The files I changed were:

  • steam-shell.pike
  • debug.pike
  • edit.pike

I shifted all the uncommon parts to the main of the respective files and then included the common file called client.pike

Issue: https://github.com/societyserver/sTeam/issues/91

Solution: https://github.com/societyserver/sTeam/pull/92

Now again after this I went back to my previous task and standardized the init function as I had changed it a lot and this would have given a merge conflict later.

Issue: https://github.com/societyserver/sTeam/issues/87

Solution: https://github.com/societyserver/sTeam/pull/89

Adding more functions to command line interface of steam-shell

sTeam allows the creation of groups and joining, leaving and listing them. However these functions were only available in the web interface. My task involved the addition of these functions to the command line interface, that is, steam-shell. The task sounded like a difficult one because it involved coding out new commands for the shell and perform actions that have never been done before from the shell. This didn’t turn out to be true.

Issue: https://github.com/societyserver/sTeam/issues/68

I began with using and understanding the group functions from the web interface. First I took up the command for the creation of groups. I listed the attributes needed by referring the web interface and then extended the create command already present in the shell to also create groups. The task turned out to be easy against what I thought earlier. This was because of the elegance of pike and modularity of the sTeam server. The code for creation of object was already present in the command and I had to pass the type of object that is group and write a few lines to accept the attributes required.

Next command was for the listing of groups, for this I created a new command called ‘group’ and inside the function called by group I switch cased on the next sub-command to find out if it was join, leave or list. After that I wrote the code to perform the action for each command in their respective cases. This is where the modularity of sTeam helped me a lot. The core portion of these functions turned out to be one liners.

Code to get a list of all groups:

array(object) groups = _Server->get_module(“groups”)->get_groups();

Code to join a group:

int result = group->add_member(me);

Code to leave a group:

group->remove_member(me);

group code 1group code 2

Soon all my command were ready. I tested these and everything seemed to be working fine. I pushed my changes and created a new pull request. It was after this that Martin asked me to change the interface. He introduced me to MUDs, Multi User Dungeon. MUDs are type of text based games. The interface for sTeam is based on the these games and these are also an inspiration for the entire project. Just like MUDs create a virtual space we at sTeam create a virtual office space. This helped me to understand not only the interface but also the project. I will be talking more about this in my next blog. Anyways the standard interface is

<action> <object> <additional attributes>

I changed the interface and now the syntax for the commands are

Create a group: create group <group_name>


 [email protected]:~/Documents/sTeam/tools$ ./steam-shell.pike
 Connecting to sTeam server...
 Password for [email protected] [steam]: *****
 Pike v7.8 release 866 running Hilfe v3.5 (Incremental Pike Frontend)
 /home/root> create group group_test
 How would you describe it?^Jtest group
 Subgroup of?^J
 /home/root>

List groups: list groups


 [email protected]:~/Documents/sTeam/tools$ ./steam-shell.pike
 Connecting to sTeam server...
 Password for [email protected] [steam]: *****
 Pike v7.8 release 866 running Hilfe v3.5 (Incremental Pike Frontend)
 /home/root> list groups


Here is a list of all groups
abcd Admin coder Everyone Groups group_test
help justnow PrivGroups sTeam testg testg;
testGroup testing test_group WikiGroups


 /home/root>
 

Join a group: join group <group_name>


 [email protected]:~/Documents/sTeam/tools$ ./steam-shell.pike
 Connecting to sTeam server...
 Password for [email protected] [steam]: *****
 Pike v7.8 release 866 running Hilfe v3.5 (Incremental Pike Frontend)
 /home/root> join group group_test
 Joined group group_test
 /home/root>
 

Leave a group: leave group <group_name>


 [email protected]:~/Documents/sTeam/tools$ ./steam-shell.pike
 Connecting to sTeam server...
 Password for [email protected] [steam]: *****
 Pike v7.8 release 866 running Hilfe v3.5 (Incremental Pike Frontend)
 /home/root> leave group group_test
 /home/root>
 

Solution: https://github.com/societyserver/sTeam/pull/77