sTeam

sTeam demo

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. Demo Visual aid can help a person to understand and grasp faster. A demo of the utility added to sTeam so far has been created to help the user base. The videos have been divided into sections based on the category of the scripts which they execute. Using a docker image. Starting the sTeam server. Running various utilities in the sTeam-shell. Import from git script. Export to git Feel free to explore the repository. Suggestions for improvements are welcomed. Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

Continue ReadingsTeam demo

Extending sTeam shell commands

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. Break the PR A lot of new commands have been added to the sTeam-shell script. The earlier PR sent  was addressing a lot of issue’s. As a result the PR was cherry picked and rebased to form smaller PR’s for each issue. Issue. Github Issue Github PR Add directory location for libxslt.so Issue-25 PR-66 Makefile.in changes to add the files in the tools directory to the installed location. Issue-27 PR-67 Import from git script The import to git script was further enhanced to support the feature whereby a user can specify the name of the object in the sTeam directory when a single object is been imported. The user can now import a single object to the steam work area using the import-from-git script. The command works for objects of all mime-types. Two formats for the command are supported. Issue. Github Issue Github PR Add utility to support single import in import-from-git script . Issue-16 PR-76 Format 1: import-from-git.pike ~/gitfolder/zy.mp3 /home/sTeam/ This would create the object in the sTeam directory. The new object would have the same name as the object in the git directory. Format 2: import-from-git.pike ~/gitfolder/zy.mp3 /home/steam/ab.mp3 This would create an object with the name ab.mp3 in the sTeam directory. If the object exists from before, the contents of it will be over written with the contents of the object from the git-folder. Note: Here ‘/’ at the end of the steam directory is used as a distinguishing factor between a directory and an object. Be careful while passing the steam directory in the command or it would throw error. List User’s The command to list all the existing user’s in the sTeam server was also added to the steam-shell. Issue. Github Issue Github PR List users. Issue-72 PR-78 The user after creation needs to be activated by the root user. Thus a user can then access his steam-shell command line by passing the parameters of user name, host name or port number. ./steam-shell.pike -u uname -h hname -p pno Passing Arguments to the sTeam-shell The sTeam shell was modified during it’s integration with vi. This had introduced a bug where by the above parameters where not been able to pass along when a command to the sTeam-shell was passed as an argument. The issue was addressed and resolved. The user can pass arguments like user name, host and port-number to the steam-shell.pike along with the steam-commands. Issue. Github Issue Github PR Add utility to support passing arguments to the sTeam-shell. Issue-71 PR-75  Eg. ./steam-shell.pike -u user -h host -p portno steam-command Modularize the tasks The commands for user manipulation were grouped under their operations like create, delete or list. Thus the commands were modularize. Issue. Github Issue Github PR The user commands have been modularized based on the actions they perform. Issue-73 PR-81 The action create, delete and list now support user operations. Example: To create a user. create user test…

Continue ReadingExtending sTeam shell commands

Writing vim scripts to open files in the steam structure

My objective was to write a script that will allow the user to open new files from inside the vim interface. There is a vim command available for normal files. However here I am not talking about normal files, these are files inside the steam structures, that is , inside rooms and containers. The commands that were doing this currently were edit.pike and the edit command from the steam-shell. Issue opened: https://github.com/societyserver/sTeam/issues/53 As I started working on it one of the issue I faced was that I could not use the code in edit.pike or the edit command in steam-shell because for that I would have had to start a new process and the vim window would have come up as an another new process. Due to these process using the same terminal window there would be an overlap and vim will not be able to function, this was one of the problems with the original implementation of the edit command, which I had solved in the first week of GsoC. Issue with edit command: https://github.com/societyserver/sTeam/issues/34 Solution: https://github.com/societyserver/sTeam/pull/36 My colleague Ajinkya Wavare had finished his task wherein we could execute pike code from a vim terminal. For this he had modified steam-shell and was passing the pike code as an argument to the call of steam-shell. One advantage of executing pike code this way is that all the steam objects and variables are available to use with the pike code. Basically it is like executing code on debug.pike. I based my solution to the problem on this newly added feature. I was able to pass in pike code to steam-shell, this would start a new process, execute the code and return me the output. The tasks that I needed to do with the pike code was: Find the required object in the steam server. Get the content of the object. Save the content in a temporary file. Once the file is saved by vim, update the file on the sever and the logs in the vim buffer. I created a new steam command ‘Open’ and passed the full path of the object as an argument. Given the full path it was easy to find the object on the sever. I was able to fetch the contents and save it in a file. Now I had a big problem. The name of the temporary file was available in the pike script, I needed this name in the vim script to open the file in a new tab. I spend more than a day trying to come up with a solution for this problem. I was able to get the name of the file in the output of the pike script and this output could be read into a vim buffer. However this output had a lot of other content including the result of execution of other pike statements also. I had to use vim search and select tools to get the file name isolated and stored in a vim script variable. Once I…

Continue ReadingWriting vim scripts to open files in the steam structure

Increasing utility of sTeam tools

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. User Utils User  convenience is an important aspect for any application to succeed. The sTeam root user should be able to create or delete a user. A command to add / delete a user was added successfully. The root user should be able to add new user’s using the command line. The parameter’s like username, password, email-id etc. should be asked and then the user should be created. Issue. Github Issue Github PR Create a user. Issue-58 PR-59 Delete an object Issue-56 PR-57 Delete a user. Issue-69 PR-70 To create a new user: create_user username password email The current users can be found by running the command _Server->get_module("users")->get_users(); Similarly a command to delete the user was added to the steam-shell. The user should be able to delete the objects created/existing inside the user area. A steam-shell command needs to be added to delete the objects from the command line. The usage of command to delete a user inside steam-shell.pike. Only the root user can delete other user. Command: delete_user username In order to delete an object which can be a container, document or a room in the current steam directory, run the command; delete test.pike The object would be deleted. Tab-Completion Module The tab completion module of the sTeam shell was analyzed during this period. The tab completion module has an issue whereby it doesn’t lists the options on pressing the tab after  ". query_attribute(" After pressing the tab after ” the options should be listed. But this is not the case. The bug needs to be resolved. Import from git script The import to git script can now import a single file into the sTeam directory. The feature for this was added. Issue. Github Issue Github PR Add utility to support single import in import-from-git script . Issue-16 PR-76 sudo ./import-from-git.pike gitfolder/xyz.mp3 /home/sTeam/ The xyz.mp3 would be imported to the sTeam directory. The future work would include resolving the tab_completion module issues and enhancing the import script to support the feature where by a user would be able to specify the name of the file in the sTeam directory. Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

Continue ReadingIncreasing utility of sTeam tools

Implementing TLS in pike

My next task was an enhancement. The command line client for sTeam was not using any kind of encryption and all the data sent over a network was in clear text. Now this was a little different from normal tasks because, 1. TLS had to be implemented in Pike. 2. TLS was to be implemented over COAL. Issue: https://github.com/societyserver/sTeam/issues/47 COAL is a home grown protocol specially for sTeam. My mentor Martin helped me to go about this task. He taught me how to break a task into small ones. I began with writing an SSL client in pike that can interact with an https server. It turned out pike has an SSL module that just made the task very simple. I had to understand how SSL works however pike made the task quite easy. I also received constant help from the pike mailing list where people actively guided me in the right direction. SSL client in pike: https://github.com/Siddhant085/SSL_client_pike Once I was done with the SSL client I approached the actual problem. I had to understand the functioning of COAL and narrow down the particular files involved. I had to wrap the COAL protocol in TLS so that it becomes COALS. Pike allows users to import programs at objects. This was used to import client_base.pike. This file involved the code for the connection. I had to throughly go through this and the files that it imported that is kernel/socket. After several experiment with these files I was able to use the SSL module and successfully make the connection use the TLS protocol. I checked this using wireshark, which captured all the packets over the network and all the data could be seen as encrypted. Solution: https://github.com/societyserver/sTeam/pull/50

Continue ReadingImplementing TLS in pike

Improvements in sTeam shell and export script

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. Logs A breakthrough in sTeam development took place. Siddhant Gupta was able to successfully implement the TLS protocol in pike. The Logs were displayed erroneously. Also the user was not able to scroll down the log to the latest message.  The golden ratio script used in the program was updated to it’s latest version. Also the editor is opened using sudo command so as to access the vim scrips in the /usr/local/lib/steam/tools directory. The files are opened using the vim command: vim -S script -c edit filename1|sp filename2 Only one buffer is accessible at a time. In order to switch the buffer to log buffer the command is CTRL+Ww. Enter this command directly without entering the vim terminal using :. The log buffer would be accessible, can be edited and scrolled down to the latest log message. Issue. Github Issue Github PR Access the log window till the end. Issue-20 PR-48 Open appropriate log window when a sTeam command is executed. Issue-49 PR-51 The logs were displayed erroneously. Ideally the log should be displayed based on the buffer where the sTeam function is called and accordingly the relevant log buffer should be called and display the output. This error was fixed. The log is displayed in the file named after the file which is opened and concatenated with the suffix “-disp”. In Vi the  :% buffer stores the value of the current file. Consequently this value was concatenated with “-disp” to display the buffer accordingly. The Vi script can be seen below. Export to git script. The  export-to-git script was tested vigorously.  All the known issue’s based on the script were replicated in the system and solution was found for them. The export to git script is now capable of exporting multiple sources at a time. If the last argument is always the target repository then any number of previous arguments can be sources. Issue. Github Issue Github PR Support Multiple Source arguments Issue-14 Issue-19 PR-54 Include Source-name in branch name and add branch description. Issue-9 PR-55 Example command : ./export-to-git.pike /home/sTeam/file1 /home/coder/file2 /home/sTeam/container3 ~/gitfolder The export-to-git script also exports the source name in the branch.  To help distinguish between the branches, we need more descriptive names: ./export-to-git.pike /sources/ /tmp/export-test/ This should create the branch sources-cur_time. Also when a file is specifically exported: ./ecport-to-git.pike /home/coder/demo1.txt /temp/export-test/ would create a branch with name home/coder/demo1.txt-cur_time. This is done to avoid ambiguity between files with same name existing in different locations. Also a description is added fo the branch name using the git command git config branch.<branch name>.description "describe branch" To view this description go to the folder where the branch is exported and then enter the git command git config branch.<branch name>.description Export to Git script executing when Multiple Source arguments passed and the modified branch name. Branch Description Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

Continue ReadingImprovements in sTeam shell and export script

steam-shell: Two processes in one terminal

Community bonding period turned out to be quite fruitful I got to know my community really well and not only that I also solved quite a number of issues which helped me understand the code base. Daily scrum meetings played a very important role in making us work professionally and cover some substantial work. Official coding period began on 23rd May and I was all set for the challenges and the sleepless nights to come. Here I will be discussing the tasks I covered in my first week. As suggested by my mentors I had changed my plans a bit by moving the work on edit command before implementing the TLS layer on COAL. I started small by fixing the edit command. The edit command opens the specified file in vi/vim/emac. In vi and vim the editor was misbehaving and not letting us work on the file. I took up this as my first task for Google Summer of Code 2016. This helped me understand steam-shell and applauncher, which is used to load the editor, in detail. Vi and vim editors have an advantage of letting the user edit the file in the same terminal window. Issue: https://github.com/societyserver/sTeam/issues/34 Looking at the issue itself it was not possible to do any kind of backtracking. The vi editor was just throwing rubbish on the screen when the user attempted to type anything. vi editor showing the garbage and the steam-shell command At first I was under the impression that it was a problem with the editor itself. I even tried approaching the vi.stackexchange.com , where the vi developers could help me. However all this was in vain. After a lot more forensics and re-reading the code multiple number of times I realized two process were active and sharing the same terminal space. How did I come to this conclusion? Well it was a very minute detail that I noticed. While in the vi editor window, with the document open and the editor throwing garbage at you when you press the up arrow the editor clears some area and show the commands executed on the steam-shell. This can be seen in the above image This simply meant that both the process for steam-shell and the vi editor was running and sharing the same terminal space. The solution was quite simple. Just called editor→wait() to suspend the calling the process till the called process was over. Solution: https://github.com/societyserver/sTeam/pull/36

Continue Readingsteam-shell: Two processes in one terminal

Enhancement of steam-shell plugin and Import from git

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. Indentation of output made independent of the screen-width. There were errors encountered in the pull request. These needed to be modified. The notable one’s were the Indentation of the output displayed by the look command in the steam-shell. The earlier display was taken into consideration that the default size of the linux terminal is 80 char long. However this can differ from individual to individual terminal settings. Therefore the module reference of pike was studied and a method to display the contents in the output of ls command format was written down. Thus,  now the output is independent of the screen width of an individual. Issue. Github Issue Github PR Indentation of output in steal-shell. Issue-24 PR-42 Also there were minor errors in the edit.pike script. These were resolved too. The edit.pike was not able to exit the program. Therefore an exit(0) call was made. This resulted in the edit.pike being able to exit successfully but a new error was introduced in the steam-shell.pike. When the edit.pike command was called inside it, the steam-shell.pike would exit on successfully completing the edit command. The steam-shell should not exit in this case. There were changes made in the VisTeam.pike, edit.pike and applauncher.pike in order to get the utility working correctly. Issue. Github Issue Github PR Edit.pike hangs on closing Issue-29 PR-44 Edit command closes the steam-shell abruptly Issue-43 PR-44 The steam-shell vim plugin was modified and the support for multi line commands was added. Now the user can type the commands in multipe lines, format them and then execute them. The output will be displayed in a new tab. Issue. Github Issue Github PR Integrate sTeam-shell into Vi. Issue-37 PR-41 An example of it can be seen below:- Output: The import-from-git.pike script is used to import contents from a normal git repository into the steam directory. The import from git script was studied and understood initially. The issue’s listed were replicated in the system and a possible solution for them was found out. The script has support for importing multiple documents at the same time. Also the objects imported supported only text mime type. The script was modified to support other mime types by initially detecting them through an external process. However later in the server/factories/Document_factory.pike file the auto detecting of mime type was found out and used in the code. Now the mime types are detected and updated successfully for documents of other mime types like MP3, jpg, png etc Issue. Github Issue Github PR Auto-detect Mime type. Issue-18 PR-45 The contents of objects of types other than text mime type were not been read by the existing process. As a result this resulted into creation of empty objects with the set mime type. The process to read the contents was modified and the contents of objects of other types were set accordingly. The content is read in the form of a string. Since every object…

Continue ReadingEnhancement of steam-shell plugin and Import from git

The Official Beginning

This blog marks the start of the coding period for Google Summer of Code. I will start by talking about the community bonding period and then my plan to begin the summer. Community bonding was an interesting and intense period. Managing college, assignments, exams and working on the project all at the same time. I think this is the capability that sets us apart. We had started out with fixing small issues in the steam-shell. Steam-shell is a command line interface that allows the user certain limited set of commands to interact with the server. I will list down the issues I solved and the improvements I made. My first task was to improve the display of commands like ‘look’. This command lists out the gates/files/documents/rooms/conatiners present in the current room. The old display showed one entry in one line. I improved this to make it something similar to the ls command of linux. I used sprintf("%#-80s",<variable>) to give the output a structure. Next I worked on the create command. I found a bug that this command was not working for containers and documents. I fixed it by adding the required conditions in the code and then I worked upon to improve the functionality of the command as suggested by my mentor Trilok. I added the feature to enter the destination where the new object is to be created and a ‘.’ to create it in the present directory. Initially this feature was only for Rooms and Containers, later I extended this to all kinds of objects. After this I spent a few days studying the COAL protocol. I have already written in detail about this in my previous blog. Though I was not successful in completely understanding the protocol but this gave me an idea of the client-server architecture. This was towards the end of the community bonding period and I decided to spend the rest of my time with steam-shell itself. Now as the coding period begins officially I will be working to fix the edit command. Currently the edit command opens the file on a vi/vim/emac instance. The vi and vim opens in the same window and it misbehaves. Then I will be extending the feature for multiple documents and in the next week move on to work on COAL again.

Continue ReadingThe Official Beginning

Integrate sTeam-shell into Vi and Indentation of output

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. Indentation of sTeam-shell output The Indentation of the output in the look command in steam-shell is formatted to be displayed in the way the output is displayed when the ls command is executed in a terminal window. The module reference provided by the pike language is used for formatting. The pike reference module can be found at Pike Ref module The screen-width of the the user is calculated using the command tput rows. This value is then passed as an argument to the write function to display it in the form of output of an ls command. The example of this can be seen in the modref by Pike. Issue. Github Issue Github PR Indentation of output in steal-shell. Issue-24 PR-42 Example: write("%-$*s\n", screen_width,"Given a\nlist of\nslosh-n\nseparated\n'words',\nthis option\n"+ "creates a\ntable out\nof them\nthe number of\ncolumns\n"+"be forced\nby specifying a\npresision.\nThe most obvious\n"+"use is for\nformatted\nls output."); Output: Given a list of slosh-n separated 'words', this option creates a table out of them the number of columns be forced by specifying a precision. The most obvious use is for formatted ls output. The following changes have been made in order to incorporate the changes: Integration of sTeam-shell into Vi For integration of sTeam-shell.pike into the Vi, the steam-shell is made compatible to be run when a command is passed as an argument to it. The changes have been made in steam-shell.pike and the applauncher.pike. Also a new file called as VisTeam.pike is created. When this is executed it validates the user and opens two vi buffers. One for typing the command and the other to display the log/ output of the executed command. A steam-shell.vim vi plugin is created so as to be able to run the Steam function in vi. Whenever this function is executed the selected text in the vi visual mode will be executed. The results are displayed in the log buffer in a new tab. Since the already existing vim scripts Goldenratio.vim and watchforchanges.vim divide the command buffer and the log buffer in the ration 2:1, the output is not visible in the logs. Thus a newtab consisting of the logs is opened. Issue. Github Issue Github PR Integrate sTeam-shell into Vi. Issue-37 PR-41 Execute the command: Display the output: Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

Continue ReadingIntegrate sTeam-shell into Vi and Indentation of output