Author: Siddhant Gupta

This author has written 15 articles

Writing vim scripts to open files in the steam structure

My objective was to write a script that will allow the user to open new files from inside the vim interface. There is a vim command available for normal files. However here I am not talking about normal files, these are files inside the steam structures, that is , inside rooms and containers. The commands that were doing this currently were edit.pike and the edit command from the steam-shell.

Issue opened: https://github.com/societyserver/sTeam/issues/53

As I started working on it one of the issue I faced was that I could not use the code in edit.pike or the edit command in steam-shell because for that I would have had to start a new process and the vim window would have come up as an another new process. Due to these process using the same terminal window there would be an overlap and vim will not be able to function, this was one of the problems with the original implementation of the edit command, which I had solved in the first week of GsoC.

Issue with edit command: https://github.com/societyserver/sTeam/issues/34
Solution: https://github.com/societyserver/sTeam/pull/36

My colleague Ajinkya Wavare had finished his task wherein we could execute pike code from a vim terminal. For this he had modified steam-shell and was passing the pike code as an argument to the call of steam-shell. One advantage of executing pike code this way is that all the steam objects and variables are available to use with the pike code. Basically it is like executing code on debug.pike. I based my solution to the problem on this newly added feature. I was able to pass in pike code to steam-shell, this would start a new process, execute the code and return me the output.

The tasks that I needed to do with the pike code was:

  • Find the required object in the steam server.
  • Get the content of the object.
  • Save the content in a temporary file.
  • Once the file is saved by vim, update the file on the sever and the logs in the vim buffer.

I created a new steam command ‘Open’ and passed the full path of the object as an argument. Given the full path it was easy to find the object on the sever. I was able to fetch the contents and save it in a file. Now I had a big problem. The name of the temporary file was available in the pike script, I needed this name in the vim script to open the file in a new tab. I spend more than a day trying to come up with a solution for this problem. I was able to get the name of the file in the output of the pike script and this output could be read into a vim buffer. However this output had a lot of other content including the result of execution of other pike statements also. I had to use vim search and select tools to get the file name isolated and stored in a vim script variable. Once I achieved this my task was almost done. I used vim command to open this file and the log in a new vim tab.

Result of execution of pike script containing the file name.
Result of execution of pike script containing the file name.

The next step was to get this file to be uploaded to the server and the logs updated. Ideally when a file is opened from the steam-shell using the edit command, there is a piece of code that gets called every one second and performs this tasks, however this couldn’t be used when the file is opened from inside vim as the steam-shell process that gets the file closes before the file is opened in vim. So I couldn’t have a pike script constantly watching the file. The solution I came up with was using auto commands to execute a pike script when the file is saved and update the file on the server and also the logs. This completed my task.

opening from vim
File being opened from inside vim.

 

file opened from vim
File opened in new tab.

Since I had a day left in the week I took up one more small task. This was due to a problem that arose due to my previous task that is letting users open multiple files for editing. There were too many vim buffers open at a time and it was a trouble closing them as :q used to be executed for each buffer. Therefore I made a used defined command in vim, :Q, that was able to close tabs at a time. :tabclose is a vim command that does the same thing however it cannot close the last tab, this shortcoming was overcome with :Q.

Issue: https://github.com/societyserver/sTeam/issues/62

solution: https://github.com/societyserver/sTeam/pull/65

Continue ReadingWriting vim scripts to open files in the steam structure

Implementing TLS in pike

My next task was an enhancement. The command line client for sTeam was not using any kind of encryption and all the data sent over a network was in clear text. Now this was a little different from normal tasks because,

1. TLS had to be implemented in Pike.

2. TLS was to be implemented over COAL.

COAL is a home grown protocol specially for sTeam. My mentor Martin helped me to go about this task. He taught me how to break a task into small ones. I began with writing an SSL client in pike that can interact with an https server. It turned out pike has an SSL module that just made the task very simple. I had to understand how SSL works however pike made the task quite easy. I also received constant help from the pike mailing list where people actively guided me in the right direction.

Once I was done with the SSL client I approached the actual problem. I had to understand the functioning of COAL and narrow down the particular files involved. I had to wrap the COAL protocol in TLS so that it becomes COALS. Pike allows users to import programs at objects. This was used to import client_base.pike. This file involved the code for the connection. I had to throughly go through this and the files that it imported that is kernel/socket. After several experiment with these files I was able to use the SSL module and successfully make the connection use the TLS protocol. I checked this using wireshark, which captured all the packets over the network and all the data could be seen as encrypted.

wireshark clear text
clear text data over network
wireshark ssl
encrypted data after implementing TLS protocol
Continue ReadingImplementing TLS in pike

steam-shell: Two processes in one terminal

Community bonding period turned out to be quite fruitful I got to know my community really well and not only that I also solved quite a number of issues which helped me understand the code base. Daily scrum meetings played a very important role in making us work professionally and cover some substantial work. Official coding period began on 23rd May and I was all set for the challenges and the sleepless nights to come. Here I will be discussing the tasks I covered in my first week.

As suggested by my mentors I had changed my plans a bit by moving the work on edit command before implementing the TLS layer on COAL. I started small by fixing the edit command. The edit command opens the specified file in vi/vim/emac. In vi and vim the editor was misbehaving and not letting us work on the file. I took up this as my first task for Google Summer of Code 2016. This helped me understand steam-shell and applauncher, which is used to load the editor, in detail. Vi and vim editors have an advantage of letting the user edit the file in the same terminal window.

Looking at the issue itself it was not possible to do any kind of backtracking. The vi editor was just throwing rubbish on the screen when the user attempted to type anything.

overlapping process
vi editor showing the garbage and the steam-shell command

At first I was under the impression that it was a problem with the editor itself. I even tried approaching the vi.stackexchange.com , where the vi developers could help me. However all this was in vain. After a lot more forensics and re-reading the code multiple number of times I realized two process were active and sharing the same terminal space. How did I come to this conclusion? Well it was a very minute detail that I noticed. While in the vi editor window, with the document open and the editor throwing garbage at you when you press the up arrow the editor clears some area and show the commands executed on the steam-shell. This can be seen in the above image

This simply meant that both the process for steam-shell and the vi editor was running and sharing the same terminal space. The solution was quite simple. Just called editor→wait() to suspend the calling the process till the called process was over.

Continue Readingsteam-shell: Two processes in one terminal

The Official Beginning

This blog marks the start of the coding period for Google Summer of Code. I will start by talking about the community bonding period and then my plan to begin the summer.

Community bonding was an interesting and intense period. Managing college, assignments, exams and working on the project all at the same time. I think this is the capability that sets us apart. We had started out with fixing small issues in the steam-shell. Steam-shell is a command line interface that allows the user certain limited set of commands to interact with the server. I will list down the issues I solved and the improvements I made.

My first task was to improve the display of commands like ‘look’. This command lists out the gates/files/documents/rooms/conatiners present in the current room. The old display showed one entry in one line. I improved this to make it something similar to the ls command of linux. I used sprintf(“%#-80s”,<variable>) to give the output a structure.

Next I worked on the create command. I found a bug that this command was not working for containers and documents. I fixed it by adding the required conditions in the code and then I worked upon to improve the functionality of the command as suggested by my mentor Trilok. I added the feature to enter the destination where the new object is to be created and a ‘.’ to create it in the present directory. Initially this feature was only for Rooms and Containers, later I extended this to all kinds of objects.

After this I spent a few days studying the COAL protocol. I have already written in detail about this in my previous blog. Though I was not successful in completely understanding the protocol but this gave me an idea of the client-server architecture. This was towards the end of the community bonding period and I decided to spend the rest of my time with steam-shell itself.

Now as the coding period begins officially I will be working to fix the edit command. Currently the edit command opens the file on a vi/vim/emac instance. The vi and vim opens in the same window and it misbehaves. Then I will be extending the feature for multiple documents and in the next week move on to work on COAL again.

Continue ReadingThe Official Beginning

Google Summer of Code: A new beginning

I am working on sTeam under FOSSASIA for GSoC 2016. sTeam is collaboration platform. It allows users to form groups and share various kinds of documents. It provides a virtual knowledge space where users can meet and interact. These spaces are called rooms, these can contain various containers to store documents in an organized manner. The rooms are connected by gates to allow the movement of users and documents from one room to another. The most interesting thing about the project is that it is in pike programming language. This is not very commonly used and I will get to learn new things and myriad of experiences.



Now moving into the technical details of my project I will be talking about the first step in my project that is implementing TLS (Transfer Layer Security) Protocol, also known as SSL. Currently all the communication between the server and the command line client is over COAL protocol. COAL is a home grown protocol developed by the original developers of sTeam. Currently COAL does not use any kind of encryption. My first task will be to integrate SSL into COAL, so just like http becomes https COAL will become COALS. After this I will be working on the improving the client interface, which I will talk about in the coming weeks.


For now in the community bonding period I have been actively participating in the discussions on the IRC and working on stash, discovering bugs in the project and getting used to the interface and the code. Recently we received the guidelines for the summer from our mentors Martin and Trilok. The guidelines include clearly mentioned milestones and the tasks we need to complete. We will also be having a team meeting everyday discussing our day’s work and issues.



I would like to thank FOSSASIA for giving me this wonderful oppurtunity to show my skills. All the discussion during this period is just making me feel more professional and getting me excited for the summers. In programmers term

printf(“let the fun begin !”);

Continue ReadingGoogle Summer of Code: A new beginning