sTeam GSoC 2016 Windup

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. sTeam-REST API repository: sTeam-REST An overview of the work done by ajinkya007 during Google Summer of code 2016 with FOSSASIA on its project sTeam. The community bonding period saw the creation of a docker image and a debian package for the sTeam server. The integration of the sTeam shell into vi, improvements in the export and import to git scripts, user and group manipulation commands, sending mails through the commandline, viewing logs and the edit script modifications were done subsequently. The later part of GSOC saw that the sTeam-rest repository was restructured, unit and api-end point tests were performed. The new web interface developed was tested. The code written during this period by me and siddhant was merged and the conflicts were resolved. The merged code was tested thoroughly as no automated test integration tool supports pike programming language. Documentation was generated using Doxygen and deployed in the gh-pages of the sTeam server repository. A trello board was maintained throughout the course of GSOC 2016. Trello Board: sTeam Accomplishments Issues Resolved and Pull Requests Submitted for the same Commits Merged Blog Posts Scrums Further Improvements Issues Reported and Resolved A list of tasks covered and all the Pull requests related to each: Tasks Issue PR Make changes in the Makefile for installation of sTeam. Issue-25 Issue-27 PR-66 PR-67 Edit script modifications Issue-20 Issue-29 Issue-43 PR-44 PR-48 Indentation of output in steal-shell. Issue-24 PR-42 Integrate steam-shell into vim or emacs. Issue-37 Issue-43 Issue-49 PR-41 PR-48 PR-51 Improve the import and export from git scripts. Issue-9 Issue-14 Issue-16 Issue-18 Issue-19 Issue-46 PR-45 PR-54 PR-55 PR-76 Create, Delete and List the user through commandline Issue-58 Issue-69 Issue-72 PR-59 PR-70 PR-78 Sending Mails through commandline Issue-74 PR-85 Generate error logs and display them in CLI Issue-83 PR-86 Create a file of any mime type from command line. Issue-79 PR-82 Add more commands for group operations. Issue-80 PR-84 Add more utility to the steam-shell Issue-56 Issue-71 Issue-73 PR-57 PR-75 PR-81 Restructure the sTeam-rest repository List of Issue's List of PR's Write test cases to test sTeam-rest api List of Issue's List of PR's Create a debian package and a docker image for easy deployment Create docker image Docker Image Document the work done Issue 149 sTeam Server Structure, sTeam Server Documentation Test the web-interface ✓ ✓ Commits Merged During the course of GSOC 2016, work was done on the sTeam and sTeam-rest repositories. 1. The work done on the sTeam repository. We have combined all the work into two branches for the ease of creating a debian package. The commits made by me in each branch can be seen here. gsoc2016-societyserver-devel gsoc2016-source 2. The work done on the sTeam-rest repository The push request's sent for the issue's are yet to be merged in the main repository. The list of PR's for the sTeam-rest repository. sTeam-rest PR's The weekly blogs The blogs summarizing the work done during the week were published…

Continue ReadingsTeam GSoC 2016 Windup

sTeam Docker Image

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. sTeam-REST API repository: sTeam-REST DOCKER IT! What is Docker? Docker is open source software to pack, ship and run any application as a lightweight container. Containers are completely hardware and platform independent so you don’t have to worry about whether what you are creating will run everywhere. In order to facilitate development in all the environments and give the user the ease from the cumbersome installation steps a docker image was made for the sTeam project. This docker image contains all the necessary dependencies to start the sTeam server. It also includes the sTeam UI and the Rest API along with it’s dependencies. These have already been installed and developer can start collaborating to it. The docker image can be found at : https://hub.docker.com/r/ajinkya007/societyserver/ It has all the necessary information for it’s usage and installation. The dockerfile for the sTeam repository: FROM ubuntu:latest RUN apt-get update RUN apt-get install -y wget git firefox nodejs nodejs-legacy npm RUN apt-get install -y build-essential mysql-server libmysqld-dev bzip2 libjpeg$ RUN apt-get install -y pike7.8 pike7.8-bzip2 pike7.8-svg RUN apt-get install -y libxml2-dev libxslt1-dev automake flex RUN npm install -g npm bower gulp coffee-script jasmine-node frisby RUN cd home RUN wget -c https://raw.githubusercontent.com/societyserver/sTeam/steam-package$ RUN git clone www.github.com/societyserver/steam.git RUN cd sTeam RUN git checkout societyserver-source RUN ./build RUN sudo ./install RUN cd .. RUN git clone https://github.com/societyserver/sTeam-web-interface-gsoc-2015.git RUN cd sTeam-web-interface-gsoc-2015 RUN npm install RUN cd .. RUN git clone https://github.com/societyserver/steam-rest.git RUN cd steam-rest RUN npm install Feel free to explore the repository. Suggestions for improvements are welcomed. Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

Continue ReadingsTeam Docker Image

sTeam Server Object permissions and Doxygen Documentation

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. sTeam-REST API repository: sTeam-REST sTeam Server object permissions sTeam command line lacks the functionality to read and set the object access permissions. The permission bits are: read,write, execute, move, insert, annotate, sanction. The permission function was designed analogous to the getfacl() command in linux. It should display permissions as: rwxmias corresponding to the  permission granted on the object. The the key functions are get_sanction, which returns a list of objects and permissions and sanction_object, which adds a new object and its set of permissions. The permissions is stored as an integer and the function should break the individual bits like getfact(). The permission bits for the sTeam objects are declared in the access.h // access.h: The permission bits #define FAIL -1 #define ACCESS_DENIED 0 #define ACCESS_GRANTED 1 #define ACCESS_BLOCKED 2 #define SANCTION_READ 1 #define SANCTION_EXECUTE 2 #define SANCTION_MOVE 4 #define SANCTION_WRITE 8 #define SANCTION_INSERT 16 #define SANCTION_ANNOTATE 32 The get_sanction method defined in the access.pike returns a mapping which has the ACL(Access Control List) of all the objects in the sTeam server. // Returns the sanction mapping of this object, if the caller is privileged // the pointer will be returned, otherwise a copy. final mapping get_sanction() { if ( _SECURITY->trust(CALLER) ) return mSanction; return copy_value(mSanction); } The functions gets the permission values which are set for every object in the server. The sanction_object method defined in the object.pike sets the permissions for the new objects. // Set new permission for an object in the acl. Old permission are overwritten. int sanction_object(object grp, int permission) { ASSERTINFO(_SECURITY->valid_proxy(grp), "Sanction on non-proxy!"); if ( query_sanction(grp) == permission ) return permission; // if permissions are already fine try_event(EVENT_SANCTION, CALLER, grp, permission); set_sanction(grp, permission); run_event(EVENT_SANCTION, CALLER, grp, permission); return permission; } This method makes use of the set_sanction which sets the permission onthe object. The task ahead is to make use of the above functions and write a sTeam-shell command which would provide the user to easily access and change the permissions for the objects. Merging into the Source The work done during GSOC 2016 by Siddhant and Ajinkya on the sTeam server was merged into the gsoc201-societyserver-devel and gsoc2016-source branches in the societyserver repository. The merged code can be found at: https://github.com/societyserver/sTeam/tree/gsoc2016-source https://github.com/societyserver/sTeam/tree/gsoc2016-societyserver-devel The merged code needs to be tested before the debian package for the sTeam server is prepared. The testing has resulted into resolving of minor bugs. Doxygen Documentation The documentation for the sTeam is done using doxygen. The doxygen.pike is written and used to make the documentation for the sTeam server. The Doxyfile which includes the configuration for generating the sTeam documentation is modified and input files are added. The generated documentation is deployed on the gh-pages in the societyserver/sTeam repository. The documentation can be found at: http://societyserver.github.io/sTeam/files.html The header files and the constants defined are also included in the sTeam documentation. sTeam Documentation: sTeam defined constants: sTeam Macro Definitions: Feel free to explore the repository. Suggestions for improvements…

Continue ReadingsTeam Server Object permissions and Doxygen Documentation

sTeam Server Peer Review and Merging.

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. sTeam-REST API repository: sTeam-REST Peer Review Peer review is done to ensure that the work performed by one's peers meets specific criteria. Peer review is used while working in groups because of the belief that one's peers are able to identify each others errors easily and thus speeding up the time it takes to identify the bugs and get them rectified. The goal of peer review is to ensure that the work satisfies the specifications and identify the aberrations from the standards, help in enhancing, modifying and providing suggestions for improvements. The Review process doesn't involve management participation. A lot of work was done in the sTeam server repository during the course of GSoC 2016. There is no support for continuous code integration and testing tools like Code Coverage, Travis CI, Circle CI or Landscape IO as the pike language is not supported by these tools. The project heavily relies on the peer reviews and feedbacks from the members of the community. The work done by Siddhant on the testing framework and the steam subshell commands was reviewed and tested for bugs. The work which was tested can be found below: sTeam Server testing framework: https://github.com/societyserver/sTeam/pull/125 sTeam Server sub shell commands: https://github.com/societyserver/sTeam/pull/135 sTeam Server structure The steam project files are segregated into various directories depending  upon the tasks performed by them. For an individual to be productively working on the project, the understanding of the directory structure is utmost. The structure needs to be documented. The documentation for the sTeam server was done to facilitate easy understanding of the project structure. https://github.com/societyserver/sTeam/wiki/sTeam-Server-Structure sTeam Server Merging the PR's Issue's reported were addressed in individual branches in the fork repositories of the collaborators. The changes made were sent in PR's to the societyserver-devel and societyserver-source repo. These needed to be merged in the source repository. For resolving the conflicts during merging these, the PR's were merged into gsoc2016 branch in the societyserver repo. GSoC 2016 societyserver-devel // Ajinkya https://github.com/societyserver/sTeam/pull/132 // Siddhant https://github.com/societyserver/sTeam/pull/133 GSoC 2016 societyserver-source // Ajinkya https://github.com/societyserver/sTeam/pull/138 // Siddhant https://github.com/societyserver/sTeam/pull/137 The merge conflicts between the two PR's must be resolved and then merged into the main societyserver branches. Also the issue for setting the permissions for sTeam objects would be addressed in the coming days. Feel free to explore the repository. Suggestions for improvements are welcomed. Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

Continue ReadingsTeam Server Peer Review and Merging.

sTeam REST API Unit Testing

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. sTeam-REST API repository: sTeam-REST Unit Testing the sTeam REST API The unit testing of the sTeam REST API is done using the karma and the jasmine test runner. The karma and the jasmine test runner are set up in the project repository. The karma test runner : The main goal for Karma is to bring a productive testing environment to developers. The environment being one where they don’t have to set up loads of configurations, but rather a place where developers can just write the code and get instant feedback from their tests. Because getting quick feedback is what makes you productive and creative. The jasmine test runner: Jasmine is a behavior-driven development framework for testing JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM. And it has a clean, obvious syntax so that you can easily write tests. The karma and jasmine test runner were configured for the project and basic tests were ran. The angular js and angular mocks version in the local development repository was different. This had resulted into a new error been incorporated into the project repo. The 'angular.element.cleanData is not a function' error is thrown in the local development repository. This error happens when the local version of the angular.js and angular-mocks.js doesn't match. The testing framework would test you if the versions f the two libraries is not the same. The jasmine test runner can be accessed from the browser. The karma tests can be performed from the command line. To access the jasmine test runner from the web browser, go to the url http://localhost:7000/test/unit/runner.html To run the karma test suite, run the following command $ karma start The unit tests of the sTeam REST service were done using jasmine. The unit tests were written in coffee script. The preprocessor to compile the files from coffee script to javascript is defined in the karma configuration file. Jasmine Test Runner Jasmine Test Failure First a dummy pass case and a fail case is tested to check there are no errors in the test suite during the test execution. The localstoragemodule.js which is used in the steam service is injected in the test module. Then the steam service version is tested. describe 'Check version of sTeam-service', -> it 'should return current version', inject (version) -> expect(version).toEqual('0.1') steam service should be injected in a global variable as the same service functions shall be tested while performing the remaining tests. Then the steam service is injected and checked whether it exists or not. beforeEach inject (_steam_) -> steam= _steam_ describe 'Check sTeam service injection', -> it 'steam service should exist', -> expect(steam).toBeDefined() The sTeam service has both private and public functions. The private functions cannot be accessed from outside. The private functions defined in the sTeam service arehandle_request and headers. describe 'Check sTeam service functions are defined.', -> describe ' Check the sTeam REST API private functions.', ->…

Continue ReadingsTeam REST API Unit Testing

sTeam API Endpoint Testing

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. sTeam-REST API repository: sTeam-REST sTeam API Endpoint Testing using Frisby sTeam API endpoint testing is done using Frisby.  Frisby is a REST API testing framework built on node.js and Jasmine that makes testing API endpoints very easy, speedy and joyous. Issue. Github Issue Github PR sTeam-REST Frisby Test for login Issue-38 PR-40 sTeam-REST Frisby Tests Issue-41 PR-42 Write Tests Frisby tests start with frisby.create with a description of the test followed by one of get, post, put, delete, or head, and ending with toss to generate the resulting jasmine spec test. Frisby has many built-in test helpers like expectStatus to easily test HTTP status codes, expectJSON to test expected JSON keys/values, and expectJSONTypes to test JSON value types, among many others. // Registration Tests frisby.create('Testing Registration API calls') .post('http://steam.realss.com/scripts/rest.pike?request=register', { email: "ajinkya007.in@gmail.com", fullname: "Ajinkya Wavare", group: "realss", password: "ajinkya", userid: "aj007" }, {json: true}) .expectStatus(200) .expectJSON({ "request-method": "POST", "request": "register", "me": restTest.testMe, "__version": testRegistrationVersion, "__date": testRegistrationDate }) .toss(); The testMe, testRegistrationVersion and testRegistrationDate are the functions written in the rest_spec.js. The frisby API endpoint tests have been written for testing the user login, accessing the user home directory, user workarea, user container, user document, user created image,  groups and subgroups. The REST API url's used for testing are described below. A payload consists of the user id and password. Check if the user can login. http://steam.realss.com/scripts/rest.pike?request=aj007 Test whether a user workarea exists or not. Here aj workarea has been created by the user. http://steam.realss.com/scripts/rest.pike?request=aj007/aj Test whether a user created container exists or not. http://steam.realss.com/scripts/rest.pike?request=aj007/container Test whether a user created document exists or not. http://steam.realss.com/scripts/rest.pike?request=aj007/abc.pike Test whether a user created image(object of any mime-type) inside a container exists or not. http://steam.realss.com/scripts/rest.pike?request=aj007/container/Image.jpeg Test whether a user created document exists or not. The group name and the subgroups can be queried. eg. GroupName: groups, Subgroup: test. The subgroup should be appended using "." to the groupname. http://steam.realss.com/scripts/rest.pike?request=groups.test Here "groups" is a Groupname and "gsoc" is a subgroup of it. http://ngtg.techgrind.asia/scripts/rest.pike?request=groups.gsoc Unit Testing the sTeam REST API The unit testing of the sTeam REST API is done using the karma and the jasmine test runner. The karma and the jasmine test runner are set up in the project repository. The karma test runner : The main goal for Karma is to bring a productive testing environment to developers. The environment being one where they don't have to set up loads of configurations, but rather a place where developers can just write the code and get instant feedback from their tests. Because getting quick feedback is what makes you productive and creative. The jasmine test runner: Jasmine is a behavior-driven development framework for testing JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM. And it has a clean, obvious syntax so that you can easily write tests. The karma and jasmine test runner were configured for the project and basic tests were…

Continue ReadingsTeam API Endpoint Testing

Generating Documentation and Modifying the sTeam-REST API

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. sTeam-REST API repository: sTeam-REST Documentation Documentation is an important part of software engineering. Types of documentation include: Requirements - Statements that identify attributes, capabilities, characteristics, or qualities of a system. This is the foundation for what will be or has been implemented. Architecture/Design - Overview of software. Includes relations to an environment and construction principles to be used in design of software components. Technical - Documentation of code, algorithms, interfaces, and APIs. End user - Manuals for the end-user, system administrators and support staff. Marketing - How to market the product and analysis of the market demand. Doxygen Doxygen is the de facto standard tool for generating documentation from annotated C++ sources, but it also supports other popular programming languages such as C, Objective-C, C#, PHP, Java, Python, IDL (Corba, Microsoft, and UNO/OpenOffice flavors), Fortran, VHDL, Tcl, and to some extent D. The Doxygen treats files of other languages as C/C++ and creates documentation for it accordingly. sTeam documentation was tried to be created with the doxygen. But empty documentation was created due to the lack of the doxygen annotations used in the project.  Doxygen doc generation. Doxygen Docs The next way to create documentation was to make use of the autodoc utility provided by the Pike. The utility to generate docs was provided in the later versions of the Pike(>=8.0.155). The autodoc files are generated and  later these are converted into  html pages. The commands used for generating the autodoc include:- pike -x extract_autodoc /source pike -x autodoc_to_html /src /opfile The autodoc_to_html utility converts a single autodoc file to an html page. As a result a shell script was written to convert all the generated autodoc files to the html file. docGenerator.sh #!/bin/bash shopt -s globstar for filename in ./**/*.pike.xml; do outputFile=doc/${filename#./} outputFile=${outputFile%.xml}."html" if [ -d $(dirname "./"$outputFile) ]; then touch "./"$outputFile else mkdir -p $(dirname "./"$outputFile) && touch "./"$outputFile fi pike -x autodoc_to_html $filename "./"$outputFile done Autodoc Documentation The documentation generated by this was less informative and lacked the referrals to other classes and headers. The societyserver project was developed long back but the autodoc utility was introduced in the later versions of pike. As a result the source files lacked the autodoc tags which are required to generate a well informative documentation with bindings to other files. Restructuring the sTeam-REST API The sTeam-REST API project made use of the angular-seed to initiate the development during the early phases. However these files still existed in the project. This had lead to a pandemonium and created difficulty in understanding the project. The files had to be removed and the app was in dire need of a restructuring. The following issues have been reported and resolved. Issue. Github Issue Github PR sTeam-REST Issues Issues PR The new UI can be seen below. Home Register About Testing the REST API The functionality to run the tests using the npm test command was added to the project.…

Continue ReadingGenerating Documentation and Modifying the sTeam-REST API

Pike

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. sTeam-REST API repository: sTeam-REST The project is written in Pike programming language. Many of us would not be aware of it. Let's take a dive into Pike and learn more about it. What is pike? Pike is a general purpose programming language, which means that you can put it to use for almost any task. Its application domain spans anything from the world of the Net to the world of multimedia applications, or environments where your shell could use some spicy text processing or system administration tools. Your imagination sets the limit, but Pike will probably extend it far beyond what you previously considered within reach. Pike is a dynamic programming language with a syntax similar to Java and C. It is simple to learn, does not require long compilation passes and has powerful built-in data types allowing simple and really fast data manipulation. Pike is released under the GNU GPL, GNU LGPL and MPL; this means that you can fetch it and use it for almost any purpose you please. Who made pike? We will not bother you here with the entire history of Pike, but in a quick summary we should credit Fredrik Hübinette, who started writing Pike (at that time called µLPC), Roxen Internet Software, who funded the Pike development during its first years, the Pike development team that continues its development at present and the Software and Systems division of the Department of Computer and Information Science (IDA for short) at Linköping University, who currently provides funding for some Pike research and development, as well as this site. Also, without the participation of the friendly community of Pike users and advocates all over the world, Pike would hardly be the same either; we are grateful for your commitment. Who uses pike? Besides those already mentioned (Roxen IS and SaS, IDA, LiU), there are many other people scattered throughout the world who have put Pike to good use. Read some of their testimonials and find out more about how they value Pike. ...and for what? Roxen Internet Software wrote the free web servers Spinner, Roxen Challenger and Roxen WebServer in Pike, as well as the highly appraised commercial content management system Roxen Platform / Roxen CMS. SaS uses Pike for their research, currently concentrated on the field of compositioning technology and language connectors. Other noteworthy applications include the works of Per Hedbor, who among other things has written AIDO, a nifty network aware peer-to-peer client/server media player and a distributed jukebox system, both in Pike. Why use pike? Pike is Powerful - Being a high-level language, Pike gives you concise, modular code, automatic memory management, flexible and efficient data types, transparent bignum support, a powerful type system, exception handling and quick iterative development cycles, alleviating the need for compiling and linking code before you can run it; on-the-fly modifications are milliseconds away from being put to practice. Pike is Fast - Most…

Continue ReadingPike

sTeam REST API

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. sTeam-REST API repository: sTeam-REST REST Services REST is the software architectural style of the World Wide Web. REST (Representational State Transfer) was introduced by Roy Fielding in his doctoral dissertation in 2000. Its purpose is to induce performance, scalability, simplicity, modifiability, visibility, portability, and reliability.It has client/server relationship with a uniform interface and is stateless. REST is most commonly associated with HTTP but it is not strictly related to it. REST Principles Resources : Each and every component is a resource.A resource is accessed by a common interface using HTTP standard methods. Messages use HTTP methods like GET, POST, PUT, and DELETE. Resource identification through URI: Resources are identified using URI. Resources are represented using JSON or XML. Stateless interactions take place between the server and the client. No context is saved for the requests at the server.The client maintains the state of the session. HTTP methods The CRUD(create, retrieve, update and delete ) operations are performed using the HTTP methods. GET It is used to retrieve information. GET requests executed any number of times with the same parameters, the results would not change. This makes it idempotent. Partial or conditional requests can be sent. It is a read only type of operation. Retrieve a list of users: GET /api.example.com/UserService/users POST POST is usually used to create a new entity. It can also be used to update an existing entity. The request will have to do something with the entity provided in the URI. Create a new user with an ID 2: POST /api.example.com/UserService/users/2 PUT PUT request is always idempotent. Executing the same request any number of times will not change the output. PUT can be used to create or update an existing entity. Modify the user with an ID of 1: PUT /api.example.com/UserService/users/1 PATCH It is idempotent. PATCH requests only updates the specified fields of an entity. Modify a user with an id of 1: PATCH /api.example.com/UserService/users/1 DELETE It can be asynchronous or a long-running request. It removes the resource. It can be removed immediately or at a later instance of time. Delete a user with an ID of 1: DELETE /api.example.com/UserService/users/1  sTeam-REST API Installing and activating the REST API The REST API is developed as an application inside the sTeam server. This simplifies development quite a lot, as we don't need to restart the server for every change. Instead just the API code gets updated and reloaded. It may eventually be integrated into the core, however the longterm plan is actually to move functionality out of the core, to make development easier. To get the current version of the API clone the steam-rest repo into your home or to any place where you keep your development repos. Then change to the tools directory of your installation and run import-from-git. git clone https://github.com/societyserver/steam-rest cd steam-rest git checkout origin/rest.pike export steamrest=`pwd` cd /usr/local/lib/steam/tools ./import-from-git.pike -u root $steamrest / Note: The new import-from-git.pike script supports importing…

Continue ReadingsTeam REST API

sTeam demo

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications. sTeam server project repository: sTeam. Demo Visual aid can help a person to understand and grasp faster. A demo of the utility added to sTeam so far has been created to help the user base. The videos have been divided into sections based on the category of the scripts which they execute. Using a docker image. Starting the sTeam server. Running various utilities in the sTeam-shell. Import from git script. Export to git Feel free to explore the repository. Suggestions for improvements are welcomed. Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

Continue ReadingsTeam demo