sTeam Docker Image

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.
sTeam server project repository: sTeam.
sTeam-REST API repository: sTeam-REST

DOCKER IT!

What is Docker?

Docker is open source software to pack, ship and run any application as a lightweight container. Containers are completely hardware and platform independent so you don’t have to worry about whether what you are creating will run everywhere.

In order to facilitate development in all the environments and give the user the ease from the cumbersome installation steps a docker image was made for the sTeam project.
This docker image contains all the necessary dependencies to start the sTeam server.
It also includes the sTeam UI and the Rest API along with it’s dependencies. These have already been installed and developer can start collaborating to it.

docker-image

The docker image can be found at :


https://hub.docker.com/r/ajinkya007/societyserver/

It has all the necessary information for it’s usage and installation.

The dockerfile for the sTeam repository:


FROM ubuntu:latest
RUN apt-get update
RUN apt-get install -y wget git firefox nodejs nodejs-legacy npm
RUN apt-get install -y build-essential mysql-server libmysqld-dev bzip2 libjpeg$
RUN apt-get install -y pike7.8 pike7.8-bzip2 pike7.8-svg
RUN apt-get install -y libxml2-dev libxslt1-dev automake flex
RUN npm install -g npm bower gulp coffee-script jasmine-node frisby
RUN cd home
RUN wget -c https://raw.githubusercontent.com/societyserver/sTeam/steam-package$
RUN git clone www.github.com/societyserver/steam.git
RUN cd sTeam
RUN git checkout societyserver-source
RUN ./build
RUN sudo ./install
RUN cd ..
RUN git clone https://github.com/societyserver/sTeam-web-interface-gsoc-2015.git
RUN cd sTeam-web-interface-gsoc-2015
RUN npm install
RUN cd ..
RUN git clone https://github.com/societyserver/steam-rest.git
RUN cd steam-rest
RUN npm install

Feel free to explore the repository. Suggestions for improvements are welcomed.

Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

Python code examples

I’ve met many weird examples of  behaviour in python language while working on Open Event project. Today I’d like to share some examples with you. I think this knowledge is necessary, if you’d like to increase a  bit your knowledge in python area.

Simple adding one element to python list:

def foo(value, x=[]):
  x.append(value)
  return x

>>> print(foo(1))
>>> print(foo(2))
>>> print(foo(3, []))
>>> print(foo(4))

OUTPUT

[1] 
[1, 2] 
[3]
[1, 2, 4]

First output is obvious, but second not exactly. Let me explain it, It happens because x(empty list) argument is only evaluated once, So on every call foo(), we modify that list, appending a value to it. Finally we have [1,2, 4] output. I recommend to avoid mutable params as default.

Another example:

Do you know which type it is?

>>> print(type([ el for el in range(10)]))
>>> print(type({ el for el in range(10)}))
>>> print(type(( el for el in range(10))))

Again first and second type are obvious <class ‘list’>, <class ‘set’>. You may  think that last one should return type tuple but it returns a generator <class ‘generator’>.

Example:

Do you think that below code returns an exception?

list= [1,2,3,4,5]
>>> print(list [8:])

If you think that above expression returns index error you’re wrong. It returns empty list [].

Example funny boolean operators

>>> 'c' == ('c' or 'b')
True
>>> 'd' == ('a' or 'd')
False
>>> 'c' == ('c' and 'b')
False 
>>> 'd' == ('a' and 'd')
True

You can think that that OR and AND operators are broken.

You have to know how python interpreter behaves while looking for OR and AND operators.

So OR Expression takes the first statement and checks if it is true. If the first statement is true, then Python returns object’s value without checking second value. If first statement is false interpreter checks second value and returns that value.

AND operator checks if first statement is false, the whole statement has to be false. So it returns first value, but if first statement is true it checks second statement and returns second value.

Below i will show you how it works

>>> 'c' == ('c' or 'b')
>>> 'c' == 'c'
True
>>> 'd' == ('a' or 'd')
>>> 'd' == 'a'
False
>>> 'c' == ('c' and 'b')
>>> 'c' == 'b'
False 
>>> 'd' == ('a' and 'd')
>>> 'd' == 'd'
True

I hope that i have explained you how the python interpreter checks OR and AND operators. So know above examples should be more understandable.

sTeam Server Object permissions and Doxygen Documentation

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.
sTeam server project repository: sTeam.
sTeam-REST API repository: sTeam-REST

sTeam Server object permissions

sTeam command line lacks the functionality to read and set the object access permissions. The permission bits are: read,write, execute, move, insert,
annotate, sanction. The permission function was designed analogous to the getfacl() command in linux. It should display permissions as: rwxmias corresponding to the  permission granted on the object.

The the key functions are get_sanction, which returns a list of objects and permissions and sanction_object, which adds a new object and its set of permissions. The permissions is stored as an integer and the function should break the individual bits like getfact().

The permission bits for the sTeam objects are declared in the
access.h

// access.h: The permission bits

#define FAIL           -1 
#define ACCESS_DENIED   0
#define ACCESS_GRANTED  1
#define ACCESS_BLOCKED  2

#define SANCTION_READ          1
#define SANCTION_EXECUTE       2
#define SANCTION_MOVE          4
#define SANCTION_WRITE         8
#define SANCTION_INSERT       16
#define SANCTION_ANNOTATE     32

The get_sanction method defined in the access.pike returns a mapping which has the ACL(Access Control List) of all the objects in the sTeam server.


// Returns the sanction mapping of this object, if the caller is privileged
// the pointer will be returned, otherwise a copy.
final mapping
get_sanction()
{
    if ( _SECURITY->trust(CALLER) )
	return mSanction;
    return copy_value(mSanction);
}

The functions gets the permission values which are set for every object in the server.

The sanction_object method defined in the object.pike sets the permissions for the new objects.


// Set new permission for an object in the acl. Old permission are overwritten.
int sanction_object(object grp, int permission)
{
    ASSERTINFO(_SECURITY->valid_proxy(grp), "Sanction on non-proxy!");
    if ( query_sanction(grp) == permission )
      return permission; // if permissions are already fine

    try_event(EVENT_SANCTION, CALLER, grp, permission);
    set_sanction(grp, permission);

    run_event(EVENT_SANCTION, CALLER, grp, permission);
    return permission;
} 

This method makes use of the set_sanction which sets the permission onthe object. The task ahead is to make use of the above functions and write a sTeam-shell command which would provide the user to easily access and change the permissions for the objects.

Merging into the Source

The work done during GSOC 2016 by Siddhant and Ajinkya on the sTeam server was merged into the gsoc201-societyserver-devel and gsoc2016-source branches in the societyserver repository.
The merged code can be found at:

https://github.com/societyserver/sTeam/tree/gsoc2016-source
https://github.com/societyserver/sTeam/tree/gsoc2016-societyserver-devel

The merged code needs to be tested before the debian package for the sTeam server is prepared. The testing has resulted into resolving of minor bugs.

Doxygen Documentation

The documentation for the sTeam is done using doxygen. The doxygen.pike is written and used to make the documentation for the sTeam server. The Doxyfile which includes the configuration for generating the sTeam documentation is modified and input files are added. The generated documentation is deployed on the gh-pages in the societyserver/sTeam repository.
The documentation can be found at:


http://societyserver.github.io/sTeam/files.html

The header files and the constants defined are also included in the sTeam documentation.

sTeam Documentation:

SocietyserverDoc

sTeam defined constants:

SocietyServerConstants

sTeam Macro Definitions:

SocietyServerMacroDefnitions

Feel free to explore the repository. Suggestions for improvements are welcomed.

Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

Accepting Stripe payments on behalf of a third-party

{ Repost from my personal blog @ https://blog.codezero.xyz/accepting-stripe-payments-on-behalf-of-a-third-party }

In Open Event, we allow the organizer of each event to link their Stripe account, so that all ticket payments go directly into their account. To make it simpler for the organizer to setup the link, we have a Connect with stripe button on the event creation form.

Clicking on the button, the organizer is greeted with a signup flow similar to Login with Facebook or any other social login. Through this process, we’re able to securely and easily obtain the credentials required to accept payments on behalf of the organizer.

For this very purpose, stripe provides us with an OAuth interface called as Stripe Connect. Stripe Connect allows us to connect and interact with other stripe accounts through an API.

We’ll be using Python’s requests library for making all the HTTP Requests to the API.
You will be needing a stripe account for this.

Registering your platform
The OAuth Flow

The OAuth flow is similar to most platforms.

  • The user is redirected to an authorization page where they login to their stripe account and authorize your app to access their account
  • The user is then redirected back to a callback URL with an Authorization code
  • The server makes a request to the Token API with the Authorization code to retrieve the access_token, refresh_token and other credentials.

Implementing the flow

Redirect the user to the Authorization URL.
https://connect.stripe.com/oauth/authorize?response_type=code&client_id=ca_8x1ebxrl8eOwOSqRTVLUJkWtcfP92YJE&scope=read_write&redirect_uri=http://localhost/stripe/callback  

The authorization url accepts the following parameters.

  1. client_id – The client ID acquired when registering your platform.required.
  2. response_type – Response type. The value is always code. required.
  3. redirect_uri – The URL to redirect the customer to after authorization.
  4. scope – Can be read_write or read_only. The default is read_only. For analytics purposes, read_only is appropriate; To perform charges on behalf of the connected user, We will need to request read_write scope instead.

The user will be taken to stripe authorization page, where the user can login to an existing account or create a new account without breaking the flow. Once the user has authorized the application, he/she is taken back to the Callback URL with the result.

Requesting the access token with the authorization code

The user is redirected back to the callback URL.

If the authorization failed, the callback URL has a query string parameter error with the error name and a parameter error_description with the description of the error.

If the authorization was a success, the callback URL has the authorization code in the code query string parameter.

import requests

data = {  
    'client_secret': 'CLIENT_SECRET',
    'code': 'AUTHORIZATION_CODE',
    'grant_type': 'authorization_code'
}

response = requests.post('https://connect.stripe.com/oauth/token', data=data)

The client_secret is also obtained when registering your platform. The codeparameter is the authorization code.

On making this request, a json response will be returned.

If the request was a success, the following response will be obtained.

{
  "token_type": "bearer",
  "stripe_publishable_key": PUBLISHABLE_KEY,
  "scope": "read_write",
  "livemode": false,
  "stripe_user_id": USER_ID,
  "refresh_token": REFRESH_TOKEN,
  "access_token": ACCESS_TOKEN
}

If the request failed for some reason, an error will be returned.

{
  "error": "invalid_grant",
  "error_description": "Authorization code does not exist: AUTHORIZATION_CODE"
}

The access_token token obtained can be used as the secret key to accept payments like discussed in Integrating Stripe in the Flask web framework.

CommonsNet’s main features are ready

Main CommonsNet features are generating a file and  generating a code script. Today I would like to provide you with a simple and short guideline how I have managed to implement them.

Generate a file

Generating a file is the most important feature of our website, and it is not only a condition of its existance but also a basis to find stakeholders – in general people who want to use CommonsNet webiste.  It is the most complicated CommonsNet feature as well even if it is not so difficult as you may think.

To implement this feature a pure JavaScript is used and for now we have chosen the simplest solution to build it, mainly because we need to have a working webiste now to start a real promotion and find many customers.

Important steps here are:

  1. Manually prepare a .fodt file
  2. Get .fodt file and modify it
  3. Generate a pdf file from a ready .fodt file
  4. Write function to download a pdf file.

 

Preparing .fodt file

My mentor Andre Rebentisch has recommended me to use .fodt file because it’s very user-friendly and let you to prepare a file’s structure very easy. Actually that’s it! I have prepared manually in Libre Office a default doc file, and then save it in .fodt extension. .Fodt is a great option because it behaves as .xml file and let you modify this file using any programming language.

So, as I have already mentioned above I have created a simple doc file and prepare the structure. It is very clear and looks like a default Libre Office file. Only thing is that it has .fodt extension which means that you can open it in a code editior like for e.g. Sublime and start modifying easily.

If you open your .fodt file in a Sublime, it looks like this. Isn’t it simply? And above all it helps you to easily manipulate your data using for e.g a JavaScript.

Screen Shot 2016-08-13 at 17.19.06

Screen Shot 2016-08-13 at 17.18.00Get .fodt file to manipulate it

In my AngularJS Wizard Controller I write a function to get data from my .fodt file

$http({
 method: 'GET',
 url: 'http://commonsnet.herokuapp.com/generatefile.fodt',
 }).success(function(data){}) .error(function(){
 alert("error");
 });

It’s very easy code sample and you can find it in many Internet resources. I hope it is understandable to you as well. I have written a function which get a .fodt file and return data. Inside .success I have written my whole code to manipulate a .fodt file.  To use it I have decided to use a simple JavaScript replace method and if-else conditions, and in dependence on user’s inputs in a wizard form create different file’s structures. Let me show you a small section :

result = result.replace("NETWORK_NAME", "The owner provides" + " " + vm.ssid + "network connection")

if ((vm.password !== "") && (typeof vm.password !== "undefined")) {
 result = result.replace("INPUT_PASSWORD", "The owner informs that password is" + " " + vm.password);

}

Generate a pdf file from a ready .fodt file

It is still in progress! I will update it as soon as I manage to successfully implement it!

Write function to save it

To save my ready file I have used save function() and put in my AngularJS Wizard Controller.

$scope.save = function() {}

Generate a code script

We have decided to implement a generating code feature, because we want our users to be able to simply fill a wizard form with all Wifi details, then copy  a generated code  and be able to simply  paste it to their website.

We believe it is a very convenient solution to them because all they need to do it is to go through a form and it is ready. Besides, we have already decided to get rid of a database and use only a JSON file to not complicate our website’s working. So, that’s also the reason why we want to make it as simple as possible.

That’s why I have decided to create a link snd store there all wizard form’s values Undoubtedly it causes to build a very long link, but in the same time I believe that it is a good solution if we don’t use a database. I have also already found some ways to shorten and encrypt the generated link.

In my WizardController AngularJS I have write a function with a condtion

$scope.gotoStep = function(newStep) {
 currentStep = newStep;
 if ( currentStep === 3) {
 var link = "commonsnet.herokuapp.com/#/file?ssid=" + vm.ssid + "&password=" + vm.password + "&security=" + vm.securitytypes + "&standard=" + vm.wifistandards + "&payment=" + vm.paymentfieldyes + "&fee=" + vm.paymentfield + "&timelimit=" + vm.timelimityes + "&limit=" + vm.timelimitfield + "&service=" + vm.serviceyes + "&specialdevices=" + vm.specialdevices + "&devices=" + vm.specialdevicesfield + "&specialsettings=" + vm.specialsettings + "&settings=" + vm.specialsettingsfield + "&acceptterms=" + vm.acceptterms + "&liking=" + vm.socialprofile + "&downloading=" + vm.downloading + "&restrictions=" + vm.country + "&country=" + vm.countries + "&law=" + vm.legalrestrictions
 vm.code = '<a href="' + link + '">CommonsNet</a>'
 }
 }

If my currentStep === 3 (actually it’s a  confirmation.html templateUrl) I create a link (a code) which is available to users to copy and paste to their website.

Then I have also created a new FileController (FileCtrl) which is called when users enter a commonsnet.herokuapp.com/#/file (if they paste in an url a copied generated link)

app.controller('FileCtrl',['$scope', '$routeParams', function ($scope, $routeParams) { $scope.ssid = $routeParams.ssid; $scope.password = $routeParams.password; }])

Finally I have created a .html structure to post data from a wizard from which are all stored in our link. It’s a normal html structure but in order to adjust the html’s content to user’s wizard inputs I have used an Angular ng-if . A little example looks like this:

ng-if="ssid && ssid != 'undefined'

And that’s it. It still needs some improvements and fixing bugs but main features work according to my plan!!!

It’ time to intensively extend the idea of CommonsNet and find stakeholders! Are you interested too? I am sure you are, because CommonsNet is a great tool which helps us to build a transparent Wifi networks!

Feel free to write your comments and suggestion!

PayPal Express Checkout in Python

As per the PayPal documentation …

Express Checkout is a fast, easy way for buyers to pay with PayPal. Express Checkout eliminates one of the major causes of checkout abandonment by giving buyers all the transaction details at once, including order details, shipping options, insurance choices, and tax totals.

The basic steps for using express checkout to receive one-time payments are:

  1. Getting the PayPal API credentials.
  2. Making a request to the API with the transaction details to get a token
  3. Using the token to send the users to the PayPal payment page
  4. Capturing the payment and charging the user after the user completes the payment at PayPal.

We will be using PayPal’s Classic NVP (Name-value pair) API for implementing this.

Getting PayPal API Credentials

To begin with, we’ll need API Credentials.
We’ll be using the Signature API credentials which consists of

  • API Username
  • API Password
  • Signature

To obtain these, you can follow the steps at Creating and managing NVP/SOAP API credentials – PayPal Developer.

You’ll be getting two sets of credentials. Sandbox and Live. We’ll just stick to the Sandbox for now.

Now, we need sandbox test accounts for making and receiving payments. Head over to Creating Sandbox Test Accounts – PayPal Developer and create two sandbox test accounts. One would be the facilitator and one would be the buyer.

PayPal NVP Servers

All the API actions will take place by making a request to the PayPal server. PayPal has 4 different NVP servers for 4 different purposes.

  1. https://api-3t.sandbox.paypal.com/nvp – Sandbox “testing” server for use with API signature credentials.
  2. https://api-3t.paypal.com/nvp– PayPal “live” production server for use with API signature credentials.
  3. https://api.sandbox.paypal.com/nvp – Sandbox “testing” server for use with API certificate credentials.
  4. https://api.paypal.com/nvp – PayPal “live” production server for use with API certificate credentials.

We’ll be using the Sandbox “testing” server for use with API signature credentials.

Creating a transaction and obtaining the token

To create a transaction, we’ll need to make a request with all the transaction details. We can use Python requests library to easily make the requests. All requests are POST.

We’ll be calling the SetExpressCheckout method of the NVP API to obtain the token.

import requests  
import urlparse

data = {  
    'USER': credentials['USER'],
    'PWD': credentials['PWD'],
    'SIGNATURE': credentials['SIGNATURE'],
    'SUBJECT': credentials['FACILITATOR_EMAIL'],
    'METHOD': 'SetExpressCheckout',
    'VERSION': 93,
    'PAYMENTREQUEST_0_PAYMENTACTION': 'SALE',
    'PAYMENTREQUEST_0_AMT': 100,
    'PAYMENTREQUEST_0_CURRENCYCODE': 'USD',
    'RETURNURL': 'http://localhost:5000/paypal/return/',
    'CANCELURL': 'http://localhost:5000/paypal/cancel/'
}
response = requests.post('https://api-3t.sandbox.paypal.com/nvp', data=data)  
token = dict(urlparse.parse_qsl(response.text))['TOKEN']

Here,

  • USER represents your Sandbox API Username.
  • PWD represents your Sanbox API Password.
  • SIGNATURE represents your Sandbox Signature.
  • SUBJECT represents the facilitator’s email ID.
  • PAYMENTREQUEST_0_AMT is the total transaction amount.
  • PAYMENTREQUEST_0_CURRENCYCODE is the 3 digit ISO 4217 Currency code.
  • RETURNURL is where the user will be sent to after the transaction
  • CANCELURL is where the user will be sent to if he/she cancels the transaction.

A URL-Encoded, Name-value pair response would be obtained. We can decode that into a dict by using Python’s urlparse modules.

From the response, we’re extracting the TOKEN which we will use to generate the payment URL for the user.

This token has to be retained since we’ll be using it in further steps of the process.

Redirecting the user to PayPal for Approval

With the token we obtained, we can form the payment URL.

https://www.sandbox.paypal.com/cgi-bin/webscr?cmd=_express-checkout&token=<TOKEN>

We’ll have to send the user to that URL. Once the user completes the transaction at PayPal, he/she will be returned to the RETURNURL where we’ll further process the transaction.

Obtaining approved payment details and capturing the payment

Once the user completes the transaction and gets redirected back to RETURNURL, we’ll have to obtain the confirmed payment details from PayPal. For that we can again use the token ID that we obtained before.

We’ll now be making a request to the GetExpressCheckoutDetails method of the API.

import requests  
import urlparse

data = {  
    'USER': credentials['USER'],
    'PWD': credentials['PWD'],
    'SIGNATURE': credentials['SIGNATURE'],
    'SUBJECT': credentials['FACILITATOR_EMAIL'],
    'METHOD': 'GetExpressCheckoutDetails',
    'VERSION': 93,
    'TOKEN': TOKEN
}

response = requests.post('https://api-3t.sandbox.paypal.com/nvp', data=data)  
result = dict(urlparse.parse_qsl(response.text))  
payerID = result['PAYERID']

A URL-Encoded, Name-value pair response would be obtained. We can decode that into a dict by using Python’s urlparse modules.

This will provide us with information about the transaction such as transaction time, transaction amount, charges, transaction mode, etc.

But, we’re more interested in the PAYERID which we’ll need to capture/collect the payment. The money is not transferred to the facilitators account until it is captured/collected. So, be sure to collect it.

To collect it, we’ll be making another request to the DoExpressCheckoutPaymentmethod of the API using the token and the PAYERID.

import requests  
import urlparse

data = {  
    'USER': credentials['USER'],
    'PWD': credentials['PWD'],
    'SIGNATURE': credentials['SIGNATURE'],
    'SUBJECT': credentials['FACILITATOR_EMAIL'],
    'METHOD': 'DoExpressCheckoutPayment',
    'VERSION': 93,
    'TOKEN': TOKEN,
    'PAYERID': payerID,
    'PAYMENTREQUEST_0_PAYMENTACTION': 'SALE',
    'PAYMENTREQUEST_0_AMT': 100,
    'PAYMENTREQUEST_0_CURRENCYCODE': 'USD',
}

response = requests.post('https://api-3t.sandbox.paypal.com/nvp', data=data)  
result = dict(urlparse.parse_qsl(response.text))  
status = result['ACK']

All the details have to be the same as the ones provided while obtaining the token. Once we make the request, we’ll again get a URL-Encoded, Name-value pair response. We can decode that into a dict by using Python’s urlparsemodules.

From the response, ACK (Acknowledgement status) will provide us with the status of the payment.

  • Success — A successful operation.
  • SuccessWithWarning — A successful operation; however, there are messages returned in the response that you should examine.
  • Failure — The operation failed; the response also contains one or more error messages explaining the failure.
  • FailureWithWarning — The operation failed and there are messages returned in the response that you should examine.

And, we have completed the PayPal transaction flow for Express Checkout. These are just the basics and might miss a few stuff. I suggest you go through the following links too for a better understanding of everything:

For Reference:
  1. PayPal Name-Value Pair API Basics – PayPal Developer
  2. How to Create One-Time Payments Using Express Checkout – PayPal Developer

Writing linux commands

sTeam includes a number of tools namely steam-shell, debug, edit, import and export from git. These tools are only accessible from the tools folder in the sTeam installation. My task is to make a steam command for linux so that all these tools can be accessible from anywhere just like any other linux application.

Problem: https://github.com/societyserver/sTeam/issues/126

The command structure decided is that we have a global sTeam command and sub commands for each of the tools. This structure can be though of as similar to the structure of the git command. The central git command has many sub commands like branch, checkout, reset, merge, etc. I started with making a pike executable and then changing the installation files to copy this executable to /usr/bin. Now this pike executable becomes accessible from anywhere in the system and acts like a linux command.

https://github.com/societyserver/sTeam/pull/127

I wrote the main function to accept the command line arguments and passed them to a function named resolveParam. In this function I am using an if-else structure to segregate the different sub commands. For shell and debug I started a child process to execute the file for these tools present in /usr/local/lib/steam/tools. The following code snippet gives an example.

void resolveParam(int argc,array(string) argv)
{
if(argv[1]=="shell")
{
array command =({"/usr/local/lib/steam/tools/steam-shell.pike"});
object shell = Process.create_process(command, (["stdin":Stdio.stdin,"stdout":Stdio.stdout,"stderr":Stdio.stderr]));
shell->wait();
}
}

I had some problem when I had to accept more arguments than just the sub command. Foe example the edit command works as ./edit <filename1> <filename2> <filename3>… Using the global command it becomes steam edit <filename1> <filename2> <filename3>… I had to pass all the rest of the arguments to the edit scripts. I had to add them in the command array. The first solution would be to loop through the arguments array and add them all, however pike has a very easy and elegant way of doing it in one sentence.

array command = ({"/usr/local/lib/steam/tools/edit.pike"})+argv[2..];

This gets our global command up and running. Rest of the time I spend on resolving merge conflicts while merging all my previous Pull Requests.

https://github.com/Siddhant085/sTeam/tree/gsoc2016

linux command
output showing the use of steam command

sTeam Server Peer Review and Merging.

(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.
sTeam server project repository: sTeam.
sTeam-REST API repository: sTeam-REST

Peer Review

Peer review is done to ensure that the work performed by one’s peers meets specific criteria. Peer review is used while working in groups because of the belief that one’s peers are able to identify each others errors easily and thus speeding up the time it takes to identify the bugs and get them rectified. The goal of peer review is to ensure that the work satisfies the specifications and identify the aberrations from the standards, help in enhancing, modifying and providing suggestions for improvements. The Review process doesn’t involve management participation.

PeerReviewA lot of work was done in the sTeam server repository during the course of GSoC 2016. There is no support for continuous code integration and testing tools like Code Coverage, Travis CI, Circle CI or Landscape IO as the pike language is not supported by these tools. The project heavily relies on the peer reviews and feedbacks from the members of the community.

The work done by Siddhant on the testing framework and the steam subshell commands was reviewed and tested for bugs. The work which was tested can be found below:

sTeam Server testing framework:

https://github.com/societyserver/sTeam/pull/125

sTeam Server sub shell commands:

https://github.com/societyserver/sTeam/pull/135

sTeam Server structure

The steam project files are segregated into various directories depending  upon the tasks performed by them. For an individual to be productively working on the project, the understanding of the directory structure is utmost. The structure needs to be documented. The documentation for the sTeam server was done to facilitate easy understanding of the project structure.

https://github.com/societyserver/sTeam/wiki/sTeam-Server-Structure

sTeam Server Merging the PR’s

Issue’s reported were addressed in individual branches in the fork repositories of the collaborators. The changes made were sent in PR’s to the societyserver-devel and societyserver-source repo. These needed to be merged in the source repository. For resolving the conflicts during merging these, the PR’s were merged into gsoc2016 branch in the societyserver repo.

GSoC 2016 societyserver-devel

// Ajinkya 

https://github.com/societyserver/sTeam/pull/132

// Siddhant

https://github.com/societyserver/sTeam/pull/133

GSoC 2016 societyserver-source

// Ajinkya

https://github.com/societyserver/sTeam/pull/138

// Siddhant

https://github.com/societyserver/sTeam/pull/137

The merge conflicts between the two PR’s must be resolved and then merged into the main societyserver branches.
Also the issue for setting the permissions for sTeam objects would be addressed in the coming days.
Feel free to explore the repository. Suggestions for improvements are welcomed.

Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.

CommonNet – how to set up tests

Setting up tests’ environment wasn’t easy

Have you ever tried to set up your tests’ environment using Protractor on Vagrant?

I must admit that it was a very difficult task for me. I have recently spent almost three days trying to prepare my tests’ environment for CommonsNet project, and read many different resources. Fortunately, I have finally done it, so now I want to share with you my experience and give you some tips how to do it.

Protractor

Firstly, I will explain you why I have decided to use Protractor. It is mainly because Proractor is especially designed for end-to-end testing  AngularJS application. Protractor runs tests against your application running in a real browser, interacting with it as a user would.

Writing tests using Protractor is quite easy because you can find working examples in AngularJS docs. Each sample of AngularJS code is enriched by Protractor’s test. It’s amazing.

Selenium

Selenium is a browser automation library. It is most often used for testing web-applications, and may be used for any task that requires automating interaction with the browser. You can download it from here. You have to choose Selenium for a language you use. I have used Selenium for NodeJS.

Vagrant

You don’t have to necessary use Vagrant to run your tests, but I have implemented it, because I run my local environment on Vagrant and it’s more comfortable for me to use it.

Setting up testing environment

Now I will share with you how to run tests. So first all of, I have created a file called install.sh and put all necessary commends there. I have put there several commands. Please take a look at this file.  It helps you to install all of these necessary dependencies using only one command instead of several ones. 

install

Next, I have prepared provision folder, where I put files to install selenium standalone and chromium driver. You can copy these file from here

Then I have created a simple test case. It’s quite easy at the beginning. You just need two files first – conf. js and next- todo_spec.js  Below, I will provide you with my conf.js As you can see it’s not complicated and really short. It’s a basic configuration file and of course you can adjust it to your needs. You can find many examples of conf.js file in Internet.

conf

And finally a simple test, which I have placed in todo_spec.js file. It’s a ProtracotrJS example available on their website

todo-spec.png

Now, let me to write a step by step todo list now.

  • Install Vagrant in your folder
vagrant up
  • Connect to Vagrant
vagrant ssh
  • Open your Vagrant folder
cd /vagrant 
  • Then run selenium file
sh selenium_install.sh 
  • Next open provision folder
cd provision
  • Install java-jar
DISPLAY=:1 xvfb-run java -jar selenium-server-standalone-2.41.0.jar 

Your selenium server should be up and running

  • Then open a new terminal – remember not to close the first one!
  • Open your CommonsNet repository again
cd CommonsNet
  • Connect to Vagrant again
vagrant ssh
  • Open Vagrant folder again
cd /vagrant
  • Then open tests folder
cd tests
  • And finally run Protractor test
protractor conf.js 

That’s it. You should see a result of your test in a terminal.

 

Allowing web-user on apache server to run scripts as root

Allowing web-user on apache server to run scripts as root

If you are new to this, you might be wondering, what the hell is a web user anyways?

So let’s say that you need a server which hosts a simple web page and does a particular task based on data entered into that web-page.

The normal way of doing this is to navigate to /var/www/html and place the web page you want to host here.

You also need to put your php script in this folder so that it is accessible from the website.
This php script will take in the data from your web-page and run the necessary commands that you need to be executed on the server.( I am assuming you are not using “The Real Dev Language” for now. :p )

I will be using a simple web page and script that I have made for this post.

<html>
<head>
  <title>Apk Generator</title>
  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" crossorigin="anonymous">
  <link href='https://fonts.googleapis.com/css?family=Roboto:400,100' rel='stylesheet' type='text/css'>
  <link href="css/main.css" rel="stylesheet">
</head>
<body>
<div class="container"><br><br>
<form name="htmlform" id="form" enctype="multipart/form-data" class="col-md-offset-4 col-xs-offset-2 col-xs-8 col-md-4 form-group generator_form" >
  <label for="name">Email</label>
      <input type="email" class="form-control" id="Email" name="Email">
      <br>
      <input type="hidden" id="theme" name="theme" value="light">

      <label for="name">App's Name</label>
      <input type="text" class="form-control" id="App_Name" name="App_Name">
      <br>
      <label> Choose your data source </label>
      <ul style="list-style-type:none">
        <li><input type="radio" name="datasource" value="jsonupload"> Upload your own JSON files </input></li>
        <li><input type="radio" name="datasource" value="eventapi"> API endpoint of event on OpenEvent </input></li>
      </ul>
      <br>
      <section id="eventapi-input" style="display:none;">
        <label for="apiendpoint">Link to Open Event API endpoint</label>
        <input type="url" class="form-control"
        id="Api_Link" name="Api_Link">
      </section>
      <br>
      <section id="jsonupload-input" style="display:none;">
        <input type="file" name="uploadZip" id="uploadZip" class="form-control"/>
        <br>
      </section>
      <br>
      <input type="hidden" name="assetmode" value="download">
      <center>
        <br>
        <div id="status"></div>
        <br>
<tr>
 <td colspan="5" style="text-align:center">
  <button type="submit">Generate and Download app</button>
</td>
</tr>
</table>
</form>
<script src="https://www.gstatic.com/firebasejs/live/3.0/firebase.js"></script>
<script src="https://code.jquery.com/jquery-3.1.0.min.js"></script>
<script src="https://code.jquery.com/ui/1.12.0/jquery-ui.js"></script>
<script>
  $('input:radio[name="datasource"]').change(
  function() {
    if ($(this).is(':checked')) {

      if ($(this).val() === 'mockjson') {
        $('#jsonupload-input').hide(100);
        $('#eventapi-input').hide(100);
      }

      if ($(this).val() === 'jsonupload') {
        $('#jsonupload-input').show(100);
        $('#eventapi-input').hide(100);
      }

      if ($(this).val() === 'eventapi') {
        $('#eventapi-input').show(100);
        $('#jsonupload-input').hide(100);
      }
    }
  });
  var $ = jQuery;
  var timestamp = Number(new Date());
  var form = document.querySelector("form");
      form.addEventListener("submit", function(event) {
        event.preventDefault();
        var ary = $(form).serializeArray();
        var obj = {};
        for (var a = 0; a < ary.length; a++) obj[ary[a].name] = ary[a].value;
          console.log("JSON",obj);
        if(obj.Email == "" || obj.App_Name ==""){
          alert("It seems like you forgot to fill up your email address or the app's name");
          setTimeout("location.reload(true);", 1);
        }
        else{
	alert("Please wait while we generate the app, meanwhile you can stick around to directly download it.The app will also be emailed to you."); 
            $.ajax({
              type: "POST",
              url: "/test.php",
              data: { timestamp : timestamp },
              success: function(response){
                console.log("Success",response);
                window.location = response;
              }
            });
        }
      });
    </script>
    </div>
</body>
</html>

This is basically a web page with some inputText widgets which accept response and send it to a php file named test.php on the server via an AJAX post.

<?php
if(isset($_POST['timestamp']))
{
    $uid = escapeshellcmd($_POST['timestamp']);
    exec("sudo sh /var/www/email.sh $uid");
}
?>

This php script will call a bash script which in turns an email to me with the user’s timestamp as the subject.

Well, here is where the problem arises, as I am trying to run the bash file as root.

You might wonder as to why is this such a big issue?
Why can’t we do that?

Well, we can surely do that on the server but the point to be noted here is that we are not running this script directly from the server.

We are running it from a web page which is hosted on a server.

So our user here is a web user aka www-data rather than being a root user.

The web user is not provided root access by default, but there are ways to get this done.

Solution 1 :

Allow the web user to run only specific scripts as root.

Please note that this is not a ideal workaround.
Ideally your web user should not have root access in any case.
Since that’s cleared up, lets proceed.

This can be done by editing your sudoers list and adding www-data to it.
Open up your terminal and enter the following command.

sudo visudo

Next up, navigate to the end of the file and add the following command there

www-data = (root) NOPASSWD: /path/to/script.sh

In case you have to execute one script as root which in turn executes some more scripts as root, you don’t need to set the path to all of them over here.
Doing it only for the parent script will do the job.

Solution 2 :

Using SuExec

DigitalOcean blog has a very good article on how to execute python scripts as root via the web user through cgi.

You can go through the article here :https://www.digitalocean.com/community/tutorials/how-to-use-suexec-in-apache-to-run-cgi-scripts-on-an-ubuntu-vps

Well, that was all about my findings on properly handling sudo requirements for your web user on your apache server.

I’ll be adding more solutions as I find them along the way.
Meanwhile feel free to comment below your thoughts, suggestions and queries.

Cheers.