AJAX

Read more about the article PSLab Remote Lab: Automatically deploying the EmberJS WebApp and Flask API Server to different domains

PSLab Remote Lab: Automatically deploying the EmberJS WebApp and Flask API Server to different domains

The remote-lab software of the pocket science lab enables users to access their devices remotely via the internet. Its design involves an API server designed with Python Flask, and a web-app designed with EmberJS that allows users to access the API and carry out various tasks such as writing and executing Python scripts. For testing purposes, the repository needed to be setup to deploy both the backend as well as the webapp automatically when a build passes, and this blog post deals with how this can be achieved. Deploying the API server The Heroku PaaS was chosen due to its ease of use with a wide range of server software, and support for postgresql databases. It can be configured to automatically deploy branches from github repositories, and conditions such as passing of a linked CI can also be included. The following screenshot shows the Heroku configuration page of an app called pslab-test1. Most of the configuration actions can be carried out offline via the Heroku-Cli   In the above page, the pslab-test1 has been set to deploy automatically from the master branch of github.com/jithinbp/pslab-remote . The wait for CI to pass before deploy has been disabled since a CI has not been setup on the repository. Files required for Heroku to deploy automatically Once the Heroku PaaS has copied the latest commit made to the linked repository, it searches the base directory for a configuration file called runtime.txt which contains details about the language of the app and the version of the compiler/interpretor to use, and a Procfile which contains the command to launch the app once it is ready. Since the PSLab’s API server is written in Python, we also have a requirements.txt which is a list of dependencies to be installed before launching the application. Procfile web: gunicorn app:app --log-file - runtime.txt python-3.6.1 requirements.txt gunicorn==19.6.0 flask >= 0.10.1 psycopg2==2.6.2 flask-sqlalchemy SQLAlchemy>=0.8.0 numpy>=1.13 flask-cors>=3.0.0 But wait, our app cannot run yet, because it requires a postgresql database, and we did not do anything to set up one. The following steps will set up a postgres database using the heroku-cli usable from your command prompt. Point Heroku-cli to our app $ heroku git:remote -a pslab-test1 Create a postgres database under the hobby-dev plan available for free users. $ heroku addons:create heroku-postgresql:hobby-dev Creating heroku-postgresql:hobby-dev on ⬢ pslab-test1... free Database has been created and is available ! This database is empty. If upgrading, you can transfer ! data from another database with pg:copy Created postgresql-slippery-81404 as HEROKU_POSTGRESQL_CHARCOAL_URL Use heroku addons:docs heroku-postgresql to view documentation The previous step created a database along with an environment variable HEROKU_POSTGRESQL_CHARCOAL_URL . As a shorthand, we can also refer to it simply as CHARCOAL . In order to make it our primary database, it must be promoted $ heroku pg:promote HEROKU_POSTGRESQL_CHARCOAL_URL The database will now be available via the environment variable DATABASE_URL Further documentation on creating and modifying postgres databases on Heroku can be found in the articles section . At this point, if the app is…

Continue ReadingPSLab Remote Lab: Automatically deploying the EmberJS WebApp and Flask API Server to different domains
Read more about the article Designing a Remote Laboratory with PSLab using Python Flask Framework

Designing a Remote Laboratory with PSLab using Python Flask Framework

In the introductory post about remote laboratories, a general set of tools to create a framework and handle its various aspects was also introduced. In this blog post, we will explore the implementation of several aspects of the backend app designed with python-flask, and the frontend based on EmberJS. A clear separation of the frontend and backend facilitates minimal disruption of either sections due to the other. Implementing API methods in Python-Flask In the Flask web server, page requests are handled via ‘routes’ , which are essentially URLs linked to a python function. Routes are also capable of handling payloads such as POST data, and various return types are also supported. We shall use an example to demonstrate how a Sign-Up request sent from the sign-up form in the remote lab frontend for PSLab is handled. @app.route('/signUp',methods=['POST']) def signUp(): """Sign Up for Virtual Lab POST: Submit sign-up parameters. The following must be present: inputName : The name of your account. does not need to be unique inputEmail : e-mail ID used for login . must be unique. inputPassword: password . Returns HTTP 404 when data does not exist. """ # read the posted values from the UI _name = request.form['inputName'] _email = request.form['inputEmail'] _password = request.form['inputPassword'] # validate the received values if _name and _email and _password: _hashed_password = generate_password_hash(_password) newUser = User(_email, _name,_hashed_password) try: db.session.add(newUser) db.session.commit() return json.dumps({'status':True,'message':'User %s created successfully. e-mail:%s !'%(_name,_email)}) except Exception as exc: reason = str(exc) return json.dumps({'status':False,'message':str(reason)})   In this example, the first line indicates that all URL requests made to <domain:port>/signUp will be handled by the function signUp . During development, we host the server on localhost, and use the default PORT number 8000, so sign-up forms must be submitted to 127.0.0.1:8000/signUp . For deployment on a globally accessible server, a machine with a static IP, and a DNS record must be used. An example for such a deployment would be the heroku subdomain where pslab-remote is automatically deployed ; https://pslab-stage.herokuapp.com/signUp A closer look at the above example will tell you that POST data can be accessed via the request.form dictionary, and that the sign-up routine requires inputName,inputEmail, and inputPassword. A password hash is generated before writing the parameters to the database. Testing API methods using the Postman chrome extension The route described in the above example requires form data to be submitted along with the URL, and we will use a rather handy developer tool called Postman to help us do this. In the frontend apps , AJAX methods are usually employed to do such tasks as well as handle the response from the server.   The above screenshot shows Postman being used to submit form data to /signUp on our API server running at localhost:8000 . The fields inputName, inputDescription, and inputPassword are also posted along with it. In the bottom section, one can see that the server returned a positive status variable, as well as a descriptive message. Submitting the sign up form via an Ember controller. Setting up a…

Continue ReadingDesigning a Remote Laboratory with PSLab using Python Flask Framework
Read more about the article Implementing Search Bar Using GitHub API In Yaydoc CI

Implementing Search Bar Using GitHub API In Yaydoc CI

In Yaydoc’s, documentation will be generated by typing the URL of the git repository to the input box from where user can generate documentation for any public repository, they can see the preview and if they have access, they can push the documentation to the github pages on one click. But In Yaydoc CI user can register the repository only if he has access to the specific repository, so I decided to show the list to the repository where user can select from the list but then it also has one problem that Github won’t give us all the user repository data in one api hit and then I made a search bar in which user can search repository and can register to the Yaydoc CI. var search = function () { var username = $("#orgs").val().split(":")[1]; const searchBarInput = $("#search_bar"); const searchResultDiv = $("#search_result"); searchResultDiv.empty(); if (searchBarInput.val() === "") { searchResultDiv.append('<p class="text-center">Please enter the repository name<p>'); return; } searchResultDiv.append('<p class="text-center">Fetching data<p>'); $.get(`https://api.github.com/search/repositories?q=user:${username}+fork:true+${searchBarInput.val()}`, function (result) { searchResultDiv.empty(); if (result.total_count === 0) { searchResultDiv.append(`<p class="text-center">No results found<p>`); styles.disableButton("btnRegister"); } else { styles.disableButton("btnRegister"); var select = '<label class="control-label" for="repositories">Repositories:</label>'; select += '<select class="form-control" id="repositories" name="repository" required>'; select += `<option value="">Please select</option>`; result.items.forEach(function (x){ select += `<option value="${x.full_name}">${x.full_name}</option>`; }); select += '</select>'; searchResultDiv.append(select); } }); }; $(function() { $("#search").click(function () { search(); }); }); In the above snippet I have defined search function which will get executed when user clicks the search button. The search function will get the search query from input box, if the search query is empty it’ll show the message as “Please enter repository name”, if it is not empty it’ll hit the GitHub API to fetch user repositories. If the GitHub returns empty array it’ll show “No results found”. In between searching time “Fetching data” will be shown. $('#search_bar').on('keyup keypress', function(e) { var keyCode = e.keyCode || e.which; if (keyCode === 13) { search(); } }); $('#ci_register').on('keyup keypress', function(e) { var keyCode = e.keyCode || e.which; if (keyCode === 13) { e.preventDefault(); return false; } }); Still we faced some problem, like on click enter button form is automatically submitting. So I’m registering event listener. In that listener I’m checking whether the key code is 13 or not. Key code 13 represent enter key, so if the key code is 13 then i’ll prevent the form from submitting. You can see the working of the search bar in the Yaydoc CI. Resources: Ajax Search Box in PHP and MySQL GitHub API Documentation jQuery AJAX Autocomplete – Country Example Build an AJAX search bar

Continue ReadingImplementing Search Bar Using GitHub API In Yaydoc CI
Read more about the article Registering Organizations’ Repositories for Continuous Integration with Yaydoc

Registering Organizations’ Repositories for Continuous Integration with Yaydoc

Among various features implemented in Yaydoc was the introduction of a modal in the Web Interface used for Continuous Deployment. The modal was used to register user’s repositories to Yaydoc. All the registered repositories then had their documentation updated continuously at each commit made to the repository. This functionality is achieved using Github Webhooks. The implementation was able to perform the continuous deployment successfully. However, there was a limitation that only the public repositories owned by a user could be registered. Repositories owned by some organisations, which the user either owned or had admin access to couldn’t be registered to Yaydoc. In order to perform this enhancement, a select tag was added which contains all the organizations the user have authorized Yaydoc to access. These organizations were received from Github’s Organization API using the user’s access token. /** * Retrieve a list of organization the user has access to * @param accessToken: Access Token of the user * @param callback: Returning the list of organizations */ exports.retrieveOrgs = function (accessToken, callback) { request({ url: ‘https://api.github.com/user/orgs’, headers: { ‘User-Agent’: ‘request’, ‘Authorization’: ‘token ’ + accessToken } }, function (error, response, body) { var organizations = []; var bodyJSON = JSON.parse(body); bodyJSON.forEach(function (organization) { organizations.push(organization.login); }); return callback(organizations); }); }; On selecting a particular organization from the select tag, the list of repositories is updated. The user then inputs a query in a search input which on submitting shows a list of repositories that matches the tag. An AJAX get request is sent to Github’s Search API in order to retrieve all the repositories matching the keyword. $(function () { .... $.get(`https://api.github.com/search/repositories?q=user:${username}+fork:true+${searchBarInput.val()}`, function (result) { .... result.items.forEach(function (repository) { options += ‘<option>’ + repo.full_name + ‘</option>’; }); .... }); .... }); The selected repository is then submitted to the backend where the repository is registered in Yaydoc’s database and a hook is setup to Yaydoc’s CI, as it was happening with user’s repositories. After a successful registration, every commit on the user’s or organization’s repository sends a webhook on receiving which, Yaydoc performs the documentation generation and deployment process. Resources: Github’s Organization API: https://developer.github.com/v3/orgs/ Github’s Search API: https://developer.github.com/v3/search/ Simplified HTTP Request Client: https://github.com/request/request

Continue ReadingRegistering Organizations’ Repositories for Continuous Integration with Yaydoc

Reset SUSI.AI User Password & Parameter extraction from Link

In this blog I will discuss how does Accounts handle the incoming request to reset the password. If a user forgets his/her password, They can use forgot password button on http://accounts.susi.ai and It’s implementation is quite straightforward. As soon as a user enter his/her e-mail id and hits RESET button, an ajax call is made which first checks if the user email id is registered with SUSI.AI or not. If not, a failure message is thrown to user. But if the user is found to be registered in the database, An email is sent to him/her which contains a 30 characters long token. On the server token is hashed against the user’s email id and a validity of 7 days is set to it. Let us have a look at the Reset Password link one receives. http://accounts.susi.ai/?token={30 characters long token} On clicking this link, what it does is that user is redirected to http://accounts.susi.ai with token as a request parameter. At the client side, A search is made which evaluates whether the URL has a token parameter or not. This was the overview. Since, http://accounts.susi.ai is based on ReactJS framework, it is not easy alike the native php functionality, but much more logical and systematic. Let us now take a closer look at how this parameter is searched for, token extracted and validated. As you can see http://accounts.susi.ai and http://accounts.susi.ai/?token={token}, Both redirect the user to the same URL. So the first task that needs to be accomplished is check if a parameter is passed in the URL or not. First import addUrlProps and UrlQueryParamTypes from react-url-query package and PropTypes from prop-types package. These will be required in further steps. Have a look at the code and then we will understand it’s working. const urlPropsQueryConfig = { token: { type: UrlQueryParamTypes.string }, }; class Login extends Component { static propTypes = { // URL props are automatically decoded and passed in based on the config token: PropTypes.string, // change handlers are automatically generated when given a config. // By default they update that single query parameter and maintain existing // values in the other parameters. onChangeToken: PropTypes.func, } static defaultProps = { token: "null", } Above in the first step, we have defined by what parameter should the Reset Password triggered. It means, if and only if the incoming parameter in the URL is token, we move to next step, otherwise normal http://accounts.susi.ai page should be rendered. Also we have defined the data type of the token parameter to be UrlQueryParamTypes.string. PropTypes are attributes in ReactJS similar to tags in HTML. So again, we have defined the data type. onChangeToken is a custom attribute which is fired whenever token parameter is modified. To declare default values, we have used defaultProps function. If token is not passed in the URL, by default it makes it null. This is still not the last step. Till now we have not yet checked if token is there or not. This is done in the componentDidMount…

Continue ReadingReset SUSI.AI User Password & Parameter extraction from Link
Read more about the article Advanced configurations in Yaydoc’s Web UI

Advanced configurations in Yaydoc’s Web UI

Yaydoc’s User Interface consists of a form with three required fields; the user’s email address, git repository’s URL, and a theme for the generated website. Specific values of these fields are the minimum requirement to generate documentation for a project. There are certain other configuration variables for whom we assumed default values. Among these, we assumed `docs/` directory or the directory specified in the `yaydoc.yml` configuration file as the default path for the documentation. Also, `Default Branch` is assumed as the branch to generate documentation website. However, this cannot guarantee the generation of docs for every other project. These configurations can have different values based on a project. Thus, there was a need to include certain input values for advanced configuration. The addition of these configurations in the UI doesn’t compel the user to specify them. In our attempt to improve user’s experience, we show the default values to the user when they are specifying custom values for these configurations. If the user doesn’t specify a value for the repository’s branch, a default value is retrieved from Github’s Repository Components API, taking repository’s URL from the required input as the input URL. /** * Setting the branch name with `default_branch` attriburte from * Github’s Repository Components API * @param gitUrl: URL of the github repository */ setDefaultBranchName: function (gitUrl) { var owner = gitUrl.split(“/”)[3] || ‘’; var repository = gitUrl.split(“/”)[4] || ‘’).split(‘.’)[0] || ‘’; $.get(‘https://api.github.com/repos/’ + owner + ‘/’ + repository, { headers: {“User-Agent”: “Yaydoc”} }).complete(function (data) { $(“#target_branch”).val(data.responseJSON.default_branch); }); } There are certain cases in which the design of the Web User Interface could have been confusing. Since we are displaying all the advanced configurations at once, it could’ve appeared to the users that they are specifying empty values for the other. Thus to handle this, inputs were enabled on toggle when a checkbox beside them was checked. This was achieved making following changes in the front end of the code. /** * Toggle editing of Branch Name input */ $(“#btnEditBranch”).click(function () { styles.toggleEditing(“target_branch”); .... .... }); /** * Toggle Enabling/Disabling an input tag * @param id: `id` attribute of input tag */ toggleEditing: function (id) { const input = $(‘#’ + id); if (input.attr(‘disabled’)) { input.removeAttr(‘disabled’); $(‘#checkbox_’ + id).removeClass(‘glyphicon-unchecked’).addClass(‘glyphicon-check’); } else { input.attr(‘disabled’, ‘disabled’); $(‘checkbox_’ + id).removeClass(‘glyphicon-check’).addClass(‘glyphicon-unchecked’); } } Introducing advanced configurations to the User Interface has opened the possibility for even more projects to generate and deploy docs with much lesser constraints. One of our main aim for this project is to have a fairly simple UI and UX and we hope to bring further updated to achieve that. Resources: Github’s Repository API: https://developer.github.com/v3/repos/ jQuery’s AJAX Requests: https://api.jquery.com/jquery.get

Continue ReadingAdvanced configurations in Yaydoc’s Web UI
Read more about the article Designing A Virtual Laboratory With PSLab

Designing A Virtual Laboratory With PSLab

What is a virtual laboratory A virtual lab interface gives students remote access to equipment in laboratories via the Internet without having to be physically present near the equipment. The idea is that lab experiments can be made accessible to a larger audience which may not have the resources to set up the experiment at their place. Another use-case scenario is that the experiment setup must be placed at a specific location which may not be habitable. The PSLab’s capabilities can be increased significantly by setting up a framework that allows remote data acquisition and control. It can then be deployed in various test and measurement scenarios such as an interactive environment monitoring station. What resources will be needed for such a setup The proposed virtual lab will be platform independent, and should be able to run in web-browsers. This necessitates the presence of a lightweight web-server software running on the hardware to which the PSLab is connected. The web-server must have a framework that must handle multiple connections, and allow control access to only authenticated users. Proposed design for the backend The backend framework must be able to handle the following tasks: Communicate with the PSLab hardware attached to the server Host lightweight web-pages with various visual aids Support an authentication framework via a database that contains user credentials Reply with JSON data after executing single commands on the PSLab Execute remotely received python scripts, and relay the HTML formatted output. This should include plots Proposed design for the frontend Responsive, aesthetic layouts and widget styles. Essential utilities such as Sign-up and Sign-in pages. Embedded plots with basic zooming and panning facilities. Embedded code-editor with syntax highlighting WIdgets to submit the code to the server for execution, and subsequent display of received response. A selection of tools that can assist with this project, and the purpose they will serve: Backend The Python communication library for the PSLab FLASK: ‘Flask is a BSD Licensed microframework for Python based on Werkzeug, Jinja 2 and good intentions.’   . It can handle concurrent requests, and will be well suited to serve as our web server MySQL: This is a database management utility that can be used to store user credentials, user scripts, queues etc WerkZeug: The utilities to create and check password hashes are essential for exchanging passwords via the database Json: For relaying measurement results to the client Gunicorn + Nginx: Will be used when more scalable deployment is needed, and the built-in webserver of Flask is unable to handle the load. Frontend Bootstrap-css: For neatly formatted, responsive UIs Jqplot: A versatile and expandable js based plotting library Ace code editor: A browser based code editor with syntax highlighting, automatic indentation and other user-friendly features. Written in JS Display documentation:  These can be generated server side from Markdown files using Jekyll. Several documentation files are already available from the pslab-desktop-apps, and can be reused after replacing the screenshot images only. Flow Diagram Recommended Reading [1]: Tutorial series  for creating a web-app using…

Continue ReadingDesigning A Virtual Laboratory With PSLab
Read more about the article Reset Password for SUSI Accounts Using Verification Link

Reset Password for SUSI Accounts Using Verification Link

In this blog, I will discuss how the SUSI server interprets the verification link sent to your email id to reset SUSI account password. The email one receives to reset password looks like this :   http://api.susi.ai/apps/resetpass/index.html?token={30 character long token} *Original link contains a token of length of 30 characters. The link has been tampered for security purpose. Taking a close look at the reset link, one would find it easy to decode. It simply contains path to an application on current SUSI Accounts hosting. Name of the application is “resetpass” for Reset Password. But what about the token in the link? As soon as a user clicks on the link, the app is called and token is passed as a GET parameter. The app in background makes a call to the server where the token is evaluated for whether the token is hashed against some user’s email id and has not expired yet. Below is code of first step the client does which is to make a simple ajax call on Ready state. $(document).ready(function() { var passerr = false, confirmerr = false, tokenerr = false; // get password parameters var regex; var urltoken = getParameter('token'); $.ajax( "/aaa/recoverpassword.json", { data: { getParameters: true, token: urltoken }, dataType: 'json', success: function (response) { regex = response.regex; var regexTooltip = response.regexTooltip; $('#pass').tooltip({'trigger':'focus', 'placement': 'left', 'title': regexTooltip}); $('#status-box').text(response.message); tokenerr = false; }, error: function (xhr, ajaxOptions, thrownError) { $('#status-box').text(thrownError); $('#status-box').addClass("error"); $('#pass').prop( "disabled", true ); $('#confirmpass').prop( "disabled", true ); $('#resetbut').prop( "disabled", true ); tokenerr = true; }, }); As you can see the call is made to /aaa/recoverypassword.json end point with token as the request parameter. Now, the client has to just evaluate the JSON response and render the message for user accordingly. If this request returns an error then the error message is shown and the entries are disabled to enter the password. Otherwise, user email id is shown with green text and user can now enter new password and confirm it. In next step, client simply evaluates the password and sends a query to server to reset it. Let us now look at how server functions and processes such requests. //check if a token is already present if (call.get("getParameters", false)) { if (call.get("token", null) != null && !call.get("token", null).isEmpty()) { ClientCredential credentialcheck = new ClientCredential(ClientCredential.Type.resetpass_token, call.get("token", null)); if (DAO.passwordreset.has(credentialcheck.toString())) { Authentication authentication = new Authentication(credentialcheck, DAO.passwordreset); if (authentication.checkExpireTime()) { String passwordPattern = DAO.getConfig("users.password.regex", "^(?=.*\\d).{6,64}$"); String passwordPatternTooltip = DAO.getConfig("users.password.regex.tooltip", "Enter a combination of atleast six characters"); result.put("message", "Email ID: " + authentication.getIdentity().getName()); result.put("regex", passwordPattern); result.put("regexTooltip", passwordPatternTooltip); result.put("accepted", true); return new ServiceResponse(result); } authentication.delete(); throw new APIException(422, "Expired token"); } throw new APIException(422, "Invalid token"); } else { throw new APIException(422, "No token specified"); } } In the above code snippet, server evaluates the received token on the basis of three parameters. First it checks whether the token is provided or not. If not, it throws APIException with error code 422 and a message “No token specified”. If it passes the check, next it checks…

Continue ReadingReset Password for SUSI Accounts Using Verification Link

Uploading a file to a server via PHP

Uploading a file to a server via PHP If you have been following my posts about my GSoC project, you would be knowing that we are making an app generator which will allow users to easily generate an android app for any event that they plan to host. So, the next thing that we wanted in our app was to allow the users to upload a zip containing the json files (in case they don’t have an API, from where app can fetch data from) and then upload it to the server where we can use these files during the app compilation. Steps below will tell you , how we achieved it : Changes to HTML First thing that we needed to do was add a file upload element to out HTML page that would allow only .zip files to be uploaded. It was pretty simple one liner code which goes as follows <tr> <td valign=”top”> <label for=”sessions”>Zip containing .json files</label> </td> <td valign=”top”> <input accept=”.zip” type=”file” id=”uploadZip” name=”sessions”> </td> </tr> PHP script to upload file on to the server Next, we needed a server sided script (I used PHP) which would upload the zip provided by the user on to the server and store it to a unique location for each user. The code for that was, <?php if ( 0 < $_FILES[‘file’][‘error’] ) { echo ‘Error: ‘ . $_FILES[‘file’][‘error’] . ‘<br>’; } else { move_uploaded_file($_FILES[‘file’][‘tmp_name’],“/var/www/html/uploads/upload.zip”); } ?> So what is happening here is basically the input arg. is first checked whether it is null or not null. If it is null, and error is thrown back to the user, else the file is renamed and uploaded to the uploads folder in the server’s public directory. Changes to the JavaScript This was the part that needed most of the changes to be done, we first had to store the file that is to be uploaded in the form data, and then make and AJAX call to the php file located on the server. var file_data = $(‘#uploadZip’).prop(‘files’)[0]; var form_data = new FormData(); form_data.append(‘file’, file_data); $.ajax( { url: ‘/upload.php’, // point to server-side PHP script cache: false, contentType: false, processData: false, data: form_data, type: ‘post’, success: function(php_script_response){ ajaxCall1(); } //Chain up another AJAX call for further operations }); So, that’s almost it! Some server sided changes were also required like allowing the web user to execute the upload.php script and making the uploads directory writable by the web user. Well, does it work? Um, yeah it does. There are a few issues with concurrent users which we are still debugging, but apart from that it works like a charm! Here you can see a folder created by each user based on his/her timestamp And here you can see the file that was uploaded y him/her Lastly our webapp (Looks stunning right?) So, that was all for this week, hope to see you again next time. Cheers and all the best :)

Continue ReadingUploading a file to a server via PHP