numpy

Read more about the article Welcome the Visdom Project at FOSSASIA Now Fully Open Source

Welcome the Visdom Project at FOSSASIA Now Fully Open Source

We are proud to announce that FOSSASIA is welcoming the Visdom project. The project is being transitioned from Facebook AI Research to the FOSSASIA Organization. As part of this transition it has been relicensed to the Apache License 2.0 as fully Open Source.

Visdom is a flexible tool for creating, organizing, and sharing visualizations of live, rich data. It aims to facilitate visualization of (remote) data with an emphasis on supporting scientific experimentation. It supports PyTorch and Numpy. 

Visdom was created in 2017 by Allan Jabri and Laurens van der Maaten of Facebook AI Research, and further developed under the leadership of Jack Urbanek. To date, 90 developers from around the world have contributed to the project with over 3000 projects depending on Visdom. It is now available on the FOSSASIA GitHub.

“I’m excited to see how Visdom continues to grow as a FOSSASIA project, as the community will set a new vision for what we all want out of it. While I’ll no longer be leading the project, I will remain engaged to provide clear context for transitions, code reviews, and direct code contributions.”

Jack Urbanek, Facebook Research Engineer and Visdom project lead

“My goal continues to be building amazing communities around state of the art AI rooted in open source collaboration. Bringing the Visdom project to FOSSASIA is a great example of this and I am extremely pleased to see the project continue this path with FOSSASIA as the new host of Visdom.”

Joe Spisak, Product Manager for Facebook’s open-source AI platform and PyTorch

FOSSASIA has been developing Open Source software applications and Open Hardware together with a global community from its base in Asia since 2009. FOSSASIA’s goal is to provide access to open technologies, science applications and knowledge that improve people’s lives stating in its mission: “We want to enable people to adapt and change technology according to their own ideas and needs and validate science and knowledge through an Open Access approach.” 

This mission perfectly aligns with the goals of Visdom as an Open Source tool that aims to:

  • Facilitate visualization of data with an emphasis on supporting scientific experimentation and 
  • Organize a visualization space programmatically or through the UI to create dashboards for live data, inspect results of experiments, or debug experimental code.

Hong Phuc Dang, OSI vice president and FOSSASIA founder says:

“We will continue the development of Visdom in cooperation with the developer and user community. We already discussed lots of ideas to move forward on an exciting roadmap with the core team and adding it to FOSSASIA’s Pocket Science Lab applications. We are looking forward to the input and involvement of the community to bring the project to the next level.”

Mario Behling, co-founder of FOSSASIA and CEO of OpnTec adds:

“We are thrilled that the Visdom project becomes fully Open Source as part of the project transition. It is fantastic to see how Facebook supports open technologies and takes an active role to foster International cooperation and development in the FOSS ecosystem by making this transition. I would like to thank Jack Urbanek who worked so hard on the project for years as well as the project creators Allan Jabri and Laurens van der Maaten, Joe Spisak whose role was essential in making this transition happen and the entire Facebook AI team.”

What are the plans for Visdom at FOSSASIA?

The short term plans are to establish all project channels to enable the developer community to actively participate in the project. On the technical side of things we are in the process of making the continuous integration work on the new deployment and adding tests and update checks. Visdom is also joining the Codeheat Coding Contest as a participating project. The contest runs until 30th June 2021.

Where is the list of issues?

The issue tracker is available on the code repository here: https://github.com/fossasia/visdom/ 

How can developers and users communicate?

Apart from adding issues in the issue tracker we invite you to join us on a dedicated chat channel here: https://gitter.im/fossasia/visdom. Login with GitHub, Gitlab or Twitter is required.

Where else is information about the next steps and the roadmap?

For technical discussions the issue tracker is the best place. To stay up to date about general project developments please follow us on:

Continue ReadingWelcome the Visdom Project at FOSSASIA Now Fully Open Source
Read more about the article Designing A Remote Laboratory With PSLab: execution of function strings

Designing A Remote Laboratory With PSLab: execution of function strings

In the previous blog post, we introduced the concept of a ‘remote laboratory’, which would enable users to access the various features of the PSLab via the internet. Many aspects of the project were worked upon, which also involved creation of a web-app using EmberJS that enables users to create accounts , sign in, and prepare Python programs to be sent to the server for execution. A backend APi server based on Python-flask was also developed to handle these tasks, and maintain a postgresql database using sqlalchemy .

The following screencast shows the basic look and feel of the proposed remote lab running in a web browser.

This blog post will deal with implementing a way for the remote user to submit a simple function string, such as get_voltage(‘CH1’), and retrieve the results from the server.

There are three parts to this:
  • Creating a dictionary of the functions available in the sciencelab instance. The user will only be allowed access to these functions remotely, and we may protect some functions as the initialization and destruction routines by blocking them from the remote user
  • Creating an API method to receive a form containing the function string, execute the corresponding function from the dictionary, and reply with JSON data
  • Testing the API using the postman chrome extension
Creating a dictionary of functions :

The function dictionary maps function names against references to the actual functions from an instance of PSL.sciencelab . A simple dictionary containing just the get_voltage function can be generated in the following way:

from PSL import sciencelab
I=sciencelab.connect()
functionList = {'get_voltage':I.get_voltage}

This dictionary is then used with the eval method in order to evaluate a function string:

result = eval('get_voltage('CH1')',functionList)
print (result)
0.0012

A more efficient way to create this list is to use the inspect module, and automatically extract all the available methods into a dictionary

functionList = {}
for a in dir(I):
	attr = getattr(I, a)
	if inspect.ismethod(attr) and a!='__init__':
		functionList[a] = attr

In the above, we have made a dictionary of all the methods except __init__

This approach can also be easily extrapolated to automatically generate a dictionary for inline documentation strings which can then be passed on to the web app.

Creating an API method to execute submitted function strings

We create an API method that accepts a form containing the function string and option that specifies if the returned value is to be formatted as a string or JSON data. A special case arises for numpy arrays which cannot be directly converted to JSON, and the toList function must first be used for them.

@app.route('/evalFunctionString',methods=['POST'])
def evalFunctionString():
    if session.get('user'):
        _stringify=False
        try:
            _user = session.get('user')[1]
            _fn = request.form['function']
            _stringify = request.form.get('stringify',False)
            res = eval(_fn,functionList)
        except Exception as e:
            res = str(e)
        #dump string if requested. Otherwise json array
        if _stringify:
            return json.dumps({'status':True,'result':str(res),'stringified':True})
        else:
            #Try to simply convert the results to json
            try:
                return json.dumps({'status':True,'result':res,'stringified':False})
            # If that didn't work, it's due to the result containing numpy arrays.
            except Exception as e:
                #try to convert the numpy arrays to json using the .toList() function
                try:
                    return json.dumps({'status':True,'result':np.array(res).tolist(),'stringified':False})
                #And if nothing works, return the string
                except Exception as e:
                    print( 'string return',str(e))
                    return json.dumps({'status':True,'result':str(res),'stringified':True})
    else:
        return json.dumps({'status':False,'result':'unauthorized access','message':'Unauthorized access'})
Testing the API using Postman

The postman chrome extension allows users to submit forms to API servers, and view the raw results. It supports various encodings, and is quite handy for testing purposes.Before executing these via the evalFunctionString method, user credentials must first be submitted to the validateLogin method for authentication purposes.

Here are screenshots of the test results from a ‘get_voltage(‘CH1’)’ and ‘capture1(‘CH1’,20,1)’ function executed remotely via postman.

 

Our next steps will be to implement the dialog box in the frontend that will allow users to quickly type in function strings, and fetch the resultant data

Resources:

 

Continue ReadingDesigning A Remote Laboratory With PSLab: execution of function strings