files

Read more about the article Saving Sensor Data in CSV format

Saving Sensor Data in CSV format

PSLab Android app by FOSSASIA provides a variety of features to its users. One of them is accessing various types of sensors both built into mobile phone and external sensors connected with PSLab device. In earlier versions users were only able to view the captured data. Moving forward, adding improvements to the app, now there is a feature to save those data displayed in graphs in csv format. This feature is important in many ways. One is educational. In a classroom, teachers can ask students to perform an experiment and prepare a report using the data collected. By just visualizing they cannot do this. Actual data points must be made available. Another use is sharing data sets related to say environmental data over different demographics. CSV, or comma-separated values file is a text file where stored data are separated by commas. The file stores these tabular data (numbers and text) in plain text format. Each line of the file represents a data record. Each data record consists of one or more fields, separated by commas. CSV files are commonly used to store sensor data because of its easy use. This post is about how PSLab device uses CSV file to write sensor data in it. In PSLab android source code, there is a dedicated class to handle read sensor data from different instruments called “CSVLogger”. Developers can easily instantiate this class wherever they want a data logging as follows; CSVLogger logger = new CSVLogger(<SUBFOLDER>); logger .writeCSVFile("Heading1,Heading2,Heading3\n");   This will create a blank folder in “PSLab” folder in device storage.  The CSV file is generated with the following convention according to the date and time where data is saved in the file. yyyymmdd-hhmmss.csv A sample file would have a name like 20180710-07:30:28.csv inside the SUBFOLDER which is specific to each instrument. Folder name will be the one used when initiating the CSVLogger. With this method, logging data is pretty easy. Simply create a string which is a comma seperated and ended with a new line character. Then simply call the writeCSVFile(data) method with the string as a parameter added to it. It will keep appending string data until a new file is created. File creation can be handled by developers at their own interests and preferences. String data = String.valueOf(System.currentTimeMillis()) + "," + item.getX() + "," + item.getY() + "\n"; logger.writeCSVFile(data);   To bring out an example let’s view how it’s implemented in Lux Meter instrument. This is a good source one can refer to when adding this feature in fragments inside a main activity. In Lux Meter, there is the parent activity named Lux Meter and inside that there are two fragments, one is fragmentdata and the other one is fragmentsettings. Data capturing and saving occurs inside fragmentdata. Menu icon controlling happens in the parent activity and we have bound a variable across the main activity and child fragment as follows; LuxMeterActivity parent = (LuxMeterActivity) getActivity(); if (parent.saveData) {/* Save Data */}   This makes it easier listening menu icon…

Continue ReadingSaving Sensor Data in CSV format
Read more about the article Uploading Files via APIs in the Open Event Server

Uploading Files via APIs in the Open Event Server

There are two file upload endpoints. One is endpoint for image upload and the other is for all other files being uploaded. The latter endpoint is to be used for uploading files such as slides, videos and other presentation materials for a session. So, in FOSSASIA’s Orga Server project, when we need to upload a file, we make an API request to this endpoint which is turn uploads the file to the server and returns back the url for the uploaded file. We then store this url for the uploaded file to the database with the corresponding row entry. Sending Data The endpoint /upload/file  accepts a POST request, containing a multipart/form-data payload. If there is a single file that is uploaded, then it is uploaded under the key “file” else an array of file is sent under the key “files”. A typical single file upload cURL request would look like this: curl -H “Authorization: JWT <key>” -F file=@file.pdf -x POST http://localhost:5000/v1/upload/file A typical multi-file upload cURL request would look something like this: curl -H “Authorization: JWT <key>” -F files=@file1.pdf -F files=@file2.pdf -x POST http://localhost:5000/v1/upload/file Thus, unlike other endpoints in open event orga server project, we don’t send a json encoded request. Instead it is a form data request. Saving Files We use different services such as S3, google cloud storage and so on for storing the files depending on the admin settings as decided by the admin of the project. One can even ask to save the files locally by passing a GET parameter force_local=true. So, in the backend we have 2 cases to tackle- Single File Upload and Multiple Files Upload. Single File Upload if 'file' in request.files: files = request.files['file'] file_uploaded = uploaded_file(files=files) if force_local == 'true': files_url = upload_local( file_uploaded, UPLOAD_PATHS['temp']['event'].format(uuid=uuid.uuid4()) ) else: files_url = upload( file_uploaded, UPLOAD_PATHS['temp']['event'].format(uuid=uuid.uuid4()) ) We get the file, that is to be uploaded using request.files[‘file’] with the key as ‘file’ which was used in the payload. Then we use the uploaded_file() helper function to convert the file data received as payload into a proper file and store it in a temporary storage. After this, if force_local is set as true, we use the upload_local helper function to upload it to the local storage, i.e. the server where the application is hosted, else we use whatever service is set by the admin in the admin settings. In uploaded_file() function of helpers module, we extract the filename and the extension of the file from the form-data payload. Then we check if the suitable directory already exists. If it doesn’t exist, we create a new directory and then save the file in the directory extension = files.filename.split('.')[1] filename = get_file_name() + '.' + extension filedir = current_app.config.get('BASE_DIR') + '/static/uploads/' if not os.path.isdir(filedir): os.makedirs(filedir) file_path = filedir + filename files.save(file_path) After that the upload function gets the settings key for either s3 or google storage and then uses the corresponding functions to upload this temporary file to the storage. Multiple File Upload elif 'files[]' in request.files:…

Continue ReadingUploading Files via APIs in the Open Event Server