archive

We in FOSSASIA’s Open Event Server project, allow the organizer, co-organizer and the admins to export all the data related to an event in the form of an archive of JSON files. This way the data can be reused in some other place for various different purposes. The basic workflow is something like this: Send a POST request in the /events/{event_id}/export/json with a payload containing whether you require the various media files. The POST request starts a celery task in the background to start extracting data related to event and jsonifying them The celery task url is returned as a response. Sending a GET request to this url gives the status of the task. If the status is either FAILED or SUCCESS then there is the corresponding error message or the result. Separate JSON files for events, speakers, sessions, micro-locations, tracks, session types and custom forms are created. All this files are then archived and the zip is then served on the endpoint /events/{event_id}/exports/{path} Sending a GET request to the above mentioned endpoint downloads a zip containing all the data related to the endpoint. Let’s dive into each of these points one-by-one POST request ( /events/{event_id}/export/json) For making a POST request you firstly need a JWT authentication like most of the other API endpoints. You need to send a payload containing the settings for whether you want the media files related with the event to be downloaded along with the JSON files. An example payload looks like this: {  "image": true,  "video": true,  "document": true,  "audio": true } def export_event(event_id): from helpers.tasks import export_event_task settings = EXPORT_SETTING settings['image'] = request.json.get('image', False) settings['video'] = request.json.get('video', False) settings['document'] = request.json.get('document', False) settings['audio'] = request.json.get('audio', False) # queue task task = export_event_task.delay( current_identity.email, event_id, settings) # create Job create_export_job(task.id, event_id) # in case of testing if current_app.config.get('CELERY_ALWAYS_EAGER'): # send_export_mail(event_id, task.get()) TASK_RESULTS[task.id] = { 'result': task.get(), 'state': task.state } return jsonify( task_url=url_for('tasks.celery_task', task_id=task.id) ) Taking the settings about the media files and the event id, we pass them as parameter to the export event celery task and queue up the task. We then create an entry in the database with the task url and the event id and the user who triggered the export to keep a record of the activity. After that we return as response the url for the celery task to the user. If the celery task is still underway it show a response with ‘state’:’WAITING’. Once, the task is completed, the value of ‘state’ is either ‘FAILED’ or ‘SUCCESS’. If it is SUCCESS it returns the result of the task, in this case the download url for the zip. Celery Task to Export Event Exporting an event is a very time consuming process and we don’t want that this process to come in the way of user interaction with other services. So we needed to use a queueing system that would queue the tasks and execute them in the background with disturbing the main worker from executing the other user…