In FOSSASIA‘s Open Event Server, we use the Python unittest library for unit testing various modules of the API code. Unittest library provides us with various assertion functions to assert between the actual and the expected values returned by a function or a module. In normal modules, we simply use these assertions to compare the result since the parameters mostly take as input normal data types. However one very important area for unittesting is File Uploading. We cannot really send a particular file or any such payload to the function to unittest it properly, since it expects a request.files kind of data which is obtained only when file is uploaded or sent as a request to an endpoint. For example in this function:
def uploaded_file(files, multiple=False): if multiple: files_uploaded = [] for file in files: extension = file.filename.split('.')[1] filename = get_file_name() + '.' + extension filedir = current_app.config.get('BASE_DIR') + '/static/uploads/' if not os.path.isdir(filedir): os.makedirs(filedir) file_path = filedir + filename file.save(file_path) files_uploaded.append(UploadedFile(file_path, filename)) else: extension = files.filename.split('.')[1] filename = get_file_name() + '.' + extension filedir = current_app.config.get('BASE_DIR') + '/static/uploads/' if not os.path.isdir(filedir): os.makedirs(filedir) file_path = filedir + filename files.save(file_path) files_uploaded = UploadedFile(file_path, filename) return files_uploaded
So, we need to create a mock uploading system to replicate this check. So inside the unittesting function we create an api route for this particular scope to accept a file as a request. Following is the code:
@app.route("/test_upload", methods=['POST']) def upload(): files = request.files['file'] file_uploaded = uploaded_file(files=files) return jsonify( {'path': file_uploaded.file_path, 'name': file_uploaded.filename})
In the above code, it creates an app route with endpoint test_upload. It accepts a request.files. Then it sends this object to the uploaded_file function (the function to be unittested), gets the result of the function, and returns the result in a json format.
With this we have the endpoint to mock a file upload ready. Next we need to send a request with file object. We cannot send a normal data which would then be treated as a normal request.form. But we want to receive it in request.files. So we create 2 different classes inheriting other classes.
def test_upload_single_file(self): class FileObj(StringIO): def close(self): pass class MyRequest(Request): def _get_file_stream(*args, **kwargs): return FileObj() app.request_class = MyRequest
MyRequest class inherits the Request class of Flask framework. We define the file stream of the Request class as the FileObj. Then, we set the request_class attribute of the Flask app to this new MyRequest class.
After we have it all setup, we need to send the request and see if the uploaded file is being saved properly or not. For this purpose we take help of StringIO library. StringIO creates a file-like class which can be then used to replicate a file uploading system. So we send the data as {‘file’: (StringIO(‘1,2,3,4’), ‘test_file.csv’)}. We send this as data to the /test_upload endpoint that we have created previously. As a result, the endpoint receives the function, saves the file, and returns the filename and file_path for the stored file.
with app.test_request_context(): client = app.test_client() resp = client.post('/test_upload', data = {'file': (StringIO('1,2,3,4'), 'test_file.csv')}) data = json.loads(resp.data) file_path = data['path'] filename = data['name'] actual_file_path = app.config.get('BASE_DIR') + '/static/uploads/' + filename self.assertEqual(file_path, actual_file_path) self.assertTrue(os.path.exists(file_path))
After this is done, we need to check if the file_path that we receive is the expected file path that we should get. Secondly, we also check whether the file was really created or is this just some dummy data sent. We get the expected path by this:
actual_file_path = app.config.get('BASE_DIR') + '/static/uploads/' + filename.
Then we assert that actual_file_path is same as the resulting path we received using the assertEqual. Thirdly, we use assertTrue to ensure that there is a file in that path. That is,
self.assertTrue(os.path.exists(file_path))
Which gives a True if file exists or False if not.
So that basically sums up the unittesting.
1) If the file is saved in the correct path, and
2) The file actually exist
The the unittest passes only if both is True and is thus successful. Else we get either an error or a failure.
Following is the entire code snippet for this unit testing function:
def test_upload_single_file(self): class FileObj(StringIO): def close(self): pass class MyRequest(Request): def _get_file_stream(*args, **kwargs): return FileObj() app.request_class = MyRequest @app.route("/test_upload", methods=['POST']) def upload(): files = request.files['file'] file_uploaded = uploaded_file(files=files) return jsonify( {'path': file_uploaded.file_path, 'name': file_uploaded.filename}) with app.test_request_context(): client = app.test_client() resp = client.post('/test_upload', data = {'file': (StringIO('1,2,3,4'), 'test_file.csv')}) data = json.loads(resp.data) file_path = data['path'] filename = data['name'] actual_file_path = app.config.get('BASE_DIR') + '/static/uploads/' + filename self.assertEqual(file_path, actual_file_path) self.assertTrue(os.path.exists(file_path))
Resources:
- Learn about python unittest library: https://docs.python.org/2/library/unittest.html
- Tutorial on Python Unittest framework: https://www.tutorialspoint.com/unittest_framework/unittest_framework.htm
- A quick video tutorial on getting started with unittest: https://www.youtube.com/watch?v=0Keq3E2bbeE&t=7s
- A blog on Unit Testing a flask application by Patrick Kennedy: http://www.patricksoftwareblog.com/unit-testing-a-flask-application/