Database Listener for User Centric Events

Badgeyay is an open-source utility developed by FOSSASIA to generate badges for conferences and events. The project is separated into two components to ease maintainability. First is the frontend part which is in ember and second part is backend which is in Flask. The choice of database to support backend is PostgreSQL.

Now comes the problem, whenever a user is registered in the database, he should receive  a verification mail, that he is successfully registered on the platform. For this case we have to listen to the database events on User model. This issue has greater extendibility than only sending greeting or verification mail to the user. We can extend this to trigger services that are dependent on user registration, like subscribing the user to some set of services based on the plan he opted while registration and many more.

These type of issues cannot be handled by normal relationship with tables and other entities, there has to be logic in place to support such functionalities. So the challenges for tackling the problem are as follows:

  • Listen to the insert_action in User model
  • Extracting the details necessary for the logic
  • Execute particular logic

Procedure

  1. Attaching insert_action listener to the User model. This function will get triggered whenever an entity is saved in the User model.

<!– HTML generated using hilite.me –>

@db.event.listens_for(User, "after_insert")
def logic(mapper, connection, target): {
......
}
  1. When the function gets triggered, extract the details of the saved user that is necessary for the logic. As currently we are sending greeting mail to the user,we only need the email of the user. Target is the actual saved user passed as argument to the listening function from the library.

<!– HTML generated using hilite.me –>

msg = {}
msg['subject'] = "Welcome to Badgeyay"
msg['receipent'] = target.email
msg['body'] = "It's good to have you onboard with Badgeyay. Welcome to " \
"FOSSASIA Family."
sendMail(msg)
  1. Now the details are passed to sendMail() function for sending mail which uses flask-mail library to send mail to the recipient.
    def sendMail(message):
    if message and message.receipent:
    try:
    msg = Message(
    subject=message.subject,
    sender=app.config['MAIL_USERNAME'], Response(200).generateMessage(
    recipients=[message.receipent],
    body=message.body)
    Mail(app).send(msg)
    except Exception as e:
    return jsonify(
    Response(500).exceptWithMessage(
    str(e),
    'Unable to send the mail'))
    return jsonify(
    Response(200).generateMessage(
    'Mail Sent'))
    else:
    return jsonify(
    Response(403).generateMessage(
    'No data received')) 'No data received'))
    
  2. This will send mail to the user who has been registered to the application.

Similarly we can use separate logics according to the need of the application.

 

The Pull Request for the above functionality is at this Link

Topics Involved

Working on the issue involve following topics:

  • Configuring mail service to allow insecure apps access.
  • Sending mail from the flask-mail to end user
  • Attaching listener to listen for database change
  • Extraction of data from saved object in database sqlalchemy.

Resources

  • Sending Mails Programmatically –  Link
  • Flask Mail Documentation – Link
  • Listening to database events – Link
  • Enabling access to GMAIL to send mails to recipient – Link

Badgeyay: Integrating EmberJS Frontend with Flask Backend

Badgeyay is a simple badge generator with a simple web UI that generates a printable badge in PDFs. The project had gone through different cycles starting from a Flask server to a CLI application then a python library and now API Interface for generation of badges.

According to latest changes in the project structure, now the frontend and backend are independent components developed in Ember JS and Flask respectively. Now there is a need to connect the frontend to the backend, which means the user should see the response on the same page without refresh, if the badge generated successfully. AJAX would fit right into the spot. Asynchronous Javascript and XML also known as AJAX, will enable us to perform asynchronous operation on the page without refreshing the page.

We can make an API call to the Server running in backend or deployed on heroku, but the server is not suitable for doing CORS(Cross-Origin Resource Sharing), ability to share the resources on server with the client having different domain names, but as the server and the frontend are not hosted on the same host  so there is a need to enable the server to accept CORS request calls.

Now the challenges were:

  • Enabling Flask Server to accept CORS requests.
  • AJAX query for sending request to the Flask server.

Procedure

  1. Giving the form an id and creating an AJAX request to the Flask server (may be localhost or deployed on heroku).
<form id=”form1″ action=”” method=”post” enctype=”multipart/form-data” onsubmit=”return validate()”>

 

When the generate button is clicked, an AJAX request is made to the server to generate badges and at the same time prevent the page from refreshing. In the AJAX request we set the CORS header to allow the domain.

 

<script type=”text/javascript”>
$(document).ready(function () {
$(‘#form1’).submit(function (event) {
event.preventDefault();
$.ajaxSetup({
headers: {“Access-Control-Allow-Origin”: “*”}
});
$.ajax({
url: “http://badgeyay-api.herokuapp.com/api/v1.0/generate_badges”,
data: $(this).serialize(),
type: ‘POST’,
success: function (data) {…},
error: function (error) {…}
})
});
})
</script>

 

  1. Import the library and enable the API endpoint to accept CORS requests.
from flask_cors import CORS
cors = CORS(app, resources={r”/api/*”: {“origins”: “*”}})

 

  1. Add Logic for appending the download link by extracting the download link from the response and replacing the static text in the template with the download link, also changing the download variable to the filename, by stripping the base url from the download link.
if (data[“response”][0][“type”] === “success”) {
$(‘#success’).css(‘visibility’, ‘visible’);
let link = data[“response”][0][“download_link”];
link = link.replace(“backend/app/”, “http://badgeyay-api.herokuapp.com/”);
$(‘#badge-link’).attr(“href”, link);
link = link.replace(“static/badges/”, “”);
$(‘#badge-link’).attr(“download”, link);
}

 

  1. Output the success on the page.
<div id=”success” style=”visibility: hidden;”>
<div class=”flash-success”>Your badges have been created successfully.</div>
<div class=”text-center”>
<a id=”badge-link” href=”http://badgeyay-api.herokuapp.com/static/badges/{{msg}}-badges.pdf”
class=”btn btn-success”
download=”{{msg}}-badges.pdf”>Download as
PDF</a>
</div>
</div>

 

  1. Frontend and Backend now are connected to each other.The Server now accepts CORS requests and response is generated after the user requests from Frontend.

 

The Pull Request with the above changes is on this Link

Topics Involved

Working on this issue (Link)  involves following topics :

  • Enabling Flask Server for CORS
  • Request Headers
  • AJAX request for CORS.

References

Badgeyay: Custom Fonts in generation of badges

Badgeyay is an open source project of FOSSASIA. The main idea for this project is to provide an open-source alternative for badge generation process for any event. It can generate badges according to a predefined config or we can also submit our own custom config for the generation of the badges. We can use custom background, text and other things. One thing that is not present is the choice for choosing a custom font for the badge. I have made a contribution for adding this functionality with selection of some common fonts in the code.

Procedure

  1. Add a Button in index.html for the choice of the font and also preview them at the same time. 
    <label>Choose your font</label>
    <ul style=“list-style-type:none”>
     <li>
        <input type=“radio” name=“fontsource” id=“custfont”> Use Custom font
                        </li>
                        <section id=“custom-font” style=“display: none;”>
        <label for=“inputFile”>Select from following fonts</label>
        <div class=“btn-group”>
           <button type=“button” class=“btn btn-default dropdown-toggle” data-toggle=“dropdown” aria-haspopup=“true” aria-expanded=“false”>
              <span class=“placeholder2”>Select a font</span>
              <span class=“glyphicon glyphicon-chevron-down”></span>
           </button>
           <ul class=“dropdown-menu”>
              {% for i in custom_fonts %}
              <li class=“font-options” style=“font-family:'{{i}}'” data-item=“{{i}}”>{{i}}</li>
              {% endfor %}
           </ul>
        </div>
     </section>
     <input type=“hidden” name=“custfont” value=“”>
    </ul>

     

     

  2. Add javascript for the toggle in the check button and CSS for the Font option button.
.$(“.font-options”).click(function () {
  var i = $(this).data(“item”);
  $(“.placeholder2”).text(i);
  $(“input[name=’custfont’]”).val(i);
});

 

.font-options {
border-bottom: 1px solid darkgray;
padding: 9px;
}

 

  1. Font list is passed in the index page.
CUSTOM_FONTS = [‘monospace’, ‘sans-serif’, ‘sans’, ‘Courier 10 Pitch’, ‘Source Code Pro’]

 

render_template(‘index.html’, default_background=default_background, custom_fonts=CUSTOM_FONTS)

 

  1. Config file for font has been created, so that it can be used by different files.
custom_font = request.form[‘custfont’]
# Custom font is selected for the text
if custom_font != :
  json_str = json.dumps({
      ‘font’: custom_font
  })
  f = open(os.path.join(app.config[‘UPLOAD_FOLDER’], ‘fonts.json’), “w+”)
  f.write(json_str)
  f.close()

 

  1. Font preference is taken from the file at the time of generation of the badge (once only for all the badges in a single run).
font_choice = None
if os.path.isfile(os.path.join(UPLOAD_FOLDER, ‘fonts.json’)):
  DATA = json.load(open(os.path.join(UPLOAD_FOLDER, “fonts.json”)))
  font_choice = DATA[‘font’]

 

  1. Changes in the SVG are made according to the preference for the PDF generation. If the user wants a custom font then it updates the svg using the config else not.
content = CONTENT
if font_choice:
  content = content.replace(“font-family:sans-serif”,
                            “font-family:” + font_choice)
  content = content.replace(“inkscape-font-specification:sans-serif”,
                            “inkscape-font-specification:” + font_choice)
  content = content.replace(“font-family:ubuntu”,
                            “font-family:” + font_choice)
  content = content.replace(“inkscape-font-specification:ubuntu”,
                            “inkscape-font-specification:” + font_choice)

 

  1. Finally the Updated SVG is used for Badge Generation with custom fonts embedded.

Resources

Resources utilised for adding this functionality

  • Fonts in SVG – Link
  • Embed fonts in Inkscape SVG – Link
  • Embed fonts in PDF and SVG – Link

 

Make Flask Fast and Reliable – Simple Steps

Flask is a microframework for Python, which is mostly used in web-backend development.There are projects in FOSSASIA that are using flask for development purposes such as Open Event Server, Query Server, Badgeyay. Optimization is indeed one of the most important steps for a successful software product. So, in this post some few off- the-hook tricks will be shown which will make your flask-app more fast and reliable.

Flask-Compress

  1. Flask-Compress is a python package which basically provides de-facto lossless compression  to your Flask application.
  2. Enough with the theory, now let’s understand the coding part:
    1. First install the module

2. Then for a basic setup

3.That’s it! All it takes is just few lines of code to make your flask app optimized .To know more about the module check out flask-compress module.

Requirements Directory

  1. A common practice amongst different FOSSASIA  projects which involves dividing requirements.txt files for development,testing as well as production.
  2. Basically when projects either use TRAVIS CI for testing or are deployed to Cloud Services like Heroku, there are some modules which are not really required at some places.  For example: gunicorn is only required for deployment purposes and not for development.
  3. So how about we have a separate directory wherein different .txt files are created for different purposes.
  4. Below is the image of file directory structure followed for requirements in badgeyay project.

  1. As you can see different .txt files are created for different purposes
    1. dev.txt – for development
    2. prod.txt – for production(i.e. deployment)
    3. test.txt – for testing.

Resources

Building the Meilix Generator with Flask

Meilix Generator is a webapp which is used to trigger the Travis build of Meilix and mail the user the link of the iso. Meilix Generator webapp is based on Flask. This blog shows that how easy is to build a webapp and take the HTML files to render it into the webapp as well as to call and pass various function. Here I used Flask, the Python framework to render the HTML templates and send requests for various purposes (mentioned later in the article) without coding everything from scratch because of import facility of the Flask.

What is Flask?

Flask is a Python micro web framework based on Werkzeug, Jinja 2 template engine. It is used as the backbone of the webapp. It features us with a whole set of Python from which we can easily generate webapp. It is micro as it has no tools and no library itself. It come up with minimum requirements and one who needs can import different library and use it. And I used several import function for Meilix Generator like render_template, send_from_directory, etc.

Implementation (The use case in Meilix Generator)

First of all, the installation process: We will do the installation in a virtual environment. We prefer virtual environment to differentiate the Python working environment since few programs are there which require different Python versions to work.
Install virtual environment 

sudo pip install virtualenv

Now go to the folder (project) and activate it using

. venv/bin/activate

Now install Flask

pip install flask
Creating your project

Now it’s time to create a simple project in the directory.
Let’s use HTML as the frontend. In the folder create styles.css for styling and index.html template for the frontend of the page.We will make one app.py file which would look similar to this: 

from flask import Flask, render_template
app = Flask(__name__)
@app.route('/')
def index():
	"""Index page"""
	return render_template("index.html")
if __name__ == '__main__':
    app.run()

Flask looks for the / (root) path and here the root return the main template (index.html) which is the main function.

Compiling it to view the page:

export FLASK_DEBUG=1 FLASK_APP=app.py
flask run

You will find your page at http://127.0.0.1:5000

More options (how more it can help you)

  • Add more HTML template options and refer it in app.py
  • Easily use Github API  from a different .py file (this file should get import to app.py) to fetch data like: https://api.github.com/users/user_name : It will fetch user name, repos, followers and many more important information.

How I used this idea for FOSSASIA (Meilix Generator)

I used Flask for the backbone of project Meilix Generator. First, I used from function to import various library needed for the project and then made several functions for the same. Let’s understand the concept using few example:

from flask import Flask, render_template
@app.route('/about')
def about():
		#About page
		return render_template("about.html")

or

from flask import Flask, send_from_directory
@app.route('/uploads/<filename>')
def uploaded_file(filename):
		return send_from_directory(app.config['UPLOAD_FOLDER'],filename)

For more details file app.py can be found here of the Meilix Generator repository where we used the above idea.

Important Links and Repositories:

Ticket Ordering or Positioning (back-end)

One of the many feature requests that we got for our open event organizer server or the eventyay website is ticket ordering. The event organizers wanted to show the tickets in a particular order in the website and wanted to control the ordering of the ticket. This was a common request by many and also an important enhancement. There were two main things to deal with when ticket ordering was concerned. Firstly, how do we store the position of the ticket in the set of tickets. Secondly, we needed to give an UI in the event creation/edit wizard to control the order or position of a ticket. In this blog, I will talk about how we store the position of the tickets in the backend and use it to show in our public page of the event.

Continue reading Ticket Ordering or Positioning (back-end)

Multiple Tickets: Back-end

In my previous post I talked about approach for Multiple Ticket feature’s user-interface [Link]. In this post I’ll discuss about Flask back-end used for saving multiple tickets.

HTML Fields Naming

Since the number of Tickets a user creates is unknown to the server, details of tickets were needed to be sent as an array of values. So the server would accept the list of values and iterate over them. To send data as an array the naming had to include brackets. Below are some input fields used in tickets:

<tr>
    <td>
        <input type="hidden" name="tickets[type]">
        <input type="text" name="tickets[name]" class="form-control" placeholder="Ticket Name" required="required" data-uniqueticket="true">
        <div class="help-block with-errors"></div>
    </td>
    <td>
        <input type="number" min="0" name="tickets[price]" class="form-control"  placeholder="$" value="">
    </td>
    <td>
        <input type="number" min="0" name="tickets[quantity]" class="form-control" placeholder="100" value="{{ quantity }}">
    </td>
    <!-- Other fields -->
</tr>

At the server

When the POST request reaches the server, any of the above fields (say tickets[name]) would be available as a list. The Flask Request object includes a form dictionary that contains all the POST parameters sent with the request. This dictionary is an ImmutableMultiDict object, which has a getlist method to get array of elements.

For instance in our case, we can get tickets[name] using:

@expose('/create', methods=('POST', 'GET'))
def create_view(self):
    if request.method == 'POST':
        ticket_names = request.form.getlist('tickets[name]')

    # other stuff

The ticket_names variable would contain the list of all the Ticket names sent with the request. So for example if the user created three tickets at the client-side, the form would possibly look like:

<form method="post">
  <!-- Ticket One -->
  <input type="text" name="tickets[name]" class="form-control" value="Ticket Name One">
  <!-- Ticket Two -->
  <input type="text" name="tickets[name]" class="form-control" value="Ticket Name Two">
  <!-- Ticket Three -->
  <input type="text" name="tickets[name]" class="form-control" value="Ticket Name Three">

</form>

After a successful POST request to the server, ticket_names should contain ['Ticket Name One', 'Ticket Name Two', 'Ticket Name Three'].

Other fields, like tickets[type], tickets[price], etc. can all be extracted from the Request object.

Checkbox Fields

A problem arose when a checkbox field was needed for every ticket. In my case, a “Hide Ticket” option was needed to let the user decide if he wants the ticket to be shown at the public Events page.

Screenshot from 2016-08-13 12:39:29

The problem with checkboxes is that, for a checkbox of a particular name attribute, if it is not selected, POST parameters of the request made by the client will not contain the checkbox input field parameter. So if I define an input field as a checkbox with the following naming convention, and make a POST request to the server, the server will receive blah[] parameter only if the input element had been checked.

<input type="checkbox" name="blah[]" >

This creates a problem for “Hide ticket” checkboxes. For instance, at the client-side the user creates three tickets with the first and last tickets having their checkboxes selected, the server would get an array of two.

<form>
  <!-- Ticket One -->
  <input type="checkbox" name="tickets[hide]" checked>
  <!-- Ticket Two -->
  <input type="checkbox" name="tickets[hide]">
  <!-- Ticket Three -->
  <input type="checkbox" name="tickets[hide]" checked>

</form>
ticket_hide_opts = request.form.getlist('tickets[hide]')

ticket_hide_opts would be an array of length two. And there is no way to tell what ticket had its “Hide ticket” option checked. So for the hide checkbox field I had to define input elements with unique names to extract them at the server.

There is also a hack to overcome the unchecked-checkbox problem. It is by using a hidden field with the same name as the checkbox. You can read about it here: http://www.alexandrejoseph.com/blog/2015-03-03-flask-unchecked-checkbox-value.html.