Designing and optimising new invoice PDFs

The Open Event project has proven to be an excellent event management application with a growing user base. With recent workflow refactors in the order process in open-event-frontend and introduction of event invoices (to be rolled out this month as a work product), the open-event-server’s invoices required a makeover. A ticket buyer is now required to give their billing information if the order is comprised of paid tickets and to accommodate this, and long information addresses, optimisation was required.

Restructuring order invoices

The new order invoices use nested tables concept instead of previously used two-cell tables. The pros of this new design is the accomodation of long-addresses and corresponding changes in billing information display.

{% if order.is_billing_enabled %}
                      <
td style="text-align:center;">
                          <
table>
                              <
tr>
                                  <
td>
                                      <
strong>Company :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.company }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td valign="top">
                                      <
strong>Tax Info :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.tax_business_info }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td valign="top">
                                      <
strong>Address :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.address }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td>
                                      <
strong>City :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.city }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td>
                                      <
strong>State :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.state }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td>
                                      <
strong>Zipcode :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.zipcode }}</strong>
                                  </
td>
                              </
tr>
                              <
tr>
                                  <
td>
                                      <
strong>Country :</strong>
                                  </
td>
                                  <
td>
                                      <
strong>{{ order.country }}</strong>
                                  </
td>
                              </
tr>
                          </
table>
                      </
td>
                      {% endif %}

This made sure that the new orders have enough space to prevent information overflow and still maintain the original structure to give a sense of uniformity in old and new PDFs.

Generating new event invoices

The new event invoices needed an overall change in structure. They will be rolling out on 1st of every month, according to current implementation. This required an overall implementation of new invoices. 

First, the published events are taken in consideration for generation of invoices for a particular user. It has been implemented as a scheduled job accordingly.

events = Event.query.filter_by(deleted_at=None, state='published').all()
      for event in events:
          # calculate net & gross revenues
          user = event.owner
          admin_info = get_settings()
          currency = event.payment_currency
          ticket_fee_object = db.session.query(TicketFees).filter_by(currency=currency).one()
          ticket_fee_percentage = ticket_fee_object.service_fee
          ticket_fee_maximum = ticket_fee_object.maximum_fee
          orders = Order.query.filter_by(event=event).all()
          gross_revenue = event.calc_monthly_revenue()
          ticket_fees = event.tickets_sold * (ticket_fee_percentage / 100)
          if ticket_fees > ticket_fee_maximum:
              ticket_fees = ticket_fee_maximum
          net_revenue = gross_revenue - ticket_fees
          payment_details = {
              'tickets_sold': event.tickets_sold,
              'gross_revenue': gross_revenue,
              'net_revenue': net_revenue,
              'amount_payable': ticket_fees
          }
          # save invoice as pdf
          pdf = create_save_pdf(render_template('pdf/event_invoice.html', orders=orders, user=user,
                                admin_info=admin_info, currency=currency, event=event,
                                ticket_fee_object=ticket_fee_object, payment_details=payment_details,
                                net_revenue=net_revenue), UPLOAD_PATHS['pdf']['event_invoice'],
                                dir_path='/static/uploads/pdf/event_invoices/', identifier=event.identifier)
          # save event_invoice info to DB

          event_invoice = EventInvoice(amount=net_revenue, invoice_pdf_url=pdf, event_id=event.id)
          save_to_db(event_invoice)

This function also required one minor modification. The function for calculating monthly revenue had to be updated as to dodge certain unseen bugs related to non completed order amount calculations hence restructuring the function as follows.

def calc_monthly_revenue(self):
      """Returns revenue of current month. Invoice sent every 1st of the month for the previous month"""
      previous_month = datetime.now().month - 1
      orders = Order.query.filter_by(event_id=self.id, status='completed').all()


      monthly_revenue = sum([o.amount for o in orders if o.completed_at and o.completed_at.month == previous_month])
      return monthly_revenue

This enabled the system to finally serve event invoice PDFs. One of whose examples are given above, With this, the open-event-server is finally able to serve event invoices accordingly which can be paid via PayPal to the Eventyay account.

Resources

Related Work and Code Repository

Continue ReadingDesigning and optimising new invoice PDFs

Open Event Server – Export Speakers as PDF File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he/she makes his event public, others can view it and buy tickets if interested.

The organizer can see all the speakers in a very detailed view in the event management dashboard. He can see the statuses of all the speakers. The possible statuses are pending, accepted, and rejected. He/she can take actions such as editing the speakers.

If the organizer wants to download the list of all the speakers as a PDF file, he or she can do it very easily by simply clicking on the Export As PDF button in the top right-hand corner.

Let us see how this is done on the server.

Server side – generating the Speakers PDF file

Here we will be using the pisa package which is used to convert from HTML to PDF. It is a html2pdf converter which uses ReportLab Toolkit, the HTML5lib and pyPdf. It supports HTML5 and CSS 2.1 (and some of CSS 3). It is completely written in pure Python so it is platform independent.

from xhtml2pdf import pisa<

We have a utility method create_save_pdf which creates and saves PDFs from HTML. It takes the following arguments:

  • pdf_data – This contains the HTML template which has to be converted to PDF.
  • key – This contains the file name
  • dir_path – This contains the directory

It returns the newly formed PDF file. The code is as follows:

def create_save_pdf(pdf_data, key, dir_path='/static/uploads/pdf/temp/'):
   filedir = current_app.config.get('BASE_DIR') + dir_path

   if not os.path.isdir(filedir):
       os.makedirs(filedir)

   filename = get_file_name() + '.pdf'
   dest = filedir + filename

   file = open(dest, "wb")
   pisa.CreatePDF(io.BytesIO(pdf_data.encode('utf-8')), file)
   file.close()

   uploaded_file = UploadedFile(dest, filename)
   upload_path = key.format(identifier=get_file_name())
   new_file = upload(uploaded_file, upload_path)
   # Removing old file created
   os.remove(dest)

   return new_file

The HTML file is formed using the render_template method of flask. This method takes the HTML template and its required variables as the arguments. In our case, we pass in ‘pdf/speakers_pdf.html’(template) and speakers. Here, speakers is the list of speakers to be included in the PDF file. In the template, we loop through each item of speakers. We print his name, email, list of its sessions, mobile, a short biography, organization, and position. All these fields form a row in the table. Hence, each speaker is a row in our PDF file.

The various columns are as follows:

<thead>
<tr>
   <th>
       {{ ("Name") }}
   </th>
   <th>
       {{ ("Email") }}
   </th>
   <th>
       {{ ("Sessions") }}
   </th>
   <th>
       {{ ("Mobile") }}
   </th>
   <th>
       {{ ("Short Biography") }}
   </th>
   <th>
       {{ ("Organisation") }}
   </th>
   <th>
       {{ ("Position") }}
   </th>
</tr>
</thead>

A snippet of the code which handles iterating over the speakers’ list and forming a row is as follows:

{% for speaker in speakers %}
   <tr class="padded" style="text-align:center; margin-top: 5px">
       <td>
           {% if speaker.name %}
               {{ speaker.name }}
           {% else %}
               {{ "-" }}
           {% endif %}
       </td>
       <td>
           {% if speaker.email %}
               {{ speaker.email }}
           {% else %}
               {{ "-" }}
           {% endif %}
       </td>
       <td>
           {% if speaker.sessions %}
               {% for session in speaker.sessions %}
                   {{ session.name }}<br>
               {% endfor %}
           {% else %}
               {{ "-" }}
           {% endif %}
       </td>
      …. So on
   </tr>
{% endfor %}

The full template can be found here.

Obtaining the Speakers PDF file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/speakers/pdf

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the speakers of the event as a PDF file. It returns the URL of the task to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}

The user can go to the above-returned URL and check the status of his/her Celery task. If the task completed successfully he/she will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the above-mentioned URL.

Resources

Continue ReadingOpen Event Server – Export Speakers as PDF File

Open Event Server – Export Sessions as PDF File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he/she makes his event public, others can view it and buy tickets if interested.

The organizer can see all the sessions in a very detailed view in the event management dashboard. He can see the statuses of all the sessions. The possible statuses are pending, accepted, confirmed and rejected. He/she can take actions such as accepting/rejecting the sessions.

If the organizer wants to download the list of all the sessions as a PDF file, he or she can do it very easily by simply clicking on the Export As PDF button in the top right-hand corner.

Let us see how this is done on the server.

Server side – generating the Sessions PDF file

Here we will be using the pisa package which is used to convert from HTML to PDF. It is a html2pdf converter which uses ReportLab Toolkit, the HTML5lib and pyPdf. It supports HTML5 and CSS 2.1 (and some of CSS 3). It is completely written in pure Python so it is platform independent.

from xhtml2pdf import pisa

We have a utility method create_save_pdf which creates and saves PDFs from HTML. It takes the following arguments:

  • pdf_data – This contains the HTML template which has to be converted to PDF.
  • key – This contains the file name
  • dir_path – This contains the directory

It returns the newly formed PDF file. The code is as follows:

def create_save_pdf(pdf_data, key, dir_path='/static/uploads/pdf/temp/'):
   filedir = current_app.config.get('BASE_DIR') + dir_path

   if not os.path.isdir(filedir):
       os.makedirs(filedir)

   filename = get_file_name() + '.pdf'
   dest = filedir + filename

   file = open(dest, "wb")
   pisa.CreatePDF(io.BytesIO(pdf_data.encode('utf-8')), file)
   file.close()

   uploaded_file = UploadedFile(dest, filename)
   upload_path = key.format(identifier=get_file_name())
   new_file = upload(uploaded_file, upload_path)
   # Removing old file created
   os.remove(dest)

   return new_file

The HTML file is formed using the render_template method of flask. This method takes the HTML template and its required variables as the arguments. In our case, we pass in ‘pdf/sessions_pdf.html’(template) and sessions. Here, sessions is the list of sessions to be included in the PDF file. In the template, we loop through each item of sessions and check if it is deleted or not. If it not deleted then we print its title, state, list of its speakers, track, created at and has an email been sent or not. All these fields form a row in the table. Hence, each session is a row in our PDF file.

The various columns are as follows:

<thead>
<tr>
   <th>
       {{ ("Title") }}
   </th>
   <th>
       {{ ("State") }}
   </th>
   <th>
       {{ ("Speakers") }}
   </th>
   <th>
       {{ ("Track") }}
   </th>
   <th>
       {{ ("Created At") }}
   </th>
   <th>
       {{ ("Email Sent") }}
   </th>
</tr>
</thead>

A snippet of the code which handles iterating over the sessions list and forming a row is as follows:

{% for session in sessions %}
   {% if not session.deleted_at %}
       <tr class="padded" style="text-align:center; margin-top: 5px">
           <td>
               {% if session.title %}
                   {{ session.title }}
               {% else %}
                   {{ "-" }}
               {% endif %}
           </td>
           <td>
               {% if session.state %}
                   {{ session.state }}
               {% else %}
                   {{ "-" }}
               {% endif %}
           </td>
           <td>
               {% if session.speakers %}
                   {% for speaker in session.speakers %}
                       {{ speaker.name }}<br>
                   {% endfor %}
               {% else %}
                   {{ "-" }}
               {% endif %}
           </td>
          ….. And so on
       </tr>
   {% endif %}
{% endfor %}

The full template can be found here.

Obtaining the Sessions PDF file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/sessions/pdf

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the sessions of the event as a PDF file. It returns the URL of the task to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}

The user can go to the above-returned URL and check the status of his/her Celery task. If the task completed successfully he/she will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the above-mentioned URL.

Resources

Continue ReadingOpen Event Server – Export Sessions as PDF File

Generating Ticket PDFs in Open Event API Server

In the ordering system of Open Event API Server, there is a requirement to send email notifications to the attendees. These attendees receive the URL of the pdf of the generated ticket. On creating the order, first the pdfs are generated and stored in the preferred storage location and then these are sent to the users through the email.

Generating PDF is a simple process, using xhtml2pdf we can generate PDFs from the html. The generated pdf is then passed to storage helpers to store it in the desired location and pdf-url is updated in the attendees record.

Sample PDF

PDF Template

The templates are written in HTML which is then converted using the module xhtml2pdf.
To store the templates a new directory was created at  app/templates where all HTML files are stored. Now, The template directory needs to be updated at flask initializing app so that template engine can pick the templates from there. So in app/__init__.py we updated flask initialization with

template_dir = os.path.dirname(__file__) + "/templates"

app = Flask(__name__, static_folder=static_dir, template_folder=template_dir)

This allows the template engine to pick the templates files from this template directory.

Generating PDFs

Generating PDF is done by rendering the html template first. This html content is then parsed into the pdf

file = open(dest, "wb")

pisa.CreatePDF(cStringIO.StringIO(pdf_data.encode('utf-8')), file)

file.close()

The generated pdf is stored in the temporary location and then passed to storage helper to upload it.

uploaded_file = UploadedFile(dest, filename)

upload_path = UPLOAD_PATHS['pdf']['ticket_attendee'].format(identifier=get_file_name())

new_file = upload(uploaded_file, upload_path)

This generated pdf path is returned here

Rendering HTML and storing PDF

for holder in order.ticket_holders:

  if holder.id != current_user.id:

      pdf = create_save_pdf(render_template('/pdf/ticket_attendee.html', order=order, holder=holder))

  else:

      pdf = create_save_pdf(render_template('/pdf/ticket_purchaser.html', order=order))

  holder.pdf_url = pdf

  save_to_db(holder)

The html is rendered using flask template engine and passed to create_save_pdf and link is updated on the attendee record.

Sending PDF on email

These pdfs are sent as a link to the email after creating the order. Thus a ticket is sent to each attendee and a summarized order details with attendees to the purchased.

send_email(

  to=holder.email,

  action=TICKET_PURCHASED_ATTENDEE,

  subject=MAILS[TICKET_PURCHASED_ATTENDEE]['subject'].format(

      event_name=order.event.name,

      invoice_id=order.invoice_number

  ),

  html= MAILS[TICKET_PURCHASED_ATTENDEE]['message'].format(

      pdf_url=holder.pdf_url,

      event_name=order.event.name

  )

)

References

  1. Readme – xhtml2pdf
    https://github.com/xhtml2pdf/xhtml2pdf/blob/master/README.rst
  2. Using xhtml2pdf and create pdfs
    https://micropyramid.com/blog/generating-pdf-files-in-python-using-xhtml2pdf/

 

Continue ReadingGenerating Ticket PDFs in Open Event API Server

Open Event Server Ticket PDF: Where and Where-not to use static frame in xhtml2pdf

One among the very important features of Open Event Server project is the tickets sales feature, where a user can buy a number of different tickets for a number of people after which he is given a link to download the ticket pdf. However, an issue concerning our Open Event Server project was that if a buyer bought different tickets at a time with different individual ticket holders, all tickets contained the same name, type and QR-Code, which in no way was acceptable since tickets and holders were different.

We use the xhtml2pdf facility in order to convert an html to pdf. An xhtml2pdf facility helps us in converting HTML contents into PDF without the use of browser ‘print’ facility. In order to do this, we use the help of pages and frames, pages being the page of the PDF document, while a frame being that part of area within the page where the contents get stored.

What is a PDF and HTML?

The basic understanding of a PDF and an HTML is that, a PDF or Portable Document Format has layout such that it is measured in terms of specific width and height. However, for an HTML or Hyper Text Markup Language, we do no not have those specific widths and heights. Rather an HTML’s width depends on a person’s device of view and height can be infinitely long as desired.

In terms of xhtml2pdf relation with pages and frames layout, we can identify with this diagram:

+-page———————+
|                                    |
|  +-content_frame-+  |
|  |                          |    |
|  |                          |    |
|  |                          |    |
|  |                          |    |
|  +———————+   |
|                                    |
+—————————-+


The stated issue was due to static frame in xhtml2pdf.

What is that you ask?

Static frames vs Content frames
xhtml2pdf uses the concept of Static Frames to define content that remains the same across different pages (like headers and footers), and uses Content Frames to position the to-be-converted HTML content.

Static Frames are defined through use of the @frame property -pdf-frame-content. Regular HTML content will not flow through Static Frames.

Content Frames are @frame objects without this property defined. Regular HTML content will flow through Content Frames.(xhtml2pdf documentation)

So, the basic idea of the use of  -pdf-frame-content  was to make the contents of the pdf static i.e. without having to continuously change alignments of the page. It made the whole pdf stay static.

This actually caused the whole pdf content to stay constant, event while the loop was going over different name, QR-code and ticket-name values. That means the first loop content stayed over even with changes in values.

Just a simple fix of not making the frame static was the solution.

Here is a few insight of what we had before that was causing the issue:

 <style>
        @page {
            size: a4 portrait;
            background-image: url('{{ base_dir }}/static/data/ticket-trans-
                                  notext.png');
            margin: 0;

            @frame col1_frame {             /* Content frame 1 */
                left: 80pt;  top: 30pt;
                height: 250pt; width: 300pt;
                -pdf-frame-content: main_content;
            }

            @frame qrcode {
                left: 455pt;  top: 50pt;
                width: 100pt; height: 120pt;
                -pdf-frame-content: qr_code;
            }
            @frame number_frame {
                 left: 455pt;  top: 25pt;
                width: 100pt; height: 30pt;
                -pdf-frame-content: number_content;
            }


             @frame col2_frame {
                left: 440pt; top: 170pt;
                -pdf-frame-content: personal_content;
            }
        }


Here,
main_content, qrcode, personal_content and number_content? are simply the class you want to make static.
The following was done to fix it:

 <style>
        @page {
            size: a4 portrait;
            background-image: url('{{ base_dir }}/static/data/ticket-trans-
                                  notext.png');
            margin: 0;

            @frame col1_frame {             /* Content frame 1 */
                left: 80pt;  top: 30pt;
                height: 250pt; width: 300pt;
            }

            @frame qrcode {
                left: 455pt;  top: 50pt;
                width: 100pt; height: 120pt;
            }
            @frame number_frame {
                 left: 455pt;  top: 25pt;
                width: 100pt; height: 30pt;
                -pdf-frame-content: number_content;
            }


             @frame col2_frame {
                left: 440pt; top: 170pt;
            }
        }



But the main question was, ‘where and where-not to use static frame’.
So the simple answer to that is, when you want the contents to stay exactly the same over the different pages of the pdf (for example the company logo or the administrator signature, etc), then you can happily get on with -pdf-frame-content. i.e:

          -pdf-frame-content: <class_name>


If not, or you need to loop over different values, then use Content frames.

Also, for more detailed understanding of PDF and HTML pages and frames, we have this good documentation written about it here: https://github.com/xhtml2pdf/xhtml2pdf/blob/master/doc/source/format_html.rst#static-frames-vs-content-frames

Continue ReadingOpen Event Server Ticket PDF: Where and Where-not to use static frame in xhtml2pdf