Using Cron Scheduling to automatically run background jobs

Cron scheduling is nothing new. It is basically a concept in which the system keeps executing lines of code every few seconds, minutes, or hours. It is required in many large applications which require some automation in their working. I had to use it for two purposes in our Open-Event application.

  • To automatically delete the items in the trash after a period of 30 days.
  • To automatically send after event mails to the speakers and organizers when an event gets completed

1. Delete items in Trash system

So the deleted items get stored in the trash of the admin. However if the items are not deleted or no action is performed on them then they should be deleted automatically if they stay in the trash for more than 30 days. I have used a framework – apscheduler (Advanced Python Scheduler) for this.The code is like this

from apscheduler.schedulers.background import BackgroundScheduler

def empty_trash():
 with app.app_context():
 print 'HELLO'
 events = Event.query.filter_by(in_trash=True)
 users = User.query.filter_by(in_trash=True)
 sessions = Session.query.filter_by(in_trash=True)
 for event in events:
 if datetime.now() - event.trash_date >= timedelta(days=30):
 DataManager.delete_event(event.id)

 for user in users:
 if datetime.now() - user.trash_date >= timedelta(days=30):
 transaction = transaction_class(Event)
 transaction.query.filter_by(user_id=user.id).delete()
 delete_from_db(user, "User deleted permanently")

 for session in sessions:
 if datetime.now() - session.trash_date >= timedelta(days=30):
 delete_from_db(session, "Session deleted permanently")


trash_sched = BackgroundScheduler(timezone=utc)
trash_sched.add_job(empty_trash, 'cron', day_of_week='mon-fri', hour=5,
                    minute=30)
trash_sched.start()

The trash_sched is initialized first. It is given an instance of BackgroundScheduler() meaning it will always run in the background as long as the application server is running.

The second line defines the trigger which is given to the scheduler meaning at what time intervals do we want the scheduler to execute the function.

trash_sched.add_job(empty_trash, 'cron', day_of_week='mon-fri', hour=5, 
                    minute=30)

Here the scheduler adds the function to the job list. The function is empty_trash and the trigger given here is ‘cron’. The following line:

day_of_week='mon-fri', hour=5, minute=30

sets the time at which the sheduler executes the job. The above line means that the job will be executed every day from Monday to Friday at 5:30 AM. Now coming to the function which is to be executed:

def empty_trash():
    with app.app_context():
        events = Event.query.filter_by(in_trash=True)
        users = User.query.filter_by(in_trash=True)
        sessions = Session.query.filter_by(in_trash=True)
        for event in events:
            if datetime.now() - event.trash_date >= timedelta(days=30):
                DataManager.delete_event(event.id)

        for user in users:
            if datetime.now() - user.trash_date >= timedelta(days=30):
                transaction = transaction_class(Event)
                transaction.query.filter_by(user_id=user.id).delete()
                delete_from_db(user, "User deleted permanently")

        for session in sessions:
            if datetime.now() - session.trash_date >= timedelta(days=30):
                delete_from_db(session, "Session deleted permanently")

There are three items in the trash: events, users and sessions. We get all the items in the trash by the query.all() method. Each model: Events, Users and Sessions has a column

trash_date = db.Column(db.DateTime)

The date on which the item is moved to the trash is stored in the trash_date column. And the following line:

 if datetime.now() - event.trash_date >= timedelta(days=30)

checks if the item has been in the trash for more than 30 days. If yes then it is deleted automatically. This function is constantly executed by the apscheduler according to the time settings given by us. Thus we do not need to manually delete the trash after 30 days.

2. Send After Event Mails

This is similar to the trash emptying function.

from apscheduler.schedulers.background import BackgroundScheduler

def send_after_event_mail():
 with app.app_context():
 events = Event.query.all()
 for event in events:
 upcoming_events = DataGetter.get_upcoming_events(event.id)
 organizers = DataGetter.get_user_event_roles_by_role_name(event.id, 'organizer')
 speakers = DataGetter.get_user_event_roles_by_role_name(event.id, 'speaker')
 if datetime.now() > event.end_time:
 for speaker in speakers:
 send_after_event(speaker.user.email, event.id, upcoming_events)
 for organizer in organizers:
 send_after_event(organizer.user.email, event.id, upcoming_events)

#logging.basicConfig()
sched = BackgroundScheduler(timezone=utc)
sched.add_job(send_after_event_mail, 'cron', day_of_week='mon-fri', hour=5, minute=30)
#sched.start()

The scheduler settings are the same as the trash scheduler settings. The function returns all the events and checks whether the event’s end_date has come or not. This check is performed against the present date by the following line:

if datetime.now() > event.end_time

If yes then the speakers and organizers for that event are obtained and after event mails are sent to them automatically.

In this way the whole system is automated. 🙂

Open Event Apk generator

So we made this apk generator currently hosted on a server (http://192.241.232.231) which let’s you generate an android app for your event in 10 minutes out of which the server takes about 8 minutes to build 😛 . So, essentially you just have to spare 2 minutes and just enter 3 things(email, Desired app’s name and Api link). Isn’t this cool?

So how exactly do we do this?

At the backend, we are running a python scripts with some shell scripts where the python script is basically creating directories, interacting with our firebase database to get the data entered by a user. So we made these scripts to first of all to clone the open event android repo, then customise and change the code in the repo according to the parameters entered by the organiser earlier(shown in the image).

Screen Shot 2016-06-14 at 12.13.12 AM
Generator Website

After the code has been changed by the scripts to customise the app according to the event the app will be used for, we move on to the script to build the apk, where we build and sign the apk in release mode using signing configs and release keys from the server because we don’t the organiser to generate keys and store it on the server to avoid the hassle and also the privacy concerns involving the keys. So this is when the apk is generated on the server. Now you have questions like the apk is generated but how do I get it? Do I have to wait on the same page for 10 minutes while the apk is being sent? The answer is no and yes. What I mean by this is that you can wait to download the apk if you want but we’ll anyways send it to your email you specified on the apk generator website. This is the sample Email we got when we were testing the system

Screen Shot 2016-06-14 at 12.08.59 AM.png

So it’s an end to end complete solution from which you can easily get an android app for your event in just 2 minutes. Would love to hear feedback and reviews. Please feel free to contact me @ [email protected] or on social media’s(Twitter, Facebook). Adios!

P.S. : Here is the link to the scripts we’re using.