Creating SMTP as a Fallback Function to Ensure Emails Work
This blog post explains the solution to scenarios pertaining to failure/ unavailability of Sendgrid service when an attempt is made to send emails in Eventyay. Eventyay is the outstanding Open Event management solution using standardized event formats developed at FOSSASIA. Currently, the Open Event Project utilizes 2 protocols namely, SendGrid and SMTP. Previously, they could be configured only be configured individually. If either protocol failed, there was no provision for a backup and the task to send the email would fail. Therefore, there was a necessity to develop a feature where the SMTP protocol, which is usually more reliable than 3rd party services, could act as a backup when sendgrid server is unavailable or the allotted quota has been exceeded. def check_smtp_config(smtp_encryption): """ Checks config of SMTP """ config = { 'host': get_settings()['smtp_host'], 'username': get_settings()['smtp_username'], 'password': get_settings()['smtp_password'], 'encryption': smtp_encryption, 'port': get_settings()['smtp_port'], } for field in config: if field is None: return False return True Function to check if SMTP has been properly configured The main principle which was followed to implement this feature was to prevent sending emails when SMTP is not configured. Ergo, a function was implemented to check if the host, username, password, encryption and port was present in the model before proceeding. If this was configured properly, we move on to determining the protocol which was enabled. For this, we have 2 separate celery tasks, one for SMTP and the other for Sendgrid. @celery.task(name='send.email.post.sendgrid') def send_email_task_sendgrid(payload, headers, smtp_config): try: message = Mail(from_email=From(payload['from'], payload['fromname']), to_emails=payload['to'], subject=payload['subject'], html_content=payload["html"]) if payload['attachments'] is not None: for attachment in payload['attachments']: with open(attachment, 'rb') as f: file_data = f.read() f.close() encoded = base64.b64encode(file_data).decode() attachment = Attachment() attachment.file_content = FileContent(encoded) attachment.file_type = FileType('application/pdf') attachment.file_name = FileName(payload['to']) attachment.disposition = Disposition('attachment') message.add_attachment(attachment) sendgrid_client = SendGridAPIClient(get_settings()['sendgrid_key']) logging.info('Sending an email regarding {} on behalf of {}'.format(payload["subject"], payload["from"])) sendgrid_client.send(message) logging.info('Email sent successfully') except urllib.error.HTTPError as e: if e.code == 429: logging.warning("Sendgrid quota has exceeded") send_email_task_smtp.delay(payload=payload, headers=None, smtp_config=smtp_config) elif e.code == 554: empty_attachments_send(sendgrid_client, message) else: logging.exception("The following error has occurred with sendgrid-{}".format(str(e))) @celery.task(name='send.email.post.smtp') def send_email_task_smtp(payload, smtp_config, headers=None): mailer_config = { 'transport': { 'use': 'smtp', 'host': smtp_config['host'], 'username': smtp_config['username'], 'password': smtp_config['password'], 'tls': smtp_config['encryption'], 'port': smtp_config['port'] } } try: mailer = Mailer(mailer_config) mailer.start() message = Message(author=payload['from'], to=payload['to']) message.subject = payload['subject'] message.plain = strip_tags(payload['html']) message.rich = payload['html'] if payload['attachments'] is not None: for attachment in payload['attachments']: message.attach(name=attachment) mailer.send(message) logging.info('Message sent via SMTP') except urllib.error.HTTPError as e: if e.code == 554: empty_attachments_send(mailer, message) mailer.stop() Falling back to SMTP when the system has exceeded the sendgrid quota Consider the function associated with the sendgrid task. The logic which sends the emails along with the payload is present in a try/catch block. When an exception occurs while attempting to send the email, it is caught via the requests library and checks for the HTTP code. If the code is determined as 429, this implies that there were TOO_MANY_REQUESTS going through or otherwise in sendgrid lingo, it means that you’ve exceeded your quota. In this case, we will not stop sending the email, rather, we would alternate…
