Dev blog of Aden Forshaw

The problem:

  • I noticed some users were signing up, then not using their API Key.
  • Someone posted on forum saying they hadn’t received their welcome email.
  • Investigated SendGrid logs
  • Noticed Hotmail/Outlook/ .edu, and some others were not receiving ~ 5% of users
  • Assumed it was either a mail config DCIM or dirty shared IP was being sent from

Screen shot of SendGrid report, showing emails which were not delivered. Domains included were hotmail, outlook.com, wp.pl, post.cz, mail.ru

The Solution

Migrate to AWS SES (Simple Email Service)

Upgrading my SendGrid tier would probably have done it, however as you get 62,000 emails free in AWS, and far more control, I’d been meaning to migrate for a while

Wanted to be sure emails were delivered so hooked up Lambda to be triggered on Send, Delivered or Bounce, and then send that to my internal Slack channel for monitoring.

Integration flow of the Cat API app with AWS services, posting events to Slack via SES, SNS and Lambda.

Setup:

  1. Add Domain & Verify email

Screen shot of dialog box to verify an email in AWS SES

2. Configuration Set & SNS Destination

Screen shot of dialog to setup an SNS destination in the Configuration set of SES

3. Create a Lambda function to be trigger by the SNS event

Screen of setting up a Lambda function to be triggered by SNS

Free NodeJs Code for Lambda to post SES events to Slack - https://gist.github.com/AdenForshaw/8180115d70a41dc16fa535232bf09aba

4. Add your Slack webhook details & channel name to the Code

Swap out the 'XXXX/XXXX/XXXXXXXXXXXXXXXX' with the last section of the Webhook URL.

Screen shot of Slack Integrations dashboard, setting up a Webhook.

**5. Test with verified Email **(must be verified whilst testing otherwise it won’t send)

Either use a test event in Lambda, or directly from your Application.

Once you're happy...

6. Raise a support ticket with AWS to unlock sending email to anyone

Screen shot of AWS support ticket creation, asking for a service limit increase for SES.

7. Go live!

Screen shot of Slack channel with SES updates sent via Lambda

Gotchas (things to be aware of):

There are no logs, only metric graphics. To keep a log of all SES events you need to setup CloudWatch, or a Kinesis Firehose in the Configuration Set. Either that of setup a Lambda to save them to DynamoDB.

SNS / Configuration Set not being used:

  • Make sure you're passing the name of the configuration Set when sending the email, it won't get used by default.
  • In the app pass it in the params object (ConfigurationSetName: 'YOUR-CONFIGS-NAME')
  • OR if using SMTP pass it as the header (X-SES-CONFIGURATION-SET) - I've done this for the discourse forum setup

Keep getting 400 errors when trying to send emails via SES?

  • You need to raise a support ticket to increase the Send limit before your can send emails to anyone outside your verified email list.

You’re not seeing the Slack message?

  • Check you have changed the SLACK_CHANNEL to your own
  • Check the SLACK_URI has been changed to the last 3 sections of your own Webhook URL

Conclusion:

Moving simple transactional emails to SES is definitely a good move. It makes it incredibly easy to monitor, and super simple to integrate events directly into your application to help onboard your users.

However:

  • Making email templates is a lot tougher than the nice editors you get inside of SendGrid.
  • You will have to manage unsubscribe links and actions yourself
  • Getting AWS support to unlock your account to send can take 12-24hrs

Anyway, feel free to send me an email if you get stuck, Aden