Reliably and speedily send mass emails via Amazon SES in C#

Amazon provides an awesome and awesomely cheap service to send emails out called Amazon Simple Email Service (Amazon SES).  However it comes with sending volume limits; a sending quota and maximum send rate. This blog post is specifically about the maximum send rate and how you can send emails at, or close to, that maximum send rate.

Your initial sending volume when you sign up is a sending quota of 10,000 emails per 24 hour period, and a max send rate of 5 emails per second. My limit is 100,000 emails per 24 hour period and a max send rate of 28 emails per second.

So if you’re a developer, why utilise Amazon SES instead of a 3rd party emailer who have everything done for you? Because 1 – you are a developer and want to build this yourself for your unique needs, and 2 – you need to send bulk customised emails, ie user details, customised product recommendations parsed from past orders or from user profiles (like Amazon does), etc.

The real reason you probably stumbled on here is HOW exactly could you achieve sending bulk emails, easily, effectively and reliably, especially since it takes 5-7 seconds to send a single email to Amazon SES? The easiest way is obviously a for loop! However sending emails 1 by 1 in a for loop, with a 5-7 second delay for each email, will take you a very long time to send a batch of emails!

The closest solution I have seen on the net is this one here: https://forums.aws.amazon.com/message.jspa?messageID=340864

It uses the techniques that I employ; AsParallel and WithDegreeOfParallelism in Linq, but I will make this simpler and more adaptable to a generic for loop scenario.

Multi-threading and sending emails in parallel without breaking your send quota

I saw a StackOverflow question where someone tried to do this:

protected void btnSendNewsletter_Click(object sender, EventArgs e)
{
Thread t = new System.Threading.Thread(new ThreadStart(SendEmails));
t.Start();
}

Even though I have used this in the past for single emails, it will not suit for sending mass emails. It will clog everything up, and you will definitely be throttled, get exceptions, won’t be able to (easily) log status of sends, etc. Similarly to ThreadPool.QueueUserWorkItem, it won’t work for the same reasons above.

So here’s my solution:

Firstly I’d assume you are building a C# Console application, or a Windows Azure Worker Role. You need to modify the App.Config file so you can control the maximum number of outgoing HTTP connections that you can initiate. Sending emails is an I/O bound operation so your CPU will barely break a sweat. This link is a guide of the recommended values: http://msdn.microsoft.com/en-us/library/ms998549.aspx#scalenetchapt06_topic8 Obviously you will want to increase this depending on your max send rate quota. Mine is set to something ridiculous like 392 on a single quad core server. Here is the App.Config entry you need.

<system.net>
<connectionManagement>
<add address="*" maxconnection="392" />
</connectionManagement>
<mailSettings>
<smtp from="form@company.com" deliveryMethod="Network">
<network host="email-smtp.us-east-1.amazonaws.com" userName="SmtpUsername" password="SmtpPassword" enableSsl="true" port="587" />
</smtp>
</mailSettings>
</system.net>

Also note the mailSettings, you can send Amazon SES email via SMTP using MailMessage in System.Net instead of the AWS SDK.

Now for the fun part!

Here is the code below to customise and send individual emails, in parallel, without getting your account throttled, or ghost threads going missing, dying, unloggable/traceable etc. As you can see, it’s just like the for loop you have always known and loved, running in a multi threaded parallel mode!

class Program
{
static readonly object syncRoot = new object();
private readonly static int maxParallelEmails = 196;

static void Main(string[] args)
{

IList<Model.SendEmailTo> recipients = _emailerService.GetEmailsToSend();
int cnt = 0;
int totalCnt = recipients.Count;


Parallel.ForEach(recipients.AsParallel(), new ParallelOptions { MaxDegreeOfParallelism = maxParallelEmails }, recipient =>
{
// Do any other logic

// Build the email HTML

// Send the email, make sure to log exceptions

// Track email, etc

lock (syncRoot) cnt++;
Console.WriteLine(String.Format("{0}/{1} - Sent newsletter email to: {2}", cnt, totalCnt, recipient.Email));
});
}
}

And there you have it! A for loop that’s trying to send 196 emails at the same time! Wow wow wow, hang on a minute… 196 emails at the same… when my Amazon SES max send rate is 28 emails per second? Well the call to send an email takes around 5-7 seconds, I wrapped that call in a C# Stopwatch to measure it. Since an email can take around 7 seconds to send (could be more could be less), 7 x 28 = 196. So we could have the Parallel ForEach loop trying to send 196 emails at the same, which will take around 7 seconds to complete, therefore keeping us within the send rate. Please note your mileage may vary, you could probably try increasing maxParallelEmails to a higher value. Especially if any other logic such as building customised product recommendations, inserting each email tracking details into a database, etc take up a few milliseconds themselves. Use a Stopwatch again to measure your mass send session, and if your calculations show you ended up sending less then your max send rate and did not receive any throttled exceptions, you could increase this on your next broadcast by a bit.

Network bursts sending email, showing the I/O bound operation while CPU is hardly affected.

Network bursts sending email, showing the I/O bound operation while CPU is hardly affected.

And there you have it! Reliably and speedily send mass emails via Amazon SES in a very easy to do way.

I suggest you also handle bounces and complaints. Here is a great blog post of how to automate that using Amazon Simple Notification Service (Amazon SNS) and Amazon Simple Queue Service (Amazon SQS); http://sesblog.amazon.com/post/TxJE1JNZ6T9JXK/Handling-Bounces-and-Complaints

Happy emailing!

P.S. If I have done something wrong, bad, dumb, or you have a suggestion, please leave a comment so we could all learn.