Observability Amazon SES

SES bounces and complaints are invisible with no event destination

This is the note that makes the other five findable. SES returns a MessageId the moment it accepts a request, and that is the last thing most applications ever learn about the message. Whether it was delivered, bounced, suppressed, or complained about is published as an event — and if no configuration set carries an event destination, those events go nowhere and the information is simply lost.

SES configuration sets Python and Node.js Fixable through the API

The short answer

A MessageId means SES accepted the request. It does not mean anything was delivered. Delivery, bounce, complaint, reject and rendering-failure outcomes are published as events, and only if the send used a configuration set with an event destination attached.

Create one with CreateConfigurationSetEventDestination pointing at CloudWatch, SNS or Kinesis Firehose, then set it as the default for the identity so every send is covered without touching application code.

The problem in plain words

Support says a customer never received their receipt. You check the logs: the send succeeded, here is the message ID. There is nowhere else to look. You cannot tell whether it bounced, whether it was suppressed before it left, whether the recipient marked it as spam, or whether it was delivered and they simply missed it.

Each of those has a different fix, and without events you cannot tell them apart. Teams end up guessing, or asking the customer to check their spam folder, which is the support equivalent of turning it off and on again.

Why it happens

Event publishing is opt-in and off by default. A fresh SES account sends perfectly well with no configuration set at all, so nothing forces the decision at setup time and it is easy to never make it.

The default configuration set is a separate setting again. Creating a configuration set does nothing unless sends actually reference it. You either pass ConfigurationSetName on every call — which means touching every code path — or set it as the identity default, which most people do not know exists.

The gap is invisible while things work. Nobody misses bounce data until the first mystery, and by then the events for that message are long gone. Events are not retroactive; you only get them from the moment the destination exists.

How to fix it

Find out what you have

List the configuration sets and check whether each one actually has a destination attached. An empty configuration set is the common trap — it exists, so it looks configured, and it publishes nothing.

aws sesv2 list-configuration-sets
aws sesv2 get-configuration-set-event-destinations --configuration-set-name default

Create a destination that covers the failures

Subscribe to the event types that mean something went wrong: BOUNCE, COMPLAINT, REJECT, RENDERING_FAILURE, and DELIVERY_DELAY. Add DELIVERY too if you want positive confirmation. CloudWatch is the least effort to start with; SNS is right if you want to act on events in code.

Make it the default for the identity

PutEmailIdentityConfigurationSetAttributes attaches a configuration set to a domain or address so every send from it is covered, including sends from code you have not touched in two years.

Keep one per traffic type

One configuration set for transactional and one for marketing means the reputation numbers are attributable. When the bounce rate moves you can see which stream moved it, which is the difference between a fix and a guess.

How to check it worked

Send a message to the AWS bounce simulator and confirm an event arrives:

aws sesv2 send-email \\
  --from-email-address you@yourdomain.com \\
  --destination ToAddresses=bounce@simulator.amazonses.com \\
  --configuration-set-name transactional \\
  --content 'Simple={Subject={Data=test},Body={Text={Data=test}}}'

Within a minute a Bounce event should appear at your destination. If nothing arrives, the destination is not attached to the configuration set the send actually used.

The full code

The script audits every configuration set for a destination that covers the failure event types, reports which identities have no default configuration set, and can create a CloudWatch destination with the right event types. It reports by default and writes only with --apply.

ses_event_destination_audit.py

"""Audit SES configuration sets for event destinations that cover failures.

A configuration set with no destination publishes nothing, which is the common
trap: it exists, so it looks configured. This reports the gap and can create a
CloudWatch destination covering the event types that mean something went wrong.
"""
import argparse
import logging
import sys

import boto3

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("ses_event_destination_audit")

# The outcomes worth knowing about. DELIVERY is optional; the rest are failures.
REQUIRED_EVENTS = {"BOUNCE", "COMPLAINT", "REJECT", "RENDERING_FAILURE"}


def missing_events(destinations):
    """Pure decision function. Which failure events is nothing listening for?

    An enabled destination is the only kind that counts -- a disabled one is
    indistinguishable from no destination at all, and is easy to miss by eye.
    """
    covered = set()
    for d in destinations:
        if not d.get("Enabled", False):
            continue
        covered |= {e.upper() for e in d.get("MatchingEventTypes", [])}
    return sorted(REQUIRED_EVENTS - covered)


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--region", default="us-east-1")
    ap.add_argument("--create-on", help="configuration set to add a CloudWatch destination to")
    ap.add_argument("--apply", action="store_true")
    args = ap.parse_args()

    ses = boto3.client("sesv2", region_name=args.region)

    names = ses.list_configuration_sets().get("ConfigurationSets", [])
    if not names:
        log.error("no configuration sets exist, so no send can publish any event")
    for name in names:
        dests = ses.get_configuration_set_event_destinations(
            ConfigurationSetName=name).get("EventDestinations", [])
        gaps = missing_events(dests)
        if gaps:
            log.error("%s: nothing is listening for %s", name, ", ".join(gaps))
        else:
            log.info("%s: all failure events covered", name)

    # Identities that do not default to a configuration set send unattributed mail.
    for ident in ses.list_email_identities().get("EmailIdentities", []):
        detail = ses.get_email_identity(EmailIdentity=ident["IdentityName"])
        if not detail.get("ConfigurationSetName"):
            log.warning("%s: no default configuration set; sends publish nothing",
                        ident["IdentityName"])

    if args.create_on:
        params = {
            "ConfigurationSetName": args.create_on,
            "EventDestinationName": "failures-to-cloudwatch",
            "EventDestination": {
                "Enabled": True,
                "MatchingEventTypes": sorted(REQUIRED_EVENTS),
                "CloudWatchDestination": {
                    "DimensionConfigurations": [{
                        "DimensionName": "ses:configuration-set",
                        "DimensionValueSource": "MESSAGE_TAG",
                        "DefaultDimensionValue": args.create_on,
                    }]
                },
            },
        }
        if args.apply:
            ses.create_configuration_set_event_destination(**params)
            log.info("created failures-to-cloudwatch on %s", args.create_on)
        else:
            log.info("WOULD create failures-to-cloudwatch on %s -- pass --apply", args.create_on)
    return 0


if __name__ == "__main__":
    sys.exit(main())

ses-event-destination-audit.mjs

/**
 * Audit SES configuration sets for event destinations that cover failures.
 *
 * A configuration set with no destination publishes nothing, which is the common
 * trap: it exists, so it looks configured.
 */
import {
  SESv2Client,
  ListConfigurationSetsCommand,
  GetConfigurationSetEventDestinationsCommand,
  CreateConfigurationSetEventDestinationCommand,
  ListEmailIdentitiesCommand,
  GetEmailIdentityCommand,
} from '@aws-sdk/client-sesv2';

// The outcomes worth knowing about. DELIVERY is optional; the rest are failures.
const REQUIRED_EVENTS = ['BOUNCE', 'COMPLAINT', 'REJECT', 'RENDERING_FAILURE'];

/**
 * Pure decision function. Which failure events is nothing listening for?
 *
 * An enabled destination is the only kind that counts -- a disabled one is
 * indistinguishable from no destination at all, and is easy to miss by eye.
 */
export function missingEvents(destinations) {
  const covered = new Set();
  for (const d of destinations) {
    if (!d.Enabled) continue;
    for (const e of d.MatchingEventTypes ?? []) covered.add(String(e).toUpperCase());
  }
  return REQUIRED_EVENTS.filter((e) => !covered.has(e));
}

async function main() {
  const region = process.env.AWS_REGION ?? 'us-east-1';
  const apply = process.argv.includes('--apply');
  const createOn = process.argv[process.argv.indexOf('--create-on') + 1];
  const ses = new SESv2Client({ region });

  const { ConfigurationSets = [] } = await ses.send(new ListConfigurationSetsCommand({}));
  if (!ConfigurationSets.length) {
    console.error('no configuration sets exist, so no send can publish any event');
  }
  for (const name of ConfigurationSets) {
    const { EventDestinations = [] } = await ses.send(
      new GetConfigurationSetEventDestinationsCommand({ ConfigurationSetName: name }));
    const gaps = missingEvents(EventDestinations);
    if (gaps.length) console.error(`${name}: nothing is listening for ${gaps.join(', ')}`);
    else console.log(`${name}: all failure events covered`);
  }

  const { EmailIdentities = [] } = await ses.send(new ListEmailIdentitiesCommand({}));
  for (const ident of EmailIdentities) {
    const detail = await ses.send(new GetEmailIdentityCommand({ EmailIdentity: ident.IdentityName }));
    if (!detail.ConfigurationSetName) {
      console.warn(`${ident.IdentityName}: no default configuration set; sends publish nothing`);
    }
  }

  if (createOn && process.argv.includes('--create-on')) {
    const params = {
      ConfigurationSetName: createOn,
      EventDestinationName: 'failures-to-cloudwatch',
      EventDestination: {
        Enabled: true,
        MatchingEventTypes: REQUIRED_EVENTS,
        CloudWatchDestination: {
          DimensionConfigurations: [{
            DimensionName: 'ses:configuration-set',
            DimensionValueSource: 'MESSAGE_TAG',
            DefaultDimensionValue: createOn,
          }],
        },
      },
    };
    if (apply) {
      await ses.send(new CreateConfigurationSetEventDestinationCommand(params));
      console.log(`created failures-to-cloudwatch on ${createOn}`);
    } else {
      console.log(`WOULD create failures-to-cloudwatch on ${createOn} -- pass --apply`);
    }
  }
}

if (import.meta.url === `file://${process.argv[1]}`) main();

Add a test

The check that matters is subtle: a destination that exists but is disabled covers nothing, and reads as configured in the console. The test pins that down.

test_ses_event_destination_audit.py

from ses_event_destination_audit import missing_events


def test_no_destinations_means_everything_is_missing():
    assert set(missing_events([])) == {"BOUNCE", "COMPLAINT", "REJECT", "RENDERING_FAILURE"}


def test_full_coverage_reports_nothing():
    dests = [{"Enabled": True,
              "MatchingEventTypes": ["BOUNCE", "COMPLAINT", "REJECT", "RENDERING_FAILURE"]}]
    assert missing_events(dests) == []


def test_a_disabled_destination_covers_nothing():
    """It exists, so the console makes it look configured. It publishes nothing."""
    dests = [{"Enabled": False,
              "MatchingEventTypes": ["BOUNCE", "COMPLAINT", "REJECT", "RENDERING_FAILURE"]}]
    assert len(missing_events(dests)) == 4


def test_coverage_is_summed_across_destinations():
    dests = [
        {"Enabled": True, "MatchingEventTypes": ["BOUNCE", "COMPLAINT"]},
        {"Enabled": True, "MatchingEventTypes": ["REJECT", "RENDERING_FAILURE"]},
    ]
    assert missing_events(dests) == []


def test_event_names_are_compared_case_insensitively():
    dests = [{"Enabled": True, "MatchingEventTypes": ["bounce", "complaint", "reject",
                                                      "rendering_failure"]}]
    assert missing_events(dests) == []

ses-event-destination-audit.test.mjs

import { test } from 'node:test';
import assert from 'node:assert/strict';
import { missingEvents } from './ses-event-destination-audit.mjs';

const ALL = ['BOUNCE', 'COMPLAINT', 'REJECT', 'RENDERING_FAILURE'];

test('no destinations means everything is missing', () => {
  assert.deepEqual(missingEvents([]), ALL);
});

test('full coverage reports nothing', () => {
  assert.deepEqual(missingEvents([{ Enabled: true, MatchingEventTypes: ALL }]), []);
});

test('a disabled destination covers nothing', () => {
  assert.equal(missingEvents([{ Enabled: false, MatchingEventTypes: ALL }]).length, 4);
});

test('coverage is summed across destinations', () => {
  const dests = [
    { Enabled: true, MatchingEventTypes: ['BOUNCE', 'COMPLAINT'] },
    { Enabled: true, MatchingEventTypes: ['REJECT', 'RENDERING_FAILURE'] },
  ];
  assert.deepEqual(missingEvents(dests), []);
});

FAQ

Does a MessageId mean the email was delivered?

No. It means SES accepted the request. The message can still be suppressed before it leaves, rejected by the receiver, or bounce. Delivery is a separate event you only see if a configuration set with an event destination was used.

I created a configuration set but still see no events. Why?

Two likely reasons. The configuration set may have no event destination attached — it exists but publishes nothing. Or the sends are not referencing it: either pass ConfigurationSetName on the call, or set it as the identity default so every send is covered.

CloudWatch, SNS or Kinesis Firehose?

CloudWatch for metrics and alarms with the least setup. SNS when you want to react in code, for example writing bounces back to your own suppression table. Firehose when you want the raw events in S3 or a warehouse for analysis.

Can I get events for messages I already sent?

No. Events are published as they happen and are not retroactive. You only get data from the moment the destination exists, which is why this is worth doing before you need it.

Do I need more than one configuration set?

One per traffic type is worth it. With transactional and marketing separated, reputation numbers are attributable to a stream, so when the bounce rate moves you can see which one moved it.

Related field notes

Sources

Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.

Stuck on a tricky one?

If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.