Field notes / Email Authentication

Repair Email Authentication

Duplicate SPF TXT records on one domain

Somewhere along the way, a domain picked up a second v=spf1 TXT record. Maybe a new mail tool's setup wizard added its own instead of appending to the one already there. Maybe a migration left an old record behind. Either way, mail servers checking SPF now see two records that each claim to be the answer, and the rule for that case is not "pick the best one." It is "fail." Here is why that happens and a script that finds it and merges the two into one.

Python and Node.js Cloudflare DNS API Safe by default (dry run)

A purple envelope — Photo by Tiffany Tertipes on Unsplash

The short answer

A domain has two TXT records that both start with v=spf1. RFC 7208 allows exactly one SPF record per domain, so when a lookup returns two, the check has to return permerror instead of guessing which record is right. This can break SPF for every legitimate sender on the domain, not just the one added last. The fix is to read both records, merge every mechanism from each into a single new v=spf1 line, delete the two old records, and publish only the merged one. Full code, tests, and a dry run guard are below.

The problem in plain words

SPF works by publishing one TXT record at a domain's apex that lists every server allowed to send mail for that domain. A receiving mail server looks up that TXT record, reads the list, and checks whether the message came from one of those servers.

That only works if there is exactly one list to read. When a domain has two separate TXT records that both start with v=spf1, the receiving server does not know which list is the real one, and RFC 7208 says it must not guess. It has to report the result as permerror, a permanent error, and most mail providers treat that as a failure. So instead of one sender being blocked, every sender can be blocked, including the ones both records agree on.

Google Workspace and SendGrid are each in their own valid-looking record. Because there are two records instead of one, the check cannot trust either, and mail from both can fail.

Why it happens

A new mail tool's setup wizard adds its own v=spf1 TXT record automatically, instead of asking you to add its include to the record you already have.
A migration to a new DNS host or registrar copies every TXT record from the old zone into the new one, including an SPF record that was supposed to be replaced, not duplicated.
Two people, or two tools, each add an SPF record for the domain without checking whether one already exists, and the DNS host lets both save with no warning.
An old mail provider's SPF record is left in place after switching providers, sitting next to the new provider's record instead of being deleted.

The key insight

RFC 7208 section 4.5 does not say "use the first record" or "use the strictest record." It says a resolver that gets more than one v=spf1 string for a domain name has an ambiguous record set and must treat the whole check as an error. There is no partial credit. Two correct-looking records are still a broken configuration, and the fix is never to just delete one at random, since that one might be the sender your other tool actually needs.

The fix, as a flow

Read both TXT records with a direct DNS query so you see exactly what is published, not what a cache remembers. Take every mechanism from both records, such as each include, drop the duplicates, and put them all into one new SPF line. Decide the qualifier at the end, the part like -all or ~all, by keeping whichever one was stricter, since a soft fail record should not quietly loosen a hard fail record. Delete both old TXT records and publish the single merged one at the domain's apex.

A domain with exactly one SPF record is left alone. A domain with two or more gets every mechanism folded into a single new record before the old ones are removed.

How to fix it

Confirm the duplicate with a direct DNS query

Query the TXT records straight from a public resolver so a stale local cache cannot hide the problem. A healthy domain returns exactly one string starting with v=spf1. Two or more lines starting with v=spf1 confirms the bug.

Terminal

check-spf.sh

dig +short TXT example.com

# bypass any local caching, ask a public resolver directly
dig +short TXT example.com @1.1.1.1

# a second opinion from a different tool
nslookup -type=TXT example.com

# count how many SPF strings came back, anything above 1 is a bug
dig +short TXT example.com | grep -c 'v=spf1'

Merge both records into one

Do not just delete one record and hope the other covers everyone. Take every mechanism from both, remove duplicates, and end with the stricter qualifier. Two SPF records like the ones below become a single merged record with both includes.

DNS record

records-before-and-after

# BEFORE (bug): two separate v=spf1 records at the apex
example.com.    TXT   "v=spf1 include:_spf.google.com ~all"
example.com.    TXT   "v=spf1 include:sendgrid.net -all"

# AFTER: one merged record, stricter -all qualifier kept
example.com.    TXT   "v=spf1 include:_spf.google.com include:sendgrid.net -all"

Publish the merged record through the Cloudflare API (or dashboard)

In the Cloudflare dashboard this is DNS, Records: delete both existing v=spf1 TXT records, then add one new TXT record at name @ with the merged content and TTL set to Auto. The same three steps work through the API.

Terminal

cloudflare-api.sh

# find both existing v=spf1 TXT records for the domain
curl -s -H "Authorization: Bearer $CF_TOKEN" \
  "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records?type=TXT&name=example.com" \
  | jq '.result[] | select(.content | startswith("v=spf1"))'

# delete each duplicate record by its id
curl -X DELETE \
  -H "Authorization: Bearer $CF_TOKEN" \
  "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/$RECORD_ID"

# then publish the single merged record
curl -X POST \
  -H "Authorization: Bearer $CF_TOKEN" \
  -H "Content-Type: application/json" \
  "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records" \
  -d '{"type":"TXT","name":"@","content":"v=spf1 include:_spf.google.com include:sendgrid.net -all","ttl":1}'

How to check it worked

Re-run the same TXT lookup and confirm exactly one line starts with v=spf1, and that it still contains every include you needed. Then validate the merged record with a syntax checker, and finally send a real test email from each authorized sender to confirm SPF actually passes.

Terminal

verify.sh

# exactly one line should start with v=spf1 now
dig +short TXT example.com | grep spf1

# should print exactly "1"
dig +short TXT example.com | grep -c 'v=spf1'

# validate the merged record with pyspf (pip install pyspf)
python3 -c "import spf; print(spf.check2('1.2.3.4','test@example.com','example.com'))"

A good result from the Python check above is Pass or None, not PermError. As a last check, send a real message through each authorized sender, for example Google Workspace and SendGrid, to a mailbox that shows authentication headers, such as Gmail's "Show original," and confirm spf=pass appears for both.

The full code

Here is the complete checker and repair script in one file for each language. It queries TXT records for the domain, flags a permerror risk when more than one v=spf1 string is found, and, only when you turn dry run off, merges the records into one through the Cloudflare API.

View this code on GitHub Full runnable folder with tests in the dns-fixes repo.

duplicate_spf_records.py

"""Detect duplicate v=spf1 TXT records on a domain, and optionally
repair the zone via Cloudflare by merging them into one record.

Safe by default. Set DRY_RUN=false to let it write.

Env vars:
  DNS_DOMAIN               the domain to check, e.g. "example.com"
  CLOUDFLARE_API_TOKEN     Cloudflare API token (only needed for repair)
  CLOUDFLARE_ZONE_ID       Cloudflare zone id (only needed for repair)
  DRY_RUN                  default "true"; set to "false" to actually write
"""
import os
import logging

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("duplicate_spf_records")

DNS_DOMAIN = os.environ.get("DNS_DOMAIN", "example.com")
CLOUDFLARE_API_TOKEN = os.environ.get("CLOUDFLARE_API_TOKEN", "")
CLOUDFLARE_ZONE_ID = os.environ.get("CLOUDFLARE_ZONE_ID", "")
DRY_RUN = os.environ.get("DRY_RUN", "true").lower() == "true"

CF_API = "https://api.cloudflare.com/client/v4"

ALL_QUALIFIERS = ("-all", "~all", "+all", "all")
QUALIFIER_STRENGTH = {"-all": 3, "~all": 2, "?all": 1, "+all": 0, "all": 0}


def merge_spf_records(spf_strings):
    """Pure decision function. No I/O.

    spf_strings: list of raw TXT record strings, each starting with "v=spf1".

    Returns None if the list is empty, returns the single string unchanged
    if there is only one, and if there are two or more, merges every
    mechanism and modifier from all of them into one new "v=spf1 ..."
    string, de-duplicated and ending with the strictest "all" qualifier
    found across the inputs.
    """
    records = [s.strip() for s in spf_strings if s and s.strip().startswith("v=spf1")]
    if not records:
        return None
    if len(records) == 1:
        return records[0]

    merged = []
    seen = set()
    strongest_qualifier = "?all"

    for record in records:
        tokens = record.split()[1:]  # drop the leading "v=spf1"
        for token in tokens:
            if token in ALL_QUALIFIERS or token == "?all":
                if QUALIFIER_STRENGTH.get(token, 0) > QUALIFIER_STRENGTH.get(strongest_qualifier, 0):
                    strongest_qualifier = token
                continue
            if token not in seen:
                seen.add(token)
                merged.append(token)

    final_qualifier = "-all" if strongest_qualifier in ("-all", "?all") else strongest_qualifier
    return "v=spf1 " + " ".join(merged + [final_qualifier])


def query_spf_records(domain):
    """Return every TXT string on the domain that starts with v=spf1."""
    import dns.resolver

    answers = dns.resolver.resolve(domain, "TXT")
    found = []
    for rdata in answers:
        text = b"".join(rdata.strings).decode("utf-8", errors="replace")
        if text.startswith("v=spf1"):
            found.append(text)
    return found


def list_spf_txt_records(domain):
    """List the id and content of every v=spf1 TXT record via Cloudflare."""
    import requests

    headers = {"Authorization": f"Bearer {CLOUDFLARE_API_TOKEN}"}
    params = {"type": "TXT", "name": domain, "per_page": 100}
    r = requests.get(
        f"{CF_API}/zones/{CLOUDFLARE_ZONE_ID}/dns_records",
        headers=headers, params=params, timeout=30,
    )
    r.raise_for_status()
    return [
        {"id": rec["id"], "content": rec["content"]}
        for rec in r.json()["result"]
        if rec["content"].strip('"').startswith("v=spf1")
    ]


def delete_record(record_id):
    import requests

    headers = {"Authorization": f"Bearer {CLOUDFLARE_API_TOKEN}"}
    url = f"{CF_API}/zones/{CLOUDFLARE_ZONE_ID}/dns_records/{record_id}"
    if DRY_RUN:
        log.info("[dry run] would delete record %s", record_id)
        return
    requests.delete(url, headers=headers, timeout=30).raise_for_status()
    log.info("Deleted record %s", record_id)


def create_txt_record(domain, content):
    import requests

    headers = {"Authorization": f"Bearer {CLOUDFLARE_API_TOKEN}"}
    body = {"type": "TXT", "name": domain, "content": content, "ttl": 1}
    if DRY_RUN:
        log.info("[dry run] would create merged TXT record: %s", content)
        return
    r = requests.post(
        f"{CF_API}/zones/{CLOUDFLARE_ZONE_ID}/dns_records",
        headers=headers, json=body, timeout=30,
    )
    r.raise_for_status()
    log.info("Created merged record: %s", content)


def run():
    records = query_spf_records(DNS_DOMAIN)
    if len(records) <= 1:
        log.info("No duplicate SPF records found for %s (%d found).", DNS_DOMAIN, len(records))
        return

    log.warning("%s has %d v=spf1 TXT records, permerror risk.", DNS_DOMAIN, len(records))
    merged = merge_spf_records(records)
    log.info("Merged record: %s", merged)

    if not (CLOUDFLARE_API_TOKEN and CLOUDFLARE_ZONE_ID):
        log.warning("No Cloudflare credentials set. Not repairing, only reporting.")
        return

    zone_records = list_spf_txt_records(DNS_DOMAIN)
    for rec in zone_records:
        delete_record(rec["id"])
    create_txt_record(DNS_DOMAIN, merged)
    log.info("Done.")


if __name__ == "__main__":
    run()

duplicate-spf-records.js

/**
 * Detect duplicate v=spf1 TXT records on a domain, and optionally
 * repair the zone via Cloudflare by merging them into one record.
 *
 * Safe by default. Set DRY_RUN=false to let it write.
 *
 * Env vars:
 *   DNS_DOMAIN               the domain to check, e.g. "example.com"
 *   CLOUDFLARE_API_TOKEN     Cloudflare API token (only needed for repair)
 *   CLOUDFLARE_ZONE_ID       Cloudflare zone id (only needed for repair)
 *   DRY_RUN                  default "true"; set to "false" to actually write
 */
import { pathToFileURL } from "node:url";

const DNS_DOMAIN = process.env.DNS_DOMAIN || "example.com";
const CLOUDFLARE_API_TOKEN = process.env.CLOUDFLARE_API_TOKEN || "";
const CLOUDFLARE_ZONE_ID = process.env.CLOUDFLARE_ZONE_ID || "";
const DRY_RUN = (process.env.DRY_RUN || "true").toLowerCase() === "true";

const CF_API = "https://api.cloudflare.com/client/v4";

const ALL_QUALIFIERS = new Set(["-all", "~all", "+all", "all", "?all"]);
const QUALIFIER_STRENGTH = { "-all": 3, "~all": 2, "?all": 1, "+all": 0, all: 0 };

export function mergeSpfRecords(spfStrings) {
  // Pure decision function. No I/O.
  //
  // spfStrings: array of raw TXT record strings, each starting with "v=spf1".
  //
  // Returns null if the list is empty, returns the single string unchanged
  // if there is only one, and if there are two or more, merges every
  // mechanism and modifier from all of them into one new "v=spf1 ..."
  // string, de-duplicated and ending with the strictest "all" qualifier
  // found across the inputs.
  const records = (spfStrings || [])
    .map((s) => (s || "").trim())
    .filter((s) => s.startsWith("v=spf1"));

  if (records.length === 0) return null;
  if (records.length === 1) return records[0];

  const merged = [];
  const seen = new Set();
  let strongestQualifier = "?all";

  for (const record of records) {
    const tokens = record.split(/\s+/).slice(1); // drop the leading "v=spf1"
    for (const token of tokens) {
      if (ALL_QUALIFIERS.has(token)) {
        if ((QUALIFIER_STRENGTH[token] || 0) > (QUALIFIER_STRENGTH[strongestQualifier] || 0)) {
          strongestQualifier = token;
        }
        continue;
      }
      if (!seen.has(token)) {
        seen.add(token);
        merged.push(token);
      }
    }
  }

  const finalQualifier = strongestQualifier === "-all" || strongestQualifier === "?all"
    ? "-all"
    : strongestQualifier;
  return "v=spf1 " + [...merged, finalQualifier].join(" ");
}

async function querySpfRecords(domain) {
  const dns = await import("node:dns/promises");
  const records = await dns.resolveTxt(domain);
  return records
    .map((chunks) => chunks.join(""))
    .filter((text) => text.startsWith("v=spf1"));
}

async function listSpfTxtRecords(domain) {
  const headers = { Authorization: `Bearer ${CLOUDFLARE_API_TOKEN}` };
  const params = new URLSearchParams({ type: "TXT", name: domain, per_page: "100" });
  const res = await fetch(
    `${CF_API}/zones/${CLOUDFLARE_ZONE_ID}/dns_records?${params.toString()}`,
    { headers },
  );
  if (!res.ok) throw new Error(`Cloudflare list returned ${res.status}`);
  const body = await res.json();
  return body.result
    .filter((rec) => rec.content.replace(/^"|"$/g, "").startsWith("v=spf1"))
    .map((rec) => ({ id: rec.id, content: rec.content }));
}

async function deleteRecord(recordId) {
  const headers = { Authorization: `Bearer ${CLOUDFLARE_API_TOKEN}` };
  if (DRY_RUN) {
    console.log(`[dry run] would delete record ${recordId}`);
    return;
  }
  const res = await fetch(`${CF_API}/zones/${CLOUDFLARE_ZONE_ID}/dns_records/${recordId}`, {
    method: "DELETE",
    headers,
  });
  if (!res.ok) throw new Error(`Cloudflare delete returned ${res.status}`);
  console.log(`Deleted record ${recordId}`);
}

async function createTxtRecord(domain, content) {
  const headers = {
    Authorization: `Bearer ${CLOUDFLARE_API_TOKEN}`,
    "Content-Type": "application/json",
  };
  if (DRY_RUN) {
    console.log(`[dry run] would create merged TXT record: ${content}`);
    return;
  }
  const res = await fetch(`${CF_API}/zones/${CLOUDFLARE_ZONE_ID}/dns_records`, {
    method: "POST",
    headers,
    body: JSON.stringify({ type: "TXT", name: domain, content, ttl: 1 }),
  });
  if (!res.ok) throw new Error(`Cloudflare create returned ${res.status}`);
  console.log(`Created merged record: ${content}`);
}

async function run() {
  const records = await querySpfRecords(DNS_DOMAIN);
  if (records.length <= 1) {
    console.log(`No duplicate SPF records found for ${DNS_DOMAIN} (${records.length} found).`);
    return;
  }

  console.warn(`${DNS_DOMAIN} has ${records.length} v=spf1 TXT records, permerror risk.`);
  const merged = mergeSpfRecords(records);
  console.log(`Merged record: ${merged}`);

  if (!(CLOUDFLARE_API_TOKEN && CLOUDFLARE_ZONE_ID)) {
    console.warn("No Cloudflare credentials set. Not repairing, only reporting.");
    return;
  }

  const zoneRecords = await listSpfTxtRecords(DNS_DOMAIN);
  for (const rec of zoneRecords) {
    await deleteRecord(rec.id);
  }
  await createTxtRecord(DNS_DOMAIN, merged);
  console.log("Done.");
}

if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
  run().catch((err) => {
    console.error(err);
    process.exit(1);
  });
}

Add a test

The merge rule is the part most worth testing, because it decides exactly what gets published in place of the two broken records. Because merge_spf_records is pure, the test needs no network and no DNS provider. It just feeds in plain strings and checks the merged result.

test_spf_merge.py

from duplicate_spf_records import merge_spf_records


def test_empty_list_returns_none():
    assert merge_spf_records([]) is None


def test_single_record_returned_unchanged():
    record = "v=spf1 include:_spf.google.com ~all"
    assert merge_spf_records([record]) == record


def test_two_different_includes_are_merged():
    records = [
        "v=spf1 include:_spf.google.com ~all",
        "v=spf1 include:sendgrid.net -all",
    ]
    result = merge_spf_records(records)
    assert result.startswith("v=spf1 ")
    assert "include:_spf.google.com" in result
    assert "include:sendgrid.net" in result
    assert result.endswith("-all")


def test_overlapping_includes_are_deduplicated():
    records = [
        "v=spf1 include:_spf.google.com ~all",
        "v=spf1 include:_spf.google.com include:sendgrid.net -all",
    ]
    result = merge_spf_records(records)
    assert result.count("include:_spf.google.com") == 1
    assert "include:sendgrid.net" in result


def test_prefers_stricter_all_qualifier():
    records = [
        "v=spf1 include:_spf.google.com ~all",
        "v=spf1 include:sendgrid.net -all",
    ]
    assert merge_spf_records(records).endswith("-all")
    assert not merge_spf_records(records).endswith("~all")

duplicate-spf-records.test.js

import { test } from "node:test";
import assert from "node:assert/strict";
import { mergeSpfRecords } from "./duplicate-spf-records.js";

test("empty list returns null", () => {
  assert.equal(mergeSpfRecords([]), null);
});

test("single record returned unchanged", () => {
  const record = "v=spf1 include:_spf.google.com ~all";
  assert.equal(mergeSpfRecords([record]), record);
});

test("two different includes are merged", () => {
  const records = [
    "v=spf1 include:_spf.google.com ~all",
    "v=spf1 include:sendgrid.net -all",
  ];
  const result = mergeSpfRecords(records);
  assert.ok(result.startsWith("v=spf1 "));
  assert.ok(result.includes("include:_spf.google.com"));
  assert.ok(result.includes("include:sendgrid.net"));
  assert.ok(result.endsWith("-all"));
});

test("overlapping includes are deduplicated", () => {
  const records = [
    "v=spf1 include:_spf.google.com ~all",
    "v=spf1 include:_spf.google.com include:sendgrid.net -all",
  ];
  const result = mergeSpfRecords(records);
  assert.equal(result.split("include:_spf.google.com").length - 1, 1);
  assert.ok(result.includes("include:sendgrid.net"));
});

test("prefers stricter all qualifier", () => {
  const records = [
    "v=spf1 include:_spf.google.com ~all",
    "v=spf1 include:sendgrid.net -all",
  ];
  const result = mergeSpfRecords(records);
  assert.ok(result.endsWith("-all"));
  assert.ok(!result.endsWith("~all"));
});

Case studies

New mail tool

The marketing tool that added its own SPF line

A team connected a new email marketing platform. Its setup wizard checked for an SPF record, did not find the exact string it expected, and published its own v=spf1 TXT record instead of asking the team to edit the existing one. Within a day, Google Workspace mail from the same domain started landing in spam, even though nothing about the Google Workspace record had changed.

A direct dig +short TXT lookup showed two v=spf1 lines. Merging both includes into one record and deleting the duplicate cleared the permerror, and mail from both sources started passing SPF again the same day.

DNS migration

The old provider's record that rode along in the migration

A domain moved to a new DNS host using a bulk zone import. The import faithfully copied an SPF record from a previous mail provider that had been decommissioned months earlier, right alongside the current, correct record. Nobody noticed until a customer reported that invoices sent from the domain were being flagged as suspicious.

Querying the zone found the leftover record immediately. Since the old provider was no longer sending mail, no includes needed to be kept from it, so the fix was simply to delete the stale record and keep the single current one, confirmed with a syntax checker before calling it done.

What good looks like

After the merge, dig +short TXT example.com returns exactly one line that starts with v=spf1, and it contains every include that either of the old records needed. A syntax checker reports a valid record instead of an ambiguous one, and a test send through each authorized sender shows spf=pass in the delivered message headers. Re-check after connecting any new mail tool, since a new wizard adding its own record is the most common way this comes back.

FAQ

Why does having two SPF records break email for everyone?

RFC 7208 requires exactly one SPF record per domain. When a resolver's TXT lookup returns two or more strings that each start with v=spf1, the record set is ambiguous and the receiving mail server cannot tell which one is authoritative. Section 4.5 of the RFC says this must return permerror, so every message from the domain can fail SPF, no matter which record is the correct one.

Can I just delete the older SPF record and keep the newer one?

Only if the newer record already includes every sender the older one covered. Usually it does not, because the second record was added by a new tool's setup wizard instead of by editing the first one. The safe fix is to merge every include and mechanism from both records into a single new v=spf1 line, then delete both old records and publish only the merged one.

How many DNS lookups can one SPF record use?

RFC 7208 section 4.6.4 caps the mechanisms that cause a DNS lookup, such as include, a, mx, ptr, and exists, at 10 total for one SPF record. When you merge two records into one, count these up and stay at or under 10, or you trade a duplicate-record permerror for a too-many-lookups permerror.

Related field notes

Citations

On the problem:

RFC 7208, Sender Policy Framework (SPF), section 4.5 on record lookup and ambiguity. datatracker.ietf.org/doc/html/rfc7208
Mimecast: multiple SPF records, causes, errors, and fixes. mimecast.com/content/multiple-spf-records
Microsoft Learn: set up SPF to identify valid email sources. learn.microsoft.com

On the solution:

Cloudflare API: DNS Records, List method. developers.cloudflare.com/api
Cloudflare API: DNS Records, Delete method. developers.cloudflare.com/api
DeBounce: multiple SPF records, how to find and fix them. debounce.com/blog/multiple-spf-records

Stuck on a tricky one?

If you have a DNS, domain, or email authentication problem you would rather hand off, this is the kind of work I do. Message me and we can work through it together.

Contact me on LinkedIn

Did this fix your mail delivery?

If this got your email out of spam or cleared up a confusing permerror, you can buy me a coffee. It is the best way to keep these field notes free and growing.

Buy me a coffee on Ko-fi

Back to all DNS and Domains field notes