How a cleaned image gets published

Where the finished set goes

By the time publishing runs, the work is done: a clean cutout passed the gate and the variant step produced the full set. All that’s left is to put the images somewhere the owner can actually reach them and to say so. The publish function writes each variant back to the same S3 bucket under a per-channel prefix — the 1600×1600 web image to out/web/<job_id>.png, the 2,000-pixel marketplace square to out/market/<job_id>.jpg, the social square to out/social/<job_id>.jpg. The layout is deliberately predictable: anyone, or anything, that wants the marketplace images for a batch knows to look under one prefix, and the filenames trace straight back to the job that made them.

Because the watched-folder lane runs in both directions, the same prefixes can be mirrored back into a Drive folder, so the finished images land next to where the owner dropped the original — no logging into a console, no downloading from a bucket. The system produces files and puts them where they’re useful; it never logs into the store or the marketplace and never publishes a listing. What goes live is always the owner’s call.

The manifest

Alongside the images, publish writes a small manifest.json for the job: the source filename, the job id, the cutout’s confidence score, and one entry per output — its channel, its S3 key, its pixel dimensions, its format, and whether it was watermarked. The manifest is what turns a pile of files into a record. It lets the owner see at a glance that a product really did get all three variants and not two; it lets a re-run detect what already exists; and it’s the thing you’d read in six months to answer “which images did we generate for this product, and from which photo?” The job row in DynamoDB is moved to status done at the same moment, so the manifest on S3 and the status in the table always agree.

One notification per job

The owner gets exactly one message per photo, and it says one of two things. On success: “Sea Salt candle: 3 images ready,” with links to the three variants. On a held cutout: “Glass vase: needs review,” with the original and the attempted cutout attached so the problem is obvious at a glance. The channel is whatever suits — an email through SES, or a push to an SNS topic the owner’s phone or a team channel subscribes to. What matters is that it’s one message with a clear outcome, not a stream of per-stage chatter. The owner doesn’t need to know the cutout ran or the variants were resized; they need to know the set is ready, or that one needs a second look.

Why nothing overwrites the original

Two rules keep the output safe to trust. First, the original is sacred: the photo that landed in inbox/ is the source of truth and is never modified. Every working file and every variant is a new object under its own key, so a re-run, a config change, or a bug can never corrupt the one thing you can’t regenerate — the photo itself. Second, the bucket is versioned, so even an output that does get rewritten — you tweak the watermark and re-run a product — keeps its previous version rather than vanishing. If a new run produces something worse, the old version is still there to roll back to.

The last guarantee is about the things that didn’t finish. A cutout sent to review is a job left deliberately incomplete, and the easiest way for it to go wrong is to be forgotten — the owner misses the notification, gets busy, and the half-done product quietly never makes it online. So an EventBridge daily sweep walks the job table for anything stuck in needs review past a day and re-notifies, the same way the other systems in this series re-surface anything left hanging. The pipeline is judged not only by the sets it ships but by its refusal to let a held one disappear.

All posts