I Built 20 Tools for AI Image Generation. The Hardest Part Had Nothing to Do With AI.

cascade-img is an open source pipeline that lets an LLM agent drive Midjourney on its own — compose prompts, generate images, inspect the results with vision, crop the winners, remove backgrounds, and log what worked. Twenty MCP tools. A complete card game where every visual asset came through this system.

Almost none of the hard engineering was about AI. It was about making Discord behave like an API. And the single most important tool in the whole pipeline is a text file.

The grind that started it

I needed game pieces for a matching game — icons, characters, UI elements, all in a consistent flat-design style. Even making ten of them is tedious. Making a full set turns it into a production line: compose a prompt, paste it into Discord, wait, inspect the 2×2 grid, pick a quadrant, click upscale, wait again, download, crop, remove the background, rename. Repeat for the next piece.

After a while I started timing myself. The creative part — deciding what style, what mood, which result to keep — was maybe 10% of the elapsed time. The other 90% was the production line. I was the bottleneck, and I wasn't doing the interesting work. What I actually wanted was to direct — talk to the AI about what I needed, evaluate its output, course-correct — not think about the prompt syntax or babysit the download-crop-rename cycle.

So: make the computer run the production line. What I didn't realize was that the production line is where all the engineering problems actually are.

Midjourney has no API

As of mid-2026, Midjourney has no public API. Every open source tool that drives it programmatically does so through a Discord user account — sending slash commands as if a human typed them, watching for response messages, clicking buttons. midjourney-proxy (5.3k stars, Java), midjourney-api (1.9k stars, TypeScript) — same mechanism. There isn't a cleaner path.

Both Discord and Midjourney's ToS prohibit this. The architecture is pluggable — Flux, DALL-E, and Imagen are on the roadmap behind the same interface — but for v0.1, the Discord bridge is it.

"Talk to Discord programmatically" sounds solved. discord.py exists. Slash commands exist. Message events exist. How hard could it be?

Routing: the problem nobody warns you about

I fire an /imagine command. Midjourney processes it. Midjourney posts the result back as a Discord message in the same channel. I need to match that result to the job I submitted.

Obvious answer: match on the prompt text. Problem: I'm generating fifty sprites in the same style. Many prompts share long substrings. Midjourney truncates prompts in its response messages. Two concurrent prompts that start with "pixel-art sprite of" are indistinguishable from the echoed text alone.

The bridge injects a per-job routing token into every prompt as a Midjourney negative prompt flag: --no cscidnocollide{token}. Midjourney echoes the full --no clause back in its response. The grid and upscale matchers key on that token substring, not on the prompt text. A user who supplies their own --no negatives gets them merged into the single clause alongside the routing token.

It's ugly. But matching on recency or prompt similarity breaks under concurrent load, and breaking means billing the wrong render to the wrong job.

The double-bill problem

Midjourney charges GPU time per generation — 0.8 to 1.3 GPU-minutes depending on resolution. Submit the same prompt twice, that's real money wasted and two grids to sort out.

What happens when a Discord interaction POST times out? Did Midjourney receive it? If I retry and the original went through, double-bill. If I don't retry and the original was lost, the job hangs forever.

The bridge handles this with a state called SUBMITTED_UNCONFIRMED. When the interaction POST exceeds the 35-second timeout, the job transitions to this state and returns HTTP 202 with a note: "poll /wait or /status — do not retry, MJ may have processed the original." The job stays claimable in the grid-matching path. If the grid arrives, it resolves normally. If it doesn't, /wait eventually returns the timeout.

Later I added idempotency keys to imagine. Reusing the key on a retry replays the existing job instead of submitting a new one. Fresh roll, fresh key. You build this after you've been burned, not before.

The grid-claim reservation

Discord delivers a grid result as a message followed by several edits, dispatched concurrently on a thread pool. Two threads can race to process the same grid — both see it arrive, both try to download and upscale. The second one is a wasted download at best and a double upscale bill at worst.

A thread claims grid_path under a lock before any download. A concurrent edit that finds it already reserved returns early. Same pattern as a database row lock, applied to a Discord message handler. You think about this after the first time you see two identical upscale charges on the same grid.

Making buttons work without clicking

After you upscale one of the four quadrants, the upscaled image carries a set of buttons: vary (subtle/strong), zoom out, pan in four directions, re-upscale, animate. A human clicks these in Discord. The bridge needs to press them programmatically.

First thought: reconstruct the button's custom_id from its type and the job UUID. Doesn't work. Midjourney's button IDs contain UUIDs that change per render, and the format isn't documented.

So when an upscale lands, the bridge records the Discord message ID. When the caller requests an action, it fetches that message, reads the target button's live custom_id off the component payload, and presses it via the same interaction primitive used for U1–U4. The UUID-bearing ID is never reconstructed — a marker substring only locates the button. A missing button returns BUTTON_NOT_FOUND rather than a wrong press.

Brittle in theory — Midjourney could change their button layout. Stable in practice for months, and matching the live payload is safer than guessing the format.

The reconnect loop

A long-running daemon holding a Discord WebSocket connection will get disconnected. WebSockets over real networks just do this. The question is what happens next.

Exponential backoff: 2 seconds, 4, 8, 16, 32, capped at 60. Transient disconnects emit DISCORD_DISCONNECTED, then DISCORD_RECONNECTING, and retry until reconnection succeeds. An auth failure — a revoked token — emits DISCORD_RECONNECT_FAILED(reason="auth") and stops the loop without burning Discord's rate limit on a dead credential.

The detail that took me longest to get right: on_disconnect clears the ready flag. Without this, a dropped gateway leaves /imagine racing the reconnect window — it fires into a dead connection and fails with an opaque error. With the flag cleared, /imagine returns 503 DISCORD_NOT_READY until the gateway is back. The caller knows what happened and when to retry.

Two stores, often confused

cascade-img keeps two separate persistent records, and I mixed them up constantly during development.

The job store is the daemon's memory. Write-through SQLite sidecar to the bridge's in-memory job table. When the daemon restarts, it rehydrates non-terminal jobs from SQLite so a grid that arrives during the restart window still matches its job. Pre-grid jobs — things Midjourney hadn't started processing yet — get failed with RESUBMIT_REQUIRED because you can't confirm whether MJ accepted them across the gap. Losing a job beats double-billing one.

The prompt log is the agent's memory. Append-only JSONL — every generation attempt recorded: the prompt, the outputs, what the agent decided (promote, reject, retry), and why. Persists across sessions.

The job store is infrastructure. The prompt log is the product.

The tool that matters most

Twenty tools. The one I'd keep if I could only keep one: read_prompt_log.

Before every generation, the agent reads the last N entries. It sees what prompts were tried for this asset, what parameters were used, which results were promoted and which rejected, and its own reasoning for each decision. It's not starting from zero. It has a creative history.

What I didn't expect: the log changes the agent's behavior qualitatively, not just quantitatively. It stops repeating failed approaches. It builds on what worked. It references its own prior decisions — "Last attempt had too much detail in the background — reducing stylize from 300 to 100 and adding 'simple shapes' to constraints." A stateless tool-caller becomes something that iterates.

The implementation is trivial. Append a JSON line to a file, read the last N lines back. No database, no schema migration, no query language. Simplest possible persistence, and it's the thing that makes the pipeline feel intelligent rather than mechanical.

The agent inspects its own work

After the grid comes back, the agent reads the PNG with vision. Looks at the four quadrants. Evaluates them against what was asked for. Picks the best one.

There's a helper tool — score_grid — that ranks quadrants on sharpness, contrast, and edge density, giving the agent something quantitative to start from. But the selection is the agent's judgment call, using the same vision capability it uses to read screenshots or evaluate UI layouts.

Every other Midjourney wrapper I've seen generates and hands the result to a human. The human looks at four options, picks one, clicks upscale. cascade-img's agent does this itself. Not always right — maybe 80-85% agreement with my own picks. But it can do fifty of these while I'm making coffee, and the prompt log means it gets better at knowing what "good" looks like for each specific asset.

The alpha keyer: when simple breaks

Crop the grid, remove the background, trim whitespace. Sounds straightforward. Mostly is. But the background removal broke in a way I didn't anticipate.

First version: global color-distance threshold. Find pixels within tolerance of the corner color, make them transparent. Works great on a blue icon with a white background. Eats a white penguin on a white background — the belly matches the background, so the keyer deletes the subject.

Fix: flood-fill from the four corners instead of a global threshold. The fill propagates outward from the edges and stops at color boundaries. A subject surrounded by a darker outline stays opaque because the outline blocks the flood, even if the subject's interior matches the background.

The tool now returns a keyed_ratio — fraction of pixels made transparent. Below 0.1 means it found no background. Above 0.9 means it ate the subject. The agent branches on a number instead of guessing.

What I'd do differently

If I were starting over, I'd spend even less time on the AI integration and more on the operational foundation. The AI parts — vision inspection, prompt composition, the judgment calls — were the easiest code to write. They work because the LLM is good at them out of the box.

The interesting engineering was all in the seams. Routing messages through a chat platform that wasn't designed for programmatic use. Preventing double-billing across network timeouts. Making a daemon survive disconnects without losing state. Giving an agent memory so simple that it's just a text file, and watching that memory change its behavior.

A brilliant prompt composer on top of a bridge that drops jobs is worse than a mediocre composer on top of a bridge that never loses a generation.

The pipeline is open source — daemon, MCP server, CLI, twenty tools, prompt log: cascade-img on GitHub. Midjourney is the first backend; Flux, DALL-E, and Imagen are next. Each one is smaller, because the hard part is done.