What breaks when you ship an AI product with payments?

Almost none of the failures we hit when shipping an AI product with live payments came from the model. They came from the boring parts around it: a payment SDK that would not import, a mail API that rejected requests without explanation, and a web framework that logged nothing in production. This post covers each failure, the fix, and the order we now do things on a new service.

Why do AI products fail in production for non-AI reasons?

An AI product has the same surface area as any paid web product, plus a model call. Payments, email, auth, and logging all have to work first. The model call is usually one HTTP request with a prompt, and it fails loudly when it fails.

The integrations fail quietly. Every issue below presented the same way: the feature was simply dead in production and perfectly fine locally. That pattern is the signal to stop looking at your prompts and start looking at your deploy.

Should you use the payment provider's SDK?

Not automatically. In one build, importing the provider's Python SDK raised ModuleNotFoundError: No module named 'pkg_resources' on a modern Python base image. Payments went down completely, and nothing about the code had changed. The SDK depended on a packaging module that newer images no longer ship by default.

The protocol the SDK wrapped turned out to be tiny. Creating an order is one authenticated POST. Verifying a payment is one HMAC. The provider documents the payment signature check as a SHA-256 HMAC over the order ID and payment ID joined by a pipe, keyed with your secret. The webhook validation guide uses the same primitive over the raw request body, and it warns you not to parse or cast the body before verifying.

That maps directly onto the standard library.

``python expected = hmac.new(secret.encode(), f"{order_id}|{payment_id}".encode(), hashlib.sha256).hexdigest() hmac.compare_digest(expected, signature) ``

hmac.compare_digest exists to compare values without leaking timing information, so use it instead of ==. Removing the SDK was strictly less code than keeping it working, and it removed a dependency we did not control.

Why does a mail API return 403 with no explanation?

In the same build, a transactional email API answered every request with HTTP 403 and a bare error code: 1010. The code is a Cloudflare error, and Cloudflare documents 1010 as the site owner banning access based on the browser signature. Our request came from a script using a default HTTP client, with no User-Agent header at all.

Sending any sensible User-Agent made it pass. The second lesson was more useful than the first. We were not logging the response body on HTTPError, so the numeric code alone told us nothing. Log the body of every failed third-party call. It costs one line and saves an afternoon.

Why did production show no errors at all?

Django ships no useful default handler for its own logger once DEBUG=False. The Django logging docs state that the default configuration only displays log records when DEBUG=True. In production, exceptions can disappear without a trace on the console.

This one made the other two hard to find. A dead payment path with no traceback looks like a mystery. The same path with a traceback is a five-minute fix.

Our checklist for a new service, in order:

1. Set LOGGING first. Route the django logger and your own loggers to stdout, at a level that includes errors, before the first deploy.
2. Replace small vendor SDKs. If the protocol is one POST and one HMAC, write it yourself and test it.
3. Always send a User-Agent. Log response bodies on every HTTP error from a third party.
4. Check env parsing. Our env reader treated the whole line as the value, so DJANGO_ENV=local # note broke a flag with no error. Keep comments on their own lines.
5. Keep secrets in the host, not the repo. Test keys locally, live keys only in the production environment, and let the frontend receive the public key from the create-order response instead of hardcoding it.

Where does the AI cost actually sit?

We ran the numbers on one paid AI product before deciding what to optimize. The model call was a rounding error next to the sale it served, a fraction of one percent of the order value. Acquisition cost and conversion were the real constraints.

That changes where engineering effort goes. Prompt compression and model routing matter at very high volume, or when a single request is expensive. For a product that charges per outcome, they were not the lever. Measure the share of revenue the model consumes first, then decide. For the cases where cost does matter, see LLM cost control and prompt caching.

What do we test before taking money?

An AI product that charges money has to survive a hostile first run. We exercise the full path in test mode and confirm each of these before live keys exist.

• Order creation and signature check. Create an order, complete a test payment, and verify the signature server-side.
• Webhook replay. Send the same webhook twice and confirm the second one changes nothing.
• Failure paths. Force a mail failure and a model failure, and confirm the user sees a clear message and the logs show the cause.
• Cold start. Hit the service after an idle period and confirm the first request still completes inside the client timeout.

For how we measure the model side of that path, read AI evals before production.

Frequently asked questions

Want this built for your team? Book a free 30-minute call.