Somebody Has to Own the API Key
The unglamorous infrastructure behind shipping AI at a company that isn't a tech company.
Everyone wants to build the AI. Nobody wants to own the API key. The second job turns out to be the real one.
I worked this out, as one does, from a billing alert. When I started shipping chatbots I assumed the hard part would be the model. It is not. The model is a paid endpoint anyone can call. The hard part is everything wrapped around it: where the thing lives, whose money pays for it, who is allowed in, and what stops it doing something expensive or embarrassing. I set out to build one bot for one project. A few months later I owned DNS records, the firm's API billing, an approval workflow, and a small body of rules about muffins. Nobody handed me any of it. It accreted, the way infrastructure always does, around the person who shipped the first thing and failed to flee.
Here is what that actually involves, since the demos never show it.
Domains, or why the link matters more than you think
The bots first went up on a host's default URLs, the kind that read something-random.platform.space. Fine for a prototype. Not fine as a link to a Fortune 500 client, who quite sensibly reads a random subdomain as a small act of phishing.
So each bot gets a proper home on the firm's own domain, which means owning the DNS and pointing a subdomain at the host with a CNAME record, the dull registry plumbing that connects inkbot.yourfirm.io to wherever the app runs. Do it once and it feels like nothing. Do it for the fifth bot and you realise you have quietly become the person who understands the firm's DNS, a sentence I never expected a strategy job to produce. I wrote the wiring into a run-book so a colleague could repeat it without me, which is the only reason I am not still doing it by hand.
The point is not the CNAME. It is that a tool a client trusts has to look like it belongs to us, and trust starts in the address bar.
The API key is a budget with no brakes
The second thing I changed was billing, and this is the one I would press on anyone building internally.
Out of the box, the bots ran on the host's own model access. Convenient, and a trap, for two reasons. You do not control which model you are on, so when a provider retires a version you discover it the moment something breaks. And you get no real view of cost, which is comfortable right up until it is not.
So I moved everything onto the firm's own provider keys. Two gains: the bots ride the current model automatically, and every call bills back to us per token, where we can see it. The cost is the one in the title. An API key is a budget with no brakes. The provider will happily let an enthusiastic user generate two hundred slide decks before lunch and present the bill afterward, complete with a cheerful little usage graph.
“An API key is a budget with no brakes.” LEARNED FROM A BILLING ALERT
Which is how cost stopped being an afterthought and became part of the design.
Cost control is a feature, not a cleanup job
A text answer is cheap. The impressive things are not. A full slide deck or a long document costs many times more per call than a plain answer, because you are paying for far more output, sometimes across several chained model calls. Leave the expensive endpoints unguarded and your cost scales with your most excited user rather than your average one.
So the governance went in at the start. Rate limits and per-user daily caps, so no single account can run the meter up. Paywalling the expensive features rather than the whole tool, so casual questions stay free and slide generation is metered. And enough accounting to see which bot and which feature is actually driving spend, rather than guessing. None of it is exotic; it is what any product company does. The difference is that at a consulting firm there is no platform team to do it for you, so "build a chatbot" silently includes "design its pricing model," whether you signed up for that or not.
Access: a password is not an access model
Early on, access control was a shared password. That is the right amount of security for an internal demo and the wrong amount for anything a client touches or anything that costs money per use.
So the serious bots got a real access layer: individual accounts behind an admin approval step, where someone, usually me, lets people in, rather than one password circulating through three companies' inboxes. It is a small change that rewrites the risk entirely. A leaked password is anonymous and uncapped. An approved account is named and rate-limited, so when something goes wrong you know whose it was and the blast radius is one user instead of everyone.
The secrets your tools leave lying around
Here is the one that actually worried me, and it is the kind of thing nobody warns you about. The build-and-deploy tools that make this so easy will, helpfully, write a config file holding your live secrets and commit it to the repository. I went looking through one of the projects and found, in plain text in version control, the database password, the signing secret, and a set of cloud access keys. Not encrypted, not in a vault, simply there, in a file the tool had created and quietly checked in.
It is a clean reminder that "it works" and "it is safe" are different claims, and the convenience layer only optimises for the first. Owning the infrastructure means periodically auditing what your tools have done on your behalf, because some of it is the security equivalent of leaving the keys in the ignition. The fix is dull and non-negotiable: keep secrets out of the repo, inject them at runtime, and rotate anything that ever touched a commit.
Guardrails are scope enforcement, not politeness
A bot pointed at your research will, left alone, answer anything. For a client-facing tool that willingness is a liability, not a charming quirk. The bot should do its job and decline the rest.
My standing test is to ask a finished bot for a muffin recipe. If it obliges, it is not done. It should decline and point back at what it is for. That sounds like a joke, and it is, but it checks something real: that the thing stays inside the scope we sold the client and does not wander off into territory where it has no grounding and every chance to be confidently wrong.
The actual job
Add it up and the AI content of running AI tools is surprisingly small. The work is domains, keys, billing, access, secrets, and scope. It is operations. It is the part that does not demo, does not screenshot well, and earns no applause, and it is most of what separates a thing you showed your team once from a thing a client uses on a Tuesday without thinking about it.
I did not plan to own any of it. But somebody has to own the API key, and it is usually whoever shipped the first bot and stayed to see what broke. ▮