AWS made it easy to call frontier models from the same account as your VPC. It also made it easy to discover a four-figure surprise at month end. The patterns below are what we bake into Forge and Embed builds on cyberds / London by default.
Practical controls
- Prefer smaller / cheaper models for classification and routing; reserve the expensive ones for hard generation.
- Cache embeddings and frequent RAG hits — most “AI spend” is repeated retrieval.
- Set hard quotas and CloudWatch alarms on token usage per environment.
- Stream when users wait; batch when they don’t.
Bedrock vs “just call OpenAI”
Bedrock wins when you need IAM, VPC endpoints, and data residency stories that procurement will accept. Direct APIs win when you need the newest model yesterday. We pick based on the customer’s constraints, not our favourite SDK.
Region note
We default to eu-west-2 for workloads. Model availability differs by region — design for fallback models, not for a single SKU forever.
Architecture choices that control spend
Bill shock is usually architecture, not model choice. The expensive patterns we see repeatedly:
- Full context re-sent on every turn instead of prompt caching or conversation summarisation.
- RAG that embeds the same chunks on every query instead of caching vectors at index time.
- One frontier model for everything — classification, routing, and generation — when a small model or regex would do.
- Agents that loop until success with no max-step guard. Token meters do not care that turn fourteen was unnecessary.
Forge builds get a cost budget per workflow before we pick models. Embed agents get per-tenant quotas from day one. Surprises belong in demos, not in AWS Cost Explorer.
Runaway spend patterns — and fixes
Three alarms we set on every Bedrock deployment:
- Daily token ceiling per environment with SNS alert at 80%. Staging should never outspend prod; if it does, someone left a loop running.
- P95 latency spike on inference — often precedes a retry storm or a model cold-start misconfiguration.
- Embedding call volume diverging from query volume — sign your index pipeline is re-embedding instead of retrieving.
Review the bill weekly in the first month. After that, monthly is fine if alarms are wired. “We'll look at cost at renewal” is how four figures become five.