Service · Engineering
API & backend development
The distance between a convincing prototype and a system you'd put in front of customers is almost entirely backend work — and almost none of it is about the model. This is the part we do.
The 90% nobody demos
Prototype → production, itemised
A notebook that produces a great answer once is maybe a tenth of the job. Here's what the other nine tenths consists of.
| The prototype | What production requires |
|---|---|
| One request, run by hand | Concurrency, queueing, backpressure, idempotency, graceful degradation under load |
| Waits for the whole response | Streaming so users see progress, with timeouts, cancellation, and reconnect that doesn't lose or duplicate output |
| Key hardcoded in a file | Secret management, rotation, per-environment scoping, and audit of who used what |
| Anyone can call it | Authentication, per-tenant isolation, authorisation on every retrieval path |
| "It worked when I tried it" | An evaluation suite, golden cases, regression gates in CI, and a rollback path |
| Cost unknown | Prompt caching, model routing, batching, budget caps, per-tenant metering and alerts |
| Fails silently | Structured logging, traces, error taxonomies, retry policy, dead-letter queues, on-call alerting |
| Untracked prompts | Prompts and tool definitions in version control, reviewed, versioned, and tied to eval results |
| One model, one provider | Provider abstraction, fallback behaviour, and a migration path when a model is deprecated |
Capabilities
What we build
Streaming AI endpoints
Server-sent events or WebSocket endpoints that stream tokens as they're generated, with proper cancellation, timeout handling, partial-response semantics, and reconnect logic. Long generations need streaming regardless of UX preference — non-streaming requests at high output limits hit HTTP timeouts.
Tool use & agent loops
Well-designed tool schemas, parallel tool execution, correct result handling, and loop termination that can't run away. Approval gates on anything irreversible, iteration and budget ceilings, and full traces — so an agent with write access is a controlled system rather than a liability.
Structured outputs & validation
Schema-constrained responses so downstream code gets valid, typed data instead of prose it has to parse. This retires an entire class of brittle workaround — regex extraction, retry-on-parse loops, and the fragile output-forcing tricks that newer models no longer accept anyway.
Retrieval & RAG pipelines
Chunking that respects document structure, hybrid search, reranking, and citations back to the source. Critically: permission filtering applied at retrieval, so a user can never be shown content from a document they don't have access to. Plus a refresh strategy, because a stale index is a confidently wrong one.
Background & batch processing
Job queues, scheduled runs, webhook handlers, and batch APIs for work that isn't latency-sensitive — often at roughly half the per-token cost. With retries, dead-letter queues, progress reporting, and partial-failure handling that doesn't lose the successful 90%.
Context & memory management
Long-running conversations that don't fall over at the context limit: compaction, pruning of stale tool results, and persistent memory across sessions. Designed so the cache-friendly parts of the prompt stay stable — which is what keeps long sessions affordable.
Evaluation harnesses
Golden datasets built from your real cases, deterministic assertions where the answer is checkable, model-graded scoring where it isn't, and regression gates wired into CI. This is what makes a prompt change, a model upgrade, or a retrieval tweak a routine deploy rather than a gamble.
Observability & cost control
Request tracing, token accounting per feature and per tenant, cache hit-rate monitoring, latency percentiles, and spend dashboards with alerts. Without per-surface cost visibility, every optimisation is guesswork — so this goes in first, not last.
The lever most teams haven't pulled
Cost engineering is engineering
AI bills are unusually responsive to architecture. We routinely find large reductions with no quality loss — and occasionally with a quality gain, because the same changes that cut spend often cut latency too.
- Prompt caching. Cached prefixes are dramatically cheaper to re-read than to re-process. But caching is a strict prefix match — a timestamp or a request ID near the front of the prompt silently invalidates everything after it. Auditing for these is often the single highest-return hour of the engagement.
- Model routing. Send classification and extraction to a small, fast model; reserve the frontier model for work that genuinely needs it. Most systems run everything on their most expensive option by default.
- Effort tuning. Modern models expose controls over how much they deliberate. Lower settings frequently match a previous generation's best output at a fraction of the tokens — but only measurement tells you where the line is for your workload.
- Batching. Anything that doesn't need an answer in seconds can go through a batch pathway at roughly half price.
- Context hygiene. Long agent loops accumulate stale tool results that are re-sent on every turn. Clearing or compacting them cuts input cost sharply.
- Prompt cruft. Instructions written for older models — emphatic all-caps rules, step-by-step scripts, format scaffolding since replaced by native features — cost tokens on every single request and can actively degrade current models. Removing them helps twice.
Cost & latency audit
A standalone engagement when the bill or the response times went sideways. Two weeks, fixed price.
- Traffic analysis: where the tokens actually go, by feature
- Cache hit-rate measurement and invalidator hunt
- Model-routing recommendations with measured quality deltas
- Prompt audit against current-generation behaviour
- Implementation of the agreed changes as reviewable PRs
- Spend dashboard, budget alerts, and a before/after benchmark
We report the honest number. If your setup is already efficient, the audit says so and we stop. That outcome has happened and we'd rather have it on record than pad a report.
Stack
What we work in
We fit your stack rather than importing ours. If your team runs Django, you get Django — not a Node service nobody there can maintain.
Languages
Python (FastAPI, Django), TypeScript / Node (Hono, Express, Next.js), Go. Java and C# where the existing platform calls for it.
Model providers
Anthropic Claude, OpenAI, Google, Azure OpenAI, Amazon Bedrock, Vertex AI, and self-hosted open models. Vendor-neutral, no referral fees.
Infrastructure
Cloudflare Workers & Pages, AWS (Lambda, ECS, Bedrock), GCP, Azure, Vercel, Fly.io, Kubernetes, or your own hardware.
Data
Postgres with pgvector, SQLite / D1, Pinecone, Qdrant, Weaviate, Elasticsearch, Snowflake, BigQuery, S3 and R2.
Queues & jobs
Celery, BullMQ, Cloudflare Queues, SQS, Temporal, and plain cron where plain cron is genuinely the right answer.
Observability
OpenTelemetry, Datadog, Grafana, Sentry, plus AI-specific tracing and token accounting wired into whatever you already run.
Testing & CI
pytest, Vitest, Playwright, GitHub Actions, GitLab CI. Eval suites run as part of the pipeline, not on someone's laptop.
Security
OAuth 2.0 / OIDC, SSO, per-tenant isolation, secret managers, least-privilege scoping, encryption in transit and at rest, audit trails.
A recurring emergency
Model migration & deprecation work
Model generations change faster than most codebases. Parameters get removed, defaults flip, tokenisation shifts, and behaviour that your prompts were tuned around simply stops being true. Teams typically discover this when something starts returning errors — or worse, when it doesn't error and quietly gets worse.
A migration done properly is not a string replacement:
- Compatibility audit. Every call site checked for parameters and patterns the target model no longer accepts.
- Token and cost re-baseline. Tokenisers differ between generations; the same text can cost meaningfully more or less. Output limits and truncation thresholds get re-measured rather than assumed.
- Prompt re-tuning. The part everyone skips. Instructions written for an older model's failure modes often over- or under-fire on a newer one, which follows them more literally. Prompting is a per-model artefact.
- Staged cutover behind evals. Run both, compare on real cases, and switch when the numbers justify it — with a documented rollback.
Signs you need this
- You're pinned to a model with a published retirement date
- Requests started returning validation errors after a provider change
- Output quality drifted and nobody can say when or why
- Costs moved sharply without a change in traffic
- Your prompts contain rules nobody can explain the origin of
- You have no way to tell whether a change made things better
That last one is the real problem, and it's fixable in about a week.
What you get
Every build engagement delivers
- Source in your repository, your licence, your CI
- Automated tests plus an evaluation suite for AI behaviour
- Infrastructure as code and a reproducible deploy
- API documentation and architecture notes
- Monitoring, tracing, and cost dashboards
- A runbook covering failure modes and recovery
- Code review sessions with your engineers as we go
- A load and cost benchmark at realistic volume
We write code your team can read. No clever abstractions that only make sense to the person who wrote them.
Common questions
Can you work alongside our engineers rather than instead of them?
That's the preferred shape. Pairing, code review, and shared standards mean the capability stays after we leave. Purely outsourced AI features tend to become the thing nobody wants to touch.
How do you handle our data?
Least data that does the job. Work happens in your environment where possible, production data isn't moved without written approval, and we document precisely what is sent where. Zero-retention arrangements, regional processing requirements, and self-hosted models are all viable — but they change the architecture, so they need deciding at the start.
What about hallucinations in something customer-facing?
You reduce it structurally rather than by asking nicely. Ground answers in retrieved sources and require citations. Use schema-constrained outputs so malformed responses fail loudly instead of passing through. Add verification steps for high-stakes claims, and route low-confidence cases to a human. Then measure the residual rate with evals — an unmeasured system is one you're guessing about.
We want to self-host. Realistic?
Often, yes — particularly for classification, extraction, and retrieval, where open models are strong and the workload is predictable. Complex reasoning and long-horizon agent work is where hosted frontier models still lead by a distance. A hybrid is common: self-host the high-volume routine work, call out for the hard cases. We'll model the actual economics rather than assuming self-hosting is cheaper, because at low volume it usually isn't.
How do you price?
Fixed price for defined scope, day rate for ongoing work. Either way you get a written scope and an estimate before anything starts, and we flag scope changes when they happen rather than at invoice time.
Have a prototype that needs to become a product?
Bring it to a scoping call. You'll get an honest read on what's missing between here and production, and roughly what closing that gap involves.