A Modern Stack for Shipping AI Products
An AI product is an application that calls a model. The model is one dependency. The product is the interface, the data, the permissions, the jobs that run after the user leaves, and the way you notice when answers get worse. This guide maps those layers and the choices that keep a first version from becoming a rewrite.
The defaults below match how we build: TypeScript, a web application framework, a relational database, and a provider-agnostic place to call models. They are defaults, not a requirement that every project copy them. The point is to know which layer owns which problem.
Frontend
The interface has to show uncertainty. Streaming tokens, a citation the user can open, a disabled button while a tool runs, and an explicit error when retrieval finds nothing are product decisions. A single spinner over a thirty-second call is how people learn not to trust the feature.
Keep the client thin. The browser should not hold API keys, and it should not decide which documents the user may see. Send the user’s action to your server, and let the server assemble context. For most products we use a React and Next.js frontend because it can render the marketing site, the signed-in app, and the server handlers in one codebase. A separate native client can call the same APIs later.
Backend and APIs
The backend is where policy lives. It authenticates the user, checks what they may do, loads or retrieves data, calls the model, validates the model’s proposed action, and writes the result. If that logic sits in a prompt, you cannot test it and you cannot audit it.
Expose a small API: one endpoint to start a task, one to fetch status, and explicit endpoints for actions such as “create the draft” or “send the message.” Do not give the model a generic “run SQL” route. API integrations with the systems you already run (CRM, billing, scheduling, a fleet TMS) belong behind those same checks.
LLM providers and model routing
Call models through one internal interface. The application asks for a capability (“draft a reply,” “extract these fields,” “embed this passage”) and a router picks a provider and a model. When a vendor changes price, latency, or terms, you change the router, not every screen.
Routing rules that earn their keep are boring: a smaller model for classification, a stronger model for long reasoning, a fallback when the primary provider errors, and a hard stop when the request contains a data class that provider is not allowed to see. Log the model name and the token counts on every call. Without that, cost reviews are guesswork.
Embeddings, vector databases, and RAG
Embeddings turn text into vectors so you can search by meaning. A vector database stores those vectors and returns nearest neighbors. That pair is the retrieval half of RAG. It is the right tool when the model must answer from documents that change and that were not in its training data.
It is the wrong tool when the question is a lookup you can do in SQL: “what is this customer’s open balance,” “which loads delivered yesterday.” Exact facts belong in the database, with the model used only to phrase the result if a sentence is even needed. The longer explanation is in RAG knowledge systems.
Queues and background jobs
Ingestion, re-embedding, evaluation runs, and outbound messages should not live inside the user’s HTTP request. Put them on a queue. A job can retry, it can resume, and it can record each step. A request that times out at a proxy leaves you with a half-written index and no record of why.
Anything multi-step (fetch documents, chunk, embed, then mark the source as ready) needs a status the operator can see. “The worker crashed” is an acceptable state if the UI says so and the job can be rerun without duplicating chunks.
Databases
Use a relational database for users, tenants, permissions, documents’ metadata, job state, and the audit log. Use object storage for the original files. Use the vector store for similarity search only. Teams get into trouble when the vector store becomes the only copy of a policy, because it is a poor system of record: updates are awkward, transactions are limited, and permissions are easy to forget.
Postgres is our usual core because it is well understood, it can store JSON when a shape is still moving, and it keeps operational data next to the metadata you will join against retrieval results. The vector index can sit beside it or in a dedicated engine. Either is fine if the document ID in the index matches a row you can authorize.
Authentication
Use the identity system the rest of the product uses. AI routes are not a side door. Session or token, role, and tenant should be available before any retrieval or tool call. Service accounts for background jobs need their own identity and a narrower scope than a human admin.
Observability, evaluation, and caching
Observability for an AI feature is traces plus content you are allowed to store. A trace should show the user, the route, the retrieval query’s document IDs, the model, the latency, the token use, and whether a tool ran. Storing full prompts is a security decision, covered in AI security and compliance. At minimum, store enough to reproduce a failure on synthetic data.
Evaluation is a set of examples with expected behavior, run when you change a prompt, a chunking rule, or a model. Include cases that should refuse, cases that should cite a specific document, and cases that should call no tools. A vibe check in a demo is not an evaluation set. You do not need hundreds of rows to start. You need rows that represent the failures you cannot afford.
Cache stable work: embeddings for unchanged chunks, and identical classification calls if the input is not sensitive and the policy allows it. Do not cache personalized answers across users. A shared cache key that omits the tenant is a data leak.
Rate limiting
Limit by user and by tenant, not only by IP. Model calls are expensive and they are an abuse path. Return a clear error when the limit hits, and apply a stricter limit to unauthenticated endpoints. Background jobs need their own budget so a re-index cannot starve the product.
Deployment, CI/CD, and security
Ship prompts and evaluation fixtures in the same repository as the code, so a change is reviewable. Run the evaluation set in CI when those files change, and block the deploy on the failures you marked as critical. Secrets stay in the host’s secret store. Preview environments should use synthetic data or a provider tier that does not retain customer prompts.
The production checklist is the same as any other web app, plus the model boundary: dependency updates, least-privilege roles, backups of the system of record, and an owner for vendor status. Enterprise web applications and custom software engagements at Vyntrix start from that production bar rather than from a notebook that happens to call an API.
Hosted models or self-hosted models
| Situation | Usually a better fit |
|---|---|
| You need to ship a feature and the data is not highly restricted | Hosted API, enterprise terms reviewed |
| Prompts must stay in infrastructure you operate | Self-hosted or a provider region and contract that meets the requirement |
| The task is narrow and high volume | A smaller hosted model, or a self-hosted model if unit cost dominates |
| You do not have people to patch and scale GPU services | Hosted, until that operating cost is real |
| You need the newest general-purpose quality | Hosted frontier model, behind your router so you can leave |
Self-hosting is an operations commitment: hardware or GPU instances, model updates, and capacity planning. Hosting is a vendor commitment: retention, subprocessors, and outages. Write down which commitment you are accepting. “We will decide later” usually means customer data is already going to whichever key was pasted into the demo.
A minimum architecture
- Web app and server API in one reviewed codebase.
- Relational database for users, permissions, and audit metadata.
- Object storage for source files.
- A single model client with routing, timeouts, and token logs.
- A queue for ingestion and other multi-step work.
- Vector search only where similarity is actually the query.
- An evaluation set that runs before you change prompts or models.
- Rate limits and an off switch per feature.
Conclusion
The stack is modern because the boundaries are clear, not because every layer uses a new product. Put policy in the server, facts in a database, files in object storage, similarity search in a vector index, and model choice behind a router you can change. AI agents sit on top of that stack. They do not replace it.
Frequently asked questions
Do we need a vector database on day one?
Only if the feature answers from a changing document set using similarity. Lookups, transactions, and workflows should use your existing database and APIs. Adding a vector index before you know the questions usually creates a second, weaker copy of the data.
Should prompts live in the database or in code?
Start in code so changes are reviewed and tied to an evaluation run. Move a prompt to a database when non-engineers must edit it and you still have a review and a way to roll back.
Is serverless required?
No. Serverless is a convenient default for spiky web traffic. Long ingestion jobs and self-hosted models often fit a worker and a queue better. Choose the runtime for the job, not for the label.
Ashir Haroon
CEO at Vyntrix Labs
Keep reading
RAG Knowledge Systems: Architecture, Security, and Implementation
How retrieval-augmented generation is built (ingestion, chunking, search, reranking, citations, permissions, evaluation) and when a knowledge base is the wrong tool.
Sufyan
AI Engineer
AI Security and Compliance: A Practical Guide for Business Systems
What AI security actually covers (data leakage, prompt injection, access control, RAG permissions, audit logs, and governance) and a checklist for deciding how strong those controls need to be.
Haseeb Tariq
Co-Founder

AI Agents vs. Traditional Automation: What's the Difference?
Automation follows rules. Agents make decisions. Here's how to know which one your business actually needs, and where they combine for maximum leverage.
Haseeb Tariq
Co-Founder