From 100+ AI Tools to 4: A Production-Tested Tech Stack for AI Systems

From 100+ AI Tools to 4: My Production Tech Stack

TLDR: After building AI systems for 8+ years and making costly architectural mistakes, the author distills production AI requirements down to just four tool categories: a unified database, an API framework, durable workflow orchestration, and LLMOps observability.

This article is refreshingly honest about a problem I've observed repeatedly in the AI ecosystem: teams chasing every shiny framework announcement and ending up with Frankenstein architectures that are harder to debug than the problems they're trying to solve. The author's admission that they "spent more time fighting abstractions than building the product" at ZTRON resonates deeply with anyone who's inherited an over-engineered AI system.

The core insight here is profound in its simplicity. With over 10,000 AI tools and 100+ frameworks competing for attention, most are optimized for demos and tutorials rather than production resilience. The field moves so fast that by the time a framework implements a feature, the underlying APIs have already moved on. This creates a peculiar kind of technical debt where your abstractions become anchors rather than enablers.

Let me walk through the four recommended categories. First, a unified database - and here the author makes a contrarian but correct argument against the common advice to spin up separate vector, graph, and relational databases simultaneously. PostgreSQL with pgvector handles embeddings well for most RAG applications while also managing JSON documents and relational data. The infrastructure overhead of multiple specialized databases rarely justifies itself before you've proven product-market fit.

Second, an API serving layer like FastAPI or the newer FastMCP. FastAPI's async-by-default architecture is genuinely critical for LLM workloads given their I/O-bound nature. FastMCP is interesting for those building MCP-compliant tools that need to be discoverable by clients like Claude Desktop or Cursor.

Third - and this is where I think the article provides its most valuable insight - durable workflow orchestration through tools like DBOS or Prefect. Agentic systems aren't static DAGs; they branch, loop, and terminate dynamically based on LLM outputs. This non-determinism means you need retries (LLM APIs are flaky), step-level caching (for debugging non-deterministic behavior), and the ability to resume from failure points rather than restarting entire pipelines.

Fourth, LLMOps observability through tools like Opik, LangSmith, LangFuse, or Pydantic Logfire. You cannot improve what you cannot measure, and with LLMs you often cannot even see what's happening without dedicated tooling. These tools make the trace the first-class citizen, providing deep insights into prompt flows, token usage, costs, and quality metrics.

What's notably absent from this list - and what the author addresses directly - are AI frameworks like LangChain and LlamaIndex. The take is nuanced: they're good for PoCs and their utility functions can save boilerplate, but they shouldn't dictate application architecture. When production breaks, and it will, you want to debug your code, not framework internals.

For architects and team leads evaluating AI infrastructure, this article provides a useful decision framework. Start with one tool from each category. Resist the urge to add complexity until you have concrete evidence that you need it. "Keep your infrastructure boring" is excellent advice that's especially difficult to follow when everyone on Twitter is announcing the next revolutionary AI framework.

Key takeaways:

• Four tool categories cover 99% of production AI needs: unified database, API framework, durable workflows, LLMOps observability
• PostgreSQL with pgvector handles most RAG use cases without needing separate vector database infrastructure
• Durable workflow tools like DBOS/Prefect are essential for handling agentic system non-determinism
• Treat AI frameworks as utility libraries, not architectural foundations

Tradeoffs:

• Unified database simplifies operations but may require migration when hitting scale limits
• Framework avoidance gives you debugging clarity but requires more boilerplate code
• Starting simple delays capability but reduces accumulated technical debt

Link: From 100+ AI Tools to 4: My Prod Stack

The information presented here is based on newsletter content and may not reflect the complete picture. Always refer to original sources for full context.