Hi OpenLIT team, thank you for building OpenLIT – an open-source AI engineering platform that puts observability, evaluations, guardrails and prompt management in one place. It is exactly the kind of tooling people need when they move from simple demos to real LLM and RAG systems.

I maintain an MIT-licensed open-source project called WFGY (~1.5k GitHub stars). A core part of it is a 16-problem “ProblemMap” for RAG and LLM pipelines, which acts as a compact taxonomy of the most common failure modes across:

• data ingestion, parsing and chunking
• embeddings, vector stores and similarity metrics
• retrievers, ranking logic and hybrid search
• LLM orchestration, tools and routing
• evaluation coverage, metrics gaps and guardrails

ProblemMap overview:
https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

It is used as a practical debugging map and “semantic firewall” for teams that need to keep complex RAG pipelines under control, and it has already been integrated or referenced by several LLM tooling and research projects that focus on robustness and reliability.

Why this might be useful for OpenLIT users

From what I can see, OpenLIT already gives users:

• telemetry and traces for their LLM / RAG flows
• evaluation and quality monitoring
• guardrails and prompt management in one loop

In practice, many of the issues that show up in those traces are exactly the 16 failure modes we track in the ProblemMap. Giving users a ready-made taxonomy makes it easier to:

• tag incidents and traces with a stable failure code (No.1–No.16)
• communicate clearly across teams about “what kind of failure” occurred
• design evaluations and alerts that target specific failure bands instead of generic “quality went down”

Concrete proposal

My suggestion is very small and fully optional for you:

1. Add the WFGY 16-problem ProblemMap as an external reference in your documentation, for example in the sections about RAG, debugging or best practices.
2. Optionally, mention it as an example of a failure taxonomy that OpenLIT users can adopt when defining their own evaluation labels or alert categories.

If this aligns with your docs philosophy, I am happy to adjust the wording or provide a short snippet that explains how to combine OpenLIT traces with the 16-problem map in a way that fits your style.