AI observability is the practice of instrumenting an AI system so you can see, after the fact, exactly what happened on any given request, which prompt went in, which tools were called, how many tokens were used, how long each step took, and what the model actually output.
When a normal web request fails, standard logging usually tells you why. An AI request has extra failure modes, a wrong tool call, a hallucinated fact, a retrieval that missed the right document, and standard logs don't capture any of that. Observability built for AI systems traces the full chain: prompt in, every intermediate step, response out.
Without it, debugging why an agent gave a wrong answer, or why costs spiked overnight, means guessing. With it, you can trace the exact prompt, retrieved context, tool calls, and token usage behind any specific response.
Every step of a request, the prompt sent, any retrieval performed, tool calls made and their results, the final generation, is logged as a trace, often visualized as a timeline showing where time and tokens were spent. Traces are typically tagged with metadata (user, session, cost, model version) so they can be filtered, aggregated, and alerted on.
Without tracing, diagnosing a quality regression after a prompt or model change means guessing at which requests were affected; with it, you can pull the exact failing traces and compare them directly against passing ones.