Tag
Observability on The Latent Daily: 1 story and 1 prompt. Latest: LLM observability: what to log and monitor in production AI apps.
What to record for every LLM call, the quality, cost and latency metrics worth watching, and how tracing, sampling reviews and alerts keep AI features reliable.
thelatentdaily.com Editorial Team3 min read
You are a site reliability engineer investigating an incident. Analyze the logs below. Symptom reported: [e.g. "checkout returned 502 errors for about 20 minutes"] When it started and ended (with time zone): [TIMES] System overview: [SERVICES INVOLVED AND HOW THEY CONNECT] Recent changes: [DEPLOYS, CONFIG CHANGES, TRAFFIC EVENTS, OR "unknown"] Logs (time-ordered if possible; note the source of each block): """ [PASTE LOG EXCERPTS] """ Please: 1. Build a timeline of significant events, citing the exact log line for each. 2. Identify the earliest event that looks abnormal, and explain why it is abnormal rather than background noise. 3. Give up to four hypotheses for the root cause, ranked by likelihood. For each: supporting evidence (quoted lines), evidence against, and what is missing. 4. For each hypothesis, suggest the next check: a log query, metric, command or config to inspect. 5. Separate symptoms (timeouts, retries, 5xx) from likely causes. 6. Say clearly if the logs are insufficient to decide. Do not state a root cause as fact unless the logs show it directly.
Works with: ChatGPT, Claude, Gemini, Any LLM