Observability — LangSmith

In one line

See every step an agent takes: traces, evals, and cost.

Tracing

Turns on with two env vars. No code change.

LANGSMITH_TRACING=true
LANGSMITH_API_KEY=your-key

Evals (tests for agents)

• Offline: run over a saved dataset and score.
• Online: grade live traffic, often LLM-as-judge.
• Trajectory: score the path (tool calls, steps), not just the answer.

Plugs into pytest, Vitest, GitHub Actions. Framework-agnostic. Alternatives: Langfuse, Arize.

New in June 2026

• Voice-trace UI. Debug audio agents with an inline audio player, and see which span was live at each moment.
• Experiment status tracking. Watch live progress for experiments run from the UI or SDK.
• Slack for Engine. Native Slack notifications for LangSmith Engine issues.
• Fleet. An On-Call Copilot template triages alerts from your code, traces, and runbooks. Fleet agents can also use an isolated virtual computer (Computer Use) for code, files, and authenticated API calls.

Source: June 2026 newsletter.

Links

• Observability
• Evaluation
• Full guide — appendix H (tracing)

My notes