Root Cause Analysis (RCA)

Trace Analysis Example

From the Sessions tab we go to our example "just build stuff". The Session traces shows the complete workflow execution.
The Writer agent produces: "In the world of software development, hands-on experience matters."
The Reviewer agent identifies: "Hook lacks energy - too generic."
The Editor agent's Tool interactions show the failure modes.

Expected vs Actual Behavior

Expected workflow:

What we expected:

1. Writer: writes initial post
2. Reviewer: "The hook is weak."
3. Editor: Calls Hook Improver tool with the draft.
4. Hook Improver Tool: Returns a punchy, optimized hook.
5. Editor: Updates the draft.

Actual trace shows:

1. Writer: writes initial post
2. Reviewer: "The hook is weak."
3. Editor: "Let's start by improving the hook"
4. Editor: Attempts hook rewrite.
5. Hook Improver Tool: Fails due to missing inputs
6. Editor: Tries to fix the hook using Content Rewrite
7. Content Rewrite Tool: Fails due to missing inputs
8. Editor: Returns final draft without change

The Power of Trace Visibility

This is a silent failure. From the outside, the system works:

• ✓ All agents complete their tasks
• ✓ No errors or exceptions
• ✓ Final output is grammatically correct

Traditional monitoring would mark this as "success." Only by examining the trace can we see the Editor's internal reasoning and discover it's avoiding tool calls.

Going through the traces we isolated the issue to the Editor's agents tools. Using a granular analysis will confirm if this issue is a singular issue or a pattern.

Analyze Property Failures

Back to the Overview tab under Tool interaction type analyze property failures for the Tool Completeness property.

The first category: "Failure to Generate Output Due to Error Handling or Missing Inputs". The category and the provided examples confirms a repeating pattern. The Editor agent fails using the tools by not providing inputs to the parameters.

Examining the tool definition:

# Version 1 (The Problem)
@tool("Hook Improver")
def hook_fix_tool(post, issue):
    ...

The tool lacks type hints and argument documentation.

Hypothesis

The LLM observes a tool named Hook Improver but cannot determine parameter types or formats. Without knowing the schema, the LLM calls the tool without providing input parameters, resulting in errors.