Graders should include native Copilot tool calls in the automatic trace payload #61178

Problem / motivation

The deterministic grader runtime in v0.88.7 builds trace.toolCalls from MCP gateway logs in actions/setup/js/trace_graders.cjs. Native Copilot tools recorded in staged events.jsonl files—such as the built-in skill tool—are therefore absent from the automatic grader payload.

This prevents skill-constraint-coverage from evaluating constraints against actual skill invocations. Although custom payloads can supply agentOutput.toolCalls, pkg/workflow/compiler_yaml_post_agent.go emits the graders step before user post-steps, so a post-step cannot normalize native events before automatic grading runs.

Expected behavior or outcome

Automatic grader payloads should expose native agent tool calls with their names, arguments, call IDs, and correlated completion status. A repository should not need a separate post-run evaluator merely to make Copilot tool activity visible to gh-aw graders.

Scope

• Reuse the existing Copilot events.jsonl parsing support.
• Normalize tool.execution_start and tool.execution_complete into grader toolCalls.
• Preserve arguments, including the target passed to the built-in skill tool.
• Avoid double-counting calls represented in both native and MCP logs.
• Alternatively, provide a documented pre-grader hook through which workflows can populate the normalized payload.

Acceptance criteria

• A native tool.execution_start produces a trace.toolCalls entry containing name, arguments, and toolCallId.
• A matching tool.execution_complete determines the entry’s success value through toolCallId correlation.
• Calls without a matching completion remain distinguishable from successful calls.
• Native and gateway-derived records are deduplicated deterministically.
• skill-constraint-coverage can match a successful invocation of a specific skill without repository-supplied grading logic.
• The unused agent-log path constants in trace_graders.cjs are either wired into preprocessing or removed.
• Tests cover arguments, failed and incomplete calls, correlation, and deduplication.

This is an enhancement to grader trace coverage rather than a regression; current behavior safely produces an unmet or not-applicable result.