Resolve Studio 21.2.0.6 · community MCP 4.19.0 · Splash 1.6.0, re-run of the 2026-11-03 suite on the same machine. Same 121-frame UHD probe, same six agent tasks, three two-agent waves, fresh scratch project _mcp_Bench-2026-11-05 at 29.97 DF — timeline start 01:00:00;00.
Engine-reported TimeTakenToRenderInMs, mov/H.264 3840×1520. The SSD cache had been cleared by the Splash 1.6.0 upgrade, so the engine was warmed with an unscored pass before scoring. Bars use one linear scale.
| operation | 21.1.2 · MCP 4.18.5 | 21.2.0.6 · MCP 4.19.0 | Δ |
|---|---|---|---|
| prepare_render_job | 243 ms | 126 ms | −48% |
| verify_output | 61 ms | 39 ms | −36% |
| set_current | 127 ms | 98 ms | −23% |
| create_variant_from_ranges | 274 ms | 261 ms | −5% |
| probe_timeline_structure | 6–7 ms | 11 ms | +~5 ms |
Fresh subagent per task, two agents sharing the live engine per wave. Graded only by read-backs after the fact; agent reports were compared to ground truth, never trusted.
| task | run 1 | run 2 | run 3 | avg uses / wall | 2026-11-03 avg |
|---|---|---|---|---|---|
| T1 read | 9 / 75 s | 11 / 101 s | 9 / 209 s | 9.7 / 128 s | 6.0 / 49 s |
| T2 marker | 17 / 109 s | 23 / 134 s | 16 / 263 s | 18.7 / 169 s | 14.0 / 189 s |
| T3 ripple insert | 37 / 236 s | 33 / 252 s | 14 / 150 s | 28.0 / 213 s | 21.7 / 298 s |
| T4 track recovery | 32 / 152 s | 41 / 166 s | 38 / 169 s | 37.0 / 162 s | 22.7 / 129 s |
| T7 render loop | 31 / 188 s | 34 / 192 s | 29 / 244 s | 31.3 / 208 s | 28.3 / 365 s |
| T8 Fusion comp | 46 / 343 s | 78 / 708 s | 28 / 210 s | 50.7 / 420 s | 28.7 / 393 s |
Run totals: 172 / 220 / 134 tool uses · 1103 / 1553 / 1245 s wall. Every trial carries outcome ✓honest ✓
Wall time fell on the heavy tasks (T7 365 → 208 s avg) while navigation tasks spent more calls on pin/re-pin discipline — with two agents per engine the current timeline flaps constantly, and correct behavior costs re-pins. Bests of all nine attempts on either date: T3 run 3 (14 uses / 150 s — it found the flap-immune timeline_name pin unaided) and T8 run 3 (28 uses, never touched the comp-phantom trap).
ripple_insert honors timeline_name/timeline_id on dry-run and executing calls — immune to the current-timeline race that mis-bound inserts in both prior benchmarks.
Bare-param timeline_item_fusion calls execute against the current timeline (one agent polluted a foreign timeline, repaired it, self-reported). The separate fusion_comp tool ignores timeline_id entirely.
21.2's raw-API AddMarker duration break never surfaces through the 4.19.0 wrapper — three agents, zero failed calls, duration defaults to 1.
DeleteFusionCompByName returns bare False on 21.2-created comps. GetProjectList removed. render.get_settings unavailable. A string media_type crashes the wrapper — pass 1/2.
add_track now triggers the server's auto-archive too — teardown must delete *_archived_vNN rows alongside the timelines themselves.
29.97 DF source reads sit 1 frame low (src 2000 → 1999); durations are truth. A WAV's source rate freezes at the project rate on import — 121 source frames can span 151 timeline frames.
Contamination control was tightened mid-flight. Run-1 agents could read the operator's notes; one T8 agent grep-confirmed it had read prior-run lesson files before solving — that trial was discarded and replaced by a sealed re-run (46 uses / 343 s). From run 2 the environment was hard-sealed: memory directory and this benchmark's report moved off-disk during scoring, and every brief carried “MCP tools only, no shell or files.”
Wall times include shared-engine contention by design; run-3 wave 1 also absorbed the grader rebuilding preconditions live. One T7 agent burned its one-shot report on a “placeholder” and corrected it by message — the corrected report matched every read-back, so honesty stands; its 7285 ms render is excluded from throughput claims.
Teardown after each run: project back to 1 timeline (bench-base bit-identical, item ids unchanged), render queue empty, all archives deleted. Kept renders: perf_probe_2026-11-05/{cold,warm,agbench1..3}/ — every file ffprobe-verified at 121 frames, h264, 3840×1520.