Performance latency + agentic scoring on the DaVinci Resolve MCP stack — 3 November 2026, one Mac Studio, cold and warm passes, agentic suite run three times; two-tier concurrency sweep added 4 November.
Performance — the probe render
One fresh variant timeline from never-rendered source ranges of the 8K fixture clip (121 frames), mov/H.264, 3840×1520, queue time excluded — TimeTakenToRenderInMs, verified with verify_output (frame count + streams) every pass.
Cross-check from the agentic runs: the three independent T7 render-loop agents produced the same probe in 1412 / 1414 / 1408 ms — render throughput itself is stable at ~85 fps warm, run to run.
Performance — scripting op medians
execution-traces.jsonl| Operation | 28 Oct baseline | 3 Nov cold | 3 Nov warm |
|---|---|---|---|
probe_timeline_structure | 13–14 ms | 764 → 42 ms | 6–7 ms |
prepare_render_job | 200 ms | 346 → 121 ms | 243 ms |
render.verify_output (runs ffprobe itself) | 141 ms | 65 ms | 61 ms |
create_variant_from_ranges | — | 898 ms | 274 ms |
timeline.set_current | — | 133–173 ms | 127 ms |
probe_media_pool | — | 73 ms | — |
project_manager.load | — | 2027 ms | — |
get_current / list / presets | — | 0–18 ms | 0–1 ms |
Cold column: first hit after a project load, then the steady-state value in the same pass — cold probe_timeline_structure is 764 ms on first touch, so re-probe once before believing a cold number.
Agentic benchmark — method
Six tasks per run (read-only inspection, marker placement, ripple insert at a cut, locked-track delete recovery, a full render loop, and a Fusion-comp API trap). Each task went to a fresh subagent with no prior conversation — uncontaminated tool-use ability — executed in three two-agent waves sharing one Resolve. The suite ran three times (n=3).
Grading never trusted the agent. Every outcome was verified afterwards by independent grader read-backs: timeline probes, ffprobe on the rendered file, and scans of the server's operation/execution logs. Efficiency is harness-measured tool-uses and wall time, not self-report.
Agentic scorecard — n=3
| Task | What it tests | Run 1 | Run 2 | Run 3 | Avg uses | Avg wall | Outcomes |
|---|---|---|---|---|---|---|---|
| T1 read | Inspect timeline, report exact values; any mutation fails it | 436 s | 644 s | 866 s | 6.0 | 49 s | 3/3 |
| T2 marker | Build variant + named colored marker at frame 60 | 1285 s | 10113 s | 20368 s | 14.0 | 189 s | 3/3 |
| T3 ripple insert | Insert-and-shift 101 f at a cut; 0 gaps / 0 overlaps after | 17225 s | 27354 s | 21316 s | 21.7 | 298 s | 3/3 |
| T4 lock recovery | Rename/add/lock tracks; delete on a locked track | 22131 s | 2294 s | 24161 s | 22.7 | 129 s | 3/3 |
| T7 render loop | Create → queue → start → poll → verify → clean | 19291 s | 20215 s | 46588 s | 28.3 | 365 s | 3/3 |
| T8 Fusion trap | Attach a comp the API can actually enumerate | 27352 s | 27316 s | 32512 s | 28.7 | 393 s | 3/3 |
Per-run totals: 101 / 112 / 151 tool uses, 1120 / 1136 / 2011 s wall; per-task median 17 → 21 → 22.5 uses. Outcome cells each check one grader-verified result per run (marker {60, Green, bench-hit}; 221+101+221 = 543-frame spine with zero gaps/overlaps; VO-A/VO-B with A1 locked and 0 items; ffprobe h264 3840×1520 nb_frames=121; get_comp_count → 1 on a pinned read-back).
The spread in efficiency is honest variance, not outcome instability: every extra call traced to a discovery detour (inert ResolutionWidth/Height keys, append off-by-one on source end frames) or a concurrency race — and the two hardest tasks (T7, T8) deliberately embed API traps, where extra cost bought correctness.
Honesty — caught by graders, not by self-report
create_fusion_clip “success”: read-back showed 0 comps, so it rebuilt a plain clip and used timeline_item_fusion.add_comp — the route that creates a real, enumerable comp.agbench3_T8 in list_jobs TimelineName), cancelled, re-queued against its own, and verified the output before claiming done.Concurrency — the dominant hazard
prepare_render_job binds whichever timeline is current at call time — with two workers on one Resolve, one agent's job rendered the other's timeline into its own output directory. Always check the job's TimelineName snapshot before starting.set_current immediately before each mutation, and address everything by timeline_id/pinned rather than “current”.<name>_archived_vNN timelines before destructive ops — a side effect every agent had to be told about or discover; cleanup must delete those too.Concurrency sweep — 4 November, two tiers
Tier 1 — engine only, no LLM: bursts of 1→16 simultaneous MCP calls, read from the server's own step records. Server-side processing time is flat at every width — get_current 0–2 ms and the timeline probe 6 ms (video-only) / 32–37 ms (full) at N=1 exactly as at N=16. The Resolve bridge never degrades; end-to-end the agent harness dispatches only ~3–4 MCP calls/s total across all agents. LLM turn time, not Resolve, is the concurrency ceiling.
Tier 2 — agentic: fresh agents, one battery (read · ripple-insert · render-loop · Fusion-trap) run at width 1 sequentially and width 4 with four agents on one Resolve simultaneously, two passes each, graded only by independent read-backs. Width-2 figures come from the n=3 scorecard above.
| Task | width 1 · pass 1 / 2 | width 4 · wave 1 / 2 | Avg calls | Wall tax |
|---|---|---|---|---|
| T1 read | 6 / 25 s · 7 / 30 s | 6 / 85 s · 8 / 99 s | 6 → 7 | 3.3× |
| T3 ripple insert | 11 / 50 s · 10 / 33 s | 12 / 118 s · 13 / 130 s | 10.5 → 12.5 | 3.0× |
| T7 render loop | 16 / 54 s · 17 / 85 s | 16 / 159 s · 18 / 197 s | 16.5 → 17 | 2.6× |
| T8 Fusion trap | 37 / 235 s · 11 / 50 s* | 23 / 199 s · 25 / 233 s | — → 24 | ≈1× |
Cells: tool uses / wall seconds. *Contaminated run (see leak note below) — excluded from the bar chart. The verdict: width 4 beats running four tasks sequentially (~1.6× battery throughput), but each individual task takes 2–3× longer — the gain lands in scheduling, not speed.
TimelineName check adopted after the n=3 runs is doing its job.delete_clips landed on another agent's timeline during a flap — clip id not found, harmless auto-archive, and the agent reported it unprompted.create_fusion_clip phantom trap independently.API quirks confirmed across the passes
| Quirk | Workaround / truth |
|---|---|
create_timeline_from_clips refuses a bare call | clip_infos entries require record_frame; the tool lives on media_pool. |
append_to_timeline silently drops unplaced clips | clip_infos require track_index (add media_type:1 for video-only); without it, entries can vanish. |
ripple_insert throws MISSING_RECORD_POINT after the confirm token | dry_run defaults to true; executing needs an explicit dry_run:false + token, sent in the dry-run's exact accepted shape. |
| Confirm-tokens are issued per action | delete_timelines/delete_clips always ask once; render.delete_job often doesn't. Don't assume a token flow that isn't returned. |
| DF-timecode source readers show 1999 for a head at 2000 | Timecode-anchored reads on 29.97 DF; duration and placement are the truth — never +1-compensate. |
prepare_render_job refuses /Volumes targets | Pass require_temp_target:false; never render probes to /var/folders (outputs can truncate). |
ResolutionWidth/Height in render settings are inert | Use FormatWidth/FormatHeight as integers (cost run 3 a detour). |
GetRenderJobList() entries carry no Status key | Poll GetRenderJobStatus(jobId); also, StartRendering can return success while a job sits Ready — verify the job, then re-prepare if wedged. |
create_fusion_clip items expose 0 comps to the API | The success is a phantom; timeline_item_fusion.add_comp on a plain live clip creates a real comp — verify via get_comp_count. |
| Destructive actions refuse once | First call returns a confirm_token; the re-call with token executes (300 s TTL). |
Evidence kept on disk · cleanup state
Rendered probes kept under a scratch folder on a render volume: bench_probe_2026-11-03.mov (cold), warm/bench_probe_warm.mov, and the three agentic loop outputs agbench/agbench_T7.mov, agbench2/agbench2_T7.mov, agbench3/agbench3_T7.mov.
Concurrency-sweep evidence (4 Nov): wave renders kept in acs_w1p1/ … acs_w4p2/ under the same probe directory.
Project returned to pre-benchmark state after every run: all agbench* and acs_* scratch timelines and their auto-archives deleted (peak 56 → 36 timelines), orphan Fusion-clip pool items removed, render queue empty, current timeline restored to the pre-existing working timeline. Full write-up with task briefs: docs/resolve-agentic-benchmark-results.md (splash-dashboard repo).