Can Gemini find the subject
and spot a broken scene?

1. Write mocks in the planner's actual shape

{
  "sceneId": "scene-7f2",
  "spokenText": "The tiled fireplace gives this living room a warm gathering place.",
  "voicePromptText": "[warm] The tiled fireplace gives this living room a warm gathering place.",
  "deliveryTags": ["warm"],
  "cameraDirectedMotion": "Move gently rightward at eye level while keeping the room in view.",
  "selectedTransitionEffect": "fade",
  "scriptText": "The tiled fireplace gives this living room a warm gathering place.",
  "durationSeconds": 5
}

2. Build eight scene/script setups

Render simple homes from colored cuboids, planes, and recognizable objects. Use a deterministic scene description, projected geometry, and a moving camera. Make the fireplace, window, island, and door visually distinct without labels, arrows, or target markers.

Each setup has three variants: clean, an obvious visible defect, and a defect wholly outside the intended crop. That makes 24 core cases. Use duplicated openings, a disappearing appliance, a warped surface, or an object popping into existence. Keep ordinary perspective changes and occlusions in clean controls.

D1–D4 are development cases. H1–H4 and the four challenges are held out. A scene/script setup is not an independent home: the paired setups intentionally share footage.

3. Separate the two skills, then combine them

If A cannot produce a valid crop, record C as unresolved and skip its model request. For impossible-framing challenges, B uses an explicitly limited diagnostic crop rather than claiming a feasible oracle solution. Do not give the fidelity reviewer the focus model's explanation or the expected answer. Each request starts fresh. Exclude infeasible and insufficient-evidence challenges from ordinary acceptance metrics, and report their expected unresolved outcomes separately.

Keep the review path real

A live capability probe comes first: image delivery, strict output schema, effective model/provider, and usage capture. Record the returned provider. If routing controls are needed, verify that the adapter forwards them. Do not silently switch models or treat the earlier offline SDK check as a live Gemini result.

Make frozen endings a specific regression case

For the tail challenge, extend four seconds of footage to five using the real cloned-frame normalization path. Compare the current pan, a fixed crop, and a proposed constrained motion path. The fixed crop is the deliberate freeze control.

For the candidate path, keep the full target box plus a 5% crop-width margin inside every crop. Continue one smooth motion direction with continuous position and velocity. Limit tail translation to 2% of crop width in total and 2% per second; allow at most 2% extra zoom only if the target and margin still fit. If no feasible moving path exists, return cannot_frame_tail. Do not expose a frozen tail as a successful result.

Decode every frame through the last 0.75-second fade. Check known object-edge displacement and target retention on frames with meaningful luminance; brightness changes alone cannot satisfy motion. Record fully black frames separately. Review the final crop samples, including a sample where the planted defect enters during the tail. These numerical bounds are proposals to evaluate visually.

Run it in four steps

1. Build and inspect the fixtures. Validate mock planner JSON against current types. Render sources, clips, masks, and oracle crops. Inspect a contact sheet and play each variant. Confirm that every core defect is unambiguous in the evidence and that each “outside” defect stays outside the complete oracle path.
2. Develop on 12 cases. D1–D4 × three variants. Run A, B, and C once. Fix basic prompt or coordinate mistakes here, then freeze fixtures, prompts, schemas, settings, and policy before opening the held-out results.
3. Evaluate 16 held-out cases twice. H1–H4 × three variants, plus four challenges. Run A, B, and C independently in each repeat. Keep repeated predictions visible; do not select the better result.
4. Review the gallery and choose the next move. Inspect every disagreement and compare model crops to the baseline. A successful synthetic test earns a cached real-scene evaluation. Weak focus and weak review can be improved independently.

Mock repair is a recorded repair_requested event with the finding and proposed positive correction text. In an optional controller smoke check, it selects a prebuilt clean counterpart and returns through focus and review. Those calls consume the reserve. They test the branch wiring, not whether a video model can follow the correction.

The model page currently lists standard input/output rates of $0.50/$3 per million tokens. Provider routes differ. Verify the actual route and pricing at execution; do not multiply request count by a guessed image price. Pricing reference

What would make this worth continuing?

Treat these as a strict smoke-test gate on deliberately clear examples. Show raw counts and both repeats. This small correlated set cannot establish production accuracy or calibrate a “75% confidence” threshold.

This test can tell us whether the reviewer follows the script and handles controlled visual evidence. Photorealism, subtle architectural errors, video-generation repair quality, real narration alignment, and production Trigger/Resume behavior require later tests.