feat: add auto-follow zoom mode with cursor tracking#257
feat: add auto-follow zoom mode with cursor tracking#257xKeCo wants to merge 4 commits intosiddharthvaddem:mainfrom
Conversation
📝 WalkthroughWalkthroughThis PR introduces an auto-focus mode for zoom regions that automatically tracks cursor telemetry. Users can toggle between manual and auto focus modes in the settings panel. When auto mode is active, cursor position data drives the zoom focus instead of manual control, with smoothing applied to reduce jitter. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant SettingsPanel
participant VideoEditor
participant VideoPlayback
participant ZoomRegionUtils
participant CursorFollowUtils
User->>SettingsPanel: Select "auto" focus mode
SettingsPanel->>VideoEditor: onZoomFocusModeChange("auto")
VideoEditor->>VideoEditor: Update selectedZoom.focusMode = "auto"
Note over VideoPlayback: During playback
VideoPlayback->>ZoomRegionUtils: findDominantRegion(cursorTelemetry)
ZoomRegionUtils->>ZoomRegionUtils: Detect region.focusMode === "auto"
ZoomRegionUtils->>CursorFollowUtils: interpolateCursorAt(cursorTelemetry, timeMs)
CursorFollowUtils-->>ZoomRegionUtils: Return interpolated cursor position
ZoomRegionUtils->>ZoomRegionUtils: Use cursor pos as focus (override manual focus)
VideoPlayback->>VideoPlayback: Apply smoothing & deadzone<br/>(smoothCursorFocus with factor)
VideoPlayback->>VideoPlayback: Update zoom camera focus incrementally
VideoPlayback-->>User: Render frame with auto-focused zoom
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contains the zoom region configuration used in the PR demo video: two auto-follow zoom regions and one manual zoom region. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 163b12d6fc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| targetProgress = strength; | ||
|
|
||
| // Apply deadzone + time-based smoothing for auto-follow mode | ||
| if (region.focusMode === "auto" && !transition) { |
There was a problem hiding this comment.
When transition is present, this condition bypasses the auto-follow update path entirely, so smoothedAutoFocus stays at its pre-transition value. In exports with connected auto zoom regions, the first non-transition frame then smooths from stale focus and produces a visible camera lurch right after the pan. Updating/resetting the smoothed state during transition frames (or before the first post-transition frame) avoids this discontinuity.
Useful? React with 👍 / 👎.
| targetProgress = strength; | ||
|
|
||
| // Apply deadzone + smoothing for auto-follow mode | ||
| if (region.focusMode === "auto" && !transition) { |
There was a problem hiding this comment.
The preview path has the same transition gating, so during connected pans in auto mode the smoothed focus ref is not refreshed while the transition runs. When transition ends, the next tick reuses stale focus and briefly pulls toward an outdated cursor position, causing a visible jump in playback that disagrees with the intended continuous auto-follow behavior.
Useful? React with 👍 / 👎.
This reverts commit 5c66212.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (1)
src/components/video-editor/videoPlayback/cursorFollowUtils.ts (1)
49-54: Clamp smoothing factor to guard against future misuse.
factoris currently assumed valid; clamping to[0,1]avoids accidental overshoot if a caller ever passes an out-of-range value.Suggested defensive tweak
export function smoothCursorFocus(raw: ZoomFocus, prev: ZoomFocus, factor: number): ZoomFocus { + const clampedFactor = Math.min(1, Math.max(0, factor)); return { - cx: prev.cx + (raw.cx - prev.cx) * factor, - cy: prev.cy + (raw.cy - prev.cy) * factor, + cx: prev.cx + (raw.cx - prev.cx) * clampedFactor, + cy: prev.cy + (raw.cy - prev.cy) * clampedFactor, }; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/components/video-editor/videoPlayback/cursorFollowUtils.ts` around lines 49 - 54, smoothCursorFocus currently uses the caller-provided factor directly; clamp factor to the [0,1] range before computing the interpolated cx/cy to prevent overshoot if callers pass out-of-range values. In the smoothCursorFocus( raw: ZoomFocus, prev: ZoomFocus, factor: number ) function, compute a clampedFactor = Math.max(0, Math.min(1, factor)) (or equivalent) and use clampedFactor in the interpolation expressions for cx and cy so ZoomFocus results are always a proper interpolation.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/components/video-editor/VideoEditor.tsx`:
- Around line 1604-1610: The focus-mode selector is currently hidden whenever
cursorTelemetry is empty due to hasCursorTelemetry={cursorTelemetry.length > 0},
which blocks switching an existing region out of "auto"; update the visibility
logic so the selector remains visible if the selected region's focusMode is
"auto" (or any persisted value) even when cursorTelemetry is empty. Concretely,
change the condition that sets hasCursorTelemetry (used where
selectedZoomFocusMode, onZoomFocusModeChange, hasCursorTelemetry are passed) to
true when either cursorTelemetry.length > 0 OR the selectedZoomId corresponds to
a zoomRegions entry whose focusMode === "auto" (or simply when a selectedZoomId
exists and zoomRegions.find(z => z.id === selectedZoomId)?.focusMode is truthy),
so handleZoomFocusModeChange remains accessible for existing auto regions.
In `@src/components/video-editor/VideoPlayback.tsx`:
- Around line 390-391: The early return in VideoPlayback (the block checking
region.focusMode === "auto" before calling onSelectZoom(region.id)) prevents
focus dragging but leaves the overlay interactive; update the overlay logic so
when region.focusMode === "auto" the overlay element used for zoom/focus is
rendered or styled with pointer-events: none (e.g., toggle a CSS class or inline
style on the overlay element) so it doesn't intercept pointer events, or ensure
the early-return path also disables the overlay (via the same prop/state used to
render the overlay) rather than only returning before onSelectZoom.
- Around line 870-903: The smoothed auto-follow state
(smoothedAutoFocusRef.current) can persist across seeks/pauses/gaps because it’s
only updated inside the active auto-region branch; clear it whenever auto mode
is not actively driving the camera by setting smoothedAutoFocusRef.current =
null in the other cases: specifically when region.focusMode !== "auto" (already
present) and also when transition is truthy or when targetProgress < 0.999 and
not isZoomingIn (i.e., not actively zooming-in), so that smoothed state is reset
before the next auto zoom; locate and update the logic around region.focusMode,
transition, targetProgress, isZoomingIn and prevTargetProgressRef to implement
this.
In `@src/lib/exporter/frameRenderer.ts`:
- Around line 540-542: The smoothing delta is currently computed from the source
timestamp (timeMs) inside renderFrame, which makes smoothing depend on playback
speed; change the dtMs calculation to use the exporter output timestamp (the
output-frame timestamp provided to the exporter callback) instead of timeMs
while leaving timeMs usage for cursor/region lookup. Specifically, in
renderFrame replace the dtMs := this.prevAnimationTimeMs != null ? timeMs -
this.prevAnimationTimeMs : 0 assignment to compute dtMs :=
this.prevAnimationTimeMs != null ? outputTimestampMs - this.prevAnimationTimeMs
: 0 (keeping the same fallback), so framesElapsed and factor (using
AUTO_FOLLOW_SMOOTHING_FACTOR) are derived from the output-frame delta; update
any references to prevAnimationTimeMs/timeMs usage accordingly so only
cursor/region logic still uses the source time.
- Around line 537-572: The code fails to clear the auto-follow accumulator when
an auto region ends, causing future auto zooms to resume smoothing from stale
state; update the branch that handles non-auto regions (check region.focusMode
!== "auto") to reset both this.smoothedAutoFocus and this.prevTargetProgress
(e.g., set smoothedAutoFocus = null and prevTargetProgress =
null/undefined/initial value) so the next auto follow starts fresh; locate the
logic around region.focusMode, smoothedAutoFocus and prevTargetProgress in
frameRenderer.ts and add the reset there.
---
Nitpick comments:
In `@src/components/video-editor/videoPlayback/cursorFollowUtils.ts`:
- Around line 49-54: smoothCursorFocus currently uses the caller-provided factor
directly; clamp factor to the [0,1] range before computing the interpolated
cx/cy to prevent overshoot if callers pass out-of-range values. In the
smoothCursorFocus( raw: ZoomFocus, prev: ZoomFocus, factor: number ) function,
compute a clampedFactor = Math.max(0, Math.min(1, factor)) (or equivalent) and
use clampedFactor in the interpolation expressions for cx and cy so ZoomFocus
results are always a proper interpolation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 2d2c2bab-0e99-4028-83fa-7c233b3c8c86
📒 Files selected for processing (15)
src/components/video-editor/SettingsPanel.tsxsrc/components/video-editor/VideoEditor.tsxsrc/components/video-editor/VideoPlayback.tsxsrc/components/video-editor/projectPersistence.tssrc/components/video-editor/types.tssrc/components/video-editor/videoPlayback/constants.tssrc/components/video-editor/videoPlayback/cursorFollowUtils.tssrc/components/video-editor/videoPlayback/overlayUtils.tssrc/components/video-editor/videoPlayback/zoomRegionUtils.tssrc/i18n/locales/en/settings.jsonsrc/i18n/locales/es/settings.jsonsrc/i18n/locales/zh-CN/settings.jsonsrc/lib/exporter/frameRenderer.tssrc/lib/exporter/gifExporter.tssrc/lib/exporter/videoExporter.ts
| selectedZoomFocusMode={ | ||
| selectedZoomId | ||
| ? (zoomRegions.find((z) => z.id === selectedZoomId)?.focusMode ?? "manual") | ||
| : null | ||
| } | ||
| onZoomFocusModeChange={(mode) => selectedZoomId && handleZoomFocusModeChange(mode)} | ||
| hasCursorTelemetry={cursorTelemetry.length > 0} |
There was a problem hiding this comment.
Keep the focus-mode control visible for already-"auto" regions.
focusMode is persisted, but hasCursorTelemetry={cursorTelemetry.length > 0} hides the entire selector as soon as telemetry is unavailable. If a project is opened without its cursor samples, the region can stay in "auto" while manual focus dragging remains disabled, leaving no UI path back to "manual".
Suggested change
- hasCursorTelemetry={cursorTelemetry.length > 0}
+ hasCursorTelemetry={
+ cursorTelemetry.length > 0 ||
+ (selectedZoomId
+ ? zoomRegions.find((z) => z.id === selectedZoomId)?.focusMode === "auto"
+ : false)
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| selectedZoomFocusMode={ | |
| selectedZoomId | |
| ? (zoomRegions.find((z) => z.id === selectedZoomId)?.focusMode ?? "manual") | |
| : null | |
| } | |
| onZoomFocusModeChange={(mode) => selectedZoomId && handleZoomFocusModeChange(mode)} | |
| hasCursorTelemetry={cursorTelemetry.length > 0} | |
| selectedZoomFocusMode={ | |
| selectedZoomId | |
| ? (zoomRegions.find((z) => z.id === selectedZoomId)?.focusMode ?? "manual") | |
| : null | |
| } | |
| onZoomFocusModeChange={(mode) => selectedZoomId && handleZoomFocusModeChange(mode)} | |
| hasCursorTelemetry={ | |
| cursorTelemetry.length > 0 || | |
| (selectedZoomId | |
| ? zoomRegions.find((z) => z.id === selectedZoomId)?.focusMode === "auto" | |
| : false) | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/components/video-editor/VideoEditor.tsx` around lines 1604 - 1610, The
focus-mode selector is currently hidden whenever cursorTelemetry is empty due to
hasCursorTelemetry={cursorTelemetry.length > 0}, which blocks switching an
existing region out of "auto"; update the visibility logic so the selector
remains visible if the selected region's focusMode is "auto" (or any persisted
value) even when cursorTelemetry is empty. Concretely, change the condition that
sets hasCursorTelemetry (used where selectedZoomFocusMode,
onZoomFocusModeChange, hasCursorTelemetry are passed) to true when either
cursorTelemetry.length > 0 OR the selectedZoomId corresponds to a zoomRegions
entry whose focusMode === "auto" (or simply when a selectedZoomId exists and
zoomRegions.find(z => z.id === selectedZoomId)?.focusMode is truthy), so
handleZoomFocusModeChange remains accessible for existing auto regions.
| if (region.focusMode === "auto") return; | ||
| onSelectZoom(region.id); |
There was a problem hiding this comment.
Don't leave the hidden auto overlay interactive.
This early return stops focus dragging, but the overlay still sits above the stage with pointer events enabled whenever a zoom is selected. In auto mode that invisible layer will eat webcam drags and other underlying pointer interactions unless the overlay-effect logic also switches it to pointer-events: none.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/components/video-editor/VideoPlayback.tsx` around lines 390 - 391, The
early return in VideoPlayback (the block checking region.focusMode === "auto"
before calling onSelectZoom(region.id)) prevents focus dragging but leaves the
overlay interactive; update the overlay logic so when region.focusMode ===
"auto" the overlay element used for zoom/focus is rendered or styled with
pointer-events: none (e.g., toggle a CSS class or inline style on the overlay
element) so it doesn't intercept pointer events, or ensure the early-return path
also disables the overlay (via the same prop/state used to render the overlay)
rather than only returning before onSelectZoom.
| // Apply deadzone + smoothing for auto-follow mode | ||
| if (region.focusMode === "auto" && !transition) { | ||
| const raw = targetFocus; | ||
| const isZoomingIn = | ||
| targetProgress < 0.999 && targetProgress >= prevTargetProgressRef.current; | ||
| if (targetProgress >= 0.999) { | ||
| // Full zoom: apply deadzone + smoothing for stable follow | ||
| const prev = smoothedAutoFocusRef.current ?? raw; | ||
| const dx = Math.abs(raw.cx - prev.cx); | ||
| const dy = Math.abs(raw.cy - prev.cy); | ||
| if (dx > AUTO_FOLLOW_DEADZONE || dy > AUTO_FOLLOW_DEADZONE) { | ||
| const smoothed = smoothCursorFocus(raw, prev, AUTO_FOLLOW_SMOOTHING_FACTOR); | ||
| smoothedAutoFocusRef.current = smoothed; | ||
| targetFocus = smoothed; | ||
| } else { | ||
| smoothedAutoFocusRef.current = prev; | ||
| targetFocus = prev; | ||
| } | ||
| } else if (isZoomingIn) { | ||
| // Zoom-in: track cursor directly so zoom always aims at current cursor | ||
| // position; keep ref in sync to avoid snap when full-zoom begins | ||
| smoothedAutoFocusRef.current = raw; | ||
| } else { | ||
| // Zoom-out: keep smoothing for continuity — avoids snap at zoom-out start | ||
| const prev = smoothedAutoFocusRef.current ?? raw; | ||
| const smoothed = smoothCursorFocus(raw, prev, AUTO_FOLLOW_SMOOTHING_FACTOR); | ||
| smoothedAutoFocusRef.current = smoothed; | ||
| targetFocus = smoothed; | ||
| } | ||
| } else if (region.focusMode !== "auto") { | ||
| smoothedAutoFocusRef.current = null; | ||
| } | ||
| prevTargetProgressRef.current = targetProgress; | ||
|
|
There was a problem hiding this comment.
Clear follow smoothing whenever auto mode is not actively driving the camera.
These refs are only updated inside the active-region branch. After a seek, a pause in the unzoomed view, or a gap between auto zooms, the next zoom-in can reuse the previous region's smoothed focus and drift from the wrong spot.
Suggested change
- if (region && strength > 0 && !shouldShowUnzoomedView) {
+ if (!region || strength <= 0 || shouldShowUnzoomedView) {
+ smoothedAutoFocusRef.current = null;
+ prevTargetProgressRef.current = 0;
+ }
+
+ if (region && strength > 0 && !shouldShowUnzoomedView) {
const zoomScale = blendedScale ?? ZOOM_DEPTH_SCALES[region.depth];
const regionFocus = region.focus;
@@
- } else if (region.focusMode !== "auto") {
+ } else if (region.focusMode !== "auto") {
smoothedAutoFocusRef.current = null;
+ prevTargetProgressRef.current = 0;
}
prevTargetProgressRef.current = targetProgress;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/components/video-editor/VideoPlayback.tsx` around lines 870 - 903, The
smoothed auto-follow state (smoothedAutoFocusRef.current) can persist across
seeks/pauses/gaps because it’s only updated inside the active auto-region
branch; clear it whenever auto mode is not actively driving the camera by
setting smoothedAutoFocusRef.current = null in the other cases: specifically
when region.focusMode !== "auto" (already present) and also when transition is
truthy or when targetProgress < 0.999 and not isZoomingIn (i.e., not actively
zooming-in), so that smoothed state is reset before the next auto zoom; locate
and update the logic around region.focusMode, transition, targetProgress,
isZoomingIn and prevTargetProgressRef to implement this.
| // Apply deadzone + time-based smoothing for auto-follow mode | ||
| if (region.focusMode === "auto" && !transition) { | ||
| const raw = targetFocus; | ||
| const dtMs = this.prevAnimationTimeMs != null ? timeMs - this.prevAnimationTimeMs : 0; | ||
| const framesElapsed = dtMs > 0 ? dtMs / (1000 / 60) : 1; | ||
| const factor = 1 - Math.pow(1 - AUTO_FOLLOW_SMOOTHING_FACTOR, Math.max(1, framesElapsed)); | ||
| const isZoomingIn = targetProgress < 0.999 && targetProgress >= this.prevTargetProgress; | ||
| if (targetProgress >= 0.999) { | ||
| // Full zoom: apply deadzone + smoothing for stable follow | ||
| const prev = this.smoothedAutoFocus ?? raw; | ||
| const dx = Math.abs(raw.cx - prev.cx); | ||
| const dy = Math.abs(raw.cy - prev.cy); | ||
| if (dx > AUTO_FOLLOW_DEADZONE || dy > AUTO_FOLLOW_DEADZONE) { | ||
| const smoothed = smoothCursorFocus(raw, prev, factor); | ||
| this.smoothedAutoFocus = smoothed; | ||
| targetFocus = smoothed; | ||
| } else { | ||
| this.smoothedAutoFocus = prev; | ||
| targetFocus = prev; | ||
| } | ||
| } else if (isZoomingIn) { | ||
| // Zoom-in: track cursor directly so zoom always aims at current cursor | ||
| // position; keep ref in sync to avoid snap when full-zoom begins | ||
| this.smoothedAutoFocus = raw; | ||
| } else { | ||
| // Zoom-out: keep smoothing for continuity — avoids snap at zoom-out start | ||
| const prev = this.smoothedAutoFocus ?? raw; | ||
| const smoothed = smoothCursorFocus(raw, prev, factor); | ||
| this.smoothedAutoFocus = smoothed; | ||
| targetFocus = smoothed; | ||
| } | ||
| } else if (region.focusMode !== "auto") { | ||
| this.smoothedAutoFocus = null; | ||
| } | ||
| this.prevTargetProgress = targetProgress; | ||
|
|
There was a problem hiding this comment.
Reset the auto-follow accumulator when no zoom region is active.
smoothedAutoFocus and prevTargetProgress are only cleared for non-auto regions. After an auto region ends, the next auto zoom after a gap can be treated as a continuation of the previous one and start smoothing from stale coordinates.
Suggested change
- if (region && strength > 0) {
+ if (!region || strength <= 0) {
+ this.smoothedAutoFocus = null;
+ this.prevTargetProgress = 0;
+ }
+
+ if (region && strength > 0) {
const zoomScale = blendedScale ?? ZOOM_DEPTH_SCALES[region.depth];
const regionFocus = this.clampFocusToStage(region.focus, region.depth);
@@
- } else if (region.focusMode !== "auto") {
+ } else if (region.focusMode !== "auto") {
this.smoothedAutoFocus = null;
+ this.prevTargetProgress = 0;
}
this.prevTargetProgress = targetProgress;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/lib/exporter/frameRenderer.ts` around lines 537 - 572, The code fails to
clear the auto-follow accumulator when an auto region ends, causing future auto
zooms to resume smoothing from stale state; update the branch that handles
non-auto regions (check region.focusMode !== "auto") to reset both
this.smoothedAutoFocus and this.prevTargetProgress (e.g., set smoothedAutoFocus
= null and prevTargetProgress = null/undefined/initial value) so the next auto
follow starts fresh; locate the logic around region.focusMode, smoothedAutoFocus
and prevTargetProgress in frameRenderer.ts and add the reset there.
| const dtMs = this.prevAnimationTimeMs != null ? timeMs - this.prevAnimationTimeMs : 0; | ||
| const framesElapsed = dtMs > 0 ? dtMs / (1000 / 60) : 1; | ||
| const factor = 1 - Math.pow(1 - AUTO_FOLLOW_SMOOTHING_FACTOR, Math.max(1, framesElapsed)); |
There was a problem hiding this comment.
Use output-frame delta for smoothing, not source-media delta.
dtMs here is derived from the source timestamp passed into renderFrame(). Inside speed regions that makes export smoothing depend on playback speed, so auto-follow becomes snappier/slower than the live preview. The exporter callbacks already have an output timestamp available; use that delta for the smoothing factor and keep source time only for cursor/region lookup.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/lib/exporter/frameRenderer.ts` around lines 540 - 542, The smoothing
delta is currently computed from the source timestamp (timeMs) inside
renderFrame, which makes smoothing depend on playback speed; change the dtMs
calculation to use the exporter output timestamp (the output-frame timestamp
provided to the exporter callback) instead of timeMs while leaving timeMs usage
for cursor/region lookup. Specifically, in renderFrame replace the dtMs :=
this.prevAnimationTimeMs != null ? timeMs - this.prevAnimationTimeMs : 0
assignment to compute dtMs := this.prevAnimationTimeMs != null ?
outputTimestampMs - this.prevAnimationTimeMs : 0 (keeping the same fallback), so
framesElapsed and factor (using AUTO_FOLLOW_SMOOTHING_FACTOR) are derived from
the output-frame delta; update any references to prevAnimationTimeMs/timeMs
usage accordingly so only cursor/region logic still uses the source time.
|
@xKeCo can you explain how this is any different from the existing auto zoom capabilities that you can apply based on the cursor telemetry and magic want button. |
@siddharthvaddem Hey! Good question — they actually solve different problems and complement each other. The existing cursor telemetry feature (Magic Wand) looks at your recording after the fact, finds moments where the cursor was relatively still, and creates zoom regions with a fixed focus point based on the average cursor position during those dwell periods. It's essentially a smart shortcut for generating zoom regions quickly. What this PR adds is a Focus Mode toggle (Manual / Auto) within an existing zoom region. Instead of locking the camera to a static point, Auto mode tracks the cursor's position frame by frame during playback — with smoothing and a small deadzone to avoid jitter — and the same logic is applied identically when exporting. Here's a side-by-side comparison to make it more concrete: Without Auto-Follow (Magic Wand only): the zoom regions are created based on where the cursor was dwelling, but the focus point stays static. If the cursor moves significantly during a zoom, the active area drifts out of frame. Screen.Recording.2026-04-02.at.11.13.52.AM.mp4With Auto-Follow: the camera continuously pans with the cursor throughout the zoom region, keeping the active area centered the whole time. Screen.Recording.2026-04-02.at.11.07.57.AM.mp4So to summarize: Magic Wand helps you create zoom regions based on cursor activity. Auto-Follow makes the camera dynamic inside those regions rather than staying locked to a single point. You can even use both together — let the Magic Wand generate the regions, then toggle Auto on the ones where the cursor moves a lot. |
|
okay this is cool and makes sense on a high level. This needs some minor tweaks as when we have sudden movements - the focus is so rigid - it feels super snappy. Happy to merge once you have this addressed. |
|
This is a nice improvement ! Well done |
|
That’s an interesting feature! I’d love to see it implemented. Thanks for making this, but I’ve noticed an issue in vertical mode. The space between a window and the container at the top and bottom is excessively large. This happens when the cursor is positioned at the top or bottom of the window. I expect the window to remain at the top of the container without much spacing, and the same should apply to the bottom. Here’s the result from your current branch: |
4 participants
Summary
FrameRendererDemo
export-1775028836571.mp4
Example project file: auto-follow-zoom-example.openscreen — load this in Openscreen to reproduce the demo (update
screenVideoPathto point to your own recording)How it works
Playback and export share the same 3-phase camera logic per zoom region:
progress < 1and increasingprogress ≥ 0.999progress < 1and decreasingExport uses a time-based smoothing factor (
1 - (1 - BASE)^(dt / 16.67ms)) so the smoothing is frame-rate-independent regardless of export resolution or speed regions.Changes
New file
src/components/video-editor/videoPlayback/cursorFollowUtils.tsinterpolateCursorAt— binary search + linear interpolation on cursor telemetry arraysmoothCursorFocus— exponential smoothing helperModified
types.ts—ZoomFocusMode = "manual" | "auto", added optionalfocusModetoZoomRegionconstants.ts—AUTO_FOLLOW_SMOOTHING_FACTOR = 0.05,AUTO_FOLLOW_DEADZONE = 0.06zoomRegionUtils.ts— threadscursorTelemetrythroughfindDominantRegion→getResolvedFocus; pre-computes cursor position once per connected-zoom transition pair to avoid a duplicate binary searchoverlayUtils.ts— hides the indicator overlay in Auto modeVideoPlayback.tsx— 3-phase smoothing in the PixiJS ticker;cursorTelemetryprop wired inVideoEditor.tsx—handleZoomFocusModeChange, forwardscursorTelemetryto both exportersSettingsPanel.tsx— Focus Mode button group (rendered only when cursor telemetry is available)projectPersistence.ts— normalizesfocusModeon project loadframeRenderer.ts— mirrors the playback smoothing with time-based factor scalingvideoExporter.ts/gifExporter.ts— forwardcursorTelemetrytoFrameRendereri18n/locales/{en,es,zh-CN}/settings.json— Focus Mode translations (nested object to fix key traversal)