Skip to content

Add optional Yandex voice-over translation (EN to RU) for player#5817

Open
mikhkz wants to merge 1 commit into
yuliskov:masterfrom
mikhkz:feature/yandex-voice-over-translation
Open

Add optional Yandex voice-over translation (EN to RU) for player#5817
mikhkz wants to merge 1 commit into
yuliskov:masterfrom
mikhkz:feature/yandex-voice-over-translation

Conversation

@mikhkz

@mikhkz mikhkz commented May 25, 2026

Copy link
Copy Markdown

Summary

Adds an optional Yandex Voice-Over Translation (VOT) feature for Android TV: translate the current YouTube video's audio to Russian while watching (similar to the browser extension workflow).

  • Player overlay button: Voice translation (RU) with OFF / waiting / ON states
  • Second ExoPlayer for the translated MP3 stream, synced with main playback (play/pause/seek, speed)
  • Mix controls: original + translation volume (settings + long-press on the button)
  • Auto-translate for non-Russian audio (skips Russian originals and YouTube ru (dubbed-auto))
  • Optional YouTube auto-dub track instead of Yandex when available
  • VotAudioTrackHelper parses YouTube audio labels (en-US (original), legacy bare [en], etc.)
  • Settings: Player -> Voice translation (Yandex) - OAuth token (lively voice), auto-translate, volumes
  • Handles AUDIO_REQUESTED via the same fallback as vot.js (fail-audio-js + empty audio upload)

Prior work (this port is based on)

Implementation ideas and API behavior are derived from the community Voice Over Translation projects:

Thank you to the authors and contributors of those repositories. This PR adds a TV-native integration for SmartTube; it is not affiliated with Yandex or the extension authors.

Motivation

Many SmartTube users watch English content on TV without a desktop browser. A built-in VOT toggle improves accessibility and matches a popular browser extension use case.

Implementation notes

  • Package common/.../vot/ - protobuf client for api.browser.yandex.ru (OkHttp), ported from vot-cli / voice-over-translation ideas
  • VoiceTranslateController - UI state, auto-translate, track selection, VotClient + TranslationAudioPlayer
  • ActionHelpers.getBitmapDrawable() - vector drawable support in TwoStateAction (fixes ClassCastException for overlay actions)
  • Button is off by default; enable in Configure player buttons

How to test

  1. Build assembleStstableDebug
  2. Open an English YouTube video (including legacy single [en] audio tracks)
  3. Enable Voice translation in player button settings if hidden
  4. Tap the button - wait for ETA - hear RU voice-over with ducked original
  5. Long-press - auto-translate / YouTube dub / volume mix
  6. Open a Russian original video - auto-translate should not start; manual shows skip message
  7. (Optional) Set Yandex OAuth token for lively voice

Limitations / risks

  • Unofficial Yandex API - may break or rate-limit without notice
  • New videos may take a long time (AUDIO_REQUESTED uses fallback, not full YouTube audio upload)
  • Video length API limits (~4 hours)
  • Ducking is main player volume multiplier, not DSP sidechain
  • OAuth token stored locally in app prefs (user-provided)

Checklist

  • Builds with assembleStstableDebug
  • Feature is opt-in (button + settings)
  • No new product flavors / package id changes in this PR
  • Strings in values and values-ru

Based on voice-over-translation, vot-cli, and vot.js (see CREDITS.md).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant