- Gate buildFfmpegJni + jniLibs packaging on `ux: enable_ffmpeg` in the
consuming app's pubspec (default off) -- no LGPL / H.264-patent
exposure unless explicitly enabled
- appInfoBuilder generates kUxEnableFfmpeg from the same flag so apps
register the FFmpeg LGPL notice eagerly, pubspec-only (no dart-define)
- Add registerFfmpegLicense() + bundled LGPL-2.1 text asset
- FFmpeg compliance docs (LICENSES-3RDPARTY.md, android/ffmpeg/README.md)
- Network video streaming: XVideoPlayerController.network
Closes H1 from the pre-ship review (the known-limit doc note added in
c0d55ba). The previous workaround was "first play-through truncates
the last ~16 frames; replay is fine because flush_buffers clears
libavcodec." That trade-off was OK for shipping but the proper fix
is to drain the reorder buffer before propagating EOS to ExoPlayer.
Media3's SimpleDecoder short-circuits the end-of-stream input buffer
and never invokes the subclass's decode(), so there's no hook to send
avcodec_send_packet(NULL). Every override worth overriding (decode
loop, queue methods, flush) is final on SimpleDecoder. So we
vendor a copy as FfmpegSimpleDecoder (Apache 2.0 attribution at the
top of the file) with one structural change: an EOS-drain state. On
EOS input, signalEndOfInput() flushes libavcodec's reorder queue,
then drainAtEndOfStream() is called on successive output buffers
until it reports DRAIN_DONE — at which point the loop attaches
BUFFER_FLAG_END_OF_STREAM and resumes normal teardown.
Everything else mirrors SimpleDecoder verbatim so upstream
improvements are cheap to pull forward.
- FfmpegSimpleDecoder.java: vendored base class.
- ffmpegVideoSignalEos JNI: sends avcodec_send_packet(NULL).
- FfmpegVideoDecoder: extends the new base; signalEndOfInput
forwards to the JNI; drainAtEndOfStream re-uses the existing
ffmpegVideoReceiveFrame so per-frame PTS recovery and the
pending_frame path from c0d55ba continue to work during drain.
Six prod-blocking issues and three correctness improvements from an
independent code review of 7243ef7. Verified on Huawei Mate 20 (EMUI
11) — playback, rotation, replay-after-end all still work.
- EAGAIN on avcodec_send_packet was silently dropping the input
packet (SimpleDecoder consumed it before we could retry).
ffmpeg_jni.cc now caches a frame drained from the output queue
into pending_frame, retries the send, and the next
ffmpegVideoReceiveFrame emits the cached frame in order before
pulling a new one.
- C.TIME_UNSET == Long.MIN_VALUE == AV_NOPTS_VALUE was an
undocumented coincidence between two upstreams. Gate it
explicitly so a future Media3 sentinel change can't scramble
display-order PTS recovery.
- supportsFormat parses the H.264 profile from format.codecs and
rejects non-8-bit profiles (High 10 / High 4:2:2 / High 4:4:4).
These initialise libavcodec cleanly and only fail at the first
receive — too late for ExoPlayer to fall through to MediaCodec.
Rejecting upfront lets the platform decoder pick them up.
- build_ffmpeg.sh wraps the whole run in a portable mkdir-based
lock and clones into a staging dir + atomic rename with a
sentinel file. Concurrent Gradle daemons no longer corrupt
each other; an interrupted clone leaves no usable state for
the next run to mistake as finished.
- FfmpegOutputSurface and VideoCompositor both used to call
eglTerminate(EGL_DEFAULT_DISPLAY) on teardown. That display is
process-global and shared — the first teardown killed the
other consumer's surface. Drop both calls; per-context cleanup
+ eglReleaseThread is sufficient. Likely cause of any "frozen
surface after second video" report.
- Rotation swap in renderOutputBuffer mutates the public
outputBuffer.width/height. Bound it to SURFACE_YUV output mode
via a currentOutputMode tracker; YUV-mode consumers
(VideoDecoderOutputBufferRenderer.setOutputBuffer) read
width/height expecting CODED dims that match yuvStrides[0] —
the swap would walk chroma off the end of the allocation.
- Fragment shader bumped from mediump to highp. The limited-range
pre-scale (y - 16/255) * (255/219) was at risk of quantizing
through 10-bit mediump and banding dark gradients on older
Mali / Adreno parts. highp on the fragment is universally
supported on GLES2 implementations Android ships post-2014.
- Threading config comment was wrong about what FF_THREAD_SLICE
does for H.264. Replace with the accurate explanation (slice
threading degenerates to single-threaded on iOS's single-slice
encodes; FRAME threading is rejected because of the input-side
latency, not because libavcodec doesn't support it).
- FfmpegVideoDecoder header documents two known limits the
review surfaced but that don't have a clean fix at this layer:
EOS tail-frame loss (~500 ms truncation on first play-through
only; replay is fine because flush_buffers clears libavcodec)
and the size-based colorspace heuristic mislabelling iPhone
6/7-era unspecified-metadata BT.601 1080p clips as BT.709.
Adds an LGPL FFmpeg-backed video renderer that slots ahead of Media3's
MediaCodecVideoRenderer via EXTENSION_RENDERER_MODE_PREFER. Resolves
playback failures on Huawei EMUI 11 (Mate 20, Kirin 980): the Codec2
HiSilicon AVC decoder initialises cleanly on iOS High@3.1 streams with
deep DPB + full-range yuvj420p, then errors on the first sample inside
MediaCodecVideoRenderer (init-failure fallback can't catch this).
Google's C2 SW AVC decoder hits its 8-frame output-delay cap on the
same shape and stalls on dequeueOutputBuffer.
Media3's own decoder-ffmpeg ships only an audio renderer;
ExperimentalFfmpegVideoRenderer has been a stub since 2020 (returns
FORMAT_UNSUPPORTED_TYPE, createDecoder returns null). NextLib is
GPL-3.0. So we vendor our own Apache-licensed JNI on top of LGPL
FFmpeg, dynamically linked at runtime.
Build flow:
- android/ffmpeg/ holds the JNI source + CMakeLists + orchestrator
script + LGPL notice. No native binaries in git.
- :ux:buildFfmpegJni Gradle task (wired to preBuild) clones
Media3 1.9.2 + FFmpeg release/6.0 into build/ffmpeg-work/ on
first run, builds h264-only static libs per ABI, links
libffmpegJNI.so per ABI into build/jniLibs/<abi>/. AGP picks
them up via sourceSets.main.jniLibs.srcDirs +=. Gradle
UP-TO-DATE skips the task when ffmpeg_jni.cc / CMakeLists /
build_ffmpeg.sh are unchanged.
Renderer:
- FfmpegVideoDecoder (SimpleDecoder) sends each packet with its
inputBuffer.timeUs as pkt->pts; the JNI overwrites
outputBuffer.timeUs with f->pts on receive so frames emitted in
display order carry their true display PTS (input PTS in decode
order scrambles ExoPlayer's drop logic and halves the render
rate on B-frame streams).
- FfmpegOutputSurface does YUV->RGB in one GLES2 pass against an
EGL window surface sized to display orientation. Y plane uses
GL_NEAREST (1:1 sized, sampling at exact texel centres
preserves luma detail); chroma uses GL_LINEAR. Pre-rotated quad
UVs (0/90/180/270) keep the YUV sampling correct when the
coded frame needs rotation for display.
- FfmpegVideoRenderer swaps the output buffer's width/height for
90/270 streams before super.renderOutputBuffer notifies size,
matching MediaCodecVideoRenderer's post-rotation reporting.
Decoder fallback:
- Renderers.kt selects FfmpegVideoRenderer first when
libffmpegJNI.so is loaded; falls through to the platform path
for formats FFmpeg doesn't handle or ABIs without the .so.
- MediaCodec selector deprioritises every HiSilicon decoder
(OMX.hisi.* and c2.hisi.*) so the platform path picks
c2.android.avc.decoder ahead of the C2 Hisi variant when FFmpeg
isn't available. Required because the C2 Hisi failure is
post-init, which Media3's setEnableDecoderFallback(true) can't
intercept.
Compositor:
- VideoCompositor.setInputSurfaceSize lets the renderer resize the
codec-input SurfaceTexture before eglCreateWindowSurface so the
EGL surface inherits matching buffer dimensions on creation
(MediaCodec sizes natively; EGL doesn't).
- VideoPlayerInstance wires Renderers.build with a sizer callback
that calls into compositor.setInputSurfaceSize from the FFmpeg
renderer thread.
Adds docs/architecture.md with the layered video pipeline diagram,
file map, renderer-selection rationale, build flow, and LGPL
boundary notes.
