c0d55babf3
Six prod-blocking issues and three correctness improvements from an
independent code review of 7243ef7. Verified on Huawei Mate 20 (EMUI
11) — playback, rotation, replay-after-end all still work.
- EAGAIN on avcodec_send_packet was silently dropping the input
packet (SimpleDecoder consumed it before we could retry).
ffmpeg_jni.cc now caches a frame drained from the output queue
into pending_frame, retries the send, and the next
ffmpegVideoReceiveFrame emits the cached frame in order before
pulling a new one.
- C.TIME_UNSET == Long.MIN_VALUE == AV_NOPTS_VALUE was an
undocumented coincidence between two upstreams. Gate it
explicitly so a future Media3 sentinel change can't scramble
display-order PTS recovery.
- supportsFormat parses the H.264 profile from format.codecs and
rejects non-8-bit profiles (High 10 / High 4:2:2 / High 4:4:4).
These initialise libavcodec cleanly and only fail at the first
receive — too late for ExoPlayer to fall through to MediaCodec.
Rejecting upfront lets the platform decoder pick them up.
- build_ffmpeg.sh wraps the whole run in a portable mkdir-based
lock and clones into a staging dir + atomic rename with a
sentinel file. Concurrent Gradle daemons no longer corrupt
each other; an interrupted clone leaves no usable state for
the next run to mistake as finished.
- FfmpegOutputSurface and VideoCompositor both used to call
eglTerminate(EGL_DEFAULT_DISPLAY) on teardown. That display is
process-global and shared — the first teardown killed the
other consumer's surface. Drop both calls; per-context cleanup
+ eglReleaseThread is sufficient. Likely cause of any "frozen
surface after second video" report.
- Rotation swap in renderOutputBuffer mutates the public
outputBuffer.width/height. Bound it to SURFACE_YUV output mode
via a currentOutputMode tracker; YUV-mode consumers
(VideoDecoderOutputBufferRenderer.setOutputBuffer) read
width/height expecting CODED dims that match yuvStrides[0] —
the swap would walk chroma off the end of the allocation.
- Fragment shader bumped from mediump to highp. The limited-range
pre-scale (y - 16/255) * (255/219) was at risk of quantizing
through 10-bit mediump and banding dark gradients on older
Mali / Adreno parts. highp on the fragment is universally
supported on GLES2 implementations Android ships post-2014.
- Threading config comment was wrong about what FF_THREAD_SLICE
does for H.264. Replace with the accurate explanation (slice
threading degenerates to single-threaded on iOS's single-slice
encodes; FRAME threading is rejected because of the input-side
latency, not because libavcodec doesn't support it).
- FfmpegVideoDecoder header documents two known limits the
review surfaced but that don't have a clean fix at this layer:
EOS tail-frame loss (~500 ms truncation on first play-through
only; replay is fine because flush_buffers clears libavcodec)
and the size-based colorspace heuristic mislabelling iPhone
6/7-era unspecified-metadata BT.601 1080p clips as BT.709.
Vendored FFmpeg video decoder for ux
This directory contains the JNI source + Gradle build wiring for
libffmpegJNI.so, the LGPL-licensed FFmpeg shared library that backs
io.swipelab.ux.video.ffmpeg.FfmpegVideoRenderer. The renderer is
slotted ahead of MediaCodecVideoRenderer so iOS H.264 streams with
deep DPB (has_b_frames > 8) and full-range YUV play on devices where
the platform decoder fails (notably Huawei Mate 20 on EMUI 11).
How it builds
The native library is produced by the :ux:buildFfmpegJni Gradle task,
wired as a dependency of preBuild. On any consumer build
(flutter build apk, ./gradlew assembleRelease, IDE sync) the task
runs automatically; Gradle's UP-TO-DATE checking skips it when nothing
relevant changed.
What the task does:
- Clones upstream Media3 (
1.9.2) and FFmpeg (release/6.0) into<ux-android-build>/ffmpeg-work/if missing — once per checkout. - Drops the vendored
ffmpeg_jni.cc+CMakeLists.txtover the upstream Media3 copies so the build produces a video-capable JNI. - Builds FFmpeg static libs (
libavcodec,libavutil,libswresample) with H.264 decoder enabled forarmeabi-v7a,arm64-v8a,x86,x86_64. Slow part — first build only (~30 min on a typical x86 host, ~2 min on Apple Silicon). Static libs are cached inffmpeg-work/and reused on subsequent runs. - Cross-compiles
libffmpegJNI.soper ABI via CMake + Ninja and writes the result into<ux-android-build>/jniLibs/<abi>/. AGP picks them up via thejniLibs.srcDirs +=line in build.gradle and bundles them in the AAR.
What ships in git:
ffmpeg_jni.cc— JNI bridge exposing the five entry pointsFfmpegVideoDecoder.javacalls: initialize / sendPacket / receiveFrame / flush / release. Adapted from Media3's audio-only template; the audio path was dropped (MediaCodec AAC works everywhere we ship to).CMakeLists.txt— links the FFmpeg static libs intolibffmpegJNI.so.build_ffmpeg.sh— orchestrates the clone + static-lib build + JNI link. Env-driven, invoked only by the Gradle task — not intended to be run by hand.LICENSE-FFMPEG.txt— LGPL v2.1 text. Required attribution.
No .so files in git. Everything under <build>/jniLibs/ is
generated on demand; /build is already in .gitignore.
License
FFmpeg is LGPL v2.1. We link to it dynamically (consumer apps load
libffmpegJNI.so at runtime via System.loadLibrary), which keeps the
LGPL boundary intact and does not impose copyleft on the consuming
app. The build configuration in upstream Media3's build_ffmpeg.sh
intentionally omits --enable-gpl and --enable-nonfree to keep the
binaries LGPL-only. Do not add codecs that require GPL configuration
(e.g. x264) — that would taint the artifact.
Pinned versions
- Media3
1.9.2(matches the version used by the rest ofux). - FFmpeg
release/6.0(matches Media3's tested compatibility window). - Android NDK is resolved via
android.ndkDirectoryfrom AGP — the NDK version your Android project pins, normally r27 on a recent AGP. Older NDKs may fail the 16-KB page-size alignment check enforced for Android 15.
Bumping either version: edit the MEDIA3_TAG / FFMPEG_TAG
constants near the top of build_ffmpeg.sh, then ./gradlew :ux:buildFfmpegJni --rerun-tasks to force a rebuild.
Why we vendor this instead of using media3-decoder-ffmpeg
Media3's published FFmpeg extension is audio-only.
ExperimentalFfmpegVideoRenderer in the same library has been a stub
since 2020 — createDecoder() returns null, supportsFormat() returns
FORMAT_UNSUPPORTED_TYPE. The community alternative (NextLib) is
GPL-3.0, which would impose copyleft on consumers. So we built our own
on top of the same JNI skeleton.