7243ef7de4
Adds an LGPL FFmpeg-backed video renderer that slots ahead of Media3's
MediaCodecVideoRenderer via EXTENSION_RENDERER_MODE_PREFER. Resolves
playback failures on Huawei EMUI 11 (Mate 20, Kirin 980): the Codec2
HiSilicon AVC decoder initialises cleanly on iOS High@3.1 streams with
deep DPB + full-range yuvj420p, then errors on the first sample inside
MediaCodecVideoRenderer (init-failure fallback can't catch this).
Google's C2 SW AVC decoder hits its 8-frame output-delay cap on the
same shape and stalls on dequeueOutputBuffer.
Media3's own decoder-ffmpeg ships only an audio renderer;
ExperimentalFfmpegVideoRenderer has been a stub since 2020 (returns
FORMAT_UNSUPPORTED_TYPE, createDecoder returns null). NextLib is
GPL-3.0. So we vendor our own Apache-licensed JNI on top of LGPL
FFmpeg, dynamically linked at runtime.
Build flow:
- android/ffmpeg/ holds the JNI source + CMakeLists + orchestrator
script + LGPL notice. No native binaries in git.
- :ux:buildFfmpegJni Gradle task (wired to preBuild) clones
Media3 1.9.2 + FFmpeg release/6.0 into build/ffmpeg-work/ on
first run, builds h264-only static libs per ABI, links
libffmpegJNI.so per ABI into build/jniLibs/<abi>/. AGP picks
them up via sourceSets.main.jniLibs.srcDirs +=. Gradle
UP-TO-DATE skips the task when ffmpeg_jni.cc / CMakeLists /
build_ffmpeg.sh are unchanged.
Renderer:
- FfmpegVideoDecoder (SimpleDecoder) sends each packet with its
inputBuffer.timeUs as pkt->pts; the JNI overwrites
outputBuffer.timeUs with f->pts on receive so frames emitted in
display order carry their true display PTS (input PTS in decode
order scrambles ExoPlayer's drop logic and halves the render
rate on B-frame streams).
- FfmpegOutputSurface does YUV->RGB in one GLES2 pass against an
EGL window surface sized to display orientation. Y plane uses
GL_NEAREST (1:1 sized, sampling at exact texel centres
preserves luma detail); chroma uses GL_LINEAR. Pre-rotated quad
UVs (0/90/180/270) keep the YUV sampling correct when the
coded frame needs rotation for display.
- FfmpegVideoRenderer swaps the output buffer's width/height for
90/270 streams before super.renderOutputBuffer notifies size,
matching MediaCodecVideoRenderer's post-rotation reporting.
Decoder fallback:
- Renderers.kt selects FfmpegVideoRenderer first when
libffmpegJNI.so is loaded; falls through to the platform path
for formats FFmpeg doesn't handle or ABIs without the .so.
- MediaCodec selector deprioritises every HiSilicon decoder
(OMX.hisi.* and c2.hisi.*) so the platform path picks
c2.android.avc.decoder ahead of the C2 Hisi variant when FFmpeg
isn't available. Required because the C2 Hisi failure is
post-init, which Media3's setEnableDecoderFallback(true) can't
intercept.
Compositor:
- VideoCompositor.setInputSurfaceSize lets the renderer resize the
codec-input SurfaceTexture before eglCreateWindowSurface so the
EGL surface inherits matching buffer dimensions on creation
(MediaCodec sizes natively; EGL doesn't).
- VideoPlayerInstance wires Renderers.build with a sizer callback
that calls into compositor.setInputSurfaceSize from the FFmpeg
renderer thread.
Adds docs/architecture.md with the layered video pipeline diagram,
file map, renderer-selection rationale, build flow, and LGPL
boundary notes.
108 lines
4.1 KiB
Bash
Executable File
108 lines
4.1 KiB
Bash
Executable File
#!/bin/bash
|
|
# Builds libffmpegJNI.so for all 4 Android ABIs from vendored JNI sources
|
|
# (ffmpeg_jni.cc + CMakeLists.txt in this directory) against upstream
|
|
# Media3 + FFmpeg cloned into WORK_DIR. Driven entirely by environment
|
|
# variables Gradle sets in the buildFfmpegJni task — no user-facing
|
|
# flags. Reruns are cheap once WORK_DIR is populated (Gradle UP-TO-DATE
|
|
# skips the script entirely when JNI sources haven't changed, and even
|
|
# when forced, the FFmpeg static libs are reused if present).
|
|
#
|
|
# Required env:
|
|
# JNI_SRC — this directory (vendored ffmpeg_jni.cc + CMakeLists.txt)
|
|
# NDK_PATH — Android NDK root
|
|
# CMAKE_PATH — directory containing the cmake + ninja binaries
|
|
# OUTPUT_DIR — where to drop the final libffmpegJNI.so per ABI
|
|
# WORK_DIR — scratch directory for upstream clones + intermediate
|
|
# build artefacts (lives under build/ so a clean wipes it)
|
|
#
|
|
# License: FFmpeg is LGPL v2.1. We link to it dynamically (consumers
|
|
# load libffmpegJNI.so at runtime), keeping the LGPL boundary intact and
|
|
# not imposing copyleft on the consuming app. The build below
|
|
# intentionally omits --enable-gpl and --enable-nonfree.
|
|
|
|
set -euo pipefail
|
|
|
|
: "${JNI_SRC:?JNI_SRC env var not set}"
|
|
: "${NDK_PATH:?NDK_PATH env var not set}"
|
|
: "${CMAKE_PATH:?CMAKE_PATH env var not set}"
|
|
: "${OUTPUT_DIR:?OUTPUT_DIR env var not set}"
|
|
: "${WORK_DIR:?WORK_DIR env var not set}"
|
|
|
|
MEDIA3_TAG="${MEDIA3_TAG:-1.9.2}"
|
|
FFMPEG_TAG="${FFMPEG_TAG:-release/6.0}"
|
|
ABIS="${ABIS:-armeabi-v7a arm64-v8a x86 x86_64}"
|
|
MIN_SDK="${MIN_SDK:-21}"
|
|
|
|
case "$(uname -s)" in
|
|
Darwin*) HOST_PLATFORM=darwin-x86_64 ;;
|
|
Linux*) HOST_PLATFORM=linux-x86_64 ;;
|
|
*) echo "Unsupported host: $(uname -s)" >&2; exit 1 ;;
|
|
esac
|
|
|
|
mkdir -p "$WORK_DIR" "$OUTPUT_DIR"
|
|
cd "$WORK_DIR"
|
|
|
|
# 1. Upstream sources — clone once, reuse on subsequent runs.
|
|
MEDIA3_DIR="$WORK_DIR/media3"
|
|
if [[ ! -d "$MEDIA3_DIR" ]]; then
|
|
echo "[ffmpeg-build] cloning Media3 @${MEDIA3_TAG}"
|
|
git clone --depth 1 --branch "$MEDIA3_TAG" \
|
|
https://github.com/androidx/media.git "$MEDIA3_DIR"
|
|
fi
|
|
|
|
FFMPEG_DIR="$MEDIA3_DIR/libraries/decoder_ffmpeg/src/main/jni/ffmpeg"
|
|
if [[ ! -d "$FFMPEG_DIR" ]]; then
|
|
echo "[ffmpeg-build] cloning FFmpeg @${FFMPEG_TAG}"
|
|
git clone --depth 1 --branch "$FFMPEG_TAG" \
|
|
https://git.ffmpeg.org/ffmpeg.git "$FFMPEG_DIR"
|
|
fi
|
|
|
|
# 2. Drop our extended JNI source + CMake config over the upstream copies
|
|
# so the build produces a video-capable libffmpegJNI.so.
|
|
JNI_BUILD_DIR="$MEDIA3_DIR/libraries/decoder_ffmpeg/src/main/jni"
|
|
cp "$JNI_SRC/ffmpeg_jni.cc" "$JNI_BUILD_DIR/ffmpeg_jni.cc"
|
|
cp "$JNI_SRC/CMakeLists.txt" "$JNI_BUILD_DIR/CMakeLists.txt"
|
|
|
|
# 3. Build FFmpeg static libs per ABI (H.264 decoder only). The
|
|
# sentinel below skips this step if all 4 ABIs already have the
|
|
# static libs from a previous run — important because the FFmpeg
|
|
# static build is the slow part (~30 min for 4 ABIs); subsequent
|
|
# Gradle runs just need to re-link libffmpegJNI.so (~5 sec / ABI).
|
|
MODULE_PATH="$MEDIA3_DIR/libraries/decoder_ffmpeg/src/main"
|
|
NEED_STATIC_BUILD=0
|
|
for ABI in $ABIS; do
|
|
if [[ ! -f "$JNI_BUILD_DIR/ffmpeg/android-libs/$ABI/libavcodec.a" ]]; then
|
|
NEED_STATIC_BUILD=1
|
|
break
|
|
fi
|
|
done
|
|
|
|
if [[ "$NEED_STATIC_BUILD" -eq 1 ]]; then
|
|
echo "[ffmpeg-build] building FFmpeg static libs (slow on first run)"
|
|
chmod +x "$JNI_BUILD_DIR/build_ffmpeg.sh"
|
|
"$JNI_BUILD_DIR/build_ffmpeg.sh" \
|
|
"$MODULE_PATH" "$NDK_PATH" "$HOST_PLATFORM" "$MIN_SDK" h264
|
|
else
|
|
echo "[ffmpeg-build] FFmpeg static libs already present, reusing"
|
|
fi
|
|
|
|
# 4. Cross-build libffmpegJNI.so per ABI via CMake + Ninja.
|
|
export PATH="$CMAKE_PATH:$PATH"
|
|
for ABI in $ABIS; do
|
|
ABI_BUILD_DIR="$WORK_DIR/jni-out/$ABI"
|
|
mkdir -p "$ABI_BUILD_DIR"
|
|
cmake -G Ninja \
|
|
-DCMAKE_TOOLCHAIN_FILE="$NDK_PATH/build/cmake/android.toolchain.cmake" \
|
|
-DANDROID_ABI="$ABI" \
|
|
-DANDROID_PLATFORM="android-$MIN_SDK" \
|
|
-DCMAKE_BUILD_TYPE=Release \
|
|
-S "$JNI_BUILD_DIR" -B "$ABI_BUILD_DIR" >/dev/null
|
|
ninja -C "$ABI_BUILD_DIR" >/dev/null
|
|
|
|
DEST="$OUTPUT_DIR/$ABI"
|
|
mkdir -p "$DEST"
|
|
cp "$ABI_BUILD_DIR/libffmpegJNI.so" "$DEST/libffmpegJNI.so"
|
|
done
|
|
|
|
echo "[ffmpeg-build] libffmpegJNI.so ready in $OUTPUT_DIR"
|