7243ef7de4
Adds an LGPL FFmpeg-backed video renderer that slots ahead of Media3's
MediaCodecVideoRenderer via EXTENSION_RENDERER_MODE_PREFER. Resolves
playback failures on Huawei EMUI 11 (Mate 20, Kirin 980): the Codec2
HiSilicon AVC decoder initialises cleanly on iOS High@3.1 streams with
deep DPB + full-range yuvj420p, then errors on the first sample inside
MediaCodecVideoRenderer (init-failure fallback can't catch this).
Google's C2 SW AVC decoder hits its 8-frame output-delay cap on the
same shape and stalls on dequeueOutputBuffer.
Media3's own decoder-ffmpeg ships only an audio renderer;
ExperimentalFfmpegVideoRenderer has been a stub since 2020 (returns
FORMAT_UNSUPPORTED_TYPE, createDecoder returns null). NextLib is
GPL-3.0. So we vendor our own Apache-licensed JNI on top of LGPL
FFmpeg, dynamically linked at runtime.
Build flow:
- android/ffmpeg/ holds the JNI source + CMakeLists + orchestrator
script + LGPL notice. No native binaries in git.
- :ux:buildFfmpegJni Gradle task (wired to preBuild) clones
Media3 1.9.2 + FFmpeg release/6.0 into build/ffmpeg-work/ on
first run, builds h264-only static libs per ABI, links
libffmpegJNI.so per ABI into build/jniLibs/<abi>/. AGP picks
them up via sourceSets.main.jniLibs.srcDirs +=. Gradle
UP-TO-DATE skips the task when ffmpeg_jni.cc / CMakeLists /
build_ffmpeg.sh are unchanged.
Renderer:
- FfmpegVideoDecoder (SimpleDecoder) sends each packet with its
inputBuffer.timeUs as pkt->pts; the JNI overwrites
outputBuffer.timeUs with f->pts on receive so frames emitted in
display order carry their true display PTS (input PTS in decode
order scrambles ExoPlayer's drop logic and halves the render
rate on B-frame streams).
- FfmpegOutputSurface does YUV->RGB in one GLES2 pass against an
EGL window surface sized to display orientation. Y plane uses
GL_NEAREST (1:1 sized, sampling at exact texel centres
preserves luma detail); chroma uses GL_LINEAR. Pre-rotated quad
UVs (0/90/180/270) keep the YUV sampling correct when the
coded frame needs rotation for display.
- FfmpegVideoRenderer swaps the output buffer's width/height for
90/270 streams before super.renderOutputBuffer notifies size,
matching MediaCodecVideoRenderer's post-rotation reporting.
Decoder fallback:
- Renderers.kt selects FfmpegVideoRenderer first when
libffmpegJNI.so is loaded; falls through to the platform path
for formats FFmpeg doesn't handle or ABIs without the .so.
- MediaCodec selector deprioritises every HiSilicon decoder
(OMX.hisi.* and c2.hisi.*) so the platform path picks
c2.android.avc.decoder ahead of the C2 Hisi variant when FFmpeg
isn't available. Required because the C2 Hisi failure is
post-init, which Media3's setEnableDecoderFallback(true) can't
intercept.
Compositor:
- VideoCompositor.setInputSurfaceSize lets the renderer resize the
codec-input SurfaceTexture before eglCreateWindowSurface so the
EGL surface inherits matching buffer dimensions on creation
(MediaCodec sizes natively; EGL doesn't).
- VideoPlayerInstance wires Renderers.build with a sizer callback
that calls into compositor.setInputSurfaceSize from the FFmpeg
renderer thread.
Adds docs/architecture.md with the layered video pipeline diagram,
file map, renderer-selection rationale, build flow, and LGPL
boundary notes.
65 lines
2.0 KiB
CMake
65 lines
2.0 KiB
CMake
#
|
|
# Copyright 2021 The Android Open Source Project
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
#
|
|
|
|
# LINT.IfChange
|
|
cmake_minimum_required(VERSION 3.21.0 FATAL_ERROR)
|
|
# LINT.ThenChange(../../../build.gradle)
|
|
|
|
# Enable C++11 features.
|
|
set(CMAKE_CXX_STANDARD 11)
|
|
|
|
project(libffmpegJNI C CXX)
|
|
|
|
set(ffmpeg_location "${CMAKE_CURRENT_SOURCE_DIR}/ffmpeg")
|
|
set(ffmpeg_binaries "${ffmpeg_location}/android-libs/${ANDROID_ABI}")
|
|
|
|
foreach(ffmpeg_lib avutil swresample avcodec)
|
|
set(ffmpeg_lib_filename lib${ffmpeg_lib}.a)
|
|
set(ffmpeg_lib_file_path ${ffmpeg_binaries}/${ffmpeg_lib_filename})
|
|
add_library(
|
|
${ffmpeg_lib}
|
|
STATIC
|
|
IMPORTED)
|
|
set_target_properties(
|
|
${ffmpeg_lib} PROPERTIES
|
|
IMPORTED_LOCATION
|
|
${ffmpeg_lib_file_path})
|
|
endforeach()
|
|
|
|
include_directories(${ffmpeg_location})
|
|
find_library(android_log_lib log)
|
|
|
|
add_library(ffmpegJNI
|
|
SHARED
|
|
ffmpeg_jni.cc)
|
|
|
|
target_link_libraries(ffmpegJNI
|
|
PRIVATE android
|
|
PRIVATE swresample
|
|
PRIVATE avcodec
|
|
PRIVATE avutil
|
|
PRIVATE ${android_log_lib})
|
|
|
|
# Additional flags needed for "arm64-v8a" from NDK 23.1.7779620 and above.
|
|
# See https://github.com/google/ExoPlayer/issues/9933#issuecomment-1029775358.
|
|
if(ANDROID_ABI STREQUAL "arm64-v8a")
|
|
target_link_options(ffmpegJNI PRIVATE "-Wl,-Bsymbolic")
|
|
endif()
|
|
|
|
# Enable 16 KB ELF alignment.
|
|
target_link_options(ffmpegJNI
|
|
PRIVATE "-Wl,-z,max-page-size=16384")
|