7243ef7de4d33099e96d925108c9dc54cc3ea18a
Adds an LGPL FFmpeg-backed video renderer that slots ahead of Media3's
MediaCodecVideoRenderer via EXTENSION_RENDERER_MODE_PREFER. Resolves
playback failures on Huawei EMUI 11 (Mate 20, Kirin 980): the Codec2
HiSilicon AVC decoder initialises cleanly on iOS High@3.1 streams with
deep DPB + full-range yuvj420p, then errors on the first sample inside
MediaCodecVideoRenderer (init-failure fallback can't catch this).
Google's C2 SW AVC decoder hits its 8-frame output-delay cap on the
same shape and stalls on dequeueOutputBuffer.
Media3's own decoder-ffmpeg ships only an audio renderer;
ExperimentalFfmpegVideoRenderer has been a stub since 2020 (returns
FORMAT_UNSUPPORTED_TYPE, createDecoder returns null). NextLib is
GPL-3.0. So we vendor our own Apache-licensed JNI on top of LGPL
FFmpeg, dynamically linked at runtime.
Build flow:
- android/ffmpeg/ holds the JNI source + CMakeLists + orchestrator
script + LGPL notice. No native binaries in git.
- :ux:buildFfmpegJni Gradle task (wired to preBuild) clones
Media3 1.9.2 + FFmpeg release/6.0 into build/ffmpeg-work/ on
first run, builds h264-only static libs per ABI, links
libffmpegJNI.so per ABI into build/jniLibs/<abi>/. AGP picks
them up via sourceSets.main.jniLibs.srcDirs +=. Gradle
UP-TO-DATE skips the task when ffmpeg_jni.cc / CMakeLists /
build_ffmpeg.sh are unchanged.
Renderer:
- FfmpegVideoDecoder (SimpleDecoder) sends each packet with its
inputBuffer.timeUs as pkt->pts; the JNI overwrites
outputBuffer.timeUs with f->pts on receive so frames emitted in
display order carry their true display PTS (input PTS in decode
order scrambles ExoPlayer's drop logic and halves the render
rate on B-frame streams).
- FfmpegOutputSurface does YUV->RGB in one GLES2 pass against an
EGL window surface sized to display orientation. Y plane uses
GL_NEAREST (1:1 sized, sampling at exact texel centres
preserves luma detail); chroma uses GL_LINEAR. Pre-rotated quad
UVs (0/90/180/270) keep the YUV sampling correct when the
coded frame needs rotation for display.
- FfmpegVideoRenderer swaps the output buffer's width/height for
90/270 streams before super.renderOutputBuffer notifies size,
matching MediaCodecVideoRenderer's post-rotation reporting.
Decoder fallback:
- Renderers.kt selects FfmpegVideoRenderer first when
libffmpegJNI.so is loaded; falls through to the platform path
for formats FFmpeg doesn't handle or ABIs without the .so.
- MediaCodec selector deprioritises every HiSilicon decoder
(OMX.hisi.* and c2.hisi.*) so the platform path picks
c2.android.avc.decoder ahead of the C2 Hisi variant when FFmpeg
isn't available. Required because the C2 Hisi failure is
post-init, which Media3's setEnableDecoderFallback(true) can't
intercept.
Compositor:
- VideoCompositor.setInputSurfaceSize lets the renderer resize the
codec-input SurfaceTexture before eglCreateWindowSurface so the
EGL surface inherits matching buffer dimensions on creation
(MediaCodec sizes natively; EGL doesn't).
- VideoPlayerInstance wires Renderers.build with a sizer callback
that calls into compositor.setInputSurfaceSize from the FFmpeg
renderer thread.
Adds docs/architecture.md with the layered video pipeline diagram,
file map, renderer-selection rationale, build flow, and LGPL
boundary notes.
ux
A Flutter toolkit for building fluid, native-feeling UIs.
XKeyboard
Frame-accurate keyboard height tracking for iOS and Android, with interactive dismiss.
Flutter's built-in MediaQuery.viewInsets.bottom lags behind the actual keyboard position
and doesn't support interactive dismiss. XKeyboard reads the keyboard height directly
from the native layer via FFI — zero channel latency, every frame.
Features
- Real-time height — reads the keyboard's actual position each frame via FFI (iOS) / JNI (Android)
- Native animation curves — sampled from
CADisplayLink(iOS) andWindowInsetsAnimation(Android), with adaptive learning that refines the curve from observations - Interactive dismiss — swipe the keyboard down like iMessage/Telegram, with snap-back or dismiss
- Scroll freeze —
isTrackingflag lets you freeze scrolling during interactive dismiss
Quick start
final keyboard = XKeyboard.instance;
// Enable swipe-to-dismiss. trackingInset is the height of your input bar.
keyboard.enableInteractiveDismiss(trackingInset: 56);
Use ListenableBuilder to rebuild when the keyboard height changes:
Scaffold(
resizeToAvoidBottomInset: false, // we handle it ourselves
body: ListenableBuilder(
listenable: keyboard,
builder: (context, _) {
final keyboardHeight = keyboard.height;
final safeBottom = MediaQuery.viewPaddingOf(context).bottom;
final bottom = max(keyboardHeight, safeBottom);
return Column(
children: [
Expanded(
child: ListView.builder(
reverse: true,
// Freeze scrolling during interactive dismiss
physics: keyboard.isTracking
? NeverScrollableScrollPhysics()
: null,
// ...
),
),
Container(
padding: EdgeInsets.only(bottom: 8 + bottom),
// your input bar
),
],
);
},
),
);
API
| Member | Description |
|---|---|
XKeyboard.instance |
Singleton instance |
.height |
Current keyboard height in logical pixels |
.systemHeight |
Last system-reported keyboard height |
.isOpen |
Whether the keyboard is visible |
.isTracking |
Whether a dismiss pan gesture is active |
.enableInteractiveDismiss({trackingInset}) |
Enable swipe-to-dismiss |
.disableInteractiveDismiss() |
Disable swipe-to-dismiss |
addListener / removeListener |
Standard ChangeNotifier API |
Key points
- Set
resizeToAvoidBottomInset: falseon yourScaffold— otherwise Flutter's built-in resize fights withXKeyboard - Use
MediaQuery.viewPaddingOf(context).bottomfor the safe area (notpaddingOf, which is consumed byScaffold) - Use
max(keyboardHeight, safeBottom)for bottom padding — the keyboard height includes the safe area when open, andsafeBottomcovers the home indicator when closed
Other utilities
- BendBox — a flexible layout widget
- Bezier — bezier curve utilities
Languages
Dart
40.2%
Swift
29.1%
Kotlin
18.4%
Java
5.2%
C++
3.1%
Other
4%