agra 7243ef7de4 video: vendor FFmpeg software AVC renderer
Adds an LGPL FFmpeg-backed video renderer that slots ahead of Media3's
MediaCodecVideoRenderer via EXTENSION_RENDERER_MODE_PREFER. Resolves
playback failures on Huawei EMUI 11 (Mate 20, Kirin 980): the Codec2
HiSilicon AVC decoder initialises cleanly on iOS High@3.1 streams with
deep DPB + full-range yuvj420p, then errors on the first sample inside
MediaCodecVideoRenderer (init-failure fallback can't catch this).
Google's C2 SW AVC decoder hits its 8-frame output-delay cap on the
same shape and stalls on dequeueOutputBuffer.

Media3's own decoder-ffmpeg ships only an audio renderer;
ExperimentalFfmpegVideoRenderer has been a stub since 2020 (returns
FORMAT_UNSUPPORTED_TYPE, createDecoder returns null). NextLib is
GPL-3.0. So we vendor our own Apache-licensed JNI on top of LGPL
FFmpeg, dynamically linked at runtime.

Build flow:
  - android/ffmpeg/ holds the JNI source + CMakeLists + orchestrator
    script + LGPL notice. No native binaries in git.
  - :ux:buildFfmpegJni Gradle task (wired to preBuild) clones
    Media3 1.9.2 + FFmpeg release/6.0 into build/ffmpeg-work/ on
    first run, builds h264-only static libs per ABI, links
    libffmpegJNI.so per ABI into build/jniLibs/<abi>/. AGP picks
    them up via sourceSets.main.jniLibs.srcDirs +=. Gradle
    UP-TO-DATE skips the task when ffmpeg_jni.cc / CMakeLists /
    build_ffmpeg.sh are unchanged.

Renderer:
  - FfmpegVideoDecoder (SimpleDecoder) sends each packet with its
    inputBuffer.timeUs as pkt->pts; the JNI overwrites
    outputBuffer.timeUs with f->pts on receive so frames emitted in
    display order carry their true display PTS (input PTS in decode
    order scrambles ExoPlayer's drop logic and halves the render
    rate on B-frame streams).
  - FfmpegOutputSurface does YUV->RGB in one GLES2 pass against an
    EGL window surface sized to display orientation. Y plane uses
    GL_NEAREST (1:1 sized, sampling at exact texel centres
    preserves luma detail); chroma uses GL_LINEAR. Pre-rotated quad
    UVs (0/90/180/270) keep the YUV sampling correct when the
    coded frame needs rotation for display.
  - FfmpegVideoRenderer swaps the output buffer's width/height for
    90/270 streams before super.renderOutputBuffer notifies size,
    matching MediaCodecVideoRenderer's post-rotation reporting.

Decoder fallback:
  - Renderers.kt selects FfmpegVideoRenderer first when
    libffmpegJNI.so is loaded; falls through to the platform path
    for formats FFmpeg doesn't handle or ABIs without the .so.
  - MediaCodec selector deprioritises every HiSilicon decoder
    (OMX.hisi.* and c2.hisi.*) so the platform path picks
    c2.android.avc.decoder ahead of the C2 Hisi variant when FFmpeg
    isn't available. Required because the C2 Hisi failure is
    post-init, which Media3's setEnableDecoderFallback(true) can't
    intercept.

Compositor:
  - VideoCompositor.setInputSurfaceSize lets the renderer resize the
    codec-input SurfaceTexture before eglCreateWindowSurface so the
    EGL surface inherits matching buffer dimensions on creation
    (MediaCodec sizes natively; EGL doesn't).
  - VideoPlayerInstance wires Renderers.build with a sizer callback
    that calls into compositor.setInputSurfaceSize from the FFmpeg
    renderer thread.

Adds docs/architecture.md with the layered video pipeline diagram,
file map, renderer-selection rationale, build flow, and LGPL
boundary notes.
2026-05-28 19:24:17 +03:00
2019-08-07 15:37:44 +01:00
2026-04-21 16:54:56 +03:00
2019-10-17 14:34:21 +01:00
zzz
2020-05-04 10:58:19 +01:00

ux

A Flutter toolkit for building fluid, native-feeling UIs.

XKeyboard

Frame-accurate keyboard height tracking for iOS and Android, with interactive dismiss.

Flutter's built-in MediaQuery.viewInsets.bottom lags behind the actual keyboard position and doesn't support interactive dismiss. XKeyboard reads the keyboard height directly from the native layer via FFI — zero channel latency, every frame.

Features

  • Real-time height — reads the keyboard's actual position each frame via FFI (iOS) / JNI (Android)
  • Native animation curves — sampled from CADisplayLink (iOS) and WindowInsetsAnimation (Android), with adaptive learning that refines the curve from observations
  • Interactive dismiss — swipe the keyboard down like iMessage/Telegram, with snap-back or dismiss
  • Scroll freezeisTracking flag lets you freeze scrolling during interactive dismiss

Quick start

final keyboard = XKeyboard.instance;

// Enable swipe-to-dismiss. trackingInset is the height of your input bar.
keyboard.enableInteractiveDismiss(trackingInset: 56);

Use ListenableBuilder to rebuild when the keyboard height changes:

Scaffold(
  resizeToAvoidBottomInset: false, // we handle it ourselves
  body: ListenableBuilder(
    listenable: keyboard,
    builder: (context, _) {
      final keyboardHeight = keyboard.height;
      final safeBottom = MediaQuery.viewPaddingOf(context).bottom;
      final bottom = max(keyboardHeight, safeBottom);

      return Column(
        children: [
          Expanded(
            child: ListView.builder(
              reverse: true,
              // Freeze scrolling during interactive dismiss
              physics: keyboard.isTracking
                  ? NeverScrollableScrollPhysics()
                  : null,
              // ...
            ),
          ),
          Container(
            padding: EdgeInsets.only(bottom: 8 + bottom),
            // your input bar
          ),
        ],
      );
    },
  ),
);

API

Member Description
XKeyboard.instance Singleton instance
.height Current keyboard height in logical pixels
.systemHeight Last system-reported keyboard height
.isOpen Whether the keyboard is visible
.isTracking Whether a dismiss pan gesture is active
.enableInteractiveDismiss({trackingInset}) Enable swipe-to-dismiss
.disableInteractiveDismiss() Disable swipe-to-dismiss
addListener / removeListener Standard ChangeNotifier API

Key points

  • Set resizeToAvoidBottomInset: false on your Scaffold — otherwise Flutter's built-in resize fights with XKeyboard
  • Use MediaQuery.viewPaddingOf(context).bottom for the safe area (not paddingOf, which is consumed by Scaffold)
  • Use max(keyboardHeight, safeBottom) for bottom padding — the keyboard height includes the safe area when open, and safeBottom covers the home indicator when closed

Other utilities

  • BendBox — a flexible layout widget
  • Bezier — bezier curve utilities
Description
No description provided
Readme MIT 1.5 MiB
Languages
Dart 40.2%
Swift 29.1%
Kotlin 18.4%
Java 5.2%
C++ 3.1%
Other 4%