- video_player: ExoPlayer (Android) / AVPlayer (iOS/macOS) backend with
PixelBufferSink, method-channel adapter, Dart-side XVideoPlayer +
testing fake.
- insets: XInsets singleton + XAnimatedInsets widget lerp the system
viewPadding over 220ms so OS bar visibility toggles
(immersiveSticky <-> edgeToEdge) slide bottom-/top-anchored UI into
place instead of snapping by the nav-bar / status-bar height.
Catch-all commit for outstanding pre-existing local changes. Mixes
several themes that would normally be split:
- Rename: UxPlugin → XPlugin across iOS, macOS, Android registrants.
- New top-level packages under lib/src/: anim/ (animated values,
panes, sheets, dock, measured), core/ (Emitter, ReactiveBuilder
scaffolding, presenter/widget/value/dispose primitives), navi/
(Screen/ScreenStack/Router/hero/transitions), reactive/.
- Edits across existing plugins (clipboard, crash, file, gallery,
keyboard, scanner, sensor, url) to align with the new core.
- Test updates and CHANGELOG/README touches accompanying the above.
Banner-tap entry point that shows the system mic prompt when the OS
will still surface one, and deep-links to Settings only on permanent
denial. Fixes the fresh-install trap where the mic entry isn't in the
Privacy pane until requestAccess has fired at least once.
Android tracks the first-asked state in SharedPreferences because
shouldShowRequestPermissionRationale returns false in two
observationally identical states (never asked vs permanently denied).
The existing initialize() request path writes the flag too, so a
banner tap after a record-then-deny correctly routes to Settings.
Refactored Android pendingPermission into PendingPermission(primary,
kind, cb) so audio-only requests check RECORD_AUDIO results instead
of always checking CAMERA.
Camera page kept the session running while the host app was
backgrounded — wastes battery, holds the hardware, and blocks
other apps from grabbing the camera. Add per-platform observers
that pause/resume the session on app foreground/background, with
a uniform `pauseForBackground` / `resumeForForeground` pair on the
shared CameraInstance.
Behaviour:
- On background: any in-flight recording is hard-cancelled
(matches every messaging app — the take ends with the app
switch). The session stops so the OS can release the camera.
- On foreground: session restarts iff it had been running.
Emits `sessionInterrupted` (`reason: appBackgrounded`) and
`sessionResumed` events so the Dart side can surface UX
affordances if needed.
iOS — `ios/Classes/Camera/CameraInstance+iOS.swift`:
Subscribes to UIApplication.{willResignActive, didBecomeActive}
notifications. Work hops onto sessionQueue so AV mutations stay
serialised. Storage uses the shared
`CameraInstance.lifecycleCleanup` closure slot — extension
doesn't need to add stored properties.
Android — added `androidx.lifecycle:lifecycle-process:2.7.0`,
observes `ProcessLifecycleOwner.get().lifecycle`. ON_STOP →
`pauseForBackground` (cancels recording + drops
CustomLifecycleOwner to CREATED → CameraX releases camera).
ON_START → `resumeForForeground`. Observer add/remove on main
thread per `ProcessLifecycleOwner` contract.
macOS — `macos/Classes/Camera/CameraInstance+macOS.swift`:
Intentional no-op. macOS desktop background semantics are
softer; the chat composer's Card dialog typically stays
foregrounded. Slot is wired so the shared
`observeLifecycle()` call still compiles.
Verified: all four platforms (iOS / Android / macOS / app tests)
build clean. Pod install picks up the new iOS extension file
once Pods/ is fresh — `flutter clean` if mid-iteration.
The two `NSLog("[ux.camera] …")` calls were debug instrumentation for
diagnosing the macOS photo rotation issue. The bug is fixed
(macOS pinned to 0° rotation, photo + preview + video all 1280x720
landscape), so the NSLog calls are now stderr noise on every shot.
Keep the per-shot `diag(photo: WxH …)` emit since it goes through
`ux.Log` (gated by level, lands in banlu.jsonl) — useful if rotation
ever regresses on a different camera / macOS version, and the cost is
one log line per photo capture.
User reports macOS photo still rotated AND no `photo:` line in
banlu.jsonl — so either the delegate isn't firing at all or the
event-channel path is broken on the success branch.
Add `NSLog` at the top of `photoOutput(_:didFinishProcessingPhoto:error:)`
and on the CGImage path. NSLog lands in `flutter run` stderr +
macOS Console regardless of channel state, so even if the diag
event drops we'll see the delegate firing + the CGImage's
dimensions. Once we see them we'll know whether it's a delegate
problem or a channel problem.
Per Apple's AVCaptureSession.h docs (line 1106), AVCapturePhotoOutput
applies connection rotation via EXIF tags rather than physical pixel
rotation. The macOS variants of `fileDataRepresentationWithCustomizer`
and `fileDataRepresentationWithReplacementMetadata` are
API_UNAVAILABLE(macos), so we can't replace the embedded EXIF
through the standard customizer.
Workaround: on macOS, grab the raw `AVCapturePhoto.cgImageRepresentation()`
— Apple documents this as "the physical rotation of the CGImageRef
matches that of the main image. Exif orientation has not been
applied." — and re-encode the JPEG via `CGImageDestination` with no
orientation metadata. Resulting JPEG has sensor-native landscape
pixels and no EXIF Orientation tag; viewers and Flutter's image
codec both display as landscape.
iOS path unchanged (still uses fileDataRepresentation).
Diagnostic now fires on both the success and failure branches of
the delegate, so when the capture fails (e.g. the macOS
"OSStatus error 13" the user observed) the failure reason — domain,
code, localized description — is captured to banlu.jsonl instead
of silently dropped.
Per Apple's macOS AVCaptureSession.h docs (line 1106), setting
`videoRotationAngle` on `AVCapturePhotoOutput`'s connection
"does not necessarily result in physical rotation of video buffers
… In the AVCapturePhotoOutput, orientation is handled using Exif
tags." So our connection-rotation tweaks only affect the EXIF
Orientation tag the JPEG carries — pixel data is sensor-native.
Yet the user keeps seeing a rotated JPEG even after `stripJpegApp1`
removes APP1 (EXIF). So either the pixel buffer IS rotated despite
the docs, or EXIF is in a non-APP1 marker, or Flutter's decoder
auto-rotates somehow.
Log the actual captured JPEG's dimensions + EXIF Orientation to
banlu.jsonl via the existing per-handle diagnostic stream:
`CGImageSourceCopyPropertiesAtIndex` reads
`kCGImagePropertyPixelWidth/Height/Orientation` from the JPEG
bytes that `AVCapturePhoto.fileDataRepresentation()` produces.
Format: `photo: WxH landscape|portrait exifOrientation=N`.
Once we see what AVCapturePhotoOutput is actually producing on the
user's Mac we'll know which side of the pipeline to fix.
Two changes that target the macOS "camera not found" leak after a
few open/close cycles. macOS's `AVCaptureDevice.DiscoverySession`
excludes devices that are still claimed by another session — even
our own zombie session that hasn't fully released its grip on the
hardware. So if dispose leaves the session in a partially torn-down
state, the next `availableCameras` returns empty.
CameraInstance.dispose now:
- Cancels the recorder (was already there) so the audio
data output's retain on the recorder drops.
- Stops the session.
- **Nils sample-buffer delegates** on the video + audio data
outputs before removing them. `setSampleBufferDelegate` holds a
strong reference to the delegate; the macOS reference to our
`SampleFanout` was transitively keeping the session alive.
- Removes inputs + outputs inside a single
`session.configure { … }` block (begin/commitConfiguration) so
AVFoundation sees the teardown as one atomic transition rather
than a sequence of partial states. Apple's docs are explicit on
this; we weren't following.
- Clears the strong references to the instance vars.
Plugin diagnostics:
- When availableCameras returns empty, native now emits an event
`{handle: -1, event: "diagnostic", message: …}` carrying the
current `devicesInUse`, `audioInUse` and `instances` keys.
Per-handle diagnostics already flow through a controller's
`_onEvent`; plugin-level ones (handle == -1) had no path to the
log_server jsonl.
- `MethodChannelUxCameraBackend` now subscribes to its raw event
stream once and pipes any handle=-1 diagnostic through
`ux.Log.tag('camera').i('plugin: …')`. The subscription kicks in
when the broadcast stream is first accessed (still lazy —
matches the prior behavior).
If the macOS "camera not found" reproduces, the jsonl will show
which side leaked: a non-empty `devicesInUse` says our claim
tracking is stale; an empty one says AVFoundation itself is
holding the hardware.
`device.activeFormat.formatDescription` reports the device's
*selected* format, but on macOS the session-preset remap means the
data output sometimes delivers a different resolution than what
the active format claims. The mismatch surfaced as a stretched
preview on macOS: camera_thumb's SizedBox was sized for a 4:3
buffer, the FittedBox(cover) was given a 16:9 texture, so the
texture stretched to fill the wrongly-shaped box.
PreviewSink now snapshots `CVPixelBufferGet{Width,Height}` from
the first sample buffer that arrives and forwards it via a
callback. CameraInstance hooks the callback to emit the existing
`previewSizeChanged` event (same shape the Android backend uses).
The Dart controller writes it into `value.previewSize`,
camera_thumb's SizedBox snaps to the real buffer aspect, and
FittedBox(cover) crops cleanly without stretching.
Identical wire shape and event name as the Android equivalent —
no Dart changes needed.
macOS preview was stretching (aspect wrong) and macOS photo capture
was rotating the landscape sensor 90° because the shared
PhotoOutput / CameraInstance code was setting
`AVCaptureConnection.videoOrientation` from the orientation snapshot
unconditionally. iOS needs that to rotate sample buffers to portrait;
macOS desktop cams are physically landscape and any rotation just
skews the result.
Moved the rotation call behind a per-platform extension on
`AVCaptureConnection`:
- `ios/Classes/Camera/AVCaptureConnection+iOS.swift` applies the
snapshot orientation (current behavior).
- `macos/Classes/Camera/AVCaptureConnection+macOS.swift` is a
no-op. macOS-flavoured photos / preview frames now flow at
native landscape orientation.
`CaptureDevice` reports sensorOrientation=0 on macOS (was hardcoded
90 for iOS); on macOS the page's `normalizeCameraCapture` math then
collapses to identity and the saved JPEG stays the landscape the
sensor produced. iOS keeps sensorOrientation=90 (matches
camera_avfoundation's reported value and the existing capture-
transform math).
Photo and video paths now both produce upright content on macOS
(video already worked because VideoRecorder's transform table maps
the always-portraitUp macOS snapshot to `.identity`).
Discovery was hard-coded to `.builtInWideAngleCamera` only — that
catches the iOS front/back cameras and the macOS FaceTime HD on
macOS 14+, but missed USB webcams (`.external` on 14+,
`.externalUnknown` before) and iPhone-as-webcam Continuity Camera
(`.continuityCamera` on 14+). On Macs whose built-in camera doesn't
expose itself as `.builtInWideAngleCamera`, the result was "no
camera detected".
Device types are now platform-conditional: iOS keeps the wide-angle
filter as-is; macOS adds the externals + Continuity (gated via
`if #available(macOS 14.0, *)` for the post-14 forms vs the
deprecated `.externalUnknown` for older macOS). The `#if os(macOS)`
guard is unavoidable — `.external` and friends literally aren't
declared as enum cases on iOS.
Reuse the AVFoundation Swift files between iOS and macOS without
sprinkling `#if canImport(UIKit)` through them. The split is:
darwin/Camera/ platform-shared (AVFoundation only)
CameraPlugin channel + instance map
CameraInstance session + outputs + texture
CameraSession AVCaptureSession + runtime-error obs
CaptureDevice front/back discovery
PhotoOutput AVCapturePhotoOutput
PreviewSink CVPixelBuffer → FlutterTexture
VideoRecorder AVAssetWriter
DeviceOrientation wire-string enum
ios/Classes/Camera/ iOS-only impls + extensions
AudioSession AVAudioSession.upgradeForRecording
DeviceOrientationBridge UIDevice.orientation listener
CameraSession+iOS AVCaptureSessionWasInterrupted obs
+ InterruptionReason decode + the
application-audio-session flags
(all iOS-only on AVCaptureSession)
CameraSettings UIApplication.openSettingsURLString
FlutterRegistrar+iOS method-form of textures/messenger
macos/Classes/Camera/ macOS no-op stubs (same surface)
AudioSession no-op (no AVAudioSession on macOS)
DeviceOrientationBridge no-op (desktops don't rotate)
CameraSession+macOS no-op setupPlatform()
CameraSettings NSWorkspace → System Settings'
Privacy_Camera pane
FlutterRegistrar+macOS property-form of textures/messenger
`CameraSession.init` now calls `setupPlatform()` which each platform
provides via an extension — keeps the iOS-only interruption observer
and the `automaticallyConfiguresApplicationAudioSession` /
`usesApplicationAudioSession` flags (both iOS-only on AVCaptureSession)
out of the shared file. Flash-mode in PhotoOutput uses
`if #available(macOS 11/13, *)` rather than `#if`, since those are
plain version gates not platform splits.
The shared files compile into the iOS pod from `ios/Classes/Camera-shared/`
and into the macOS pod from `macos/Classes/Camera-shared/`, each a
mirror populated by a `prepare_command` in the podspec:
rm -rf Classes/Camera-shared && cp -R ../darwin/Camera Classes/Camera-shared
Symlinks and `../` source globs both fail — Pathname.glob bails on
symlinks, and CocoaPods silently drops paths that escape the pod
directory. The mirror destinations are .gitignore'd.
macOS UxPlugin now registers CameraPlugin alongside the others.
