camera: log captured JPEG dims + EXIF Orientation post-shot

Per Apple's macOS AVCaptureSession.h docs (line 1106), setting
`videoRotationAngle` on `AVCapturePhotoOutput`'s connection
"does not necessarily result in physical rotation of video buffers
… In the AVCapturePhotoOutput, orientation is handled using Exif
tags." So our connection-rotation tweaks only affect the EXIF
Orientation tag the JPEG carries — pixel data is sensor-native.

Yet the user keeps seeing a rotated JPEG even after `stripJpegApp1`
removes APP1 (EXIF). So either the pixel buffer IS rotated despite
the docs, or EXIF is in a non-APP1 marker, or Flutter's decoder
auto-rotates somehow.

Log the actual captured JPEG's dimensions + EXIF Orientation to
banlu.jsonl via the existing per-handle diagnostic stream:
`CGImageSourceCopyPropertiesAtIndex` reads
`kCGImagePropertyPixelWidth/Height/Orientation` from the JPEG
bytes that `AVCapturePhoto.fileDataRepresentation()` produces.

Format: `photo: WxH landscape|portrait exifOrientation=N`.

Once we see what AVCapturePhotoOutput is actually producing on the
user's Mac we'll know which side of the pipeline to fix.
This commit is contained in:
agra
2026-05-13 20:26:24 +03:00
parent 41c3fab7b5
commit cb2b57f661
2 changed files with 44 additions and 2 deletions

View File

@@ -102,6 +102,13 @@ final class CameraInstance {
session.onInterrupted = { [weak self] reason in
self?.emit(["event": "sessionInterrupted", "reason": reason])
}
photoOutput.onCapturedDiagnostic = { [weak self] message in
self?.emit([
"event": "diagnostic",
"message": message,
])
}
session.onResumed = { [weak self] in
self?.emit(["event": "sessionResumed"])
}

View File

@@ -1,5 +1,6 @@
import AVFoundation
import Foundation
import ImageIO
/// Wraps `AVCapturePhotoOutput`. One instance per
/// [CameraInstance]; gets added to the session at create time and
@@ -17,6 +18,15 @@ final class PhotoOutput {
private var inFlight: PhotoCaptureDelegate?
/// Forward post-capture diagnostics (encoded JPEG dimensions +
/// EXIF Orientation) through `CameraInstance.emit` so they land
/// in `banlu.jsonl`. macOS-only debug aid for "photo is rotated
/// 90° CW" cases Apple's docs say
/// `AVCapturePhotoOutput`'s connection rotation only affects EXIF
/// tags, not pixel data, but observation contradicts that. Logging
/// the actual file metadata helps tell which side is rotated.
var onCapturedDiagnostic: ((String) -> Void)?
/// Capture a single still. [orientation] applies to the photo
/// connection. [flashMode] is applied to the per-shot
/// `AVCapturePhotoSettings`. [completion] is invoked on `.main`
@@ -59,7 +69,11 @@ final class PhotoOutput {
}
}
let delegate = PhotoCaptureDelegate { [weak self] result in
let delegate = PhotoCaptureDelegate(
diag: { [weak self] message in
self?.onCapturedDiagnostic?(message)
}
) { [weak self] result in
// Reset orientation back to portraitUp on the photo
// connection so a follow-up shot without an explicit
// snapshot defaults cleanly. No-op on macOS (the
@@ -81,9 +95,14 @@ final class PhotoOutput {
/// `NSTemporaryDirectory()`, invokes the completion. The plugin
/// retains it via [PhotoOutput.inFlight] across the async hop.
private final class PhotoCaptureDelegate: NSObject, AVCapturePhotoCaptureDelegate {
private let diag: (String) -> Void
private let completion: (Result<String, NSError>) -> Void
init(completion: @escaping (Result<String, NSError>) -> Void) {
init(
diag: @escaping (String) -> Void,
completion: @escaping (Result<String, NSError>) -> Void
) {
self.diag = diag
self.completion = completion
}
@@ -104,6 +123,22 @@ private final class PhotoCaptureDelegate: NSObject, AVCapturePhotoCaptureDelegat
)))
return
}
// Parse the produced JPEG's actual pixel dimensions + EXIF
// Orientation so we can see in `banlu.jsonl` whether the
// pixel buffer is sensor-native (landscape) or rotated. The
// dimensions come from `CGImageSourceCopyProperties` reading
// the SOFn marker, EXIF orientation from kCGImagePropertyOrientation.
if let src = CGImageSourceCreateWithData(data as CFData, nil),
let props = CGImageSourceCopyPropertiesAtIndex(src, 0, nil)
as? [CFString: Any] {
let w = (props[kCGImagePropertyPixelWidth] as? NSNumber)?.intValue ?? -1
let h = (props[kCGImagePropertyPixelHeight] as? NSNumber)?.intValue ?? -1
let orient = (props[kCGImagePropertyOrientation] as? NSNumber)?.intValue ?? -1
let aspect = w > h ? "landscape" : (h > w ? "portrait" : "square")
diag("photo: \(w)x\(h) \(aspect) exifOrientation=\(orient)")
}
let url = URL(fileURLWithPath: NSTemporaryDirectory())
.appendingPathComponent("ux_camera_\(UUID().uuidString).jpg")
do {