使用AVCaptureSession调用系统的摄像头、麦克风来捕获视频、音频，包括二维码解析。
AVCaptureSession
An object that manages capture activity and coordinates the flow of data from input devices to capture outputs.
sessionPreset
A constant value indicating the quality level or bitrate of the output.
| Symbol | Resolution | Comments |
|---|---|---|
| AVCaptureSessionPresetHigh | High | Highest recording quality. This varies per device. |
| AVCaptureSessionPresetMedium | Medium | Suitable for Wi-Fi sharing. The actual values may change. |
| AVCaptureSessionPresetLow | Low | Suitable for 3G sharing. The actual values may change. |
| AVCaptureSessionPreset640x480 | 640x480 | VGA. |
| AVCaptureSessionPreset1280x720 | 1280x720 | 720p HD. |
| AVCaptureSessionPresetPhoto | Photo | Full photo resolution. This is not supported for video output. |
beginConfiguration && commitConfiguration
在对已经运行的session进行配置修改都需要放在中间，最后会调用commitConfiguration来使配置生效。
1 | session.beginConfiguration() |
如果session还没有运行，可以直接配置。
startRunning && stopRunning
在调用之前我们可以用isRunning来判断session有没有运行。
调用会阻塞当前线程，需要异步到其他线程中执行。
AVCaptureDevice
A device that provides input (such as audio or video) for capture sessions and offers controls for hardware-specific capture features.
这里我觉得写成下面获取视频的Device比较方便。
1 | /// Camera position. |
iPhone 8 Plus输出以下结果。
lockForConfiguration && unlockForConfiguration
Before you attempt to set properties of a capture device (its focus mode, exposure mode, and so on), you must first acquire a lock on the device using the lockForConfiguration() method. You should also query the device’s capabilities to ensure that the new modes you intend to set are valid for that device. You can then set the properties and release the lock using the unlockForConfiguration() method. You may hold the lock if you want all settable device properties to remain unchanged. However, holding the device lock unnecessarily may degrade capture quality in other applications sharing the device and is not recommended.
formats
The capture formats supported by the device.
An AVCaptureDevice.Format object describes in detail the video, image, or audio parameters of a specific mode of capture. If you need access to capture settings not covered by an AVCaptureSession preset, you can set the activeFormat property to any of the formats in this array.
下面是iPhone 8 Plus后置摄像头输出结果。
1 | (lldb) po device |
下面是iPhone 8 Plus前置摄像头输出结果。
1 | (lldb) po device |
activeFormat
The currently active media data format of the capture device.
一般我们使用AVCaptureSession的sessionPreset来配置捕获配置。使用AVAudioSession来配置音频。当我们设置之后会自动配置activeFormat这个参数。
但是可能我们需要特殊需求，比如高帧率视频，不能从sessionPreset配置。
1 | /// Apple 提供设置高帧率视频捕获例子。 |
activeDepthDataFormat
The currently active depth data format of the capture device.
focusMode
The device’s focus mode.
focusPointOfInterest
The point of interest for focusing.
This property’s CGPoint value uses a coordinate system where {0,0} is the top left of the picture area and {1,1} is the bottom right. This coordinate system is always relative to a landscape device orientation with the home button on the right, regardless of the actual device orientation.
1 | if ([currentDevice isFocusModeSupported:AVCaptureFocusModeContinuousAutoFocus]) { |
AVCaptureDevice.Format
A set of media format and capture settings (such as video resolution and frame rate) that can be used to configure a capture device.
AVCaptureDevice.FocusMode
AVCaptureInput
The abstract superclass for objects that provide input data to a capture session.
AVCaptureDeviceInput
A capture input that provides media from a capture device to a capture session.
AVCaptureOutput
The abstract superclass for objects that output the media recorded in a capture session.
AVCaptureVideoDataOutput
A capture output that records video and provides access to video frames for processing.
videoSettings
| Format | YUV | Discussion |
|---|---|---|
| kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange | ✅ | Bi-Planar Component Y’CbCr 8-bit 4:2:0 video-range (luma=[16,235] chroma=[16,240]). baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct. |
| kCVPixelFormatType_420YpCbCr8BiPlanarFullRange | ✅ | Bi-Planar Component Y’CbCr 8-bit 4:2:0 full-range (luma=[0,255] chroma=[1,255]). baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct. |
| kCVPixelFormatType_32BGRA | ❌ | 32 bit BGRA. |
1 | videoDataOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as AnyHashable: kCVPixelFormatType_32BGRA] |
摄像头输出的视频流原始格式是YUV的格式，如果手动配置为BGRA的格式，系统会在内部进行YUV转BGRA的格式，GPUImage采用的是输出YUV，然后用OpenGL来转换。具体的转换矩阵可以之前的文章：GPUImage-Mini灬哆啦
alwaysDiscardsLateVideoFrames
Indicates whether video frames are dropped if they arrive late.
参数默认是
true，如果不想丢帧，设置为false。
AVCaptureVideoDataOutputSampleBufferDelegate
1 | optional public func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) |
将回调帧放到指定的队列中执行，为了保证帧的顺序，队列一定要是串行队列。
AVCaptureMetadataOutput
A capture output for processing timed metadata produced by a capture session.
metadataObjectTypes
An array of strings identifying the types of metadata objects to process.
最常用的元数据类型：二维码AVMetadataObjectTypeQRCode和脸部AVMetadataObjectTypeFace。
AVCaptureMetadataOutputObjectsDelegate
1 | optional public func metadataOutput(_ output: AVCaptureMetadataOutput, didOutput metadataObjects: [AVMetadataObject], from connection: AVCaptureConnection) |
Capture
使用AVCaptureSession捕获视频数据。
首先我们先定义一个基类，主要负责启动和停止AVCaptureSession：
1 | /// Simple system camera to capture video. |
Video
仅捕获视频。
1 | open class VideoCamera: SystemCamera, AVCaptureVideoDataOutputSampleBufferDelegate { |
在这里
videoSettings设置为kCVPixelFormatType_32BGRA，这样就不用自己将YUV的数据转换成BGRA的格式了。
Media
捕获音频和视频。
1 | open class MediaCamera: VideoCamera, AVCaptureAudioDataOutputSampleBufferDelegate { |
QR Decode
直接捕获元数据。
1 | open class QRCamera: VideoCamera, AVCaptureMetadataOutputObjectsDelegate { |
Display
摄像头采集的视频并不是可以直接显示在视图中，而是由于摄像头设备的朝向，所以有如下的转换。
| AVCaptureDevicePosition | Mirror | Rotation |
|---|---|---|
| front | ✅ | 90 |
| back | ❌ | 90 |
首先我们先定义一个VideoFrame来承接即将到来的视频帧。
1 | /// Core video frame. |
使用GLKView来显示视频帧。
1 | /// Renderer video frame by GLKView, default is AspectRatio. |
视频帧是采用推流的方式获取，所以在设置
GLKView的时候，我们需要设置enableSetNeedsDisplay = false表示我们采用推流的方式绘制视频，在调用display()表示需要绘制视频帧。由于前置摄像头捕获的数据是一个镜像数据，所以需要进行一次transform，最后使用
CIContext绘制到GLKView。