2

I am trying to access samples for further processing (visual mostly) from CMSampleBuffer, here is my code:

 func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

    var audioBufferList = AudioBufferList()
    var blockBuffer : CMBlockBuffer?

    CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer, nil, &audioBufferList, MemoryLayout<AudioBufferList>.size, nil, nil, 0, &blockBuffer)

    let buffers = UnsafeBufferPointer<AudioBuffer>(start: &audioBufferList.mBuffers, count: Int(audioBufferList.mNumberBuffers))

    var data  =  audioBufferList.mBuffers.mData

    let pointer = data?.assumingMemoryBound(to: Float.self)

    let floatPointer = data?.bindMemory(to: Float.self, capacity: 1024)
    let floatBuffer = UnsafeBufferPointer(start: floatPointer, count: 1024)
    let outputArray = Array(floatBuffer)
    print(outputArray[0..<20])

}

I am getting the results but they seems like in the strange (for me) kind of form, like this (output from print statement):

[-snan(0x1dff94), -nan(0x15ffba), 2.93871626e-39, 2.29594486e-39, 8.26537682e-40, -nan(0x1affef), -nan(0xeffe0), 2.84690739e-39, 4.95919386e-39, 4.86737658e-39, 1.19389788e-39, -nan(0x8ffe4), -snan(0xaffa6), -3.29645785e+38, -snan(0x17ff85), -nan(0x5ffae), 2.66320417e-39, 2.11228928e-39, -nan(0x2fff6), -nan(0x19ffd6)]
[-2.45903224e+38, -3.13694095e+38, -snan(0xdff85), -nan(0xbffb4), -snan(0x11ffba), -2.83122967e+38, -2.41915276e+38, -2.67170487e+38, -2.83122724e+38, -2.73817357e+38, -2.39257084e+38, -1.50864781e+38, -1.34249056e+38, -1.71464408e+38, -2.53878186e+38, -2.48561761e+38, -2.64512051e+38, -3.19011393e+38, -snan(0x12ff96), -snan(0x1bffb7)]

The question is how can I access CMSampleBuffer samples in the way that "normal" AVAudioPCMBuffer?

FloatChannelData Or what am I doing wrong here?

1 Answer 1

2

When I create and AVCaptureAudioDataOutput the samples I get are Int16s (not sure if you can configure for Floats), and when printed with your modified code, they make perfect sense:

let intPointer = data?.bindMemory(to: Int16.self, capacity: 1024)

Is the audio Int16ness documented anywhere? If not you can always inspect the sample buffer's format by looking at its AudioStreamBasicDescription

let asbd = CMAudioFormatDescriptionGetStreamBasicDescription(CMSampleBufferGetFormatDescription(sampleBuffer)!)!.pointee

or start using swift 4 for a slightly more readable

let asbd = sampleBuffer.formatDescription!.audioStreamBasicDescription!.pointee
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.