9

I am using Audio Queues to playback audio files. I need precise timing on the finish of last buffer. I need to notify a function no later than 150ms-200 ms after the last buffer is played...

Thru callback method I know how many buffers are enqueued

I know the buffer size, I know the how many bytes last buffer is filled with.

First I initialize a number of buffers end fill the buffers with audio data, then enqueue them. When Audio Queue needs a buffer to be filled it calls the callback and I fill the buffer with data.

When there is no more audio data available Audio Queue sends me the last empty buffer, so I fill it with whatever data I have:

            if (sharedCache.numberOfToTalPackets>0)
            {
                if (currentlyReadingBufferIndex==[sharedCache.baseAudioCache count]-1) {
                    inBuffer->mAudioDataByteSize = (UInt32)bytesFilled;
                    lastEnqueudBufferSize=bytesFilled;
                    err=AudioQueueEnqueueBuffer(inAQ,inBuffer,(UInt32)packetsFilled,packetDescs);
                    if (err) {
                        [self failWithErrorCode:err customError:AP_AUDIO_QUEUE_ENQUEUE_FAILED];
                    }
                    printf("if that was the last free packet description, then enqueue the buffer\n");
                    //go to the next item on keepbuffer array
                    isBufferFilled=YES;
                    [self incrementBufferUsedCount];
                    return;
                }
            }

When Audio Queue asks for more data via callback and I have no more data , I start to countdown the buffers. If buffer count equals to zero, which means only one buffer left on the flight to be played, the moment playback is done I try to stop the audio queue.

-(void)decrementBufferUsedCount
{

    if (buffersUsed>0) {
        buffersUsed--;
        printf("buffer on the queue %i\n",buffersUsed);
        if (buffersUsed==0) {
            NSLog(@"playback is finished\n");
            // end playback
            isPlayBackDone=YES;
            double sampleRate = dataFormat.mSampleRate;
            double bufferDuration = lastEnqueudBufferSize/ sampleRate;
            double estimatedTimeNeded=bufferDuration*1;
            [self performSelector:@selector(stopPlayer) withObject:nil afterDelay:estimatedTimeNeded];
        }
    }
}  

-(void)stopPlayer
{
    @synchronized(self)
    {
        state=AP_STOPPING;
    }
    err=AudioQueueStop(queue, TRUE);
    if (err) {
        [self failWithErrorCode:err customError:AP_AUDIO_QUEUE_STOP_FAILED];
    }
    else
    {
        @synchronized(self)
        {
            state=AP_STOPPED;
            NSLog(@"Stopped\n");
        }

However it seems I can't get precise timing here. Above code stops player early.

if I do following audio cuts early too

double bufferDuration = XMAQDefaultBufSize/ sampleRate;
double estimatedTimeNeded=bufferDuration*1;

if increase 1 to 2 since the buffer size is big I get some delay, seem 1.5 is the optimum value for now but I dont understand why lastEnqueudBufferSize/ sampleRate is not wotking

Details of the audio file, and buffers:

Audio file has 22050 sample rate
#define kNumberPlaybackBuffers  4
#define kAQDefaultBufSize 16384
it is a vbr file format with no bitrate information available
1
  • Have you tried the method I posted below? You can get a sample accurate prediction of when the last sample will leave the speaker.
    – dave234
    May 31, 2015 at 3:31

3 Answers 3

6
+100

EDIT:

I found an easier way that gets the same results (+/-10ms). After you set up your output Queue with AudioQueueNewOutput() you initialize a AudioQueueTimelineRef to be used in your output callback. (ticksToSeconds function is included below in my first method) don't forget to import<mach/mach_time.h>

//After AudioQueueNewOutput()
AudioQueueTimelineRef timeLine;     //ivar
AudioQueueCreateTimeline(queue, self.timeLine);

Then in your output callback you call AudioQueueGetCurrentTime(). Caveat: queue must be playing for valid timestamps. So for very short files you might need to use the AudioQueueProcessingTap method below.

AudioTimeStamp timestamp;
AudioQueueGetCurrentTime(queue, self->timeLine, &timestamp, NULL);

The timestamp ties together the current sample playing with the current machine time. With that info we can get an exact machine time in the future when our last sample will be played.

Float64 samplesLeft    = self->frameCount - timestamp.mSampleTime;//samples in file - current sample
Float64 secondsLeft    = samplesLeft / self->sampleRate;          //seconds of audio to play
UInt64  ticksLeft      = secondsLeft / ticksToSeconds();          //seconds converted to machine ticks  
UInt64  machTimeFinish = timestamp.mHostTime + ticksLeft;         //machine time of first sample + ticks left 

Now that we have this future machine time we can use it to time whatever it is that you want to do with some accuracy.

UInt64 currentMachTime = mach_absolute_time();
Uint64 ticksFromNow = machTimeFinish - currentMachTime;
float secondsFromNow = ticksFromNow * ticksToSeconds();
dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(secondsFromNow * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{
    //do the thing!!!
    printf("Giggety");
});

If GCD dispatch_async isn't accurate enough there are ways to set up a precision timer

Using AudioQueueProcessingTap

You can get fairly low response time from an AudioQueueProcessingTap. First you make your callback that will essentially put itself in-between the audio stream. The MyObject type is just whatever self is in your code(this is ARC bridging here to get self inside the function). Inspecting ioFlags tells you when the stream starts and finishes. The ioTimeStamp of an output callback describes time that the first sample in the callback will hit the speaker in the future. So if you want to get exact here's how you do it. I added some convenience functions for converting machine time to seconds.

#import <mach/mach_time.h>

double getTimeConversion(){
    double timecon;
    mach_timebase_info_data_t tinfo;
    kern_return_t kerror;
    kerror = mach_timebase_info(&tinfo);
    timecon = (double)tinfo.numer / (double)tinfo.denom;

    return  timecon;
}
double ticksToSeconds(){
    static double ticksToSeconds = 0;
    if (!ticksToSeconds) {
        ticksToSeconds = getTimeConversion() * 0.000000001;
    }
    return ticksToSeconds;
}

void processingTapCallback(
                 void *                          inClientData,
                 AudioQueueProcessingTapRef      inAQTap,
                 UInt32                          inNumberFrames,
                 AudioTimeStamp *                ioTimeStamp,
                 UInt32 *                        ioFlags,
                 UInt32 *                        outNumberFrames,
                 AudioBufferList *               ioData){

    MyObject *self = (__bridge Object *)inClientData;
    AudioQueueProcessingTapGetSourceAudio(inAQTap, inNumberFrames, ioTimeStamp, ioFlags, outNumberFrames, ioData);
    if (*ioFlags ==  kAudioQueueProcessingTap_EndOfStream) {
        Float64 sampTime;
        UInt32 frameCount;
        AudioQueueProcessingTapGetQueueTime(inAQTap, &sampTime, &frameCount);
        Float64 samplesInThisCallback = self->frameCount - sampleTime;//file sampleCount - queue current sample
        //double secondsInCallback = outNumberFrames / (double)self->sampleRate; outNumberFrames was inaccurate
        double secondsInCallback = * samplesInThisCallback / (double)self->sampleRate;
        uint64_t timeOfLastSampleLeavingSpeaker = ioTimeStamp->mHostTime + (secondsInCallback / ticksToSeconds());
        [self lastSampleDoneAt:timeOfLastSampleLeavingSpeaker];
    }
}

-(void)lastSampleDoneAt:(uint64_t)lastSampTime{
    uint64_t currentTime = mach_absolute_time();
    if (lastSampTime > currentTime) {
        double secondsFromNow = (lastSampTime - currentTime) * ticksToSeconds();
        dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(secondsFromNow * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{
            //do the thing!!!
        });
    }
    else{
        //do the thing!!!
    }
}

You set it up like this after AudioQueueNewOutput and before AudioQueueStart. Notice the passing of bridged self to the inClientData argument. The queue actually holds self as void* to be used in callback where we bridge it back to an objective-C object within the callback.

AudioStreamBasicDescription format;
AudioQueueProcessingTapRef tapRef;
UInt32 maxFrames = 0;
AudioQueueProcessingTapNew(queue, processingTapCallback, (__bridge void *)self, kAudioQueueProcessingTap_PostEffects, &maxFrames, &format, &tapRef);

You could get the end machine time as soon as the file starts too. A little cleaner too.

void processingTapCallback(
                 void *                          inClientData,
                 AudioQueueProcessingTapRef      inAQTap,
                 UInt32                          inNumberFrames,
                 AudioTimeStamp *                ioTimeStamp,
                 UInt32 *                        ioFlags,
                 UInt32 *                        outNumberFrames,
                 AudioBufferList *               ioData){

    MyObject *self = (__bridge Object *)inClientData;
    AudioQueueProcessingTapGetSourceAudio(inAQTap, inNumberFrames, ioTimeStamp, ioFlags, outNumberFrames, ioData);
    if (*ioFlags ==  kAudioQueueProcessingTap_StartOfStream) {

        uint64_t timeOfLastSampleLeavingSpeaker = ioTimeStamp->mHostTime + (self->audioDurSeconds / ticksToSeconds());
        [self lastSampleDoneAt:timeOfLastSampleLeavingSpeaker];
    }
}
7
  • I am doing something like double estimatedTimeNeded=(lastbufferPacketCount*dataFormat.mFramesPerPacket)/dataFormat.mSampleRate; which gives close results to your float secondsFromNow = ticksFromNow * ticksToSeconds(); however float secondsFromNow = ticksFromNow * ticksToSeconds(); not accurate enough when the buffer size is small because audio is being played fast for some reason. when I choose very large buffer size and apply your first solution it seems somewhat accurate, but I need to test it. Jun 1, 2015 at 20:07
  • processingTapCallback if condition kAudioQueueProcessingTap_EndOfStream is only kicks in after Queue stops being played. Also samplesInThisCallback and secondsInCallback lines are kinda confusing and not accurate for me could you elaborate little bit. For example why multiply this double secondsInCallback = * , you declared local frameCount and also ask from class self->frameCount ? which framecount do you refer? Jun 1, 2015 at 20:10
  • The local variable frameCount is not being used , it is just required by the function AudioQueueProcessingTapGetQueueTime(), but it refers to the number of samples that are being called for in the callback. The value I need from the function is sampleTime, which refers to the playback position in frames at ioTimeStamp. The value self->frameCount is referring to total number of frames in the audio file and I use it to get the remaining number of samples in the buffer. You can pull all of this logic outside of the if (*ioFlags == kAudioQueueProcessingTap_EndOfStream) and it will work too.
    – dave234
    Jun 1, 2015 at 20:57
  • As far as using the first method, AudioQueueGetCurrentTime() doesn't need to be called from within the callback, the only requirement is that the audio is playing, so it actually will work for short files, and it will be much more accurate than the method you posted above. Another thing you can do if you like the second method is change kAudioQueueProcessingTap_EndOfStream to kAudioQueueProcessingTap_StartOfStream and it will calculate the last sample time right when the audio starts. Basically once you get a sample time tied to a machine time you just do the math.
    – dave234
    Jun 1, 2015 at 21:06
  • I think processingTapCallback is a neat solution, I changed the logic little bit like you said and it seems working fine. Jun 1, 2015 at 21:14
1

For me this worked really well for what I heeded:

  • stopping the queue in callback when data is over using AudioQueueStop(queue, FALSE), while:
  • listening to actual stop using kAudioQueueProperty_IsRunning property (happens later than AudioQueueStop() is called, actually, when last buffer gets actually rendered)

after stopping the queue You can get prepared for action You need to execute on audio ending, and when listener fires - actually execute this action.

I am not sure about time precision of that event but for my task it behaved definitely better than using notification straight from callback. There is buffering inside AudioQueue and output device itself so definitely IsRunning listener gives better results as to when AudioQueue stops playing.

0

If you use AudioQueueStop in asynchronous mode, then stopping happens after all queued buffers have been played or recorded. See doc.

You're using it in a synchronous mode, where stopping happens ASAP, and playback cuts out immediately, without regard for previously buffered audio data. You want precise timing, but only because audio is cutting off. Right? So rather than go synchronous + add additional timing/callback code, I recommend going asynchronous:

err=AudioQueueStop(queue, FALSE);

From docs:

If you pass false, the function returns immediately, but the audio queue does not stop until its queued buffers are played or recorded (that is, the stop occurs asynchronously). Audio queue callbacks are invoked as necessary until the queue actually stops.

enter image description here

2
  • Good answer but audio cut off is not my exact problem here, problem is to be able to tell that last buffer is finished playing exactly in milliseconds so I can do an important action the moment of the finish, I use AudioQueueStop(queue, TRUE); on purpose to prove that my calculations are not working May 29, 2015 at 19:54
  • Okay, I see. It sounds like you want the precise time (wall-clock) when the actual last sample goes through the output unit (and produces sound), versus the timing at any time prior, enroute thru the audio plumbing. Is that right? What's your Q callback situation like and do you have a stripped down project to run? My thought offhand is a rendercallback to observe/count the samples flying by, but need more context. May 29, 2015 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.