0

I'm working on an Android app that plays back audio. To minimize latency I'm using C++ via JNI to play the app using the C++ library oboe.

Currently, before playback, the app has to decode the given file (e.g. an mp3), and then plays back the decoded raw audio stream. This leads to waiting time before playback starts if the file is bigger. So I would like to do the decoding beforehand, save it, and when playback is requested just play thre decoded data from the saved file. I have next to no knowledge of how to do proper file i/o in C++ and have a hard time wrapping my head around it. It is possible that my problem can be solved just with the right library, I'm not sure.

So currently I am saving my file like this:

bool Converter::doConversion(const std::string& fullPath, const std::string& name) {

    // here I'm setting up the extractor and necessary inputs. Omitted since not relevant

    // this is where the decoder is called to decode a file to raw audio
    constexpr int kMaxCompressionRatio{12};
    const long maximumDataSizeInBytes = kMaxCompressionRatio * (size) * sizeof(int16_t);
    auto decodedData = new uint8_t[maximumDataSizeInBytes];

    int64_t bytesDecoded = NDKExtractor::decode(*extractor, decodedData);
    auto numSamples = bytesDecoded / sizeof(int16_t);
    auto outputBuffer = std::make_unique<float[]>(numSamples);

    // This block is necessary to get the correct format for oboe.
    // The NDK decoder can only decode to int16, we need to convert to floats
    oboe::convertPcm16ToFloat(
            reinterpret_cast<int16_t *>(decodedData),
            outputBuffer.get(),
            bytesDecoded / sizeof(int16_t));

    // This is how I currently save my outputBuffer to a file. This produces a file on the disc.
    std::string outputSuffix = ".pcm";
    std::string outputName = std::string(mFolder) + name + outputSuffix;
    std::ofstream outfile(outputName.c_str(), std::ios::out | std::ios::binary);
    outfile.write(reinterpret_cast<const char *>(&outputBuffer), sizeof outputBuffer);

    return true;
}

So I believe I take my float array, convert it to a char array and save it. I am not certain this correct, but that is my best understanding of it. There is a file afterwards, anyway. Edit: As I found out when analyzing my saved file I only store 8 bytes.

Now how do I load this file again and restore the contents of my outputBuffer?

Currently I have this bit, which is clearly incomplete:

StorageDataSource *StorageDataSource::openPCM(const char *fileName, AudioProperties targetProperties) {

    long bufferSize;
    char * buffer;

    std::ifstream stream(fileName, std::ios::in | std::ios::binary);

    stream.seekg (0, std::ios::beg);
    bufferSize = stream.tellg();
    buffer = new char [bufferSize];
    stream.read(buffer, bufferSize);
    stream.close();

If this is correct, what do I have to do to restore the data as the original type? If I am doing it wrong, how does it work the right way?

6
  • 1
    What is decodedData? Just a pointer? If decodedData is a pointer then sizeof decodedData gives you the size of the pointer itself (which may very well be 8 bytes). But I'm a bit confused as to why you're passing decodedData to outfile.write. Weren't you supposed to store the floating point data (which is in outputBuffer)?
    – Michael
    Nov 16, 2020 at 9:42
  • You are right, this was an error on my side when transferring the example. I also added a few more lines to the code example, so the part where decodedDatais defined is now in there too. But yes, what I want to save (and load) is outputBuffer.
    – michpohl
    Nov 19, 2020 at 5:48
  • Again, sizeof outputBuffer will not give you the size of the memory that outputBuffer points to. You want something like numSamples * sizeof(float).
    – Michael
    Nov 19, 2020 at 6:51
  • So does that mean I cannot just read the size of my buffer, I always have to calculate it?
    – michpohl
    Nov 20, 2020 at 10:27
  • You're dealing with pointers, and the size of a pointer is not the same as the size of the data it points to. You could put your data in a container instead, e.g. a std::vector, which does have a size() member function.
    – Michael
    Nov 20, 2020 at 10:44

2 Answers 2

0

I figured out how to do it thanks to @Michael's comments.

This is how I save my data now:

bool Converter::doConversion(const std::string& fullPath, const std::string& name) {

    // here I'm setting up the extractor and necessary inputs. Omitted since not relevant

    // this is where the decoder is called to decode a file to raw audio
    constexpr int kMaxCompressionRatio{12};
    const long maximumDataSizeInBytes = kMaxCompressionRatio * (size) * sizeof(int16_t);
    auto decodedData = new uint8_t[maximumDataSizeInBytes];

    int64_t bytesDecoded = NDKExtractor::decode(*extractor, decodedData);
    auto numSamples = bytesDecoded / sizeof(int16_t);
 
    // converting to float has moved to the reading function, so now i save decodedData directly.

    std::string outputSuffix = ".pcm";
    std::string outputName = std::string(mFolder) + name + outputSuffix;
    std::ofstream outfile(outputName.c_str(), std::ios::out | std::ios::binary);
    
    outfile.write((char*)decodedData, numSamples * sizeof (int16_t));
    return true;
}

And this is how I read the stored file again:

    long bufferSize;
    char * inputBuffer;

    std::ifstream stream;
    stream.open(fileName, std::ifstream::in | std::ifstream::binary);

    if (!stream.is_open()) {
        // handle error
    }

    stream.seekg (0, std::ios::end); // seek to the end
    bufferSize = stream.tellg(); // get size info, will be 0 without seeking to the end
    stream.seekg (0, std::ios::beg); // seek to beginning

    inputBuffer = new char [bufferSize];

    stream.read(inputBuffer, bufferSize); // the actual reading into the buffer. would be null without seeking back to the beginning
    stream.close();

    // done reading the file.
    
    auto numSamples = bufferSize / sizeof(int16_t); // calculate my number of samples, so the audio is correctly interpreted
    
    auto outputBuffer = std::make_unique<float[]>(numSamples);

    // the decoding bit now happens after the file is open. This avoids confusion
    // The NDK decoder can only decode to int16, we need to convert to floats
    oboe::convertPcm16ToFloat(
            reinterpret_cast<int16_t *>(inputBuffer),
            outputBuffer.get(),
            bufferSize / sizeof(int16_t));


    // here I continue working with my outputBuffer

The important bits of information/understanding C++ I didn't have or get were

a) the size of a pointer is not the same as the size of the data it points to and b) how seeking a stream works. I needed to put the needle back to the start before I would find any data in my buffer.

0

Another alternative is using libsndfile. I'm maintained this forgotten fork that has this library implemented with Oboe library. For example I'm using it in my microphone callback like this:

class MAudioRecorderCallback : public DefaultDataCallback {
  oboe::DataCallbackResult onAudioReady(oboe::AudioStream *oboeStream,
                                        void *audioData, int32_t numFrames) {
    auto result =
        DefaultDataCallback::onAudioReady(oboeStream, audioData, numFrames);

    auto *inputInts = static_cast<const int16_t *>(audioData);
    ...
      mSoundRecording->write(inputInts, numFrames * oboeStream->getChannelCount());
    ...
    return result;
  }

  AndroidAudioDeviceManager *m_aadm;
  SoundRecording* mSoundRecording = nullptr;

public:
  MAudioRecorderCallback(AndroidAudioDeviceManager *pManager)
      : m_aadm(pManager) {
    mSoundRecording = &m_aadm->mSoundRecording;
  }
};

And the file will be starting to write with a simple:

mSoundRecording.initiateWritingToFile(path.c_str(), mOutputChannelCount, mSampleRate);

The complete implementation of SoundRecording class is here.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.