Our application uses the Twilio voice SDKs for iOS, Android, and Web. Our use case relies on precise device synchronization and time stamping. We are playing an audio stream on multiple adjacent devices (in a Twilio conference call) and we need that audio playback to be in sync. Most of the time, it works great, but every now-and-then, one of the devices falls a little bit behind and throws off the whole experience. We want to detect when a device is falling behind (receiving packets late) so we can temporarily mute it so it does not throw off the user experience we are going for.
We believe that Twilio voice uses real time communication (web RTC) and real-time transport protocol (RTP) under the hood. We also believe RTP has time stamping information for when packets are sent out and when packets are received.
We are looking for any suggestions for how we might read this timestamp information (both sent & received) to detect device synchronization issues.
Our iOS and Android clients are built using Flutter & Dart, so any way to look at this packet information using Dart would be great. If not, we can use native channels through Swift and Kotlin. For the web, we would need a way to look at this timestamp data using javascript.
If possible, we'd like to access this information through the SDK. I don't see anything about timestamps in Twilio's voice documentation. So, if not possible, we might have to sniff for packets on the devices? This way, we could look at the RTP packets coming from Twilio to see what information is available. As long as this does not break Twilio terms of service, of course :)