Media Source Extension (MSE) needs fragmented mp4 for playback in the browser.
1 Answer
A fragmented MP4 contains a series of segments which can be requested individually if your server supports byte-range requests.
Boxes aka Atoms
All MP4 files use an object oriented format that contains boxes aka atoms.
You can view a representation of the boxes in your MP4 using an online tool such as MP4 Parser or if you're using Windows, MP4 Explorer. Let's compare a normal MP4 with one that is fragmented:
Non-Fragmented MP4
This screenshot (from MP4 Parser) shows an MP4 that hasn't been fragmented and quite simply has one massive mdat
(Movie Data) box.
If we were building a video player that supports adaptive bitrate, we might need to know the byte position of the 10 sec mark in a 0.5Mbps and a 1Mbps file in order to switch the video source between the two files at that moment. Determining this exact byte position within one massive mdat
in each respective file is not trivial.
Fragmented MP4
This screenshot shows a fragmented MP4 which has been segmented using MP4Box with the onDemand
profile.
You'll notice the sidx
and series of moof
+mdat
boxes. The sidx
is the Segment Index and stores meta data of the precise byte range locations of the moof
+mdat
segments.
Essentially, you can independently load the sidx
(its byte-range will be defined in the accompanying .mpd
Media Presentation Descriptor file) and then choose which segments you'd like to subsequently load and add to the MSE SourceBuffer.
Importantly, each segment is created at a regular interval of your choosing (ie. every 5 seconds), so the segments can have temporal alignment across files of different bitrates, making it easy to adapt the bitrate during playback.
-
9A concise specification of what fMP4 is can also be found in the HLS specification in section 3.3.– slhckSep 13, 2017 at 8:23
-
Is sidx stored in mp4 header or some byte range? I would like to fetch the all info about sidx but don't want to download whole mp4 file and parse. thanks– 鄭元傑Sep 10, 2018 at 10:07
-
I wonder where one could find the specifications of the moof frames. There seems to be a counter and timestamp information which I need to access. Jan 10, 2019 at 20:08
-
1So sending an moof+mdat and appending that to SourceBuffer works? Is that what media source expecting. So for example can I just concatenate 10 Frames of h264 data one after another and create only one mdat and make that wrap that 10 frame concatenated h264 data. so something like moof+mdat(10*h264 data) Feb 1, 2019 at 22:41
-
@EvrenBingøl , Did u tried your approach - moof+mdat(10*h264_data). Does Sourcebuffer works with that data ?– KumarSep 16, 2020 at 8:56