HLS vs MPEG-DASH vs pure MP4/WebM
What's the best format (?) for delivering video contents online?
Googling tells me that HLS and MPEG-DASH are mostly similar, and I should use one of those for adaptive bitrate streaming, yet YouTube seems to be sending raw MP4 or WebM files. Most video hosting companies seems to use HLS (or at least some kind of their own implementations on top of it).
If anyone has any insight on this, please help me out choosing the best format.
Thanks!
Comments
There's no one format that works on every device.
MPEG-DASH is supported widely but the problem is it is not supported by some Apple devices - the devices that use iOS. So one would want to deliver HLS to iOS devices and MPEG-DASH to other devices. This requires the content be encoded twice and store content in both formats.
To avoid encoding twice and reduce the storage requirement while supporting both HLS and DASH, content can be encoded to "fragmented MP4" and generate both the HLS and DASH manifest files to use the same fragmented MP4. Fragmented MP4 approach is different to splitting one file into many smaller files and serving one for each HTTP request. A fragmented MP4 file is a single file with the full content. A player generates HTTP range requests to fetch different parts of the same file. The web server needs to support HTTP range requests - many do.
However, some old Android devices don't support fragmented MP4s even though they support MPEG-DASH in split files format.
One needs to think about the player too. While HTML5 players are preferred now, old devices don't support HTML5. So to show videos on many devices, one may want to consider using a player that supports both HTML5 and Flash. Formats supported by the player needs to be taken into consideration too.
So as said, there's no one format that works on every device. A trade-off analysis needs to be done to pick the best approach. If it is OK to not support some old devices, fragmented MP4s with both HLS and DASH manifest files may be selected with an HTML5 player. If storage is not a concern, twice encoded approach can be used. Based on the player(s), some server side scripting and/or client side scripting may be needed to detect the device and guide to use the best format.
The wise receive respect. The dumb demand respect.
A raw MP4 file does not involve any bitrate adaptation (just byte range fetching from one encoded file). The answer will be DASH or HLS. For higher resolutions (where there is actually a benefit from adaptation) YouTube is typically using DASH and many closed networks are either using DASH already or moving to DASH.
In terms of the difference between HLS and DASH, there isn't a lot. Each of them involves fragmenting an H.26x video bitstream. DASH is more widely supported these days, and has a lot of companies involved whereas HLS is really "owned" by Apple. DASH has more features but you probably won't touch most of them.
HLS supports both TS (transport stream) and fragmented MP4 whereas DASH only supports fragmented MP4. DASH typically requires audio to be in a separate file to the video. If you are streaming live (as opposed to on-demand) then nginx's RTMP module can produce HLS pretty easily, whereas on-the-fly DASH is more iffy (in my experience). For VoD you can use tools like MP4Box to prep all the DASH files offline and then it is pretty easy.
One thing you need to consider is the client base. Older devices don't support either DASH or HLS so you end up needing to have some type of plain MP4 anyway, depending on what browsers and mobile devices you want to address.
One more thing: If the video file size is small, there's no need to go for HLS or MPEG-DASH, the whole MP4 can directly be sent to the player as a normal file. If different bit rates should be supported, the file can be encoded accordingly and there needs to be a means to pick the correct file to be delivered. This approach however is not taken for larger files since it would be a waste to send the full content when a viewer wants to watch only a part.
The wise receive respect. The dumb demand respect.
That's a good point about length.
So to figure the right setup, considerations include length of video, resolution you are capturing it at, whether it is live or on-demand, what devices you want/need to support, storage availability/cost, and so on.
@KamalW @tetech
Thanks to both of you for the insight! really appreciate it!
It seems like fragmented MP4 with both DASH and HLS support seems to be the best way to go?
If I understood correctly, fragmented MP4 is just an regular MP4 file, and I just need to create an m3u8 or mpd file with range and timestamp? Can the MP4 file be played directly (be served as a fallback for old browsers) as well, or does the MP4 file need some kind of conversion that should make it hard (or impossible)?
Thanks again!
MP4 divides content into "boxes" and the box ordering is somewhat arbitrary. Unfortunately this means that when a MP4 file is being produced, an encoder typically writes some high-level information at the end of the file (after it is known). To play back progressively, the boxes need to be ordered so that the information is at the start of the file (MP4Box can do this, or ffmpeg with certain options).
Then for DASH, certain information needs to be packaged at the start of each fragment. And as mentioned earlier, audio is stored in a separate file for DASH. Generally this means it is not possible to simply concatenate (or split) a MP4 file for DASH. However, the video and audio content is the same, so re-packaging it is fairly lightweight and can even be done on the fly with a script (not recommended) if storage is a huge issue - it does not require transcoding/heavy CPU.
TS (as opposed to fragmented MP4) is a bit more convenient because there is no box re-ordering. However, you need to have audio integrated into each rate, and audio bitrate typically doesn't vary as much as video, so this means you duplicate the audio.
Another thing to bear in mind is that older devices/browsers can check the H.264 profile indication and some won't play anything not marked as Baseline Profile. But this produces bigger files than High Profile for a given quality.
To be clear, I don't think there's much advantage of HLS/DASH unless you are willing to encode video at multiple bit rates and resolutions so that clients can dynamically adapt. If you're not willing to incur the storage overhead of video at multiple resolutions/rates, then just go with MP4 and byte ranges, no need to fragment. If you're willing to go the full-blown adaptive streaming route, then you'd realistically need 3-4 versions of the video, where the higher ones are encoded using High Profile and the lowest one is encoded using Baseline Profile and also re-packaged as a monolithic MP4 for backward compatibility. A service like Netflix produces literally dozens of variants to suit different devices and platforms - storage cost is an issue for them, but it is a paid service so working across devices is considered vital.
There's no right or wrong way to do it though, because it depends on stuff I wrote earlier about the type of video and your viewers, as well as storage budget and so on.
I just use mp4 with ffmpeg faststart option. (one less seek)
May be a few years down the line we may have one best way to go with. Due to the way technologies evolve, as of now we don't have one specific way that can be considered the best that would suit all circumstances. Multiple formats and devices with varying support don't allow a simpler solution.
What I suggest is that to study different scenarios with their pluses and minuses. That will lead you to pick what would be the best for you.
They are different. Read what @tetech has written. Here are some references that might be useful:
https://stackoverflow.com/questions/35177797/what-exactly-is-fragmented-mp4fmp4-how-is-it-different-from-normal-mp4
https://www.pcmag.com/encyclopedia/term/fragmented-mp4
https://www.bento4.com/documentation/mp4fragment/
When a regular MP4 is fragmented with mp4fragment and DASH and HLS manifests are generated with mp4dash, two MP4 files get created - one for the audio stream and one for the video stream. You will end up with more MP4s if streams with different bit rates are used. A player is given the DASH or HLS manifest file. The player can dynamically pick one audio MP4 and one video MP4 and subsequently make HTTP range requests to fetch chunks of content.
So to play such MP4 content, a player needs at least three files: the manifest file, an audio MP4 and a video MP4. So those MP4 files cannot be served to old browsers that accept MP4 files with both audio and video streams packed together. What I don't know however is if the multi-stream fragmented MP4s produced by mp4fragment can be played by old browsers directly. I wonder even though mp4fragment can produce multi-stream fragmented MP4s, whether the tool is intended to be used that way. In my case however I don't keep them but the multiple MP4s produced by mp4dash.
Recreating a regular MP4 from those MP4 content is easy and doesn't consume much CPU. You will provide the DASH or HLS manifest file to the converter. The converter will pick the audio and video MP4s and generate a regular MP4. You may want to use the switches of the converter to specify which audio MP4 and which video MP4 should be used in case you have multiple streams mentioned in the manifest file. While I am not sure, it may also be possible to directly provide the two MP4s to a converter rather than providing the manifest and specifying the streams.
Note: Updated the original comment to avoid some possible errors.
The wise receive respect. The dumb demand respect.
Original comment above was updated. Now the description is more specific to what I have experienced with some tools.
The wise receive respect. The dumb demand respect.
Having a single set of media files referenced by a number of descriptors like .mpd and .m3u8 for Dash and HLS respectively comes under what is known as the Common Media Application Format (CMAF).
The wise receive respect. The dumb demand respect.
Thanks, I'll search on that as well