H.264 AVC has been the king of codecs for years, but now video service providers have a wide choice of next-generation codecs from MPEG (HEVC, EVC, VVC, LC-EVC) and AOM (AV1). In this episode we talk with Brian Alvarez, until recently with Amazon Prime Video, and Vittorio Giovara from Vimeo, about the codec selections they have made and their outlook for the future. Navigating the codec landscape becomes much easier when guided by these Video Insiders!
Brian Alvarez LinkedIn profile
Vittorio Giovara Blog
---------------------------------------------------
Join our LinkedIn Group so that you can get the latest video insider news and participate in the discussion.
Email thevideoinsiders@beamr.com to be a guest on the show.
Learn more about Beamr
Announcer (00:00):
The video insiders is the show that makes sense of all that is happening in the world of online video, as seen through the eyes of a second generation codec nerd and a marketing guy who knows what I frames and macro blocks are. Here are your hosts - Mark Donnigan and Dror Gill.
Dror Gill (00:16):
Hello and welcome to another episode of the video insiders. I'm Dror Gill, and with me is my co-host Mark Donnigan. Hi, Mark.
Mark Donnigan (00:25):
Hello Dror. It's great to be back and talking about codecs today.
Dror Gill (00:30):
Yeah, it's wonderful to talk about codecs, um, ever-changing landscape of codecs, and today we have two video insiders who are really experts and can help us navigate through this maze of codecs, especially...
Mark Donnigan (00:45):
The minefield...
Dror Gill (00:46):
Minefield. Yeah, that's right...
Mark Donnigan (00:49):
Through the jungle...
Dror Gill (00:50):
Through the jungle... You touch one of them. You want to use it, it looks good and shiny, but then boom - it explodes in your face...
Mark Donnigan (00:57):
Oh boy. Okay. All right. Okay. This is not, this is not a podcast about war by the way. There are no, there are no codec Wars on this podcast.
Dror Gill (01:07):
No, Dan Rayburn always says there are no codec wars, no platform wars, whatever. Okay. So, I would like to welcome to the podcast, Bryan Alvarez and Vittorio Giovara. Brian was with Amazon until recently and Vittorio is with Vimeo. So hi, Brian and Vittorio, and welcome to the podcast.
Brian Alvarez (01:28):
Hey guys. So, I'm Bryan Alvarez. I was the Principal Product Manager for Encoding, Prime Video, and then was a Principal Product Manager on Platforms and some advanced capabilities with Fire TV, uh, prior to leaving Amazon. Good to be here.
Vittorio Giovara (01:48):
Hello everybody, I'm Vittorio. Thanks for having me. I lead the video and encoding team at Vimeo. We are the research team, um, and we take care of anything that happens to the video once it is uploaded. So, I deal with the codec configuration, uh, streaming and storage and all this sort of stuff.
Mark Donnigan (02:11):
Welcome Victoria. It's really great to have you.
Dror Gill (02:14):
Yeah. Yeah. And, um, and I'm sure we're going to have a wonderful discussion about codecs. So I'd like to start with your personal perspectives on the codec landscape. Tell us about, the codecs that you are actually using in production and how are they split in terms of the encodes you do, or the streams that you deliver. What's the breakdown between those codecs. Brian, maybe you'd like to start.
Brian Alvarez (02:37):
Sure. So, you know, prime video supports AVC and HEVC. Originally, right, the service was all AVC, um, and then brought on HEVC to start to cover UHD and then HDR, but increasingly, right, we, our strategy there, when I was product managing, is we would send HEVC wherever we could that was practical, right. For both to offer higher quality and to reduce, you know, delivery costs.
Dror Gill (03:06):
And today you're still with AVC and HEVC.
Brian Alvarez (03:09):
Correct. Prior to, to my, you know, leaving prime video and moving over to Fire TV, right, we were definitely evaluating all the future options, right, that will we'll discuss.
Dror Gill (03:20):
Yeah, definitely. And Vittorio, how about Vimeo?
Vittorio Giovara (03:25):
So Vimeo has of course, a full lineup of H.264 streams that are universal for video. And then, depending on the type of content we receive, we switch to HEVC or AV1. If the video has HDR meta data, we use HEVC Main 10 and use HDR 10 to provide a colorful, um, videos. And, if otherwise, if the video is a staff pick, which means it's one of the videos that is, highlighted in, on the homepage, uh, we also provide AV1 encoding for all of them since July.
Dror Gill (04:13):
What is your positioning around AV1? I mean, what is the, what's the logic behind doing the featured videos in AV1? Is it because, they will be streamed a lot more than other videos, so you want to reduce their bit rate, or is it a kind of, let's say, an experimental feature or an initial feature that you're starting with the featured videos, and then you'll see how it goes, and then expand it to more videos.
Vittorio Giovara (04:45):
Uh, it's a little bit of both. We use AV1 exactly because, uh, it, it is able to deliver a better quality than, uh H.264, and, the videos, the Staff Pick videos are usually the ones that are watched the most. So it makes sense that we are able to provide a better viewing experience for those videos. And it was also a way to be able to use the, the codec itself because, the encodings of AV1 take a much longer time than H264. And even if the, the gap is, is closing, it's better to focus the encoding efforts on the content that has been going, that is going to be watched the most.
Dror Gill (05:29):
Yeah, definitely.
Mark Donnigan (05:30):
And you're, you're also primarily distributing into the browser? Or what's the playback environment?
Vittorio Giovara (05:38):
Well, it's predominantly in the browsers, yes. We have a lot of mobile apps and the devices that we also support.
Mark Donnigan (05:48):
And so on the Apple ecosystem is, is AV1 supported or is that just limited to Windows and Chrome?
Vittorio Giovara (05:58):
AV1 right now is in Firefox and Chrome. And they, they also represent the majority of our viewership on Apple devices. We use, uh, HEVC when, when the video meets the requirements.
Dror Gill (06:13):
When it has HDR.
Vittorio Giovara (06:15):
Exactly. Yeah.
Brian Alvarez (06:16):
Hey Vittorio I've got a question for you. Did you guys go straight to AV1, or were you using VP9 before you went to AV1?
Vittorio Giovara (06:23):
We went straight to AV1. We believe it's a much better codec. And it offers us a few features that VP9 does not. Primarily the color properties, VP9 is, is very limited in that regard, and needs to rely on the container for signaling correctly all the properties. Whereas AV1 is a full fledged codec with everything that we need. And we knew it was, when we were looking at VP9, AV1 was really close to release. So we just thought that we would be waiting for it to be out.
Dror Gill (06:59):
Okay. That's nice. I mean, that's one, less codec to worry about...
Mark Donnigan (07:03):
Now, Brian, you know, I'm curious - in your ecosystem, at Amazon, so how did you look at codecs based on devices, um, that you needed to support? Was there a bit of a matrix there where, you know, clearly not all devices support all the codec standards, right? So how, how did you think about that when you're trying to decide?
Brian Alvarez (07:26):
Yeah, I mean, you know, our goal obviously, right, is to get content to every possible, you know, Prime Video user, right? Regardless of device. So by default, right, that means supporting AVC and I foresee in my own personal opinion, right, that AVC is going to have a long shelf life because there'll be so many customers with older devices, right, well, into the future that only support AVC. And device deprecations are painful, right? If you try to deprecate a class of devices you're likely to lose customers, right. Even if you have efforts to retain them by either giving them a device or some incentive to upgrade. And then, you know, for us, in Prime Video was really about reach. So we would just look at the device landscape and what gives us the greatest amount of reach. Compression efficiency is obviously nice. Um, there's also the factor of just being able to offer new features, right? So we, we wouldn't be able to really effectively do UHD and HDR with AVC, which led to adopting HEVC. It's also nice that like HEVC really started to grow, right, from the early living room devices into mobile. Um, so then there was the advantage of being able to give a better viewing experience to people on mobile, especially in data constrained countries like India. Um, so it's, it's, uh, it is a matrix. Um, but I think it's, it's largely guided by how can we reach the largest group of customers and give them the best experience possible.
Mark Donnigan (08:56):
Sure.
Vittorio Giovara (08:57):
Yeah. I, I think I agree with this, this sentiment. The codec itself should be used as an enabler more than, you know, a tag that you have - "I use this codec". It's more, what can I do with this codec and how much audience can I make happier?
Dror Gill (09:13):
Yeah. That's an important point because I think in the early days of AV1, you know, maybe a year ago, 18 months ago, uh, people would kind of wave their, um, AV1 support saying: "Hey, I do AV1. I can do AV1 Live, you know. Of course I'm using a hundred computers at the same time on the cloud, but hey, I can do AV1 in live". And, and, uh, you know, it was more of a marketing spin around your coding service. But, uh, but now I think it's, it's, it's starting to mature, you know, and Vittorio you're talking about providing a better experience, um, to those browsers, uh, who support AV1 for those, uh, premium video that your editors, select. So it's really kind of a practical consideration and not just, you know, use it because, because you can.
Brian Alvarez (10:06):
And I think that's, you know, when I was in Prime Video, that's where AV1 started to become very attractive. Especially as the decode, um, requirements started to lessen, right, with things like David. There was a gap in giving customers high quality video at low bit rates in the browser, right? HEVC was not that available in browsers, and so, um, from my perspective, when I was in Prime Video AV1 really had a sweet spot of being able to deliver that next gen codec performance, right, to customers looking at Prime Video on browsers.
Mark Donnigan (10:39):
And I know that Prime Video is really invested in HEVC, you know, in terms of whether it's AV1 or VVC. Can you comment at all on, uh, just what, what the thoughts are there? Can you give some guidance like, uh, you know, are we going to see HEVC for some time to come? Is it possible that, you know, AV1 or VVC or LC EVC is, is going to begin to be rolled out?
Brian Alvarez (11:07):
You know, I, I don't see HEVC going away for a couple reasons, right. Even if you could run, say software AV1 decoding on some living room devices, which looks increasingly possible, you're not going to be able to satisfy the studio level DRM requirements that somebody like Prime Video has, right? So, so there's still going to be, you know, millions if not tens of millions of devices in the marketplace, that the only way you can get 4k to in a secure way, right, with full DRM protection is going to be HEVC. And people hold on to their devices for a surprisingly long time, right? So you can't just abandon those, those folks. I think the other practicality too is, you know, as I'm sure we all are aware, you know: AV1, VVC, EVC - substantially heavier compute overhead. And especially if you're trying to move into, you know, on-demand encoding for long tail content or content that gets seldom used, it's less attractive, right? And we're, we're, I think starting to get into that sweet spot with HEVC where potentially on-demand encoding, right, makes a lot of sense. And for sure it does with AVC, right, another reason why AVC, I think, will have a long life. I don't know that the delta in quality between HEVC and AV1 is really that big, at least not from what I've personally observed. It has its benefits, but I wouldn't say it's that much better, right? That you would want to just abandon HEVC for AV1.
Dror Gill (12:37):
Yeah. Codec comparisons we know are in the eye of the beholder, uh, there are so many parameters you can play with and content and settings, and you'll see two different, you know, um, academic, um, papers, you know, one saying it's 40% better, and one saying it's exactly the same in terms of compression efficiency, and, and, you know the truth is probably somewhere in between. Um, so yeah, it's, it's interesting, you know, the, the fact you mentioned, regarding the software decode, and, and I wanted to touch this point because I was thinking of it from a perspective of performance - does your device have, enough CPU power in order to decode in software or does it need a hardware decoder. And then if it needs a hardware decoder, it means you need to wait for a generation of that device that supports a new codec, like AV1, or maybe in the future VVC. But you're bringing on another perspective to that, which is - software decode is less secure than hardware decode. So for premium content, you want hardware decode, and it's not only because of the performance, it's because of the security, right?
Brian Alvarez (13:51):
It's a huge factor for Prime Video, right? And I would imagine for other services, right, that are either creating their own, you know, professional content, right, or have a mix of professional and Hollywood and other premium content. All of those video operations have to happen in a trusted execution environment.
Mark Donnigan (14:09):
Does that ultimately Brian come down to, um, because there needs to be a hardware root of trust and that needs, you know, and that's in the Silicon?
Brian Alvarez (14:18):
Yeah, I mean, so, you know, we're, we're bound by a couple of different constraints, right? So one of them is MovieLabs security requirements, right? Especially in places with prime video that offers the Hollywood content. And when you look at the studios, Right, there, the MovieLabs requirements and even individual studios' requirements, or that there has to be this, you know, below the how, uh, handling of video information, right? So all the decode has to happen basically in the trusted execution environment at the T. So it's to protect, you know, I mean, everything, even down to the video frame buffer is basically operating at that level, right, so you don't expose the bits or the pixels until essentially it goes to be drawn on the screen, or it goes to the HDMI output. And so, you know, there's liability implications, obviously, if you don't do that. Um, and then, you know, in general, because of the DRM involved, the DRM also operates in that, that level typically. Um, and so for security, right? Like you, you just have to support that. And, and I've yet to see, maybe the industry might move to address this, the ability to do software video decoding in the T, right? Typically it's tied to either hardware or a very low level software decode that's, you know, part of the operating system.
Mark Donnigan (15:40):
Vittorio does Vimeo. I mean, Vimeo must have some content that, you know, essentially can be free and in the clear, but you must also have some that needs to be protected, right?
Vittorio Giovara (15:53):
I have very controversial ideas on this topic. Uh, I believe that DRM is not, nothing secure is just give the illusion of security and protection, but it's, anything in software or hardware can be broken with enough skill. Having said that, we try to not be involved in, in that vehicle and rely on a third party, uh, to supply that, uh, when a client really needs that level of compliance.
Dror Gill (16:22):
When you moved to AV1, when you added support for AV1, did you run into issues with, uh, supporting it, packaging it in streaming protocols or applying DRM to it? Um, because it's a new codec, was all of the other ecosystem compatible with the new codec or you ran into issues with, with those, um, components that, that compliment the video encoding?
Vittorio Giovara (16:47):
Uh, I have to say Vimeo is in a really unique situation because most of the software that we use is built in-house. So everything except the encoder software itself and the decoder, uh, which is FFmpeg, is built in-house and can be modified on the fly, uh, for any kind of, uh, need that we might have - HDR, HEVC, AV1, we were able to modify the, our packager to support, uh, all the, all the features that we needed.
Mark Donnigan (17:17):
Now we've been talking about, uh, on the, on the decode side. So the, the player side, you know, software versus hardware. Um, but it's, it's interesting that on the encoding side, um, as codecs are getting more complex and just requiring a whole lot more compute, ironically, we've been of course spending the last, um, seven, eight, even maybe 10 years, sort of switching from black box encoders to software. And, uh, now there's kind of a path where we can see that hardware is going to begin playing a role again. Um, obviously that's going to look very different. Those are not just going to be black box, you know, purpose-built encoders, but there's, you know, GPU approaches, there's FPGA, there's ASICs. Talk to us about, you know, what, what you guys are seeing in your respective, um, networks and even just sort of the industry at large, like, is it true that GPUs and FPGAs, ASICs are starting to be thought about, or are you just totally focused on CPU or, you know, what, what do you think?
Vittorio Giovara (18:23):
Uh, Vimeo is is mostly focused on the cloud. Um, and we use, uh, CPU encoding. Uh, it's, it's true that the GPU, uh, are more and more, um, performant, but, um, I feel like that the whole data model is kind of a stretch and it's not working very well in that regard. Even so, like having a GPU in the cloud, uh, for these kinds of experiments would be, uh, outrageously expensive. And, and so we do focus only on CPU encoding,
Dror Gill (19:00):
But do you think if you had an order of magnitude more videos coming in, right, because you are a platform of user generated content. As the volume grows, you know, does it make sense to consider other platforms that are, let's say more power efficient? Because you know, you can outgrow the CPU if you have like too many videos.
Vittorio Giovara (19:24):
Well, you can have, we can take a different approach. And we were mentioning this before, uh, maybe tweak your encoder configuration. For example, one great solution is to enable chunked encoding, and, uh, being able to spin off smaller machines that they can be discarded after use, uh, makes a lot of sense for when, uh, when encoding at scale.
Brian Alvarez (19:47):
And I think, you know, I, I can speak a bit to this, too. The, I'm very much on the David Ronca school of thought that we're reaching this non-sustainable level of increased encoder complexity, right? For, uh, decode efficiency. And, well, I can't speak to what Prime Video is doing. I can say that like, while I was there, and I'm still a firm believer of this, um, I definitely see a future, right, in alternates to CPU, let's even say x86 CPU, right? Like, I think there's absolutely the, a good opportunity in both moving to ARM, um, as well as, you know, for really high-performance - FPGA and, and ASICs. I mean, fundamentally, you know, I look at, you know, chunked coding and distributed encoding, or parallel encoding as a brute force solution. We're just throwing a bunch of boxes, right, and instances at a problem and just subdividing down, right, to smaller and smaller chunks and like, sure, we could keep doing that, but in my mind, that's not the elegant way to do this. Right. If we can do more work per given resource, that makes a lot more sense to me.
Dror Gill (20:58):
Eventually it's all down to cost, right? How much does it cost to encode the video? Uh, all of the videos you need to encode into all of the formats and ABR layers, et cetera, that you need to create, um, on a CPU versus a GPU or an FPGA. And if you don't run your own data center, then that translates into the cost of those, uh, platforms that are offered on the cloud.
Brian Alvarez (21:23):
One thing to consider though, you know, in, in the case of, uh, you know, Prime Video, right? The, you look at the cost of encoding versus the cost of delivery and delivery is, you know, by far the bigger cost, right? So there are significant savings to be gained, right, by reducing the delivery costs and they more than cover, right, the encode costs.
Mark Donnigan (21:48):
That's interesting because I know there are certain regions of the world, uh, where delivery costs are just significantly higher than other parts of the world. And I'm thinking of Dan Rayburn's latest, um, CDN pricing survey that he does, like - bandwidth is getting down to fractions of a penny. So at the extreme high end, in terms of volumes, so like Amazon prime video and Netflix, and, you know, the other services and Vimeo, I'm sure you must be in, you know, certainly a similar sort of range - bandwidth is, is meaningful even at fractions and fractions of a penny per gig transferred. But there's still so many distributors that are so far below that, you know, that at the end of the day, um, you know, on a percentage basis, you save them 30, 40%. And that translates to maybe tens of thousands of dollars, you know. Meanwhile they're running these computer operations, um, that, uh, are getting more and more expensive, um, because their volume of videos can, you know, might be high in terms of the number of videos coming in to their, to their service or, you know, their platform. Um, they're just not getting played a lot. And it seems to me like this is where these, these new hardware architectures, um, whether it's ASICs, FPGA, you know, or just more efficient GPU, um, on x86, it could play a role, but I do see a chicken and the egg, because it's true - if you're going to own your own infrastructure, you know, you have to go install all of these, and now they're setting idle and, you know, or you have to find a cloud that, a public cloud that has a sufficient number deployed that you can access. So I suppose that's also going to be a little bit of a, of an, of an initial challenge here.
Vittorio Giovara (23:46):
On that regard the decoding is usually very little compared to the encoder costs because that's, that's by design. And the, most of the, of the problem relies in, in the, encoding as the, the decoder side will follow to whatever the encoder will decide. Uh, and yeah, it's, it's a self fulfilling prophecy because, uh, the more, the more content is produced in a certain encoder, the more devices that will be able to digest it. And, uh, I don't know if you, if the, um, the specification of the new MacBooks are already, uh, popular. They do mention a wide support in, for hardware decoding in a variety of codecs and HEVC, AV1 and H.264 are all there already.
Brian Alvarez (24:37):
You know, the other part of it too, is I see a general move away from, you know, CPU. Because we also have to do an increasing amount of machine learning and computer vision analysis on video. And that's also fairly inefficient to run on a CPU. So, you know, I see the need in the future to really have specialized instance types in our hardware, right, for each step of the chain, if you will, right? Like, obviously you got to do some analysis with, with ML and CV. You might have to do some image processing or some other type of like SDR to HDR conversion, for example, right, or some kind of color space mapping, and then the, the raw encode itself. And it just, to me, seems inherently more efficient to have dedicated architectures for each of those steps versus trying to cram it all into a CPU.
Dror Gill (25:28):
And then the video needs to pass between different machines.
Brian Alvarez (25:32):
Yeah. I mean, and that's the other challenge, I would say, we could discuss it if you want, but there's definitely also the problem in cloud compute right now that you're moving data around between instances far too much.
Mark Donnigan (25:43):
You know, I would infer from your most recent comment there, Brian, that the benefits of hardware, again, whether it's FPGA, ASIC et cetera, apply to VOD equal to Live,. Even though I think a lot of us, we kind of, you know, sort of delineate like, okay, Live - yes, hardware is definitely, but Hey, if it's VOD - we can just throw CPU at it. But what I heard you just say is that, you know, there's complexities that are coming or even are in the workflow now where it doesn't really matter. Is that true? That we shouldn't just be splitting this discussion based on Live versus VOD?
Brian Alvarez (26:21):
Absolutely. And, and the lines are blurring too, right? I mean, it doesn't happen often, but like one of the challenges right, with premium content is you have a release date, right, that's fixed and immovable, and it doesn't always mean that you get the master a month before that., right? Uh, sometimes the stuff , literally they're working in post-production to, you know, a few days before release. So like throughput is really important in VOD. And I think sometimes people don't realize that, um, also the sheer volume, right? Like you look at like the catalog size of somebody, like, you know, Prime Video, um, there's a lot of content coming in and you can't always just queue it and wait for availability. So being able to do things quickly is, is definitely beneficial. But what we're also seeing, right, um, is there's a blurring between Live and VOD. There, there's typically a lot of VOD content now that is used as a enhancement for Live and linear. Um, and that stuff also has to be processed very quickly. Sometimes you're getting files, you know, post produced files that are coming in hours before basically a Live event, right, or, or some linear feed. So, you know, it's, it's a combination of everything. You know, increasingly, uh, I don't know about, you know, other streaming services, but for, for Prime Video, in particular, since we're offering such a wide selection of services, right, we have, you know, channels for VOD, right, SVOD, linear and live sports, That architecture, basically, it looks like everything that's in the video industry.
Dror Gill (27:55):
And, and you mentioned Brian before, the concept of on-demand encoding. And I think this is sometimes called just in time encoding, for that, um, segment of the content that is not viewed a lot. So, you keep it, instead of keeping all of the different versions and encodes for hundreds of types of devices, you just keep the master, and then when somebody requests this title, you encode it on demand. Right?
Brian Alvarez (28:24):
Yeah. I, I really see this as a very good solution for somebody like Prime Video, right, that has a really large library. Um, I've traditionally found that like the 80/20 rule really holds, right, like roughly 20% of your content generates 80% of your views, yet we're expending the same amount of storage and compute, right, on the other 80% of the content. It doesn't logically make a lot of sense. It's intuitive that we should be moving to a model where the resources are allocated to the things that get watched the most. And if something increases in popularity, right, or consumption, then, then you go in and add the bit rates or formats as needed. So on demand and just in time encoding I think it's just as attractive as packaging in just-in-time.
Dror Gill (29:09):
And Vittorio, in Vimeo, do you use any JIT encoding?
Vittorio Giovara (29:14):
Um, we're, we're working on it. It's in the, in the pipeline. I wanted to add to the previous discussion about VOD and Live. It's almost much, much harder to make a good Live than making a good VOD, because for VOD, you can throw as much CPU as, as needed, while for Live. You have so much to take into consideration, uh, latency, delay, and of course, good quality. And from the encoder's perspective it's much, much more complex than VOD.
Dror Gill (29:51):
Yeah. I agree. The live stream just keeps coming in. Mark, you remember who said "it just keeps coming in"?
Mark Donnigan (30:00):
Uh, I'm at a loss.
Dror Gill (30:01):
It was Newman, Newman.
Mark Donnigan (30:02):
Oh yeah, yeah. That's right.
New Speaker (30:05):
Because Seinfeld asked him, you know, why is it, why is it that every time somebody starts shooting around in a McDonald's, it turns out to be a mailman. And then Newman says, well it's because the mail, it just keeps coming in every day., all the time, keeps coming in. You can't do anything about it. So it's exactly the same with the live stream. It just keeps coming in, right? In real time. And you have to process each one of those packets, no delay, you can't stop it. It's just 24/7, right? It can drive you crazy.
Mark Donnigan (30:39):
Well, Vittorio, you have a team, a Codec team. And I'm curious, are you working with vendors to optimize solutions that you've licensed or maybe even help them in development? Or are you primarily leveraging open source? What does that look like for you?
Vittorio Giovara (31:00):
Historically, Vimeo has been always very close to, uh, communities and most of the team members are part of the open source community. FFmpeg is basically almost what defines this industry because of being able to upload anything, and you are able to decode it. It's just, just thinking about this is incredible if you think about it. And, um, we found that, we found actually that the encoder support from the open source community, uh, such as x264, x265 and RAV1 are among the best in class, uh, we can provide. And, uh, it also integrates, they integrated very well with our approach of everything in house so we can modify it if needed. And it has also been the case that the community itself helped resolve, uh, a bug that we found because when everybody was working there was improving the software. It's a nice, it's a nice feeling.
Mark Donnigan (32:00):
So in terms of these, you know, when you're looking at new standards and, and you've already adopted AV1, uh, then, uh, can you share, which like, you know, because there's a couple of different, open source projects for AV1, which one you're using and then, you know, how you chose that, or why you chose that particular project over another one.
Vittorio Giovara (32:22):
Yeah. We chose Rav1. And we didn't go with libaom or SVT-AV1. There are a couple of reasons about it. The first one is that we wanted a clean room implementation. Um, Rav1 is, uh, completely written from scratch and it's also written in, a safe and robust language, Rust. And we don't have to explore how that would work. And then there is the, the team that is, uh, backing the, actually that was backing the project. Mozilla has been working very closely with the, uh, with the xiph team was the Codec research team, uh, in Mozilla before they were let go in, in August, uh, due to, you know, Mozilla restructuring. And they're still supporting, uh, the project. It's, um, it's still ongoing. The results are there. It's probably true that Rav1 could be better and I think it will, it will get there with enough resources, and we will see, uh, where this is going.
Mark Donnigan (33:37):
Interesting. And, uh, you know, Brian, how did you guys look at this in terms of leveraging commercial solutions or even just developing in house or leveraging open source projects across the various codec standards?
Brian Alvarez (33:50):
Yeah. This is one of those areas, right? Where, um, Prime Video is famously very secretive. And so there's not really that much I can share, but what I can say is like, you know, as many things in Amazon, we're really driven by giving customers the best experience. So we don't have, um, any sort of dogma, right, around codecs or, you know, open source versus, you know, in-house versus, you know, third-party developed codecs. It's really about finding the best way to deliver good quality video to customers.
Mark Donnigan (34:25):
Yeah. I think at the end of the day, that's a, that's a good approach for all of us. Right? You know, if we're delighting our customers.
Dror Gill (34:31):
Exactly.
New Speaker (34:32):
You know, the tools we use, how we get there, obviously there's, you know, more than one way. Um, but that's what really matters is our customers being, you know, served well.
Dror Gill (34:43):
Yeah. We need to make the customers happy. So, uh, Brian, Vittorio, this has really been an amazing conversation. Uh, I'm sure our listeners learned a lot about which codecs are being used and why, which is also very important. And now I would like to kind of shift the conversation a few years ahead in time, if you would, uh, take out the crystal ball and try to, uh, envision how the codec landscape will look like, let's say three, four, five years from now. We are not going to, uh, check you, you know, in our 267th episode, which will air in July 2024. But, just interesting to get your feeling on, you know, based what we have now and what is emerging, because we have AV1, AV2 will be coming out and then MPEG, uh, this year released VVC and is going to release EVC and there's LC-EVC, which can be added to different codecs. And of course the things you've already deployed, AVC, HEVC and AV1. So the question is: How do you see it, uh, going from here and where will we be in a few years in terms of codec adoption?
Vittorio Giovara (36:08):
My crystal ball is using data that, uh, we have. So, uh, it's saying that the problem we are facing with future codecs will not be technical. It will be mostly a matter of how, how bad is the licensing scheme. Like HEVC, I love HEVC, it's one of the most advanced technical codecs. Uh, it was the first one to release 10 bit in the Main profile and remove interlacing video, which everybody needs to be thankful for that, but it got plagued with the, uh, such an unclear licensing scheme and the current status that VVC is coming out and what's happening in MPEG is not showing much difference for the future. We will either see, uh, a continuous separation between browser vendors and the broadcasting industry, or we will see some broadcasting industries start to adopt uh, additional codec support.
Dror Gill (37:16):
Additional, you mean like, open source codecs.
Vittorio Giovara (37:19):
Yes.
Mark Donnigan (37:20):
I think that's a really astute observation. I totally agree. I see the lines of demarcation splitting, the broadcast on one side and the browser on the other. And it seems like, you know, obviously H.264 cuts across all of those platforms, but, um, you know, in terms of AV1 and potentially AV2 and, you know, other variants that might be coming, there's a high probability that they are going to sort of own the browser. Um, and then in the broadcast side, uh, it seems like there's a very strong possibility that that's going to continue in the direction of the MPEG, um, standards.
Dror Gill (38:02):
Interesting, interesting, Brian, what do you think?
Brian Alvarez (38:05):
Yeah, my, my take is a slightly different perspective, you know, we saw, right, like, for example, the lifespan of MPEG-2 versus the lifespan of AVC or, or not the lifespan, but the, the time to introduction, right. And then it shortened between AVC and HEVC. And it seems basically like we're going to be getting new generations of codecs in a shorter and shorter timeframe, which I think is going to put us in a position where we're going to have quite a lot of codecs coexisting, right? And from my perspective, especially when you're delivering premium content, primarily into living rooms, it's going to come down to what is supported in the Silicon. And it's, it's unclear to me right now. Right? What say somebody like MTK, right. Or M-logic's going to put in their living room SOCs. It seems, I think intuitive that they would probably continue down the MPEG route because they're very familiar with that. But now we've seen that they've had to add AV1, right, in, in 2020.
Mark Donnigan (39:08):
To support YouTube, that's right.
Dror Gill (39:10):
Right. YouTube, and 8K YouTube and things like that, yeah.
Brian Alvarez (39:13):
Exactly. And so, you know, I think we're just going to see a really fractured landscape where for lots of different reasons, right, services are going to pick their own codec., that may not be necessarily, like, the dominant one. And they'll coexist, it'll add a lot of complexity and I'm pretty sure a lot of people's storage costs are going to increase, but, you know, uh, I'll reiterate it again for somebody like Prime Video, storage and encoding are expensive, but nowhere near as expensive as delivery, right, so they'll always be, I think, an acceptance to having a multi codec world so long as you're delivering savings and quality, basically to customers and delivery.
Vittorio Giovara (39:55):
Maybe it's also a problem that will solve itself because, um, H.264 will be, uh, royalty-free in a few years when the last patent expires. And so maybe that will be forever the universal codec that every browser, every device will have to support, and then there will be additional codecs on top of it.
Dror Gill (40:17):
That's right, but it kind of becomes the lowest common denominator, you know, like you can broadcast MPEG-2 or even H.263 today, you know, it, it will work, but nobody wants to use it because as, as we said, we want to delight our customers and give them the premium experience that they expect. Um, so AVC will be there, but a service provider will have to make a choice, you know, do I go, do I fall back to the lowest common denominator or do I have a solution that can give me more, uh, with a codec with better compression efficiency? And I think Mark, this division you, you presented between broadcast and, uh, and browser, uh, the problem is, is convergence, you know. The, the broadcasters are doing OTT streaming to mobile devices and to browsers. And, uh, and then you have, um, you know, Netflix and Amazon Prime and all of those, uh, OTT providers are streaming to televisions, um, that are, you know, broadcast reception devices.
Mark Donnigan (41:20):
Well, it's a multi codec world.
Dror Gill (41:23):
Yeah. Yeah.
Mark Donnigan (41:24):
And it always will be moving forward.
Vittorio Giovara (41:27):
It's great. Actually, the, the browser, uh, what they implemented in the recent, um, last year or so, uh, they, they're they're offering ways to, um, actually switch, codec between, uh, the coding session, um, in, so you can, for example, have a base layer of H.264 up to 1080p and then you could mix and match with the different codecs, the 2K and 4K profiles, for example. And it would be really great if these kind of functionalities could be exported to, to the, to the devices as well. It's a much more complex than a browser of course, but it's something I'm looking forward to.
Dror Gill (42:08):
Interesting. So not just spatial and temporal scalability, you can have codec scalability.
Vittorio Giovara (42:14):
Yeah.
Brian Alvarez (42:15):
Yeah. My understanding is actually like, um, I believe iOS devices do this, right. You can switch between AVC and HEVC.
Vittorio Giovara (42:22):
Yeah. All the browsers support it now.
Mark Donnigan (42:24):
Yeah. Yeah. Well, it's, it's super exciting, uh, we're all gonna, um, have a lot to do for many, many more years. So that's what I hear...
Dror Gill (42:33):
Job security. Great.
Mark Donnigan (42:37):
Job Security, yes, yes. For sure. For sure. Well, this was really a great discussion and, uh, you know, thank you for, for, sharing your, your candid thoughts and your observations. I know that this is, you know, really valuable to our audience. So thank you again, gentlemen, for joining us. It's really great to have you on the show.
Brian Alvarez (42:59):
This was a real honor. Thanks for inviting me on.
Vittorio Giovara (43:01):
I really appreciate the time you spent with us.
Dror Gill (43:04):
Thank You Brian and Vittorio. And we'll meet all of you in the next episode of The Video Insiders.
Announcer (43:10):
Thank you for listening to the video insiders podcast. If you'd like to appear on the show, just send an email to thevideoinsiders@beamr.com. That's B E A M R.com with a brief description on what you're working on and why you think it's interesting for our audience. This podcast is sponsored by Beamr imaging. The views expressed by guests are their own and their appearance on the program does not imply an endorsement of them or any entity that they represent.