VP9

Transcription

MATT FROST: Welcome to the first session on WebM and the New VP9 Open Video Codec. We figured that there's no way to really add a little excitement to a presentation than to change it at the last minute, and so what we've spent this morning doing is encoding some VP9 video and H.264 video and putting together a side by side demonstration just to give you a taste of what we're working on. So what you're going to see is a video. The video is going to be the same on either side. It's going to be VP9, the new codec on the left, H.264 on the right. And H.264, we used the X264 open video encoder, which is commonly regarded as the best encoder out there. We used the highest possible settings. So we've done everything we can to favor H.264 here. All of this is at the same data rate, so both of the videos are going to be at the same data rate. And the bit rate varies. In some cases, we're using 500K. In other cases, we've dropped the bit rate down to bit rates that are actually banned by certain UN conventions for the compression of HD video. And so with that, I think that's everything, Ronald? RONALD BULTJE: Yes. So like Matt said, what you're looking at here is shots that we just took this morning. We've encoded those in just a couple of hours and basically, what you're looking at here, on the left, VP9 and on the right, H.264, is what an amazing job we can actually do at video compression if we're using the very latest technologies. MATT FROST: So you can see the blockiness on the right. On some of this, it's a lot more evident than others, and especially evident, if you want afterwards to come up and take a look at this running on the screen, we can freeze frames. But you see there on the right especially, all this blockiness and how much it clears up as it moves into VP9 territory. RONALD BULTJE: And a point here really is that for high definition video, H.264 can do a reasonable job, but we can do a lot better than that. And so having said that, let's actually get started on the presentation. MATT FROST: So the way that we're going to handle this presentation is I'm going to do a quick introduction on why we care about open video, both why does Google-- which has historically been involved with developing applications around video-- has gotten down deeply into actually helping work on these next generation compression technologies. After we talk about that and why, in general, improving video compression is good for everybody, I'm going to turn it over to Ronald for really the meat of this presentation, which will be to show you some more demonstrations, to talk a little bit about how we measure video quality, talk about some of the techniques that we're exploiting to really make this dramatic improvement in compression. And then finally, after you've seen this, and I hope that you've started to get a little excited about what this technology can do for you, we'll go and talk about the last stages, how we're going to wrap up this project and how we're going to get these tools into your hands as quickly as possible. So to start off with, just taking a quick look at how Google got into video. Video at Google started in the same way that so many big projects at Google start, as an experiment. And we launched these efforts with just a single full time engineer and a number of engineers working 20% of their time on video, really focusing on video-related data. And then over the last 10 years, obviously, video at Google has exploded, not only with YouTube but with Google Talk, Hangouts, lots of applications where you wouldn't necessarily think of video as playing a core role, like Chromoting, which is Chrome Remote Desktopping. But if you look at the really motivating factors for getting into video compression, there are a couple that are really of note. One, of course, is the acquisition of YouTube. And with the acquisition of YouTube, we all of a sudden started to focus very heavily on both improving the experience for users, improving video quality, but also about the costs associated with all aspects of running a service like YouTube. There are costs associated with ingest, transcode of video formats, storage of multiple different formats, and then distribution of the video, both to caches and to the edge, and ultimately to users. The second was the move from HTML4 to HTML5, which came at the same time, pretty much, as our launch of Chrome. And of course, in HTML4, although to the user, it appeared that video could be supported in a browser, in fact, video was supported through runtimes and plug-ins. With HTML5, video becomes a native part of the browser. And so with the move towards HTML5, we see it filtering through the addition of the video tag in Chrome and the launch of HTML5 video for YouTube. So these are the two factors-- the focus on quality and reducing cost with YouTube, the need to build a high quality codec into Chrome and other browsers for the video tag-- that sparked the acquisition in 2010 of On2 Technologies, the company that I came from and many members of the WebM team came from, and the launch of the WebM project. The WebM project is an effort to develop a high quality, open alternative for web video. We're very focused on web video, not on video for Blu-ray discs, not on video for cable television, but about solving the problems that we find in web video. In addition, we're very focused on having an open standard because we believe that the web has evolved as quickly is it has because it is based on open technologies. And clearly, multimedia communication has become such a core part of how we communicate on the web that we need open technologies that are rapidly evolving to allow us to keep pace and to make sure that we can develop the next generation of killer video applications. We wanted something simple as well. So we used the VP8 Open Codec, the Vorbis Open Audio Codec, which was a long existing open audio codec, and then them the Matroska File Wrapper. With the launch of VP9 in a matter of months, we're going to be adding the VP9 Video Codec as well as the brand new Opus Audio Codec, which is another open audio codec, very performant and high quality. So since our launch, obviously, web video has continued to grow. And if we just look at what we know very well, which is YouTube, YouTube has grown to be a global scale video platform capable of serving video across the globe to these myriad connected video enabled devices that we're all using. It supports a billion monthly users, and those users are looking at video four billion times a day for a total of six billion plus hours of video viewed monthly. Just to think about that number, that is an hour of video for every person on the planet consumed on YouTube. And on the creation side, we're seeing exactly the same trends. 72 hours of video is uploaded per minute, and that video is increasingly becoming HD video. So if you look at the graph on the right, blue is 360p standard definition video, which is slowly declining, but quickly being matched by uploads of HD video. And the key here of great importance is that HD video is obviously more complex. There's more data for a given HD video than there is for-- unless, of course, you're encoding it in VP9-- than there is for a standard resolution video. In addition, I think we can all agree that the better the video is, the higher the resolution, the more watchable it is. And then finally, the other trend that's driving both creation and consumption is the increase in mobile devices and the move towards 4G networks. So even this morning, there was an article when I woke up and was checking my email saying that YouTube video accounts for 25% of all downstream web traffic in Europe. And I think BitTorrent accounted for 13%. So there alone, between just two web video services, we're looking at close to 40% of all web data in Europe being video related data. And that accords with what we see from the latest Cisco forecasts, for instance, which is that consumer web video is going to be close to 90% of all consumer data on the web within the next three years. So it's remarkably encouraging to see the growth in video, but it also represents a real challenge. Of course, the good news is that we have a technology that is up to this challenge, and that is VP9. With next generation video codecs, with the codecs as good as VP9, we can effectively significantly increase the size of the internet and we can significantly increase the speed of the internet. So obviously, if you're taking VP9-- which, as Ronald will say, halves the bit rate you need for the very best H.264 to deliver a given quality video-- you're going to be able to speed the downloaded of a download and play a video, you're going to be able to speed, obviously, the buffering of these videos. So we have the tools to effectively dramatically increase the size of the internet. But of course in doing that, in improving the video experience, in improving the ability to upload video quickly, we're going to just create the conditions for even more consumption of video. And so it's not going to be enough for us to rest on our laurels with VP9. We're going to have to turn to VP9 and keep on doing it, keep on pushing the boundaries of what we're capable of with video compression. So with that, I'm going to turn it over to Ronald to show you some really remarkable demonstrations of this new technology. RONALD BULTJE: Thank you. So to get started, I just briefly want to say some words about video quality. So how do we measure quality? Well, the most typical way to measure quality is to just look at it, because at the end of the day, the only thing that we care about is that the video that you're looking at looks great to your eyes. But that's, of course, not all there is to it because as we're developing a new video codec, we cannot spend our whole day just watching YouTube videos over and over and over again. That would be fun, though. So in addition to visually analyzing and inspecting video, we're also using metrics. The most popular metric in the field for measuring video quality is called PSNR. It stands for Peak Square Noise Ratio. And the graph that you're looking at here on the left is a typical representation of PSNR on the vertical axis and video bit rate on the horizontal axis to give you some sort of a feeling of how those two relate. So the obvious thing to note here is that as you increase the bit rate, the video quality, as measured by this metric, increases. So at the end of the day what that means is that it doesn't really matter what code you use, as long as you've infinite bandwidth, you can accomplish any quality. However, our goal is to make it easier and faster and simpler to stream video. So how does PSNR actually compare to visual quality? So for that, there's a sample clip. So what you're looking here is a very high penalty shot of the New York skyline. I believe that this is the Empire State Building. And this clip has a lot of detailed textures all across. So what we've done here is that we've encoded it at various bit rates, and then every couple of seconds, we're dropping the bit rate and the metric quality of the video will slowly decrease. So this is 45 dB, and what you're seeing slowly at 30 dB is that some of the detail, or actually a lot of the detail, in the backgrounds of the buildings just completely disappears. And that was the case at 35 dB already also. As you go to 25 dB, you can see-- we can go really low in quality, but you do not want to watch this. Here's a different scene. Same thing, we start with the original 45 dB. 40 dB looks pretty good. 35 dB starts having a lot of artifacts, and then 30 and 25 are essentially unwatchable. So what does that mean for video quality? Well, the typical target quality for high definition video on the internet lies rounds 40 dB. You were just looking at the video, and a 40 dB looked really quite good. So if you go to YouTube and you try to stream a 720p video, that's actually about the quality that you will get. In terms of bit rate, what you should expect to get is a couple of megabits a second. For this particular clip, that's one to two megabits a second, but that's very source material dependent. So what we've done, then, is we have taken, I think, about 1,000 YouTube CCL licensed uploads, just randomly selected from whatever users give us, and we've then taken out particular material that we're not really interested in, such as stills or video clips that contain garbage video content. And then we were left with, I think, about 700 CCL licensed YouTube uploads, and we've encoded those at various bit rates-- so at various quality settings-- with our VP9 Video Codec or with H.264 using the X264 encoder at the very best settings that we are aware of. Then for each of these clips, we've taken the left half of the resulting compressed file and the right half of the 264 one and we've stitched those back together, and then you essentially get what you're looking at here. So left here is VP9, right is 264, and those are at about the same bit rate. You will see graphs here on the left and on the right, and those are actually the effective bit rate for this particular video clip. And as you can see, it starts being about equal. Now, you saw it just jumping up, and that's because we're gradually increasing the bit rate to allow the 264 encoder to catch up in quality. And as you can see, it slowly, slowly starts looking a little bit better. And at this point, I would say that it looks about equal on the left and on the right. But if you look at the bit rate graphs, you can basically see that we're spending about two and a half times the bit rate on a 264 file versus the VP9 file. So those are the compression savings that you can get if you do same quality encodings but you use VP9 instead of 264. So what you're looking at here is a comparative graph for the clip that you were just looking at. The blue line is the 264 encoded version and the red line is the VP9 encoded version. And as I said in the beginning, vertical axis is PSNR as a metric of quality, and the horizontal axis is bit rate. So the way that you compare these is that you can pick any point from the red line-- or from the blue line, for that matter-- and then you can do two things. Either you can draw a vertical line and find the matching point on a blue line that matches the points on the red line that you're looking for and look at what the difference in quality is. But what we usually do is we do it the other way around. So we're drawing a horizontal line for the point on the red graph, and we're finding the point that matches the horizontal line on the blue. And what you're looking at here is that for the point that we were just looking at, that is, a quality metric point of about 37.1 dB, the VP9 version takes an average of 328 kilobits a second to reach that quality, and for H.264, you need to go up to essentially 800 kilobits a second to get exactly the same quality. So what that means is, again, the metrics tell us you can get a two and a half times lower bit rate and effectively get the same quality by using VP9 instead of 264. If you look to the higher end of the graph, you will see that the differences in quality for the same bit rates might go slightly down, but that's basically just because at the higher end, there's a diminishing returns for bit rate. So if you look at the high ends of both of those graphs and you do the horizontal line comparison, so what is the different bit rate that accomplishes the same quality? You will see that it about comes down to 2x over the whole graph. So let's look at the difference video because I could just be cheating you with this one video and we could have optimized our codec for this one video. So what you're looking at here is, again, the same thing, VP9 on the left, 264 on the right, live bit rate graphs and we start at the same bit rate. Then as we do that, we're slowly increasing the bit rate for the 264 portion video so that it can actually catch up in quality. And what you're looking at is that on the right, the floor is pulsing a lot. You can actually see, if you focus on the pants of little boy here or on the plastic box, that it's very noisy. But eventually, it catches up in quality. Guess what happened to the bit rate? It's almost 3x for this particular video. So here is the [INAUDIBLE] graph for the material that we were just looking at. The red line is VP9, the blue line is H.264. And if we do the same quality different bit rate comparison at the point that we were just looking at, which is about 38.6 dB, for VP9, you arrive at about 200 kilobits a second, and for H.264, you need to interpolate between two points because we don't have an exact match, and it ends up being around 550 kilobits a second. So almost 3x more bit rates to accomplish the same quality, just because you can use VP9 to save this. So we've done this over many, many clips. I told you we had about 700 clips that we tested this on at various bit rates and various quality settings, and overall, you can save 50% bandwidth by encoding your videos in VP9 instead of H.264 at the very best settings that we are aware of. So how did we do this? So let's look a little bit at the techniques that we're using to actually get to this kind of compression efficiency. So a typical video sequence consists of a series of video frames, and then each of these video frames consist of square blocks. So for current generation video codecs, like H.264, these blocks have a size of a maximum 16 by 16 pixels. We've blown this up a lot. We have currently gone up to 64 by 64 pixels for each block, and then at that point, we introduce a partitioning step. And in this partitioning step, we allow you to do a vertical or horizontal partitioning, a four-way split, or no partitioning at all, resulting in different size sub-blocks. If you do a four-way split and you have four 32 by 32 blocks, then for each of these blocks, you go through the same process again of horizontal, vertical split, four-way split, or no split at all. If you do the four-way split, you get down to 16 by 16 pixels, do the same thing again to get to eight by eight, and eventually four by four pixels. So what this partitioning step allows you to do is to break up the video in such a way that it's optimized for your particular content. Stuff that has a very stable motion field can use very large blocks, whereas video content where things are moving all across all the time, you can go to very small video blocks. So what do we you do after that? So after this partitioning step, we're usually doing motion vector coding, and basically what that does is that you pick a reference frame, and you pick a motion vector, and then the block of that particular size that you selected in your partitioning step will be coded using a motion vector pointing in one of the previously coded reference frames. These reference frames in VP8 were usually frames that had previously been encoded, and were therefore temporarily before the current frame. What we've added in VP9 is that we have multi-level alt reference frames, and what that allows you to do is encode the video sequence in any frame order, and then you can use any future frame as a reference frame for a frame that you encode in order, decide to encode after that. So for this series of frames in the left, this is six frames. I could, for example, choose the first thing encode frame one, then frame six, and then frame three using both a future as well as a past reference. And then, now that I have encoded three, I can encode one and two really efficiently because they have a very proximate future and past reference. After I've encoded two and three, I go to five, which has four and six as close neighbors. And so that allows for very temporally close reference frames to be used as a predictor of contents in the current block. So once you have a motion vector, you can use subpixel filtering, and subpixel filtering allows you to basically pick a point in between two full pixels and this point in between is then interpolated using a subpixel interpolation filter. In VP8, we had only a single subpixel interpolation filter. Most codecs use just a single subpixel interpolation filter. We've actually added three in VP9, and those are optimized for different types of material. We have a sharp subpixel interpolation filter, which is really great for material where there's a very sharp edge somewhere in the middle. For example, that city clip that we were looking at in the beginning, if you're thinking of a block that happens to be somewhere on the border between the sky and a building, we consider that a sharp edge, and so using an optimized filter for sharp edges actually maintains a lot of that detail. On the other hand, sometimes there's very sharp edges but those are not consistent across video frames across different temporal points in the sequence that you're looking at. And that point, this will cause a very high frequency residual artifact, and so for those, we've added a low pass filter. And what the low pass filter does is that it basically removes sharp edges, and it does exactly the opposite as a sharp filter. Lastly, we have a regular filter, which is similar to the one that VP8 had. After this prediction step, you have predicted block contents and you have the actual block that you're trying to get as close as possible to, and then the difference between these two is the residual signal that you're going to encode. So in current generation video codecs, we usually use four by four or eight by eight cosine based transforms called DCTs to encode this residual signal. What we've added in VP9 is much higher resolution DCT transforms all the way up to 32 by 32 pixels, and in addition to using the DCT, we've also added an asymmetric sine based transform called ADST. And the sine based transform is optimized for a signal that has a near zero value at the edge of the predicted region, whereas the cosine is optimized for a residual signal that has a zero signal in the middle of the predicted signal. So those are optimized for different conditions, and together, they give good gains when used properly. Basically, the take home message from all of this is that we've added big resolution increments to our video codecs, and what that leads to is a codec that is highly, highly optimized for high definition video coding. But at the same time, because it is very configurable, it still performs really well at low resolution content, for example, SIF-based 320 by 240 video as well. So I'll hand it back to Matt now, who will take over. MATT FROST: Thanks, Ronald. So I just want to give you a quick recap of what we've discussed and sort of the highlights of this technology, and then to tell you about the last steps that we're going through to get VP9 in your hands. As Ronald said, we're talking about technology here that is 50% better than literally everything that everybody else out there is using. And actually, we made a point to say we were using the very best encoder out there at the very best settings, settings which I really think you're not seeing very often in the real world because they're very difficult to use in a real world encoding environment. So I hope that there are a number of people in this audience now who are out there, either with existing products with video or products to which you're looking to add video, or just you're thinking about how you can use these tools to launch a new product and to come out with a start-up. This technology has not been used by anyone right now. YouTube is testing it and we'll talk about that in a little bit, but if you adopt VP9, as you can very quickly, you will have a tremendous advantage over anybody out there with their current offering based on 264 or even VP8. It's currently available in Chrome, and the libvpx library on the WebM project is out there for you to download, compile, and test. It's open source. You will have access to source code. The terms of the open source license are incredibly liberal so that you can take the code, improve it, optimize it, modify it, integrate it with your proprietary technology, and you're not going to have to give back a line of code to the project. You're not going to have to be concerned that you will inadvertently open source your own proprietary code. And then finally, it's royalty free. And obviously, this is something that was of great importance to us as we sought to open source a video technology for use in HTML5 and the video tag. We believe that the best is still to come in terms of video products on the web, and that in order to make sure that people are free to innovate and that start-ups are free to launch great new video products, we have to make sure that they're not writing $5 or $6 million checks a year to standards bodies. We're working very hard on putting this technology into your hands as soon as possible. We did a semi freeze of the bit stream just a couple of weeks ago, and at that time, we said that we were taking comments on the bit stream for 45 more days. Specifically, we're looking for comments from a lot of our hardware partners to some of the software techniques that we're using just to make sure that we're not doing anything that's incredibly difficult to implement in hardware. At the end of the 45 day period on June 17, we're going to be bit stream frozen, which means that after June 17, any VP9 encoder that you use is going to be compliant with any VP9 decoder, and that if you're encoding content with an encoder that's out after June 17, it's going to be able to play back in a decoder after the bit stream freeze. Obviously, getting VP9 in Chrome is very important to us. The beta VP9 which you've been seeing today is already in Chrome. If you download the latest development version of Chrome and enable the VP9 experiment, you'll be able to play back VP9 content immediately. As soon as we've frozen the bit stream as of June 17, we're going to roll it into the Dev Channel of Chrome as well with this final version of VP9, and then that's going to work through the beta channel and through the stable channel. And by the end of the summer, we are going to have VP9 in stable version of Chrome rolling out to the hundreds of millions of users. I think [INAUDIBLE] today said that there are 750 million users of Chrome right now. VP9 is going to be deployed on a massive scale by the end of summer. In terms of final development activities that we're going to be working on, after the bit stream is finalized in the middle of June, we're going to be focusing on optimizations both for performance and for platform. So what that means is we'll be working on making sure that they encoder is optimized for a production environment. Obviously, something that's very important to YouTube as YouTube moves to supporting VP9, that the decoder is sufficiently fast to play back on many of the PCs that are out there. We're also going to be working on platform optimizations that will be important to Android developers, for instance, and to people who want to support VP9 on embedded devices. These are ARM optimizations and optimizations for other DSPs. We have hardware designs coming out. For those of you who may work with semiconductor companies or are thinking about a technology like this for use in something like an action camera, these are hardware designs that get integrated into a larger design for a semiconductor and allow for a fully accelerated VP9 experience. Real time optimizations are obviously incredibly important for video conferencing, Skype style applications, and also for new applications that are coming out like screencasting and screen sharing. By the end of Q3, we should have real time optimizations which allow for a very good real time performance. Those optimizations should then allow VP9 to be integrated into the WebRTC project, which is a sister project to the WebM project and basically takes the entire real time communication stack and builds it into Chrome, and more broadly into HTML5 capable browsers. And so what this means is that when VP9 is integrated into WebRTC, you will have tools that are open source, free for implementation that used to, even four years ago, require license fees of hundreds of thousands of dollars. And you, with a few hundred lines of JavaScript, should be able to build the same sort of rich video conferencing style applications and screencasting applications that you're seeing with products like Hangouts. And finally, in the end of this year moving into Q1 2014, we're going to see, again, hardware designs for the encoder. So just to give you an idea of how usable these technologies are, we have a VP9 demonstration in YouTube. If you download the Development Version of Chrome and flip the VP9 tag, you can play back YouTube VP9 videos. And one thing this should drive home is this was a project that was done over the course of two weeks, that VP9 was built into YouTube. Obviously, we have very capable teams. Obviously we have people on the WebM team and people on the YouTube team who know a lot about these tools, but this demonstration is VP9 in the YouTube operating environment. There's nothing canned here. This is VP9 being encoded and transmitted in the same way that any other video is. So this, I hope, again, will give you guys pause to say, god, we could do this as well. We could come out very quickly with a VP9 based service that will be remarkably better than anything that's out there right now. So I just want to leave you with some thoughts about what I hope that you're thinking about coming away from this presentation. The WebM project is a true community-based open source project, and obviously, these sorts of projects thrive on contributions from the community. We are coming out of a period where we've been very intensively focused on algorithm development. Some of this work is certainly very complicated stuff that not every-- even incredibly seasoned-- software engineer can work on. But we're moving into a point where we're focusing on application development, we're focusing on optimization, we're focusing on bug fixes and patches, and that's the sort of thing that people in this room certainly can do. So we encourage you to contribute and we encourage you to advocate for use of these technologies. We build open source technologies, and yet simply because we build them, that doesn't mean that people adopt them. It takes work to get communities to focus on adopting these sorts of open technologies. So advocate within your project in your company, advocate within your company for use of open technologies, and advocate within the web community as a whole. We think that with VP9, we've shown the power of a rapidly developing, open technology, and we hope that people are as excited about this as we are and that you go out and help spread the word about this technology. But most important, we'd like you to use the technology. We're building this with a purpose, and that is for people to go out, take advantage of these dramatic steps forward that we've made with VP9. And so we hope you will go out, that you'll be charged up from this presentation, and that you'll immediately download the Development Version of Chrome and start playing around with this and start seeing what you can do with this tool that we've been building for you. So there are just a couple of other things I'd like to say. There are a couple of other presentations related to this project. There's a presentation on Demystifying Video Encoding, Encoding for WebM VP8-- and this is certainly relevant to VP9-- and then another on the WebRTC project. And again, if you're considering a video conferencing style application, screensharing, remote desktopping, this is something that you should be very interested in. Sorry. I shouldn't be using PowerPoint. So with that, we can open it up to questions. Can we switch to just the Developers Screen, guys? Do I do that? AUDIENCE: Hey there. VP8, VP9 on mobile, do you have any plans releasing for iOS and integrating with my iOS applications-- Native, Objective C, and stuff? Do you have any plans for that? MATT FROST: He's asking if VP8 is in iOS? AUDIENCE: VP9 on iOS running on top of Objective C. RONALD BULTJE: So I think as for Android, it's obvious Android supports VP8 and Android will eventually support VP9 as well. For iOS-- MATT FROST: When I was talking about optimizations, platform optimizations, talking about VP9, that's the sort of work we're focusing on, ARM optimizations that should apply across all of these ARM SOCs that are prevalent in Android devices and iOS devices. There aren't hardware accelerators and iOS platforms right now. Obviously, that's something we'd like to change, but presently, if you're going to try to support VP8 in iOS, you're going to have to do it through software. AUDIENCE: Thank you. RONALD BULTJE: Yep? AUDIENCE: Bruce Lawson from Opera. I've been advocating WebM for a couple of years. One question. I expect your answer is yes. Is it your assumption that the agreement that you came to with MPEG LA about VP8 equally applies to VP9? MATT FROST: It does apply to VP9 in a slightly different way than it does with VP8. The agreement with MPEG LA and the 11 licensors with respect to VP9 covers techniques that are common with VP8. So obviously, we've added back some techniques we were using in earlier versions, we've added in some new techniques, so there are some techniques that aren't subject to the license in VP9. But yes, the core techniques which are used in VP8 are covered by the MPEG LA license, and there will be a VP9 license that will be available for developers and manufacturers to take advantage of. AUDIENCE: Super. Follow up question. About 18 months ago, the Chrome team announced they were going to drop H.264 being bundled in the browser, and that subsequently didn't happen. Can you comment further on whether Chrome will drop H.264 and concentrate only on VP9? MATT FROST: I can't really comment on plans going forward. What I can say is that having built H.264 in, it's very difficult to remove a technology. I think when you look at the difference between VP9 and H.264, there's not going to be any competition between the two. So I think with respect to VP9, H.264 is slightly less relevant because there was nothing-- we didn't have our finger on the scale for this presentation. And especially, we were hoping to drive home with that initial demonstration which we put together over the last few hours that we're not looking for the best videos. We're just out there recording stuff. So even if 264 remains in Chrome-- which I think is probably likely-- I don't think it's going to be relevant for a next gen codec because there's just such a difference in quality. AUDIENCE: Thanks for your answers. AUDIENCE: Hi there. I have a question about performance. Besides the obvious difference in royalty and licensing and all that, can you comment on VP9 versus HEVC, and do you hope to achieve the same performance or proof of [INAUDIBLE]? RONALD BULTJE: So the question is in terms of quality, how do VP9 and HEVC compare? AUDIENCE: Yeah, and bit rate performance, yeah. RONALD BULTJE: Right. So testing HEVC is difficult. I'll answer your question in a second. Testing HEVC is difficult because there's currently no either open source software or commercial software available that can actually encode HEVC unless it's highly developmental in nature or it is the development model. The problem with the alpha and beta versions that are currently on the market for commercial products is that we're not allowed to use them in comparative settings like we're doing. Their license doesn't allow us to do that. Then the problem with the reference model is it is a really good encoder, it gives good quality, but it is so enormously slow. It can do about 10 frames an hour for a high definition video. That's just not something that we can really use in YouTube. But yes, we've done those tests. In terms of quality, they're currently about equal. There's some videos where HEVC, the reference model, is actually about 10%, 20% better. There's also a couple of videos where VP9 is about 10%, 20% better. If you take the average over, for example, all of those CCL licensed YouTube clips that we looked at, it's about a 1% difference. I think that 1% is in favor of HEVC if you so wish, but 1% is so small that really, we don't think that plays a role. What does that mean going forward? Well, we're really more interested in commercial software that will be out there that actually encodes HEVC at reasonable speed settings. And like I said, there's currently nothing on the market but we're really interested in such products, so once they are on the market and we can use them, we certainly will. AUDIENCE: Follow-up question about the performance. Is this any reason to not expect this to scale up to 4K video or [INAUDIBLE]? RONALD BULTJE: We think that the current high definition trend is mostly going towards 720p and 1080p. So if you look at YouTube uploads, there is basically no 4K material there, so it's just really hard to find testing materials, and that's why we mostly use 720p and 1080p material. MATT FROST: But certainly when we designed the codec, we designed it with 4K in mind. There aren't any limitations which are going to prevent it from doing 4K. RONALD BULTJE: Right. You can use this all the way up to 16K video if that's what you were asking. MATT FROST: Sir? AUDIENCE: Yeah. Have you been talking to the WebRTC team, and do you know when they're going to integrate VP9 into their current products? MATT FROST: We talk with the WebRTC team regularly. As I said, we've got to finish our real time enhancements in order to actually have a codec that works well in a real time environment before we can expect it to be integrated into WebRTC. But I think we're looking at Q4 2013. AUDIENCE: Great, thanks. MATT FROST: We're in 2013, right? RONALD BULTJE: Yeah. AUDIENCE: Hi. I just wanted to talk about the rate of change in video codecs. I think maybe we can see like VP8, VP9, we're talking about an accelerating rate of change. And that's great, and I really wanted to applaud the efforts to getting this out in Chrome Dev quickly, or Chrome Stable quickly. I just wanted to ask about maybe some of your relationships with other software vendors that are going to be relevant, like we're talking Mozilla, IE, iOS was, I think, previously mentioned. As this kind of rate of innovation in codecs increases, how are we going to make sure that we can have as few transcode targets as possible? My company is working on a video product. We don't want to have eight different codecs. And if we can imagine, let's say, that Version 10 comes out relatively soon, sometime down the road. How can we make sure that devices stick with a relatively small subset of compatible decodings? MATT FROST: I guess I'm a little unsure of what you're asking. In terms of how we get support on devices as quickly as possible, or how we solve the transcoding problem? AUDIENCE: And just keeping the number of transcoded formats as small as possible. If IE only supports H.264, I have to have an H.264 encoding. So I was just wondering what kind of relationships you guys are working on to make sure that as many devices and platforms as possible can support something like VP9. MATT FROST: We're certainly working very hard on that, and as I said in the slide on next steps showing the timeline, our focus on having hardware designs out there as quickly as possible is an effort to try to make sure that there's hardware that supports VP9 more rapidly than hardware has ever been out to support a new format. We had a VP9 summit two weeks ago, which was largely attended by semiconductor companies. Actually, some other very encouraging companies were there with great interest in these new technologies. But we're working very hard with our hardware partners and with OEMs to make sure that this is supported as quickly as possible. I think internally, what we're looking at is probably relying on VP8 to the extent that we need hardware now and we don't have it in VP9. So I think what we've talked about is always falling back to an earlier version of an open technology that has very broad hardware support. But we're trying to think very creatively about things like transcoding and things that we can do to ensure backwards compatibility or enhancement layers. So part of the focus of this open development cycle and process that we have is to really try to think in very new ways about how we support new technologies while maintaining the benefits of hardware support or device support for older technologies. AUDIENCE: Excellent. Thank you. AUDIENCE: So a key point in any solution is going to be performance. Hardware acceleration really solves that, and that was one of the challenges with the adoption of VP8 in timing versus H.264, which has broad spectrum hardware acceleration. I understand the timing, the delays, and the efforts you guys are doing to really achieve that hardware accelerated support for VP9. But until then, what's the software performance in comparison to H.264, for either both software, software, or software, hardware? RONALD BULTJE: So we've only done software, software comparisons for that. Let me start VP8 264. Currently, VP8 decoding is about twice as fast as 264 decoding using fully optimized decoders. VP9 is about twice as slow currently as VP8, decoding, and that basically means that it's exactly at the same speed as H.264 decoding. That's not what we're targeting as a final product. We haven't finished fully optimizing the decoder. Eventually, what we hope to get is about a 40% slowdown from VP8 decoding, and that will put it well ahead of the fastest 264 decoders that are out there in software. AUDIENCE: Great. Thank you. AUDIENCE: Hello. I was just wanting to get some background on the comparison between H.264 and VP9. For H.264, what were you using-- CVR, BVR, and what QP values? RONALD BULTJE: This is two path encoding at the target bit rate. So it's preset very slow. Since we're doing visual comparison, there is no tune set. It's paths one or two, and then just a target bit rate. We tend to choose target bit rates that are somewhere between 100 and 1,000 kilobits a second, and then we just pick the same point for the VP9 one as well to start with. AUDIENCE: So in both of the comparisons, you were trying to be very generic so you weren't tuning the encoder in any way to make it a better quality at that bit rate. You were just giving it two paths to try to figure it out. RONALD BULTJE: So you mean visual quality, or-- AUDIENCE: Yes. RONALD BULTJE: So we haven't tuned either one of them for any specific setting. For 264, the default is that it optimizes for visual experience, and so that's why we optimized it to 6414. So it's not optimized for SSIM or PSNR in the visual displace that we did here. VP9 encoding does not have any such tunes, so we're not setting any item, of course. AUDIENCE: So you just used the default settings of [INAUDIBLE]? RONALD BULTJE: We're using the default settings, and we've actually discussed this extensively with the 264 developers. They agree. They support this kind of testing methodology, and as far as I'm aware, they agree with it. They fully expect the kind of results that we're getting here. AUDIENCE: Right. OK, thanks. AUDIENCE: Hi. One more question about performance. I think you mentioned a little bit about the real time. So do you think in the future, you can manage to bring an application like application desktop into the web? I mean like putting three, four windows in the same browser, high definition, things like that? RONALD BULTJE: In terms of decoding or encoding? AUDIENCE: Both. RONALD BULTJE: So for encoding, yes. So there will be real time settings for this codec eventually. For no codec will that get you exactly the types of bit rate quality ratios that you're seeing here. These are really using very slow settings, and that is by far not real time. But if you set the VP9 codec to real time settings, then yes, eventually it will encode in real time. It will be able to do four full desktops all at once, and it will be able to decode all of those also. You'll probably need a multicore machine for this, obviously, but it will be able to do it, yes. AUDIENCE: And you're using the graphics card and other things like that. You didn't mention about the hardware, OpenGL or-- RONALD BULTJE: It's future software. There's no hardware involved. AUDIENCE: No using the hardware, the card hardware. RONALD BULTJE: We're not using GPU or anything like that at this point. AUDIENCE: Thank you. AUDIENCE: Hi. I just want to know, how does a VP9, now or later, compare to VP8 and H.264 when we're talking about single task CBR, low bitrate, real time encoding? Little background is we are part of the screen sharing utility that currently uses VP8, and we've been successfully using it for a year, but the biggest gripe with VP8 is that it doesn't respect bit rate, especially on low bit rates, unless you enable frame dropping, which is unacceptable. So we have to do a bunch of hacks to actually produce quality and it doesn't behave like H.264 would in that situation. So how will VP9 address that problem, or is that even on the roadmap? RONALD BULTJE: So in general, desktop sharing and applications like this, also real time communications, yes, they're on the roadmap, and yes, they will all be supported. In terms of your specific problem, I guess the best thing to do is why don't you come and see us afterwards in the Chrome [INAUDIBLE], and we can actually look at that. AUDIENCE: OK, awesome. RONALD BULTJE: As for VP9, VP9 currently does not have a one pass mode. We've removed that to just speed up development, but it will eventually be re-added, and it will be as fast as the VP8 one but with a 50% reduction in bit rate. AUDIENCE: Do you have a timeline for that? Is it going to this year, or next year? RONALD BULTJE: Like Matt said, that will happen-- MATT FROST: Late Q3. RONALD BULTJE: Q3 2013, around then. We're currently focusing on YouTube, and those kind of things will come after that. AUDIENCE: Awesome. Thank you. AUDIENCE: I have two questions, unrelated questions to that. What is the latency performance of VP8 compared to VP9 in terms of decoding and encoding? And the second question is, how does VP9 compare to H.265? RONALD BULTJE: So I think H.265, I addressed earlier. So do you want me to go into that further, or was that OK? AUDIENCE: More in terms of the real time performance. RONALD BULTJE: So in terms of real time performance, I think for both, that's really, really hard to say because there is no real time HEVC encoder and there is no real time VP9 encoder. So I can sort of guess, but this is something that the future will have to tell us. We will put a lot of effort into writing real time encoders or adapting our encoder to be real time capable because that is very important for us. MATT FROST: But in terms of raw latency, it should be faster than VP8. You can decode the first frame, right? RONALD BULTJE: I think it will be the same as VP8. So VP8 allows one frame in, one frame out, and VP9 will allow exactly that same frame control model. AUDIENCE: So you mentioned that you've asked hardware manufacturers for any concerns or comments. Have you gotten any yet? MATT FROST: Sorry. Are considering supporting it? AUDIENCE: Well, in terms of the algorithms and how you would actually-- MATT FROST: They're working on it quickly. AUDIENCE: But there's no concerns or comments or anything yet? MATT FROST: No concerns. AUDIENCE: You said you opened up for comments. MATT FROST: No. We have received comments. We have a hardware team internally that took a first pass at comments. We've received a couple of comments additionally just saying, here's some stuff you're doing in software that doesn't implement well, and hardware. I don't foresee a lot of additional comments from the hardware manufacturers. The other work that we're doing over the next 45 days is we had a bunch of experiments that we had to close out, and so we're doing some closing out as well and just finishing the code. Absent act of God, this is bit stream final on June 17. RONALD BULTJE: So we have actually received comments from some hardware manufacturers, and we are actively addressing the ones that we're getting. AUDIENCE: OK, thanks. AUDIENCE: Hi. I might have missed this, but when did you say the ARM optimizations for VP9 are going to come out? MATT FROST: Actually starting now really, we're focusing on doing some optimizations by ourselves and with partners. So I would say that's going to be coming out second half of the year, and it'll probably be sort of incremental where you may get an initial pass of ARM optimizations and then some final optimization. It's obviously very important for us for Android to be able to get VP9 working as well as possible, and obviously, ARM is incredibly important for the Android ecosystem, so that's an area of significant focus. AUDIENCE: And in terms of real time encoding, so in order to blend into WebRTC, you're going to have to get that working. So is this going to coincide with the assimilation of VP9 into WebRTC? MATT FROST: It'll be real time optimizations, which I think we were sort of thinking about end of Q3, beginning of Q4, and then integration into WebRTC will follow on that. Obviously, the one thing I'd say, it's an open source project. If you guys think that you see an opportunity, you can go out and do the optimizations yourselves. There are contractors who can do it. So I encourage you guys to think about that, that you can take the code and you can start working on some of this stuff yourselves. Obviously, we'd love it if you'd contribute it back but we're not going to force you to. Yeah, I guess last question. AUDIENCE: This is a question about how VP9 relates to what the Android team talked about with Google proxy and the speedy proxy. You alluded to transcoding real time for backwards compatible device support. Do you see Google doing the same thing they're going to do with images in this proxy and doing video transcoding to adapt this and use this for compression mode in the Google proxy? RONALD BULTJE: That's a really interesting application, and that's something that we'll have to look into the future. It's not as easy as it sounds because video transcoding actually takes some time. So that would mean that you would actually have to wait a minute while the video is transcoding until you can visit that website, and that might not be quite what you're looking for. But it's an interesting application and we might look into that in the future. MATT FROST: I think that's it. I think we're out of time. Sorry, but we're happy to talk to you afterwards. [APPLAUSE]