Audio and Video

Video: Artificial Intelligence Enables Smoother Video Streaming

14 August 2017

An example of how Pensieve reduces rebuffer in streaming video. Source: MIT CSAILAn example of how Pensieve reduces rebuffer in streaming video. Source: MIT CSAILOne of the world's universal truths is that everyone hates buffering while watching a streaming video.

Pixelation and buffering of videos happen because videos are broken into small chunks that load as you view. If your internet is slow, the video might take a few seconds to load, resulting in low resolution or interruption of streaming.

Video sites such as YouTube and Netflix use adaptive bitrate (ABR) algorithms to try to give users a more consistent viewing experience. ABR algorithms adjust buffering to suit resources available on the destination computer, a technique that reduces buffering. Because people don’t watch videos all the way through and with literally 1 billion hours of videos streamed every day, buffering thousands of long videos for all users at all times would be a major waste of resources.

MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) has developed an artificial intelligence program called Pensieve. It uses machine learning to pick different streaming algorithms depending on network conditions. This program demonstrably delivers a higher-quality streaming experience with less rebuffering than existing systems.

MIT CSAIL found in experiments that the system could stream video with 10 percent to 30 percent less rebuffering than other approaches and at levels that users rated 10 percent to 25 percent higher on key quality-of-experience metrics.

MIT says Pensieve can be customized based on a content provider's priorities. For example, if a user is on a subway and entering a dead zone, YouTube could turn down the bitrate so that it can load enough of the video to prevent rebuffering during the loss of network connectivity.

“Our system is flexible for whatever you want to optimize it for,” says Hongzi Mao, Ph.D. student at MIT. “You could even imagine a user personalizing their own streaming experience based on whether they want to prioritize rebuffering versus resolution.”

How It Works

Researchers combined two types of ABR algorithms: One measures how fast networks transmit data and the other is a buffer-based algorithm, both of which ensure a certain amount of future video is already buffered.

However, both are limited by the fact that they aren’t using information about both rate and buffering. As a result the algorithms often make poor bitrate decisions and require hand-tuning by human experts to adapt to different network conditions.

Pensieve doesn’t need a model or any existing assumptions about network speed. The AI features an ABR algorithm as a neural network and repeatedly tests it in situations that have a wide range of buffering and network speed conditions, MIT says.

Pensieve tunes its algorithms through a system of rewards and penalties. It might get a reward anytime it delivers a buffer-free, high-resolution experience but a penalty if it has to rebuffer.

“It learns how different strategies impact performance, and, by looking at actual past performance, it can improve its decision-making policies in a much more robust way,” Mao says.

Researchers say the AI’s reward system can be customized by content provides using whatever metrics they want to prioritize.

The team tested the AI in several setting such as WiFi at a café and an LTE network while walking down the street. Results showed that Pensieve could achieve the same video resolution as other algorithms but with a reduction of 10% to 30% in the amount of rebuffering.

MIT says the next project the team will work on will be tested Pensieve on virtual reality video.

To contact the author of this article, email

Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the Engineering360
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter
Find Free Electronics Datasheets