The goal of this column is simply to inform and enlighten, and rest assured, I’m definitely not trying to spoil your television watching experience at home. However, you may have been sold a bill of goods by your friendly video service provider, so a little knowledge will go a long way.
When I decided to dive into the topic of video compression, I thought that things would be fairly easy to learn and explain. Little did I know that when the layers of the video compression onion were peeled back, the complexity grew exponentially — and, unfortunately, my knowledge of advanced math stopped abruptly at geometry, and didn’t venture any farther.
It turns out, however, that there’s a fairly easy analogy that explains it.
Terabytes at the Cinema
When we’re at a concert or a business convention where large-screen displays and projection systems are used, we have the luxury of watching uncompressed video. That is, the quality of the video signal, as delivered from the camera (or video server) to the screen, is transmitted at its highest quality — and not compressed in any manner from its original format and resolution. Video data is not tossed out in order to reduce bandwidth and lower the cost of transmission. Granted, the signal is probably going across the arena, not across the nation.
This isn’t to say that compressed video is a degraded signal, but it is certainly not a pure copy of the original. When we’re at a theater that shows movies in digital format, the film is actually a compressed signal transmitted from the server to the projector. A big screen digital movie requires terabytes of storage, so high quality JPEG 2000 compression is used to fit it all on the hard drive that’s delivered to the theater. This is extremely high quality compression (and one that’s quite acceptable to the Hollywood crowd), but that’s not quite the same as the signal that hits your living room.
The Price of Quality
The quality of an uncompressed signal is visually superior, but that comes at a price, and that price is bandwidth — the size of the pipeline that carries the signal from source to destination. When we’re talking about home delivery (including streaming, cable and satellite methods), video compression is required to get a video signal through a very small data pipe, in order to transport that signal (at a low cost) with high efficiency. The goal is simple: reduce the bandwidth as much as possible, degrade the signal as little as one can, and still retain some semblance of the original image.
Sure, the cable companies could reduce the compression ratios and increase the quality, but you wouldn’t be pleased when the monthly bill arrives — because that kind of bandwidth into the home would be unbelievably expensive. Remember, with millions of data streams being transmitted simultaneously at this very moment, there is an amazing demand for bandwidth today. (Side note: sadly, the U.S. lags behind other countries in getting high bandwidth into the home.)
Big Pipes and Soda Straws
When we’re watching high definition video at home, it ain’t full bandwidth, Sparky. It’s a highly compressed signal, with lots of bits tossed out in order to get that signal into your living room — essentially through a tiny soda straw, instead of a fire hose. Video compression uses “algorithms” to do the heavy lifting. An algorithm is a highly complex, well-defined set of instructions that either runs in hardware or software, at millions of times per second. These calculations are performed on blocks of pixels in every video frame — and you can essentially think of them as little compression robots or little decision-making engines.
These algorithms are remarkable, and on most home transmissions, the layman can’t tell that the signal is compressed, but those of us with eagle eyes can spot the difference immediately. From the director’s chair inside the television truck, or inside the control room at the television or network studio, the video is simply gorgeous and uncompressed. All sources such as cameras, recorders and graphics generators are putting out pristine 1080p. Once that signal leaves the truck and shoots up to the satellite or the cable television “head end” (and eventually back down into our living room), the signal is often toast in terms of quality. When the compression algorithms work perfectly, you can’t even tell — the video is simply beautiful. When the algorithms are having a hard time, well, it’s not a pretty sight.
Watch any NFL game or any golf tournament, and check out the quality of the green grass. Look closely. The grass isn’t solid — in fact, it often appears to be crawling. Here, you’re actually seeing the compression algorithms making serious compromises in the quality of the picture, in order to keep the bandwidth within limits (sorry about that, NFL fans).
On a complex scene with lots of color and contrast, the compression is not very noticeable, but on a scene with solid colors, the blocky artifacts can appear, and on a scene with very fast motion, sometimes the algorithms can’t even catch up — and you’re left with image break-up.
So, what in fact is going on, in order to compress high-definition video down a small data pipe, at perhaps 10 Mbps (megabits per second) or less?
An Easy Analogy
The easy answer is that video compression is just like concentrated orange juice. If you wanted to move a huge quantity of orange juice from Florida to California, but you didn’t want to transport all the volume, all you need to do is find the “essence” of the juice, extract it, and send it. The common elements are left behind — namely (in this case), water. At the destination, you add the water back in — and you have something very close to the original (similar, quite tasty, but not perfect).
Video compression can be looked at in the same way. For the algorithms to work at their optimum, something in the image needs to be simplified and tossed out — just like the water in the OJ. The common element most often used to reduce bandwidth in video is “pixel redundancy,” and a lot of math is required for the task.
For example, the compression algorithm measures and samples the first video frame, and compares that frame to the next one in sequence. If its value is the same, a very small instruction is sent to repeat the first frame (the redundant information), for as many times as it remains identical — thus saving a lot of data. In a cartoon or an animated feature with lots of solid colors and lines, there is a tremendous amount of redundant pixel data, as far as the compression robots are concerned. Similarly, in a feature film in which the camera and actors are fairly motionless, the redundant pixel data is easy for the algorithms to spot. In fact, a solid full screen field of color has almost no data in it at all, from the standpoint of the compression robots.
To continue along those lines, if the second frame sample is not quite the same as the first, but pretty close, you subtract the two, and only transmit the difference — again, greatly saving data. Slow pans, slow zooms, actors standing still on the set — all these equal lots of redundant data. The key here is that the algorithms try their best to avoid sending the entire frame. However, if there is big difference between frames (e.g., cuts, fast action, etc.), an entire frame is measured and sent, and the comparison process starts again. If something gets corrupted along the data lines, the algorithms start duplicating bad frames, and the quality can diminish rapidly.
The Next Level
The topic of compression becomes more important as we enter the “4K” era, with four times the pixels of HD. Transmitting native 4K would clog up the nationwide bandwidth almost immediately, but a solution is on the horizon. The successor to the ubiquitous H.264 is H.265, also known as High Efficiency Video Coding (HEVC). This remarkable algorithm is now being tested by a host of broadcast and streaming companies, and demonstrations have been held at broadcast trade shows. HEVC is said to double the compression ratio of H.264 — yet maintain the same level of quality.
I’ll have more about HEVC in a future column, but in the meantime, have a glass of OJ on me, enjoy the broadcast, and ponder what those little compression robots are doing to every frame, as you watch.
