On this page
When you stream a physics lecture or a cricket match, your phone rebuilds most of each picture from the one before it. That trick owes a great deal to Arun Netravali, an engineer from India who spent decades at Bell Labs in New Jersey.
Who was Arun Netravali?
Netravali earned a B.Tech from IIT Bombay in 1967 and a PhD in electrical engineering from Rice University in 1970, according to the Engineering and Technology History Wiki. After two years at NASA, he joined Bell Labs in 1972 as a member of technical staff and became head of its Visual Communications Research Department in 1978. He later served as the ninth president of Bell Laboratories, and he died in 2021.
The Marconi Society gave him the 2017 Marconi Prize "for leadership and technical contributions to digital video systems and services". He also received the US National Medal of Technology and India's Padma Bhushan.
Why does video need compressing at all?
Work it out yourself. One full-HD frame is 1920 x 1080 pixels, and at three bytes per pixel (red, green, blue) that is 6,220,800 bytes, about 6.2 MB. At 30 frames a second, one second of raw video is about 187 MB, and a ten-minute lecture would be about 112 GB. That is far more than a typical home connection or mobile data pack can carry.
But two frames a thirtieth of a second apart are usually nearly identical. Netravali's key insight, as the Marconi Society describes it, was that most video changes only slightly from frame to frame, so a coder can send mainly the differences linked to moving objects, after compensating for motion.
What is motion compensation in video?
Motion compensation means predicting a new frame by shifting parts of the previous frame along their estimated motion, then sending only the motion and the small leftover error.
Earlier coders already sent "frame differences". In Motion-Compensated Television Coding: Part I (Bell System Technical Journal, 1979), Netravali and J. D. Robbins estimated how far moving objects had shifted using a recursive algorithm that seeks to minimise the prediction error, with simplifications to make it practical in hardware. In computer simulations on two image sequences, motion-compensated prediction gave bit rates 22 to 50 percent lower than simple frame-difference prediction, according to the paper's abstract.
The Marconi Society credits him with the first iterative algorithm to estimate the motion of objects on a TV screen. He also worked on models of human vision so that coding noise is placed where it is least visible.
A worked example: send the ball, not the picture
Picture a red ball rolling across a still scene, like the diagram above. Split each frame into blocks of 16 x 16 pixels. Whole blocks are easier to picture than Netravali's step-by-step estimate, but the idea is the same.
Frame 1 has already been sent in full.
In frame 2, most blocks (the background and the hill) match frame 1 exactly, so the coder sends a tiny "same as before" signal for each.
For the ball's block, the coder searches frame 1 and finds that the ball moved two blocks right and one down: a motion vector of (+2, +1).
It copies the old block to the new position and subtracts. The difference is almost all zeros.
It sends the vector and that small difference instead of the raw block, which would cost 16 x 16 x 3 = 768 bytes.
The receiver makes the same copy and adds the correction back.
How did his work reach the videos you stream today?
The Marconi Society's fellow profile says Netravali helped launch video coding and compression research at Bell Labs in the early 1980s and HDTV and video networking research in the early 1990s, and that his research team proved the viability of HDTV, resulting in the first commercially viable HDTV system. It describes his pioneering work on video compression standards as the key base technology for MPEG 1, 2 and 4, which enabled digital TV, HDTV and streaming video. He also received an Emmy for the HDTV Grand Alliance in 1994.
What can a student take from this?
Calculus pays off. Netravali's algorithm searches step by step for the smallest prediction error, an idea close to finding where a derivative is zero.
Sampling comes first. Before video can be compressed it must be sampled; read how Harry Nyquist found the sampling rate for digital audio.
There is a career here. Video coding sits inside signal processing; see signal processing careers for ECE students.
Save data. See how to study online with slow internet or a small data pack.
If frames, vectors and prediction error are hard to picture, the Fermi Simulation Lab has interactive simulations for ideas that are easier to see than to read.
Next time a video loads smoothly on a weak signal, count the frames you are not being sent.
