The Physical Jpeg
30 Jun 2014Progress: Animated
First, for those unaware, a little info on how Jpeg compression works. The image is cut into 8x8 blocks, which are then transformed to give their spacial frequency components. Then, depending on the quality desired, a certain amount of unwanted data is thrown away. Finally the data undergoes a number of lossless compression techniques.
The Physical Jpeg is giant electromechanical display. Instead of pixels, it uses tiles corresponding to 8x8 blocks.
To illustrate the effect, I have constructed an animation of what the display might look like. The masses of mechanisms are not shown.
Half a croissant, on a plate, with a sign in front of it saying '50c' replay
On the first day of the installation, members of the public are enticed to send in pictures via social networks and mobile gizmos. The Physical Jpeg gladly springs to life and the carriage at the top zooms back and forth, dropping tiles into the grid like a game of Connect-4. The end result, however, is terrible, because the magazine contains only 8 different types of tile.
Day 1
Half a croissant, on a plate, with a sign in front of it saying '50c' replay
On the second day, the magazine is extended. More tiles are added, each printed with an 8x8 section of image. The choice of which tiles to print is dictated by the 'quality' parameter of the Jpeg algorithm, which has been artificially extended well below the normal range.
By the third day, images are almost identifiable on the display. There is no colour info (we are not totally mad) and the quantization is evident. As we move from having a huge number of tiles in a small number of bins to a large number of bins with relatively few tiles, we can optimize by overprinting: almost all tiles will be a linear combination of the few basis tiles we started out with.
Upon finishing each picture, the tiles are released and fall down to a tray where they are collected and meticulously sorted by the machine. For some people, this may be the most exciting part of the exhibit.
65,536 types of tile are present by the end of the week - a number dictated by the sorting mechanism more than anything. With individual bins impractical, the carriage now has to root through buckets of tiles before finding one that it needs. The end result is still a greyscale picture of embarrassing bit-depth.
The Physical Jpeg is efficient at a microscale yet grossly inefficient at its overall goal. It is a triumph of construction and a tribute to the algorithms it is trying to describe. It is a Rube Goldberg machine that, paradoxically, is much simpler than the real method of decoding a Jpeg. Principally it is a poignant reminder of the futility of mechanical engineering in the 21st century.