Trevor Paglen Machine Readable Hito, détail | detail, 2017.
Photo : permission de | courtesy of the artist & Metro Pictures, New York

The Automation of Empathy

Grant Bollmer
Trevor Paglen’s print Machine Readable Hito (2017) is composed of hundreds of images of artist Hito Steyerl’s face. Each image has Steyerl making a different facial expression, and each is captioned with the output of computational algorithms designed to detect age, gender, or emotion. The captions in Machine Readable Hito are similar to those generated by software developed by Microsoft — a suite of programs formerly called Project Oxford, which are now part of the Microsoft Azure machine-learning platform. Along with a range of programs designed to match faces and identify age, gender, and emotion, Microsoft Azure includes algorithms that can recognize voices, engage in real-time language translation, and perform content moderation — the automated identification and removal of pornographic images and videos, for instance.

Much of the algorithmic output that Paglen includes is, in one way or another, questionable. In one image, the algorithm identifies Steyerl as 59.58 percent likely to be male and 40.42 percent likely to be female. This interpretation of Steyerl’s face shows us how these algorithms identify only probabilities within rigid categories, whereas a great deal of research on gender tells us of its fluidity and most certainly does not establish a definitive link between facial expression and gender. In other images, what would appear to a human eye as similar photographs are classified as representing completely different emotional states. Steyerl rolling her eyes back into her head, the algorithm tells us, expresses something between the states of “neutral” and “sadness.” But these seemingly incorrect identifications aren’t the point. Instead, Paglen is showing us that computational systems “see” in radically different ways than humans do. Each image in Machine Readable Hito is, effectively, a double representation. One is the visual representation of a face that can be understood and interpreted by human capacities of vision. The other is the representation — oblique, indirect, via text — of what a computer sees when it classifies a digital image. The second series of representations are the important ones here, imbued as they are with an assumed objectivity, if belied by the judgments at which the AI programs arrive.

You must have a valid Digital or Premium subscription to access this content

Subscribe to Esse now to read the full text!

This article also appears in the issue 95 - Empathy

Suggested Reading