Over the past few weeks, if you’ve seen your friends post their pictures alongside famous Renaissance paintings, here’s what’s going on. The Google Arts and Culture app (available on iOS and Android), which has been around for two years, shot to the top of major app stores because of a new – hard to find – feature that it just added.
This feature lets you take a selfie and utilises the database of art collections available to Google’s “Cultural Institute”. In a matter of seconds, using artificial intelligence and facial recognition techniques, the app serves you a portrait that is the most similar to your face. In short, the Google Arts and Culture app shows you various museum portraits you most resemble along with a match percentage on each. The results have been delightful, fun, educational, hilarious and problematic.
But as a diverse range of users started reporting their results, some started noticing how the Google Arts & Culture app has a race problem.
What they meant is that regardless of the user’s racial or ethnic background, and often gender, the app overwhelmingly matches light skinned users to portraits of European white men. In other cases, it matched many users with any Asian heritage to a limited range of Asian portraits available to Google’s collection. In other grey cases, as the Mashable article linked above shows, the app served mixed results, matching a Mexican-American to a European white portrait and then in lesser degree, to the picture of a Japanese man. Predictably, given that nobody knows what exactly is “under the hood” of this algorithmic app, the Internet is full of people testing the app’s limits, producing ‘folksonomies’ or collaboratively pieced understandings of how the app works and why they are seeing the results they do.
In my own experience of using the app, I was matched to the ‘Venetian Girl’, an 1880 oil painting by American painter Frank Duveneck. After moving my face, trying different angles and lighting, I kept getting the same portrait but with higher confidence (I stopped at 60% match).
After a few minutes of meddling to eke out an Indian portrait match, I succeeded, as you can see below, only to get a painting by the famed colonial aristocratic painter, Raja Ravi Varma. For those who might not be familiar, Ravi Varma’s style and iconography – often described as embodying “European technique with Indian sensibilities” – represent what is commonly identified as “Indian” in the Western imagination even today. In that sense, to be matched to a Ravi Varma painting is akin to being recast as a colonial subject: to be re-placed in a world that continues to reduce non-white bodies to a handful exotic representations; a world that while making space for non-human imagination, somehow retains select engagement with what human looks like.