On the bleeding edge of heritage conservation: “Deep Fakes” at EPFL Pavilions, Lausanne

EPFL Pavilions, view towards the lake, North entrance of Pavilion A, with the Deep Fakes: Art and Its Double exhibition poster. Photo credit: Sarah Kenderdine.

Though some techniques of performance conservation, such as oral history and body-to-body transmission, may sometimes avoid technological mediation altogether, there are also technologies of mind-boggling sophistication now in development – and in use! – for the conservation, study, and transmission of both tangible and intangible heritage. Some of these technologies and their potential applications are  currently on view in the exhibition Deep Fakes: Art and Its Double at the EPFL Pavilions on the campus of the École polytechnique fédérale de Lausanne (which has been extended until May 1, 2022). I recently visited the exhibition with my colleagues Valerian Maly and Emilie Magnin of the research project Performance: Conservation, Materiality, Knowledge, where we were graciously received by curator Sarah Kenderdine, professor of Digital Museology at EPFL and director of EPFL Pavilions. She gave us a thorough tour of the exhibition, telling us the stories and hidden complexities of the various exhibits. We were awed by the possibilities of what we saw, but sometimes skeptical of the applications to which it was put – and ultimately left wondering how we might adjust such potential to the scale of bespoke performance conservation.

The exhibition revolves around digital facsimiles of artworks, and includes projects by artists, universities, start-ups, and research labs. Only a few would be considered artworks in the sense in which this is usually understood – by which I mean that they emerge from, and are intended to be exhibited, purchased, and discussed, in what is called the art world. Rather, many exhibits in Deep Fakes represent the results of complex research projects that seek to gather data on, digitally duplicate, or otherwise mediate precious art objects, culturally significant places, and intangible heritage scenarios. Each project holds much more – more data, more configurations, and more sheer possibilities – than what is available through the interfaces that viewers access in the exhibition. In essence, each individual exhibit is a snapshot of a much larger, dynamic project, one of many possible points of entry for the data collected, and each project is also a kind of advertisement for the further potential of the techniques used.

Most people are at least somewhat familiar with augmented reality, which is accessible in a limited way via smartphones. But this updated version of Jeffrey Shaw’s The Golden Calf (1994-2020) incorporates cameras that allow the digital sculpture’s shiny surface to accurately reflect the room around it. (Shaw discussed this work and more in a conversation with EPFL Pavilions’ Monica Antohi.) Exhibition view: The Golden Calf, 1994-2020 by Jeffrey Shaw. Photo credit: Catherine Leutenegger.

The exhibition was frankly stupefying. Even though I have a passing familiarity with the use of digital technology in heritage projects, and even though I try to follow developments in digital art, the projects in Deep Fakes represent the bleeding edge, and I was often startled by what is already possible. Seeing and interacting with these technologies up close made the possibilities of their implementation vivid and tangible. In an updated version of a 1994 work by the influential media artist Jeffrey Shaw, The Golden Calf, the unique veining on a slab of marble functions like a QR code, locating a gleaming gold bovine on the image of the marble that appears on the tablet’s screen. That technology may not be especially complex, but the interface between the virtual and material is more sophisticated than in most augmented reality projects. Even more impressive, the actual room in which we stood was reflected in the calf’s shiny surface, creating a live, sited experience of the work. Nonetheless, Emilie, who is experienced in the conservation of new media work, was somewhat skeptical of the choice to update Shaw’s piece with the newest bells and whistles available to digital art. With digital art, the temptation of making it newer and better is always there, but conservation’s role is also to determine what aspects of the original 1994 experience might be lost – and whether they should, or could, be conveyed to an audience used to today’s sophisticated media installations.

Observed with bare eyes, the detailed 3D images of the sculptures in Bamberg’s Abbey St. Michel resemble what you see when you take off your 3D glasses in an IMAX cinema. But I have never seen a 3D film nearly as functional or seamless as the image that resolved once I had put the glasses on. Exhibition view: The Abbey St Michel, Bamberg, 2021 – by Digital Projection UK, Consensive, and ArcTron3D. Photo credit: EPFL Pavilions.

Elsewhere, I put on a pair of unusual 3D glasses to appreciate the capabilities of what is surely the most expensive projector I have ever experienced, the INSIGHT 4k HFR 360 Multi-View 3D projector. This is a digital heritage conservation project undertaken by the Bauhaus-Universität Weimar along with ArcTron3D, a company specialized in advanced 3D scanning, and presented to the public here in collaboration Consensive, which focuses on visualizing such scans (click here for an interview about the project). Plunged into a digital facsimile of the interior of the Abbey St. Michel in Bamberg, Germany, I flew like a drone high above ground level to contemplate the perfectly-rendered face of an angel. Such technology does not only reproduce material heritage, it provides entirely new access to it. Sometimes, the access provided by new technologies has a profoundly corporeal aspect.

At another exhibit, I watched the futuristic-looking scanner of CultLab3D’s CultArm3D, a robot that produced a scan used to create the most detailed, precise, and carefully-colored 3D print I have ever seen (the printer itself was not on view). The print’s original, a mask on loan from Berlin’s Samurai Art Museum, was also available for comparison. While the print would hardly pass as the real mask, the implications for performance are profound: Such copies (which are, of course, improving all the time) allow precious, fragile artifacts like the mask to come alive once more in dance and ritual.

The robotic arm of CultLab3D’s sophisticated scanner moves to capture every angle of an object, creating a digital model of extraordinary fidelity that can then be printed as an impressive copy of the original. Exhibition view: CultArm3D, by CultLab3D. Photo credit: Catherine Leutenegger.

Ritual is the subject of an installation that especially captivated Valerian’s attention, and that set a new standard for us in the possibilities of documenting complex group performance. “Remaking Confucian Rites” is a project of high historical ambition. It was initiated by Peng Lin, professor at the Centre for Chinese Ritual Studies of Tsinghua University Beijing, and includes a number of collaborators, Kenderdine among them. RCR not only seeks to painstakingly reconstruct lost Confucian rituals through careful analysis of textual sources, but also to document these reconstructions in endlessly recombinable ways. “Deep Fakes” includes a three-channel video installation presenting the results of RCR’s “Provincial Archery Competition,” which hopes to bring back to life a ritual described in the 2,500-year-old Book of Etiquette and Rites. Certainly, this timespan seems to offer a great deal of hope for performance works merely a few decades old!

Visitors to Deep Fakes: Art and Its Double watch The Confucian Body: Rite of Archery (2021), a three-channel video installation that presents the results of the ambitious heritage preservation project “Remaking Confucian Rites.” Photo credit: Catherine Leutenegger.

I commented on the cinematic quality of the installation, which uses conventions that I recognized from the language of contemporary video art: multiple simultaneous perspectives, close-ups, cuts, pans. This was particularly striking given that the archery competition presumably would not have had an audience beyond the participants themselves (and no audience is visible in the videos). But Kenderdine explained that what we were watching did not really represent the results of this project, but merely one possible mediation of it, suited to this particular exhibition setting. In fact, each performer, each movement is captured individually, and essentially in front of a green screen – the version we saw featured a flawless temple setting, which I certainly would not have identified as a copy-and-paste job – so that the information captured is not tied to any one particular presentation. That is: RCR’s output is not a fixed work of video documentation, but a whole interactive library of motion-captured ritual movements with near limitless potential. For Valerian, this fantastically broadened the horizon of what is thinkable in terms of the documentation and re-presentation of performance art. Still, we reasoned, not every performance work would benefit from this treatment, and fewer still would be able to inspire the astronomical expenditure of money and manpower necessary to bring the archery competition back from ancient history.

Though our minds were buzzing with the implications of these technologies for the conservation of performance art, speaking with Kenderdine about her work also emphasized the distance between her sphere and our own. While some of the projects in Deep Fakes are designed around the mediation of singular artworks, each is of singular importance and value, often distinguished by UNESCO – like in the case of St. Michel, or in Double Truth, a clever presentation of data from the Atlas of Maritime Buddhism, which collects powerful scans of 15 of the world’s most precious Buddhist sculptures. This is also the case with RCR’s archery competition, which has the further support of a Chinese cultural establishment eager to reconnect to lost national patrimony. This approach would not map neatly onto the performance works of the present and recent past; it is, at least for now, reserved primarily for cultural objects and activities considered of the utmost significance not only by scholars, but also, often, governments that understand their nationalistic value. One can hardly ignore the staggering costs involved. Yet Kenderdine, who is adroit at finding industry partners, remarked that the work often “pays for itself;” technologies and systems she helps develop for cultural projects can sometimes be employed profitably elsewhere. It is a model with which Valerian, Emilie, and I have little experience. In our world of humanities departments, performance venues and festivals, and public museums, art is almost exclusively something that costs money, rather than a producer of it.

Double Truth is an installation which uses an interface that Kenderdine designed for the exquisitely detailed digital scans of important Buddhist sculptures that comprise the Atlas of Maritime Buddhism. Viewers physically turn the round platform to spin the image of the sculpture on view. When the platform is turned in the other direction, however, the images do not rotate like physical objects, but rather dissolve in ways that emphasize their digitality. Photo credit: Catherine Leutenegger.

Still, as Kenderdine points out, these technologies – like all technologies – quickly become more accessible. Even if a precise performance reconstruction seems like a mere fantasy for our own research project, we ought to recognize that we might have access to such possibilities sooner than we think – and that we will then have to consider what we might actually do with them. Some of the projects we saw in Deep Fakes, though certainly impressive, struck us as kitschy in their execution; though visual, spatial, and historical information was preserved with amazing fidelity, the simulation failed – at least for us – to summon the grandeur and beauty of the original. If we wish to use such tools, or the descendants and spin-offs they are likely to engender, for the conservation of performance art, we will need to carefully consider what might get lost in the translation.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s