Conserving software: Unity, Macromedia Director, Max/MSP and time-based media in ACMI’s Collection
As the 2021 Grimwade and ACMI Time-Based Media Conservation Fellow I have had the opportunity to work in the Blackmagic Design Media Preservation Lab as part of the ACMI Collections team. The research project I have developed investigates the use of Unity, a graphics engine and software development environment, by artists and creators, as well as ACMI’s approach to documenting and acquiring works made using Unity, which include virtual reality (VR) artworks such as Did you ask the river? by Joan Ross.
The idea for my research project came from noticing the Unity engine being used in a number of artworks — particularly installation works which have been recently commissioned. I wondered how one might approach the acquisition, and further in the future, the conservation of these works. During the fellowship I have been interviewing artists and software developers about their professional opinions on the kinds of preservation which might be possible.
This year, for the first time, the Fellowship has been awarded in partnership between the University of Melbourne’s Grimwade Centre for Cultural Materials Conservation and ACMI to support a graduate to undertake a research project in the conservation of time-based media (TBM) artworks — these are works that have a durational aspect, such as video installations, software, and sound art. The ACMI Collection contains a range of such artworks, some of which were commissioned or acquired by the organisation, and others which have been donated or licensed for research use. Altogether the ACMI Collection is a great resource for researchers in many fields, such as art history, conservation and digital humanities. As part of my research project I have engaged in discussions and interviews with creatives and professionals working with Unity both at ACMI and externally.
One of the projects I have been working on at ACMI is imaging prioritised works from the optical disc collection — largely CDs and DVDs. The optical disc collection includes software-based works from the 1990s and 2000s, as well as significant video artworks, short films and ACMI productions. Because of the risk of deterioration poses to data stored on optical discs, the Canadian Conservation Institute has compiled a list of these causes of deterioration, ACMI is undertaking the migration of works stored on discs to its archival digital storage.
There are two works in particular which have provided me with valuable insights into the kinds of challenges that software-based artworks face as they age. This has in turn informed my predictions about the preservation challenges which artworks made with Unity might present in the future. The two works are MUTO by Kat Mew, which was built using Macromedia Director software, and Swarm by Craig Walsh and Michael Yuen, which was built using the Max/MSP environment with Jitter extension.
Software development environments
Other than being interesting and complex works, I wanted to investigate these artworks because of the software used in their production. I considered how this software may have influenced the creative process and aesthetics, as well as its implications for the works’ conservation.
Many software packages for editing video, audio and photographs directly recreate tools and workflows from their analogue counterparts. For example; the layout of DaVinci resolve and other video editing software is inspired by that of a Steenbeck film editing desk. The film strip and magnetic sound track run parallel to each other across the centre of the desk with a measurement scale below and the viewer located above. The largest Steenbeck editors had two viewing screens and space for multiple audio tracks running parallel to each other, a configuration which is recreated in many software editors.
Unity’s interface is a step further removed from these kinds of physical metaphors. Instead, Unity borrows its interface concepts and tools from earlier software development environments, such as Max/MSP and Macromedia Director.
Artist and software developer Max Piantoni made the following observation about Unity when I interviewed him as part of my fellowship research:
“You would expect it to articulate a first step, but there isn’t one and of course it turns out that that’s really great once you get into it, because the thing about Unity is you can use it for all kinds of different, varying kinds of projects.” (Piantoni, p. 3)
Because software development environments allow artists to work outside of the more conventional forms of media it is important to understand both their process and intent in creating an artwork so that it can be conserved and displayed in the future in a way which is faithful to the concepts within the work and the context it was originally produced in. However, the proprietary nature of each of these development environments presents significant challenges to the conservation of works made using them — for technical and other reasons. Additionally, most of the artworks discussed here also have external hardware requirements, such as microphone or video input, or specialised display parameters.
Macromedia Director was popular in the 1990s as a tool for creating interactive media, particularly the genre of multimedia CD-ROMs. Like Unity, Macromedia Director was first developed for the Macintosh and later released for Windows as well. Macromedia Director users with a knowledge of coding were able to use the integrated Lingo programming language to extend the capabilities of the environment — the language C# plays a similar role in developing works in Unity.
Max/MSP has been in development since the 1980s and has an enduring popularity, particularly with sound artists and musicians. It offers a visual interface for ‘patching’ or creating interactive instruments and artworks which can then function as a standalone application. Unity’s visual scripting tool has clearly drawn inspiration from the Max/MSP interface.
Artists’ documentation in Swarm
Commissioned by ACMI as part of the exhibition Eyes, Lies & Illusions in 2006, Swarm is a time-based media installation by Craig Walsh and Michael Yuen. It uses video projection and audio to create an illusory swarm of insects.
The work was created for a PowerPC-based Mac running OS 10.4.8 shortly before Apple had completed transitioning the Mac lineup to Intel processors. The Swarm application won’t open at all on versions of MacOS after 10.7 because the Rosetta translation layer, which enables PowerPC applications to run on Intel-based Macs, was removed at this point. There were audible glitches while running it in Rosetta — using a more powerful machine may have been enough to compensate for the inefficiency of Rosetta. The work was also tested using the emulator QEMU via Emulation-as-a-Service Infrastructure (EaaSI). The application ran slowly, much like when using it in Rosetta, and because OSX emulation in QEMU does not yet fully support audio, I wasn’t able to verify that this aspect was functional.
ACMI has Swarm licensed for internal research purposes but not for display or exhibition. Because the intent was to verify and archive the copy in the ACMI Collection rather than to re-exhibit it, for this purpose it was not necessary to find the hardware to run it natively. I was able to see from running the application in emulation that the artists have left extensive comments and instructions about the function of the Swarm application within the interface. Similar to the way programmers leave comments in their code, within the graphical interface of Max/MSP, these instructions are valuable documentation of the way the application functions and of the way in which the work should behave. These kinds of comments are valuable in the conservation of a work because they provide direct insight into the artist or programmer’s creative intent and process.
Hardware optimisation in You, Me, Things
During our discussion Piantoni explained, about the work You, Me, Things, created with Uyen Nguyen and Matthew Riley, that he had hoped to move the software components away from a large Windows PC that had been used during previous exhibition to a small Mac. It could then fit inside a plinth, when the work toured in the Experimenta Life Forms: International Triennial of Media Art. Apple had introduced the first of its Macs with Apple designed processors with an ARM architecture, which would provide the necessary processing and graphics power to run the work. However, during testing Piantoni realised that optimisation for the new ARM processors was lacking in Unity at that time:
“I tested it [You, Me, Things] on an ARM Mac mini and the Unity application couldn’t do the post processing effects because Unity hadn’t ported that yet and it wasn’t working right in the Rosetta translation layer. And forget about getting all of the dependencies in the machine learning application to run correctly.”
Though the artwork was functioning using the Rosetta 2 translation layer, it was not up to an exhibition standard.
“We had to stick to, in the end, the original version run on Windows. It runs on the Mac fine, but not well enough to be put into the gallery context, so we had to keep it on Windows.”
Emulating classic operating systems and MUTO
MUTO is a software-based work from 1996 by Kat Mew in which the user navigates between, and creates hybrid lifeforms out of microscopic organisms and the data stream of cyberspace — accompanied by a genre spanning soundtrack by Steve Law aka. Zen Paradox.
In a similar way to Unity, Macromedia Director projects could be built for Mac or Windows operating systems. This offers greater flexibility in terms of choice of emulation or other strategies for redisplaying a work. However, through imaging and documenting MUTO, it became clear that some elements of the work suffered when run on Windows compared to a Mac environment.
Because MUTO was produced for Macintosh System 7.5.3 it won’t run natively on a Mac with MacOSX or later. For several years, Apple included support for older software by emulating MacOS 9 in the ‘Classic’ environment. This was later removed from MacOSX so running MUTO requires the use of an emulator — in this case using SheepShaver via EaaSI it was possible to produce results comparable to running the work natively on legacy hardware.
Building Unity projects for different operating systems
Because Unity allows projects to be built for a number of target operating systems from the same project, experimenting with running You, Me, Things in a Mac environment rather than Windows didn’t require porting the entire work. However, as Piantoni pointed out during the interview, the version of Unity which he was using at the time did not support optimisation for ARM equipped Macs. If the works needed to be run on ARM hardware the project would have to be migrated to a newer version of Unity which has support for these computers.
However, there are challenges involved in moving Unity projects between versions, particularly where there have been new features added, such as support for ARM processors would involve. Piantoni describes running into this difficulty when working on a different project which was in a 2019 version of Unity:
“If I was to open it in 2020 say, there would probably be a lot of things that have changed and there would have to be, at the very least, a lot of testing to ensure that nothing is broken and then also probably a bit of work to move things that aren’t working properly into whatever the new paradigm is.”
Conclusions
The experience of undertaking quality control on the disc images of MUTO and Swarm has demonstrated to me how important documentation is in facilitating the preservation of these software-based artworks. Without it, it is difficult to assess whether an emulator is faithfully displaying the work. Creators and conservators producing thorough documentation, particularly in the form of artist preservation interviews, are not only enabling the preservation of these works, but also enhancing the research value of the collection.
Any software development environment which allows a project to be built for multiple operating systems provides an advantage for the work’s preservation. Acquiring multiple different builds of a work will provide more options in terms of redisplay and emulation. Because of Apple’s history of phasing out support for older hardware and operating systems it seems advisable that a Windows copy of a work be acquired wherever possible.
Although optical discs are now rarely used to deliver artworks, a conceptually similar approach is taken in the preservation of artworks made with Unity and delivered on a hard drive. The drive is imaged — capturing all the drivers and other software required to run the work — although the process itself is different to the one undertaken with optical discs. The experience of working with the ACMI Collection has informed the discussions I have had with artists and creators about how to preserve works made using Unity.
Testing MUTO and Swarm in emulation has highlighted to me how useful descriptions of the subjective qualities of an interactive experience are. A screen capture and image stills aren’t always sufficient to give a sense of the ‘feel’ of engaging with a work — and it is easy to make assumptions about how responsive and stable software should be based on one’s experience of contemporary systems rather than what was usual at the time the work was made.
The author would like to thank the Grimwade Centre for Cultural Materials Conservation, Grimwade Conservation Services and ACMI for sponsoring the Time-Based Media Conservation Fellowship.
