5. conclusion
It has been just over two years since I first created AIYA.
Each work I have made with AIYA has been a transformation and revision of AIYA itself; in no two works was AIYA functioning technically in the same way, nor was AIYA regarded in the same way across these works. Across the lifespan of AIYA I have tried to represent it, but I cannot...at least I cannot define the machine by a single attempted representation. It will continue to be an itinerant, multifaceted entity, and every attempted representation will be more of an interpretation of AIYA's black box, a fleeting glimpse inside its dubious character.
When AIYA was originally created, it functioned more as a sound analysis tool and elaborate live sound module than an actual agential entity. In this state, it was perceptibly unable to be emancipated from my inputs. As the machine evolved, however, to a more complex state, the rules I had programmed the machine to follow no longer occupied the forefront of my attention. In their multiplicity, the rules became their own black box. On the stage, I performed with the machine as a musician, bouncing ideas with the machine at light speed, and not as a system designer trying to remember the programming behind these ideas. This shift in mindset and behavior allowed me to perceive the machine as being emancipated from myself.
From the audience's perspective, however, it proved to be difficult to instill the perception of the machine as being emancipated from the human performer. I attribute this significantly to the tendency for the human audience to focus their attention on the human subject (i.e., myself). I was once told after performance I made with the Chaosflöte (without the improvisation machine) in 2017 at the Zürich Digital Festival, “The visual projections are competing with you as a human performer. On the one hand, I am fascinated by the video; on the other hand, I can’t unsee a human performer on the stage as anything but the center of attention.” This phenomenon also seemed to occur in the 360° video work, black box fading, where one viewer of an early version of the work gave the feedback, "I got so scared watching your piece because I got super disoriented moving it [the 360° video] around, and kept shouting 'where's Melody?! Where's melody!'" (The participant, however, noted that she nevertheless enjoyed the work.). Despite my efforts to convolute the human presence, it was still seen as a reference point for orienting oneself in the work.
I have used immersive production techniques and the exposure of the technical production process as an effort to outweigh the inherent focus/attention on the human subject. For example, I attempted to bring the focus of the black box fading 360° video on the inner workings of the machine, to create a feeling that one was being entangled in the virtual space and body of the machine, almost in an overwhelming way. At times, the audience perceives the human performer (me) in the video as somehow having control over the machine, but at other times being swallowed up by it. Perhaps the question became less about machine _agency and more about hierarchy between man and machine: "Who has the power?" or "Who seems to have the power?"
Allowing the prevalence of _glitch(es) in AIYA's software also afforded a certain _emancipation of the machine from myself. This was especially apparent to me in aiya meets self. As I sometimes did not know the cause of the glitch, it made me regard the machine as more of a separate entity of myself, as opposed to an extension of myself. Because I could not anticipate its next moves, it became someone to negotiate with, someone to be reckoned with. I did not have the same _control or influence over the machine in its glitching behavior as I might have had when it functioned "normally."
Indeed, the issue of _control was a prominent and prevailing issue to negotiate when working with the improvisation machine. Having too much control and predictability would undermine the machine as an agential improvisation entity. Too little control would cause perhaps too much of a disconnect, to the point where it would no longer be considered an extension of my body or even as an _instrument. In the end, the balance between more or less control was correlated with the balance between perceiving the improvisation as an extension of my body or as a separate/distinct improvisation entity.
The machine's physical presence (which refers not only to the software of the machine, but also all connected hardware components, such as the projectors and speakers) also had an effect on whether I would perceive it as either a _distinct-entity or an extension of my own body. In diaphragma, my body was quite removed, distance-wise, from the visual representation of the machine, which contributed to my perception of the machine as being a separated entity. I could physically turn to the screen to have a _dialogue with the machine, or turn my back to the machine and walk away to transform the machine into a background texture of the performance. Or I could sit down on the floor with my body almost entirely hidden, in a dark part of the stage and away from the audience's attention; in this setting, the visual projections of the machine could come to the forefront of the audience's attention instead. With my own physical presence away from the center of focus, I could finally feel as if the machine and I were perceived as a single unit, and as if the machine was a digital "ghost" speaking above my sitting body.
In my works, AIYA's physical presence tended to occupy a distinctly 2-dimensional form (i.e., visual projections on the screen). In A->B I tried to resolve what was otherwise a mismatch between the machine's 2-dimensional form and my 3-dimensional body by superimposing my 2-dimensional shadow onto the 2-dimensional visual projections. My initial conclusion is that if two performing entities can be perceived as occupying the same number of dimensions (in this case, 2), this helps to reinforce the impression of the two entities as being extensions of each other. Of course, this alone is not enough to create this impression, but it is a starting point.
In black box fading I tried to test this conclusion, but in the opposite way: using the immersive box projections as an illusion of giving the improvisation machine a three-dimensional character, in order to match the three-dimensionality of my body. On top of this, I also embedded a two-dimensional _representation of myself into the projections, seen on the right.
The 360° video format in black box fading also yielded interesting visual possibilities for playing with the perception of bodily dimensions. As seen in the last picture on the right, just by layering multiple two-dimensional videos of myself playing in the black box, I could create an even stronger illusion of three-dimensional _space that both the machine and I appeared to exist in. Using immersive production techniques in general helped to create impressions of physical "embeddedness" between my human form and the form of the machine.
What is also equally as important for perceiving the human and machine as extensions of each other is the _transparency of interaction design between the two. In bad decisions, the use of many fixed media elements (i.e., a fixed video running underneath the live projection visuals and the fixed audio track running underneath the live audio elements) seemed to convolute the connections between my actions on the stage and the reactions of the machine. The reduced _transparency of the human-machine interaction reduced the perception of the machine as being an extension of my body, where the machine at times looked like simply a vivid background element that only sometimes correlated with my actions.
Finally, I think it is important to also acknowledge the human-machine interaction design in the other direction as well: it could be effective to program "rules" for the human to react to the machine. If the rules are executed in a consistent way, it mimics the programming paradigm of mapping human actions to machine behavior and closes the circle of bidirectional human-machine interaction. Gioti argues that a truly interactive system only exists when this bidirectional relationship is established. Otherwise the system is simply "reactive"—where the machine's electronics are reactive to the human actions, but not the other way around (Gioti, 2019). Of course, one can argue that such interaction already exists through the human's intuitive reactions to the improvisation machine. But perhaps having specific instructions for human reactions to the machine—for example those that are exhibited in Gioti's work for human and robot percussionist, Imitation Game—could yield new, hybrid possibilities for the human-machine relationship.
The bidirectional relationship between human and machine exists on the production level as well. I have come to see the technical and artistic preparation of creating a work with AIYA as being inherently intertwined. As I have previously described, the conception of black box fading at the Immersive Arts Space was a constant dialogue between my own initial ideas and the ideas coming from the characteristics and/or limitations of the technology itself. Instead of fighting against the nature of the technology and space (for example, the low-fidelty of the 360° video recordings as caused by the low light of the space), I incorporated it into the aesthetic development of the work. This allowed more of the machine's character to permeate the work and give it more presence within the work.
Preparing a performance with the improvisation machine also somehow felt distinctly different to the usual practices of rehearsing for a classical music recital. Aside from the obvious presence of electronics and the need to thoroughly test their functionality before the performance, the process of rehearsing with the improvisation machine felt more as if I was testing an interactive _installation work, as opposed to rehearsing with a musician on a predetermined _composition. Not only was the machine a program I created; I created an interactive system to be tested under a multitude of artistic conditions. Each rehearsal period was a time for me to "get to know" the machine, in the sense that I could not reproduce exact behaviors every time I would play, but I would simply come to get a sense of the machine's tendencies and "character." The machine represented a set of possibilities in a way similar to how an interactive installation might do so, and the thought that each moment with the machine could yield ever so slightly new possibilities proved to be a very attractive situation.
Indeed, I try to convey this sense of human-machine discovery to the _audience, because the audience also needs to become acquainted with the machine (and arguably also the human). While some may find this restricting to the _narrative arc of the work (that the form of the work tends to develop from sparseness/simplicity to denseness/complexity), I have found it to be a somewhat necessary dramaturgical arc to use, given the relative "newness" of improvisation machines and sensors in music performance (Given the significant lag between the introduction of a technical device in music performance and the recognition of that device as "standard practice" it is unfortunate that when I say "newness" I refer to a practice that has been in place within the last 50 years). This was especially apparent to me in the interactive installation black box finding, where participants soon got confused once the density of the space and activity increased; it was helpful to begin with a sparse texture first to show the mappings between the rigid bodies and audiovisual output in a more transparent way. Creating a work with the improvisation machine is a balance between the audience understanding the full interaction design/relationship (i.e., programmed mappings) between human behavior and machine behavior and the convolutedness of this relationship. Understanding the human-machine relationship in the technical sense helps to establish a stronger relationship between human and machine, reinforcing the idea that the two are extensions of each other. However, this can also undermine the impression of the machine as a distinct, autonomous entity, and convoluting the technical mappings between human and machine can help to tip the perception in the other direction (this is especially apparent in moments of _glitch).
Accordingly, it appears there is a certain obsession with the idea of convolution as a human quality and that improvisation itself is a _black box that is perhaps better left untouched. As George Lewis recalls from an interview with the jazz magazine Downbeat, his interviewer wrote, "[regarding the improvisation practice] We draw upon our vocabulary, our experiences, our associations—and we spontaneously speak to each other. I think that happens in music, but there's a particular mindfulness to the craft of that in music. Maybe that's a mystery—and should remain a mystery—if it's beyond what we can say in words...I'd be disappointed if we could actually map spontaneity" (Lewis, 2018 p. 3). How fascinating it is, that the black box—a metaphor typically reserved for the complexity of technical devices—is a descriptor that we ourselves wish to possess and preserve when referring to the practices and processes we consider as sublime or inherently "human" (such as improvisation). Is it because there is something about the black box concept that emanates the aura of agency or subversion of expected power structures? Or that the black box allows us to transcend our seemingly fixed identities as "human" and "machine"? Perhaps by constructing and deconstructing the black boxes of ourselves and our machines, we can describe this phenomenon...through actions rather than words...
So welcome to this black box. It was a pleasure having you here.