On the 23rd of June, 2018, amidst daily theater rehearsals, I presented the performance piece I worked on during a semester for the class of Art and Technology of the PhD in Arts.
Matt Makes Games' Celeste hit me hard. The game's theme was strongly familiar, and from the moment I understood that I knew it would find a way into my work.
Breathe is a performance piece that is inspired by and utilizes the game Celeste. In it, after sleeping under the desk for many uncomfortable minutes, I climb into the chair with difficulty, my stressed breathing betraying my lack of strength in doing such an apparently simple task, put on my mask, and begin playing the game. The projection displays my progress in the final stages, only, when I lose control of my breath, when the stress and anxiety over dying for the 50th time on the same spot begins to show, the screen changes, the audio suffers as I do. If I breathe faster than I should, the visuals garble, suffer interference, get more and more broken while the audio's distortion is aggravated as I try to regain some level of calm. I finish the game despite its difficulty, and at the peak I am given strength to finally stand up, get dressed, and move on with my life.
At the end of this private performance, the teacher asked if I could do it again, as I had explained the technicalities of the system I created, the theme of the game and its importance to me in creating this cathartic experience.
I could not do it.
I spoke before of how art feeds more art, how a composer is not a composer, is a performer, a painter, an actor, a designer... Videogames influence my music, my performance pieces, my art, in ways I cannot (or want to) stop. As they should.
No artist is an island,
Entire of itself.
Each is a piece of the continent,
A part of the main.
(altered part of "For whom the bell tolls", John Donne's "Meditation XVII - Devotions upon Emergent Occasions")
Music is art - a branch of art, a piece of the continent, and all art part of the main.
Composers must not deny the influence of other pieces of music on ourselves, or of other pieces of art, much like we wouldn't deny how our life, and how it was shaped by forces beyond us, influences our work.
Every story I read, every movie I saw, every poem I muttered, they are in my music - there is no "me" in it besides the gathering of all that happened to me.
On the 27th of June, 2018, I was invited to speak on the CYSMUS's - Group of Advanced Studies of Music and Ciberculture - 3rd Workshop, at NOVA FCSH - Faculty of Social Sciences and Humanities of NOVA University. I chose to give a somewhat personal talk about the influences videogames had on my concert music, as well as on the music of a colleague - Pedro Finisterra, a path that seemed rarely taken, as most academic research focuses on the influences of concert music on videogame soundtracks. For half an hour (which everyone who attended agreed was not enough to fully experience all the examples), I showed four pieces of concert music I had created where the influence of videogames (not just their soundtracks, but how they changed me enough to create something because of them) was identifiable, as well as three by P. Finisterra, and one by Hiatus Kaiyote (Atari; though I have a feeling any of the songs in that album could have been picked) - that was just a personal tale from a composer with still many, many, more examples to create.
From the 4th of July to the 6th, 2018, me and some colleagues from Lisbon's University of Theater and Cinema presented, after many months of work, a theater piece. The process this team went through was unusual (one of the actor's teachers even commented "this is the first time you work like this and will surely be the last"), as there was no leader, only an horizontal structure where anyone, actors and musicians alike, could shape the work being done. We started with a mash-up of W. Shakespeare's "The Tempest" with W. A. Mozart's "The Magic Flute" that quickly evolved into work over only "The Tempest". From there, "actions", or small moments where each group of 4-5 people analyzed a scene and represented it as they saw fit (a magical dining table used to torture castaway sailors turned into a silent family dinner, the hidden meeting of forbidden lovers turned into a woman escaping from full-body duct-tape bindings...). "The Tempest", but not the "actions" already created, were let go and we began to work on T. S. Eliot's "The Wasteland", finally splitting up our show into four scenes - an introduction, a disk grinder creating sparks in the dark while a wall of noise filled the room; the poem, spoken in the dark; "The Tempest's" "actions" glued next to each other; the family dinner with everyone who participated, in silence, followed by tonal music (at last!).
(Photography by Alípio Padilha. Many were not added due to nudity)
I was not just a composer, but an actor, a sound designer, a musician... I shaped their world with the screeches of my hurdy-gurdy and computer, the cellist shaped the noise that came before with beautifully (and disturbingly) calm music, the pianist dropped the piano on her leg which changed scenes where risks were too great.
All along this process I kept seeing the similarities between this process and some of videogame development. Concert music, in fact the whole scene of "classical" contemporary music, is highly hierarchical - the composer, the score, is the one true ruler and everyone else must comply. In videogames the designers change the visuals, the visual artists change the music; everyone is a piece of the continent, a part of the main.
Composers aren't alone and shouldn't try to be. The image of the secluded, misunderstood artist isn't real, never was. We are better when we work together, and in a medium such as videogames, composers who are only that, composers, - not painters, actors, directors, writers, game designers, programmers, sound designers... - who do not speak even the very basics of the world they want to live in... they're not videogame composers.
They compose for videogames.
This GGJ18's theme was "Transmission," something some felt was close to last year's theme of "Waves". Many, however, found ideas beyond the closeness of these themes, creating games about a Dr. Gill's Radio Show (a fish giving life advice on his radio show), about soul transmission, about hacking T.V. broadcasts and, in my team's case, about hacking someone's mind via a nanobot mosquito that sucks thoughts from your nerve cells - an outlandish idea, fit for a game jam.
Even if we had finished the game - as Thorsten suggested, we didn't - the friendships gathered, the inside jokes ("a round of applause, everyone!"), the 5AM thoughts fueled only by coffee ("sofas are just pillows with legs"), the failures and successes, they made this weekend, this Global Game Jam, worth the 4 hours of sleep.
Sound-wise, I found myself working on more than music, but on sound design as well. As opposed to what I was told (this being my first in loco jam), I wasn't "stolen" by many teams to work on their game, but stayed only with one.
At the end of this jam, I have a few suggestions for anyone working or playing with sound design and music for videogames in game jams:
- Bring a good computer (I had to freeze/pre-render whole projects to continue working on them);
- Bring good food (burgers and energy drinks may not be enough);
- Don't expect a decent internet connection (but expect the GGJ's site to crash during the last hour);
- While waiting for others, work for others (if you're waiting for some part of your game to work on, then offer your services to some other team);
- HAVE FUN! Make .gifs of D. Trump dancing, go fill someone's white board with cheer, laugh at the bugs you caused (especially if they happen during the final presentation), go exercise in front of the camera, dance and sing with others!
You can play what came out of this year's Global Game Jam at globalgamejam.org/2018/games
You can play games made at IADE at globalgamejam.org/2018/jam-sites/iade-%E2%80%93-universidade-europeia/games
To play my team's work during this great (the greatest) weekend, you can find it at globalgamejam.org/2018/games/insecton
If you'd like to hear the short soundtrack of INSECT!ON, you can find it here:
I find myself in a lucky position. A former teacher has allowed me to participate in his class where a group of performers improvise at will as the composers manipulate the sounds they create. Composers can learn a lot about live electroacoustic music performance, and players get to enjoy dealing with the odd minds that come up with software that turns them into demonic robots.
An inside look at our experiments:
In the middle of July, my dissertation "Non-interactive Interactivity: building a seemingly interactive installation" was defended.
As noted by myself and the jury, one of the weaker points of the project that accompanies my dissertation is the game design of the installation. Its puzzles are too hard, with vague, obtuse, or unfair solutions, as I found myself designing it by thinking "what is the player supposed to do?" and not "what can the player do?".
Critical investigation of interactive works requires extensive cross-disciplinary knowledge in a diverse range of fields including software programming, hardware design, instrument design, composition techniques, sound synthesis and music theory.
(Drummond, 2009, p.125)
Drummond, J. (2009). Understanding Interactive Systems. Organised Sound, 14(2), 124-133, doi:10.1017/S1355771809000235.
While not directly related to video-game audio design and music, I leave you with a chapter of my dissertation "Non-Interactive Interactivity: building a seemingly interactive installation" that talks about choice in games and how the labeling of low-interactivity video-games has cultural and semantic importance.
This came after playing Everything for the first time and wondering about the discussion of whether or not it is a game.
"The choices you make (...) influence what that character becomes in your mind. Even if you will see little actually change, the act of choosing (or experiencing the illusion of choice) really does have an impact."
- Anonymous (at gaming.stackexchange.com ).
The inclusion of video-games where the player's actions have little to no real mechanical influence on the game or the progression of its story is still a hotly debated topic among video-game enthusiasts, players, developers and critics. It is easy to understand how such games can disturb our understanding of the genre itself and the expectations one forms of them; they shake the very foundations of how we define them.
Interactivity has been one of the hallmarks of video-game experiences, their selling-point. As, in defining terms, what something is is considered at least as important as what it isn't, this genre separates itself from books, movies, theater and even interactive film by the complexity of choices the user has in shaping their experience - its interactivity. Yet, video-games such as Beyond: Two Souls, The Walking Dead, Kentucky Route Zero, Firewatch and The Beginner's Guide are generally still considered games, despite lacking complexity of choice and consequence.
Beyond: Two Souls by Quantic Dream is certainly the closest, of the examples, to interactive movies. The game presents the player simple challenges (known as quick-time events) where precision and speed allow them to progress into the story. In this particular example, failing such challenges provides no in-game consequences, simply an infinite number of chances - the game and its story will not progress until such test is surpassed. After easily beating the challenge the game continues its story, with no real user choice, essentially stripping the player of essential aesthetics (as explained in the paper MDA: A Formal Approach to Game Design and Game Research (Hunicke , Leblanc , Zubek, 2004)), such as Challenge, Expression and, after understanding that most of the game consists of such events, Discovery, aspects the player expects from the genre Beyond: Two Souls is part of. In fact, the only aesthetics provided by this game are Narrative and Fantasy, which are also solely present in books and movies. Beyond: Two Souls also contains moments where the player is free to explore their surroundings in a 3D environment, being given little to no challenge, choice or agency.
The Walking Dead, by Telltale Games, unlike the previous example, shows a non-linear story and presents the player choices on how to progress in it; depending on what the player decides, they will be taken through different paths. "Various choices make one or two changes in dialogue and can alter how other characters treat you, but it does not change the main plot significantly" (Elise Favis - gameinformer.com, 2015). However, those paths return to common points after diverging, resulting essentially in a linear story presented as a non-linear. We see, then, an issue similar to the first example - player's agency is removed, in a genre where it is expected. Like in the first example, quick-time events are common, although more challenges, similar to first-person on-rails shooter video-games, where the player has limited time to aim and shoot enemies in a fixed camera environment, are present.
In the Kentucky Route Zero example, by Cardboard Computer, no challenges appear, only a story and the means to follow it, which blurs the line between video-game and movies, interactive movies and books even more. The player follows a linear story, told by setting, non-playable characters (NPCs), and descriptions of their surroundings, and has no in-game control other than how much of the story they wish to engage with (by following alternate, hidden, paths), and the speed that the story unfolds in front of them. A main character is guided by the player in 3D sets, frames of action, moving in or out of those sets as the story commands. They can speak to NPCs to progress, or click on buttons to read a description of various items and characters. As the player picks which lines specific characters say, such choices have little to no influence in the inner workings of the game, but they may affect the player's perception of the characters and their situations, as show by this interaction where two characters watch an unspecified television program together and the player much choose the character Shannon's answer:
EZRA: What are they singing about?
- SHANNON: They're singing about travel.
- SHANNON: They're singing about going home.
- SHANNON: They're singing about hard times.
This is perhaps the example which most closely resembles theater, as intended by the developers, who inform the player of the story's divisions - acts and scenes. Tamaz Kemenczy, one of the developers, in the 2014 Game Developer Conference, demonstrates how his team took cues from and were inspired by theater, in his presentation "The Scenography of Kentucky Route Zero":
Since we're talking about a videogame environment and not a physical theater space, for us performer and spectator start to mean different things for us in KRZ [Kentucky Route Zero], they kind of collapse and become one thing, simply the videogame player. Part of this collapse has to do how we observe, as spectators, theatrical events in a theater space and how we can observe a theatrical event in a videogame. So, in a conventional theater space, as a scenographer, you often design for the proscenium, which basically means the front of the scenery, but in a videogame. from a scenographic perspective. you can design for cinematic frame.
The spectator can be dislodged and lifted from the seat and can move in all directions, so the idea of a proscenium kind of changes or just drops away in some instances.
In the 2013 Game Developers Conference, Jake Eliott, one of the designers of Kentucky Route Zero, in his presentation "Designing For Mystery In Kentucky Route Zero", describes the process of creating this experience:
You're making software that is reactive to somebody, like how an installation is reactive to a viewer; in the context of performance it is reactive to the performer (you're doing instrument building) and in the context of a game it is reactive to the player and the simulation, and it all runs in real time.
You can see some of these similarities between these different practices kind of permeating the boundaries between game, performance and installation, like in recent games like "Proteus" or "Panoramical", you can see some of these borders kind of collapse, and in these old games like "LSD Dream Emulator" or "QQQ" or works of JODI, they start to collapse these performance videogames.
Jake Eliott even shows that his game has no, or limited, interaction, by saying that "a lot of games offer players great chances to be strategists, to be powerful strategists, but this game, Kentucky Route Zero, is about disempowered people and it's about not having that option of behaving strategically", which demonstrates how a lack of consequence (no opportunity to be "powerful strategists") can enhance the theme of a game.
In the case of Firewatch, by Campo Santo, the player controls one character, being allowed to explore a 3D open-world environment. In here, the story is also linear, and choices often do not create noticeable consequences. No challenges are present, only prompts by the character's one companion to explore in certain areas which allow them to progress through the story. The player simply follows a story, with no agency on how it develops, having, however, extreme freedom to explore the landscape without following the story.
In the last example, The Beginner's Guide, by Davey Wreden under the studio name Everything Unlimited Ltd. a narrator (the author, his voice and his presented persona) explains the user that he will take them in a journey through someone else's game, a friend named Coda, much like in a museum trip. The user walks through a short level of a game Coda allegedly created, while the narrator explains what is happening. The levels have no challenge, the user walks the path that is displayed to them, and nothing more. It is a story and a though experiment where no choice is given to the player.
With these examples in mind, we must ask why are these considered video-games? If they reduce player agency to very low levels, having their choices essentially ignored or outright refused, what still places them in a group whose defining feature is player agency and complexity of choice and consequence, if their choices do not matter?
A common argument for those who wish to expel low player agency story driven games from the video-game category draws on the importance of non-mechanical effects of choice. While one's decisions in a Telltale game may not have deep consequence on the in-game workings, they can shape our understanding of situations, scenarios and characters presented to us - choosing to downplay the character Conway's pain or admit its reality in the videogame Kentucky Route Zero may have no effect on the game character's movement speed, health or any future interactions, but it portraits him as someone who keeps things hidden for the sake of a quicker and smoother social play or as a man who admits how he feels and describes his situations with honesty. Given multiple similar choices, the player's mental representation of Conway can have surprising diversity and complexity. In this sense, the typical person-machine interaction processed at the machine level is replaced by a person processing of character traits, scene setting and problem framing.
Following this path, these games, while still incredibly close to movies, interactive film, theater or books, can be defined by how the interaction is processed not primarily by the machine, but by the player, allowing the standard video-game definition to remain intact and allowing this genre to subsist inside the label. However, one can transport this definition over to, for example, movies. In a film, viewers often attempt to guess the character's choices, as they are engaged in their story and interested in seeing it go through; in this way, the movie invites the viewers to process information in similar ways that low interaction video games do, by creating mental representations of the character's personality. Still, unlike videogames, movies collapse into a reality - the character does or doesn't do what was predicted, they answer in one way or another. In videogames, the choice, even if it has no effect (and precisely because it has no effect), remains open - the developers did not answer the players' expectations, so they're allowed to keep them. In this way, the inclusion of low-interactivity games in the videogame category seems to be, at least, plausible and effective.
We can establish a parallel with the ideas in Jorge Luis Borges' short story "Pierre Menard, author of the quixote", where it is suggested that authorship comes from the reader at least as much as from the writer. As Howard Ginskin puts it (2005, p. 1):
It is a deeply profound revisioning of how meaning is created through the interaction of man and text. Through Menard’s recreation of the Quixote in a different time and place from Cervantes’ original, Borges implies the simple yet disturbing supposition that the meaning of literary works is entirely dependent on the varying historical and social contexts in which they are read.
As "meaning is created through the interaction of man and text" (Ginskin, 2005, p. 1), the choices a player makes in a videogame, even if they do not alter the game mechanically, do create a relation between the reader's interpretation of the story and the author's code, images and text, in much the same way a mechanically significant choice creates an interaction between the author's work and the player's decisions.
"Pierre Menard, author of the quixote" also explores the idea that perceived authorship (and perceived inclusion in specific genres) can also alter the way someone reads a work of art, which is explicative of the importance of calling these non-interactive experiences videogames, as calling them something else would certainly change the ways they are perceived, used and experienced.
In fact, if we do a more sociological analysis of the cultural roles of these types of videogames, we see that they are played and experienced as games, - they run in computers, with a hands-on active engagement by the player, with some expectation of the presence of interactivity - as opposed to, for example, a movie - a more passive engagement by the viewer, and no interactivity. One could even forget the analysis we did firstly and focus on the way users utilize these low-interactivity experiences to classify and categorize them - an analysis not by semantic definitions but by use an cultural significance.
This situation is comparable to the one we face when discussing the public perception of interactivity; as it has become a buzzword, what we could describe as merely reactive is, in fact, interactive in the eyes of many, who are not aware of these differences. We face the dilemma, then, of defining terms by their more useful, technically correct meaning - reactivity as a system that responds to input in a predictable manner, creativity as a system that responds to input in an unpredictable and apparently creative manner - or by the socio-cultural standing, by how the word was shaped by forces such as marketing efforts - reactivity and interactivity as one, as a system that responds to input. The best solution is unclear.
It is clear that this separation of objects into distinct sub-genres or even full genres behaves likes an in-group/out-group situation, where individuals define what they see as "games" by their own experiences and personal taste, often relying on semantics secondarily. A gamer may proclaim that a game like Proteus is not a game but a "walking-simulator" (term often used to describe games whose main and sometimes only mechanic is simple and minimalistic - like walking through a virtual world for the sake of exploration or progression through the story only) or an interesting artistic piece devoid of importance to those who play games with "deeper" mechanics; this semantic rationalization often falls flat, in these cases, as the cultural element was primordial in its creation, as they often consider objects with similar minimalistic approaches games. When a certain sub-cultural group tries to culturally exclude various low-interactivity video-games, their platonic essentialist ideal of a "video-game" is not the only factor influencing why they think the way they do.
There is another interesting comparison to make with the genre of music, where interactivity may be matched up with silence. In John Cage's famous silent piece, 4'33'', the composer has given the audience, theorists and historians pause, as the implications that silence is extremely important were ground-breaking, at the time. Much in the same way we do not think pauses in a piece of music are not music, which would create an odd understanding of most music created so far, demanding from videogames a high level of interactivity, particularly if it has to be constant and ever-present, would remove many works of art currently, and without controversy, considered video-games from that label.
Analyzing the cultural significance of the label of "video-game" alone, in relation to the use of interactivity, reminds us to consider not only the semantic, creator-based sense of the term, but also the cultural significance that may come from the narrowing or enlarging of the label, as lesson to apply to "interactivity" as well.
- Anonymous; Do choices matter in Beyond: Two Souls? - Arqade - https://gaming.stackexchange.com/questions/134004/do-choices-matter-in-beyond-two-souls
- Elise Favis; Opinion – Your Choices Don't Matter In Telltale Games - http://www.gameinformer.com/themes/blogs/generic/post.aspx?WeblogApp=features&y=2015&m=02&d=03&WeblogPostName=why-your-choices-dont-matter-in-telltale-games&GroupKeys=
- Hunicke, R., LeBlanc, M., Zubek, R. (2004). MDA: A Formal Approach to Game Design and Game Research. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.79.4561
- Tamas Kemenczy; The Scenography of Kentucky Route Zero - https://youtu.be/nh_o8JEmVdw
- Jake Elliott; Jake Elliott - 'Designing For Mystery In Kentucky Route Zero' - https://youtu.be/XLYAxYGzhBw
- Giskin, H. (2005). Borges’ Revisioning of Reading in “Pierre Menard, Author of the Quixote”. Variaciones Borges, 19, 103-123.
At last, a personal website. Here I will post my adventures in putting sound and music into media - videogames and film.
I hope you like it and find it informative.