Published: Thursday, 27 October 2022 15:36 | Written by Jamie Ruby
The new documentary from filmmakers Sebastian Pardo and Riel Roch-Decter, The Computer Accent, follows the journey of the innovative and boundary-pushing band YACHT, as they attempt something unique: handing over creative control of their album to artificial intelligence. Working with researchers in the field, the group uses cutting edge machine-learning to create a one of a kind collaboration between humans and machine in all aspects of the creative process, including music, lyrics, artwork, videos, and more.
Recently, SciFi Vision talked to the musicians of the Grammy-nominated dance-pop band about their ground-breaking journey creating their album Chain Tripping. The spark of the idea first came from band member Jona Bechtolt. “I've always been interested in interrogating new technologies and trying to break them and trying to misuse them to expose what's hidden from us,” he explained. “…This was the newest tool available, and it seemed like it was going to be important for the future. I wanted to know more about it, and I wanted us to learn about it through making things with it.”
“If we want to understand something, really, the best way to do it for us is just to try to figure out how to make something with it,” added vocalist Claire L. Evans, “because then we're going to come up against all those challenges and have to solve all the problems and get a deeper understanding of the thing.”
Although it was something new, they weren’t worried, said Robert “Bobby Birdman” Kieswetter. “I think we’ve worked together enough on enough projects to know that…we're probably going to be able to be happy with [the album] just based on our working relationships and our history and past results…I don't think there was a concern that we wouldn't end up with something interesting at the end of it.”
Initially, the plan was to release the album without any context and see the response and use the documentary as the reveal of how the album was created, but for various reasons, including that the documentary took longer to make than the album, they decided not to do that and to be transparent from the beginning. “[T]here was so much that we were learning along the way, and there was the spirit of discovery,” Evans told the site. “We were becoming aware of the enormous complexity of this stuff. I mean, going into it, we really thought like, ‘We'll just train an algorithm [and] press some buttons,’ but then the more we got into it, we realized we’re actually working a lot, and we're doing a lot of stuff. We're doing something that no one's ever done before, really, in this specific way, and we have to try to explain what we're doing to people, otherwise, they're not going to get [it].”
Still, there is controversy about using artificial intelligence in any form of art, because some people think that that means that it’s effortless, but there was actually quite a bit of human involvement in their process. “We were involved at every step,” said Bechtolt, explaining that at the time, the models they were using couldn’t generate an actual song with structure, just clips. “What we ended up doing was generating thousands upon thousands of riffs, like two to four bars, maximum, of notated music,” he explained.
They then had to put all of those pieces back into the computer and test them out on synthesizers and cherry-pick which parts they would use for all aspects of the songs, such as the vocal melody or drum pattern, and then put them together.
“It ended up like a scotch tape method, like a cut up method of just putting all of these pieces together to make songs since there wasn't any model available to create structure,” said Bechtolt.
They did the same thing with the song lyrics. “We had this massive corpus of stuff that we trained this text generating model on,” said Evans. “We got this huge document, this big thing that we printed out. Then, I was literally highlighting lines that I liked, and we would collage. It was essentially a collage process.”
Creating the music and they lyrics were two separate processes, which made it even more difficult, making the words fit the notes syllable by syllable. “We had to transplant the lyrics that Claire chose on to the melodies that we chose,” Bechtolt told SciFi Vision. “They're like two incompatible things that we then had to break language apart on top of these melodies.”
“We didn't have to do this, but we were very stringent about how we wanted this to be and how this worked,” he added, also saying that it was important that the music not only sounded good but sounded and felt like their own.
Although they probably would not embark on a journey this way again, getting assistance from AI changed the way they look at their music. “[J]ust going through the process of using those tools so often and for such a long time, I think it has sort of shifted our perception of what's possible,” Kieswetter told the site. “…We were playing things or singing things that we wouldn't have normally thought to do. So, I think now we can't help but approach songwriting through that lens at this point.”
The documentary allows you to see their entire process, and the band performs live after some of the screenings, allowing viewers to see the results, which has worked really well. “People were able to see the entire [thing],” said Evans. “I mean, it’s not often you get this experience and can see the entire process from inception to the final performance. So, we've been trying to do a lot of that around the film.”
Although the time to complete the film went beyond the album production, Bechtolt is actually grateful because the climate with AI has shifted. “[N]ow, the greater consciousness around AI art-making has caught up with where we're at, and it's a completely different landscape. Now people understand what these tools can do, and now, like DALL-E-2 exists, and is so widely used. People understand it a little bit better.”
“It's how it was with synthesizers and drum machines,” expressed Evans. “Some people are going to fall in love with this new technology; some people are going to reject it. Some people are going to exist in a kind of in between space, which I sort of feel like that's where we are. There are things that are interesting about it. There are things that are scary about it. I want to be able to take what's good and make it part of our work in a way that protects us or insulates us from the threat of being displaced. Like, if we bring it on to our team, it can't hurt us.”
Screenings and performances will be taking place throughout October and November in major cities.
For the full conversation, watch the interview or read the transcript below.
***Edited for clarity and length***
SCIFI VISION: I know it's obviously not the same thing, but I've been into making digital art using AI. So, I when I found out you guys did this, I was really interested and I wanted to watch the documentary and wanted to talk to you. CLAIRE EVANS:I mean, it's very similar. JONA BECHTOLT:I mean, we did we made all the the album art [that way], that’s digital art They released the model open source now, though, so you can do it on your own computer. CLAIRE EVANS:It’s the beginning of the end. Yeah, exactly.So, the one thing I wanted to know, and obviously you talked about doing this because it's different, but what was sort of, I guess, the spark that made you guys want to do it? Like, where did that come from? What was your interest in it? JONA BECHTOLT:I think it was my spark originally. I've always been interested in interrogating new technologies and trying to break them and trying to misuse them to expose like, what's hidden from us, I guess. I'm really interested in breaking tools, and this was the newest tool available, and it seemed like it was going to be important for the future. I wanted to know more about it, and I wanted us to learn about it through making things with it. CLAIRE EVANS:Yeah, we're not very good at book learning or whatever. If we want to understand something, really, the best way to do it for us is just to try to figure out how to make something with it, because then we're going to come up against all those challenges and have to solve all the problems and get a deeper understanding of the thing. JONA BECHTOLT:Yeah, and since we saw that AI was not just one tool for one specific medium, that it was touching every aspect of art making, we felt like the album was the perfect medium to showcase all of the different things that it would touch, from typography to visual art, to the audio itself, to language, to the music composition. So, it felt like a great way to show all of the things. Were the two of you kind of scared when he came up with this idea at first? CLAIRE EVANS:Yeah, I mean, JONA has a tendency of coming up with a really big idea, and then being like, “Let's do it!” Then, it's like a roller coaster; the only way out is through. We've got to just go through the entire experience. Oftentimes, it sounds so ambitious and overwhelming that we're like, “Oh, God,” but once you begin the process, it's always rewarding. There's always something, and oftentimes, these grand ambitions seem beyond our scope. Then, once we get into it, we realize that we're exactly the people that should be doing this. It's not the computer scientists and the coders necessarily who should only have access or understanding of this stuff. It's the people who are overwhelmed by it, who don't fully understand it, who feel like the consequences might displace their careers and muddle up their world. We're the people that should be messing around with this technology and trying to figure out how to make sense of it and how to make it part of what we do, so that what we do can never be taken away from us. What about you, Rob? Were you apprehensive? ROBERT KIESWETTER:I mean, I think we’ve work together enough on enough projects to know that the mission statement was there, like let's hand over the reins to AI, but we're going to end up with something that's off that mark, and we're probably going to be able to be happy with it just based on our working relationships and our history and past results. So, I wouldn't say fear is the right word, or apprehension. I don't think there was a concern that we wouldn't end up with something interesting at the end of it. Obviously, the album is out now and the documentary is coming out, but did you announce to your fans ahead of time that you were doing this? I was kind of curious of the reactions, because I know just from the little bit I've done how you get all those people who are like, “It's not art if the computer [made it], and all that stuff. What were the first, I guess, positive and negative reactions? CLAIRE EVANS:Well, initially we wanted to release the album with no context. Sort of like we would release the album and see how people respond[ed] to it, and then use the documentary as the reveal of like, “Hey, this was actually all made AI.” But for various reasons, we didn't want to do like a gotcha type thing. Also, the documentary took a lot longer to make than the album did. So, there ended up being a different sort of more staggered rollout of stuff, where we kind of wanted to be much more transparent about what we were doing, because there was so much that we were learning along the way, and there was the spirit of discovery. We were becoming aware of the enormous complexity of this stuff. I mean, going into it, we really thought like, “We'll just train an algorithm [and] press some buttons,” but then the more we got into it, we realized, no, we're actually working a lot, and we're doing a lot of stuff. We're doing something that no one's ever done before, really, in this specific way, and we have to try to explain what we're doing to people, otherwise, they're not going to get [it]. I mean, they're going to have that reductive, knee-jerk response of saying it's not really art, because AI made it. So, we tried to be as open and transparent about it as possible and explain it. It is complicated. JONA BECHTOLT:Yeah, with our, like, super fans, we've had incredible conversations about what this all means and what it means for the future and the kind of processes that we’re using to create music. It was really great. I think all of the like close conversations we had - the wider press didn't understand it, or even touch it at all, and now I'm grateful that the movie took longer to come out, because now, the greater consciousness around AI art-making has caught up with where we're at, and it's a completely different landscape. Now people understand, what these tools can do, and now, like DALL-E-2 exists, and is so widely used. People understand it a little bit better. CLAIRE EVANS:They have a better sense of it. They're not just thinking like Skynet, you know, HAL 9000. That’s where people were at in 2016, 2017. There wasn't a lot of like literacy culturally about what these tools were. I mean, it's still really complicated, and it's still difficult for people to understand, and a lot of the press talks about it in really reductive terms. JONA BECHTOLT:The right conversations still aren't happening, and I hope that our film can help that a little bit.
The one thing I want to ask you about, I mean, you make the comment in [the documentary] about how you didn't add, you subtracted [things]. But I assume that there was also kind of like, to some extent, [cherrypicking] and choosing which lines you used. I know just for making prompts [with digital art AI], people don't understand, like you said, it's not just “push button, make art.” It's not that easy. What are some things, kind of the human involvement that you had? JONA BECHTOLT:We were involved at every step. So, if at the time we were making lists, the models that we were using, for one, couldn't generate any kind of structured song. It could only generate clips. So, what we ended up doing was generating thousands upon thousands of riffs, like two to four bars, maximum, of notated music. So, just sheet music, not any kind of audio or anything like that. So, once we had these thousands of little pieces of sheet music, we had to then audition them by putting them into the computer and testing them out on synthesizers and being like, “Okay, this will be a baseline, this will be a vocal melody, this will be a drum pattern, this will be a keyboard line, a guitar line,” and like any of that kind of stuff. So, it ended up like a scotch tape method, like a cut up method of just putting all of these pieces together to make songs since there wasn't any model available to create structure. CLAIRE EVANS: There wasn't even a model available at the time that could generate chords. So, there’s not that many chords on the album. JONA BECHTOLT:Yeah, it's all monophonic. It's pretty crazy. CLAIRE EVANS:We did the same thing for the song lyrics. We had this massive corpus of stuff that we trained this text generating model on. We got this huge document, this big thing that we printed out. Then, I was literally highlighting lines that I liked, and we would collage. It was essentially a collage process. JONA BECHTOLT:And since those two models didn't talk to each other at all, it's like, you know, Photoshop and Premiere or something like that. We had to then transplant the lyrics that Claire chose on to the melodies that we chose. So, they're like two incompatible things that we then had to break language apart on top of these melodies. CLAIRE EVANS:Like syllable by syllable to make them fit each in individual note… JONA BECHTOLT:And we didn't have to do this, but we were very stringent about how we wanted this to be and how this worked. You wanted it to sound good too. [laughs] JONA BECHTOLT:We wanted to sound good and sound like us, not just like this new alien form. It needed to feel like us. Which, I mean, inherently, since we were making it, it will sound like us, but yeah, we were very conscious of it fitting into our [body of work]. CLAIRE EVANS: I mean, everything's performed live. There's like synthesizers and stuff, but we're playing all these parts live. I'm singing all these melodies; we’re producing and arranging everything. We're putting everything in the studio in Marfa. So, the human component is significant. I mean, you could give fifteen bands the same giant chunk of text and the same four-hundred audio clips of different MIDI notes and tell them to make a record, and they’d make a completely different record. It's like, we're basically just working from generated source material and kind of in collaboration with these systems, rather than letting the systems write the songs for us, which wasn't possible at the time and is not desirable to us even now. JONA BECHTOLT:Yeah, I guess the best analogy is it's like a multi-dimensional magnetic poetry. [laughs] [laughs] kind of, yeah. I kind of wonder now; there's probably a lot more can do this many years later. You probably could do most of it [with AI]. Rob, do you have anything to add? ROBERT KIESWETTER:Yeah, there definitely, I mean, even in the film, we show the end, these tools that are you're able to sort of like create songs from whole cloth, but, if those tools existed at the time, I don't think it would have been any fun. If we were able to just push a button and create a record, I don't think we would have been happy with the product. I think we're fortunate that we had to sort of like, wade through the muck to get to where we got to. JONA BECHTOLT:Even the tools today couldn't make Chain Tripping. They couldn't make any of those songs, because even though the songs are still pulled from our back catalogue and stuff, like part of us, they're also different than something we would have made just on our own. So, yeah, I don't think there's any model that could create any of the songs on Chain Tripping. CLAIRE EVANS:Because people change. A lot of AI is structured on this idea that if you take this historical data, and then you project it into the future, you just get more of the same of what came before, but artists don't work that way. Artists are evolving all the time based on their relationships with one another and what's going on in their lives, and, you know, their relationships with their own past work. So, you can never mimic that process of transformation that occurs when you sit down to make something new. You make me think about what I do [with artwork], because I take a lot of pieces and Photoshop them together. That's what I like to do, so it's not just pure output from [AI]. JONA BECHTOLT:Yeah, we like to say, “The output is not the end.” So, I know you said in the documentary, obviously, you wouldn't do this the same way again, because you do everything different, but is there any part of it that you could see reusing again when you make another album to help you?Do you think that would happen? JONA BECHTOLT:Well, I don't think it could help us at all, because it definitely doesn't make anything easier, but in terms of like, yeah, wanting to generate things that sound a little out of character, a little bit wilder, maybe? I don't know, Rob, what do you think? ROBERT KIESWETTER:Yeah, I don't know if we would even need to necessarily go back to the tools, but just going through the process of using those tools so often and for such a long time, I think it has sort of shifted our perception of what's possible in terms of, I think we talked about in the film, how some of this output challenged our physical capabilities. We were playing things or singing things that we wouldn't have normally thought to do. So, I think now we can't help but approach songwriting through that lens at this point. So, in that way, you know, the tools are always with us, you know? You guys talked about how it makes you look at your music different and everything, but was there anything specifically, that you kind of learned about your music that you didn't, I guess, recognize before you started that now you're thinking about more? JONA BECHTOLT:I mean, yeah, what Claire said a minute ago about how a lot of AI tools, they require lots of training of material. Just the fact that there isn't a form - the only algorithm that is inside of YACHT is just each of us have our own algorithm that is constantly evolving. There is no formula for how we make songs other than just like time. CLAIRE EVANS:Yeah, and I think trying to play and sing, specifically for me, these melodies that are not necessarily super tricky; they're just weird. They're not created by an entity which has a body so they have no relationship to embodiment. When we tried to play those, we realized how much our experiences of playing music are rooted in like, what actually feels good and what we're physically accustomed to doing, like these patterns, these habits that are just deeply ingrained and deeply physical. Trying to step outside of those is extremely challenging, because, it just makes you realize how…many habits you actually have. Then, when you do try to break them, you build new habits, I hope. It's like it really pushed us up against our limitations, I guess, our physical limitations. Yeah, I listened to it, and I like it; it's really cool, but the one thing that I kind of wonder about is how usually when you listen to music, at least for me, I'm kind of looking for the artist’s meaning in the lyrics. So, obviously, you got it from the computer, have you after the fact, I guess, sort of found meaning in that part of it and connected to it in that way? CLAIRE EVANS:Yeah, I think that's a really interesting question. It's something I thought about a lot. I mean, obviously, there's a level of meaning that came through just the process of collaging and arranging little scraps of lyrics together. We were thinking a lot about making it fit within the algorithmic structure. Of course, there were lots of constraints there, but in my mind, the songs all mean different things. It's just that that meaning is kind of evolving all the time, and every time I sing them, I'm like, “Oh, maybe that's what it means.” There's a kind of mutation that occurs, which happens normally anyway. Like there are songs we've been performing for fifteen years that when we wrote them we were children, and we were thinking about very different things…Now, when I sing them, they're about something else in my mind. They're also colored by the experience. Also other people are projecting their own meaning on them. That’s what we do with art; we project, our own thoughts and experiences. We think that these songs are written about us, but they're about the artist’s experience. So, there's always this shifting meaning that's already happening in music, and I think with these songs, my own interpretation might be different from theirs is different from the audience's, and I kind of like this sort of collective meaning making thing that we're all doing together. It's true that it's trickier to grab on to. It's much more slippery and elusive, but there's something really interesting about that for me. I mean, it's not necessarily the way I want to write songs forever, but it's a different relationship to meaning that was fun to play with for this album. Rob, do you have anything you want to add? ROBERT KIESWETTER:Well, I think Claire had undoubtedly the most intimate relationship with the with the words. She spent the most time on the ground highlighting and cutting up and pasting together. I think she summed it up perfectly. Now, what about documentary part? Is it strange? I mean, obviously, you guys have been performing. So, I guess maybe it's not as strange for you as it would be for me, but is it weird having somebody follow you around all the time? Did that ever get on your nerves, or was it just a cool experience? Like, how did that go? JONA BECHTOLT:Yeah, I never became comfortable with them around, [laughs] because, yeah, we perform, but that's like a very specific version of ourselves that we get to show. We hadn't ever let anyone in on the creative process, and especially [at] such a vulnerable time in the creative process where we're trying a whole new way of creating, and we're fucking up a lot. And we didn't look good a lot of the time, and we were having a hard time. So, no, I didn't like it most of the time. I loved the filmmakers and everyone involved. It was really great to have them around just as friends and people that that we got to know, but the actual documentation of it was awkward for me most of the time. CLAIRE EVANS:I found that the experience, I mean, the film obviously follows us in the studio and all that, but also we went and traveled. We did interviews with lots of people, and there was like a level of access that we got, just by virtue of having a film crew. We were able to have conversations with people that I don't think would have given us the time of day necessarily if we had just come in like, “Hey, we're a band.” So, in that sense, like, I think it was really beneficial, because we went into the studio with a lot more knowledge and understanding of the landscape and rudimentary understanding of the technology. I mean, obviously, we're not coders, but we at least talk to enough people to feel like we kind of get what's going on here, like behind the scenes under the hood, which I think was really an instructive part of the process. Then, by the time we got into the studio, we'd been [filming all these] interviews. We'd been doing a lot of stuff with filmmakers. I personally felt pretty comfortable with them, because we'd just been hanging out for like, two years at that point. So, it was just kind of they were there. But also, there's a lot of stuff that you don't see in the film. Obviously, we were in the studio for like much longer than the two days that you see. I think it was in a press release…you were showing the film and then performing for the people [who saw] the film; that was my understanding. That’s cool. I was waiting to see kind of what it came out like, that's why I went and listened to it after, because I wanted to know what it sounded like. CLAIRE EVANS: Yeah, it's been fun. When the film first premiered at CPH: DOX in Copenhagen, we did a screening and then immediately afterwards played a show and that worked super well, because people were able to see the entire [thing]. I mean, it’s not often you get this experience and can see the entire process from inception to the final performance. I feel like, yeah, it was good. So, we've been trying to do a lot of that around the film. We're doing that in New York and Portland and a few other places. Very cool. Well, like I said, I liked listening to it; I think it sounds interesting, and I want to listen to some of your other music to to compare it. I haven't had a chance to do that yet, but I think it's pretty cool. And I'm curious to see kind of where this goes and who, I guess, adopts it and who doesn't, because I know right now, at least in even in digital art, there's kind of a big [debate]. You’ve got both sides, people fighting back and forth with it. CLAIRE EVANS:Yeah, I mean, it's always how it's been. It's how it was with synthesizers and drum machines. Some people are gonna fall in love with this new technology; some people are going to reject it. Some people are going to exist in a kind of in between space, which I sort of feel like that's where we are. There are things that are interesting about it. There are things that are scary about it. I want to be able to take what's good and make it part of our work in a way that protects us or insulates us from like, the threat of being displaced. Like, if we bring it on to our team, it can't hurt us.