Jochem Paap & Scott Pagano: Spatial Experiments in Hyperanimation

Jochem Paap & Scott Pagano: Spatial Experiments in Hyperanimation

An interview with Jochem Paap and Scott Pagano

We meet Jochem Paap and Scott Pagano at NWE Vorst theater in Tilburg where the Playgrounds audiovisual arts festival is taking place. Rupture curated a program at this festival, containing the world premiere of Region Free 77, a collaboration between Jochem Paap, aka Speedy J and Scott Pagano, visual artist and curator of the Reline dvd series.
Though separated by the North Atlantic ocean – Jochem lives and works in Rotterdam while Scott is based in Los Angeles – they teamed up to create both Umfeld, a one hour long 5.1 surround sound hyperanimation film and Region Free 77, a collection of 77 randomly playing micro-compositions of surround audiovisuals.

JP: It all started when I decided to do something with surround sound some three years ago. I had no idea what it implied so I just started experimenting. When things started to shape up I started playing with the idea that I should make it an audiovisual project so I began looking for visual artists. I knew Scott’s work from his website and approached him with the question whether he wanted to work with me on this. He was very excited, said yes and that’s how we hooked up.

SP: I was living in San Francisco at the time when I got Jochem’s email. At first I was surprised: “What? I’ve got an email from Speedy J?”. We met in Rotterdam and I started with making a music video for one of Jochem and Chris Liebings “Collabs 300”. In the meanwhile Jochem had been talking about this idea of making a dvd. We started thinking about going down this rabbithole together and doing a large scale project each taking care of the image and the sound respectively.

What are ‘Umfeld’ and ‘Region free 77’?

JP: They are the result of a lot of experimenting within the audiovisual field, it was never pre-designed to be what it is now. I felt there were so many possibilities for audiovisual productions with surround sound that I just wanted to dive in. Surround has been around for over 30 years and is being used in a lot of movies. In electronic music there are so little people touching the surround domain, although the format provides a lot of possibilities which are very interesting to experiment with. I wanted to make something more structured than typical academic works but also less easy to consume than big Hollywood productions.

SP: I see Umfeld as both a final product and a process. It is an hour long graphics flim in concert with a really intense 5.1 soundtrack. The name is German and means ‘environment’ or ‘surrounding’ and essentially that is what it is: a series of environments engulfing the listener/viewer. Instead of guiding them or wanting them to have a specific reaction or emotional response, we provide different worlds of experimentation which they can explore, perhaps enjoy. There’s very aggressive section that might be a little discomforting, sections that are challenging to consume, but I also think there’s a kind of meditative nature to the whole arc of the piece. What I like about this sort of work is that you can let go of the expectations you have in dominant cinema, music videos and all these tropes we’re being fed.
One of the most challenging things was how to deal with composition over so much time. I’ve done a lot of experimental animation work, but never before did I produce a movie this long. One way I deal with this is to look back at the history of experimental animation, at the whole lineage of the medium. I try to understand what has been done and make some sort of movement forward or contribution to the historical arc of this kind of work.
I’ve learned a lot as an artist, producing that much content. I really focus on the synchronicity of image and sound, both in specific hyperanimation [A new and expanded form of animation using advanced digital technologies] and in mood, texture and tone.
We finished Umfeld over a year ago and in its aftermath we created Region Free 77, which I thought was going to be like a short 10-15 minute piece…Instead it expanded into a 45 minute massive undertaking. I remember when I first got the audio from Jochem and I thought…Ok… I’m going to have to block out a couple of months for this one…
It was really nice to go from the linear format and make it into individual segments with an esthetic consistency and an overall visual and oral concept.

This poses a whole new dramaturgic problem. How do you go about solving that?

JP: To be honest, when we first started this piece we didn’t really understand the technical issues around it. It went through a couple of incarnations.Because of technical reasons, some esthetic decisions couldn’t be made. Since it is a dvd we had to program it in order to have dvdplayers play it in random order.

SP: Exactly. This has been done before to a certain degree, but never with this many sections.

JP: It was a trial and error project for everyone involved. We had to alter the nature of the composition while we were making it. It ended up as a series of micro-compositions which clearly have things in common. We were considering making it tighter, have the pieces merge without gaps, but now we’ve built silence and blackness into the composition. It was an experiment but I’m happy with the way it turned out. We had to deal with the limitations of the dvd format and we were disapointed about the possibilities and the amount of freedom. It works on every dvd player we’ve tested in on so far, but we had to scale back since it’s not a computer and you can’t do certain things like cross-fades for instance.

SP: It’s nice because it functions differently on different systems. On the dvd it randomizes differently than in the installation version which runs from a computer. There’s also a bit more gap because players can’t jump to the next section that fast.
It’s good to have it here at playgrounds in the installation form where the randomization is more true to how we’ve intended it originally. It’s also much more instantaneous.
Even though the pieces are playing in a random order, they are designed specifically for that. Jochem experimented with using silence, using gaps, changes in intensity,… It’s hard to describe the myriad of subleties in the music, but they are all done with the randomisation in mind. On the image front there’s a huge range of images, almost even moren than in Umfeld. Pure cinematography, complex audio-synchronised computer generated work and everything in between.

What software do you use and how important is it to use this specific software?

JP: For the audio it was quite hard because most of the things I wanted to do with surround were not possible with existing software. I started out doing things in very crude ways. Do one task in this program, take it over to another program, another platform,…
Every sound in every stage of the production has gone through different platforms and a series of programs, both custom and commercial.
My driving force is always the idea and I’ll do and use whatever it takes to realize what I have in mind. In some cases this was impossible with existing software so I worked with Telcosystems who developed some tools for these specific tasks.

SP: I use a range of 2d and 3d animation programs. To a degree there were less format challenges for me. The main thing for me to figure out was new ways to work with my tools and create a level of synchronicity to sound which I never achieved before. Although the work is delivered on dvd, I’ve developed everything in HD. This brings about a whole new set of issues for an individual artist in a small studio: having to deal with massive rendering and storage needs. None of that is really interesting, you just need a lot of computers and spend a lot of time dealing with them.
As the end of Umfeld and the beginning of Region Free 77 approached I came up with, for me, new ways of audio analysis and mapping audio to animation. It’s been very educational. I’ve misused existing tools in order to get some effects. I’ve also been using Houdini, a high-end 3d animation application. It’s very good in taking any sort of information and turning it into other information. Using that for audio-synchronous 3d has been a real eye-opener.

umfeldIn terms of cinema, where do you stand? Go to a random big budget, action-laden Hollywood movie and you see that they are more and more built around special effects. It’s basically just a thrill, the dramatic experience is based around audio-visual effects and in a way it is really abstract. The story is so thin that when the action sequences start it becomes completely irrelevant.

JP: In a sense that is true. It’s very abstract if you’re used to seeing film as a story. It becomes more like a rollercoasterride. You’re bombarded with audiovisual impressions and eye-candy. Take a movie like Transformers for instance. Technically they’re absolutely brilliant, but besides that,… well…
In the end that’s just about money. It’s taking the safest possible route to getting as much revenue as possible.

Do you think it’s possible that people start to realise that what they’re watching is not longer the story but just the audiovisual ride?

JP: I find it an interesting route to take. If you would completely abandon the narrative and make a production that is as technically advanced, with an equally high production value, but completely abstract. I would definitely be into something like that…

SP: …and a lot of people with you. Overall we’re very well trained, very coded to wanting to see representational images. Even if there is a paperthin storyline we still need that suture in the end to provide the satisfactory “I understand what’s going on, these images make sense to me, these emotions are triggered because I’m used to having them triggered in this way.”

JP: This subject has been investigated by so many artists. For years people have been trying to find the balance. How abstract can I go untill people are completely lost and lose their attention?

SP: What we’re doing is outside the commercial cinema realm, but it’s also outside what’s being done in terms of music video. It think there’s space for this and that interest will grow because there’s enough people out there looking for exciting audiovisual content that isn’t triggering all the conventional responses and that isn’t just a marketing tool or a corporate message.

JP: It definitely has a chance. If you consider the fact that a mere 15 years ago people would dance to very ‘defined’ music, with a song structure like we’re used to for over 50 years. And now, hundreds of thousands of people go out every weekend to dance to music played by a guy behind a computer screen, made by other guys behind computer screens. It’s very abstract, very repetitive, but we’ve become used to dance music. If you would play this to people 20 years ago and tell them this is the future, they would respond with “no way, this is sci-fi!”.
The same thing could happen with other disciplines and artforms in the next decade. It’s very difficult to predict how peoples minds get adjusted to new formats and new ways of consuming. It can happen really fast, look at mobile phones and texting for instance. I you consider that I have two kids, of 5 and 8, their first mobile phone they will own will be this really flat touchscreen device capable of playing video, audio,… I think audiovisual media will go through a very big change in the next decade, where you’ll be able to consume and access it everywhere. Things change a bit slower in popular cinema than in music but we have no idea how people in 50 years will be able to understand certain ways of storytelling or experience film and music. Those shifts can go really fast.

SP: One other thing I want to bring up is that all this brings about whole new distribution modes. Our project for instance is not just this beautiful packaged dvd with amazing artwork by Keepadding, but the main piece is available as a free downloadable dvd image. We’re trying to bring this work out in the world, but it’s also with a total understanding that there are huge barriers. A lot of people wont go “oh there’s an abstract hour-long crazy surround sound dvd? I want that!”.
So by offering it on our website, people will be able to experiment with the idea,…

JP: …get familiarised with the work. That’s just the way people’s minds work these days. If you stumble upon something which you think you might like, you want access it instantly, sample it for free. And if you support the idea, you pay the people behind it some money or you buy the physical product. Specifically this project is not something that would work if it were only available as a dvd in shops. So the way to reach an audience is to make it as easily available as possible. As a consumer I think like that as well. If I like some work and I can’t get to it legally, I’ll look for other ways to get to see it. I can’t order a dvd and wait for 2 weeks for it to arrive… that’s the past.
We’ve spent so much time on this project and we’re are very happy with the way it turned out. And we both know that a lot of people around the world who would be interested in this sort of work are completely unreachable through traditional distribution mechanisms. So by making it available as a download, which is the most accessible way for most people who would be interested in this kind of work, they can sample it, enjoy it, and that’s the most important thing. We want to reach them with this work.

More info, the downloadable dvd image of Umfeld or information on how to order the dvd which includes a short documentary and Region Free 77 can be found at