PEMcast 11: Immersive Part 2

[background music]

Chip Van Dyke: Welcome to the "PEMcast," conversations and stories for the culturally curious. From the Peabody Essex Museum in Salem, Massachusetts, I'm Chip Van Dyke.

Dinah Cardin: I'm Dinah Cardin. Today, we bring you the second installment of our series on immersive environments.

Chip: There's just something about a space that envelops us completely that is impossible to ignore. When something immersive takes over our senses, we're not just looking at something, we're in something, having a 360-degree, full-body experience. In the episode we have for you today, we explore this transportive power of immersion.

Dinah: First up, we'll hear from Jason Reiner, a sound designer who has a lot to say about exciting new developments in immersive sound.

Chip: Then we'll join Dinah as she goes inside of virtual reality world where no one wants to be.

Dinah: Finally, we'll hear from a dedicated LARPer, that's live-action role-player, about a recent LARP she played in Finland, about Tulsa, Oklahoma.

Chip: Just like last episode, we're going to step away from the mic in order to let the stories speak for themselves.

Chip: We'll be back at the end of the show to tell you more about what you've heard.

Dinah: We hope you enjoy it.

Jason Reiner: Ambisonic is a name given to surround recording, three dimensions of sound. Stereo is basically binaural -- two ears. That gives you the ability to pan things left and right.

What three-dimensional or ambisonic audio does, is allows you to work with above and below, up and down, and depth. Left and right, side to side, forward and back. There's always been ways of creating three-dimensional audio. The main thing that you're able to do now that was difficult in the past was have the video and audio track your head moment.

Chip: So, if I hear a dish break behind me in this recording, if I've got my headphones on or whatever, I guess it would be headphones?

Jason: Yes.

Chip: If I've got my headphones on and in that recording, I hear a dish break behind me, I can turn my head in that direction that I heard it from, and probably hear in that direction somebody sweeping up the pieces.

Jason: Exactly. Normally, if you think about what a binaural recording that you may have that's reproduced in a headset, and you put on the headset, and you walk around, anywhere you turn, the binaural recording is turning with you. You're not in a congruent space.I like to use that word because, if I'm sitting in a room and I've got a congruent soundscape and I turn my head, then all the sounds are going to stay where they should be. That's going to be a psychological reassurance of like, "OK, I know that the sound over there. That light's being turned on. I can hear it turning on. See the light turning on. Reality."

Chip: It's orienting.

Jason: It's orienting, and I think there's a fundamental principle of congruency. The reason why we're seeing so much 360 emphasis right now is because people are starting to say, "We can make a game, we can make a million bucks. We can make a movie, we can make a billion dollars." That's true. It's going to happen. Virtual, augmented...I'm in favor of reality, the big R. It's like what we're trying to do is reshape what we already know is real. I'm interested in taking us closer, understanding more deeply, feeling more empathy with what's already there.

[pause]

Dinah:

VR Demonstrator: Is that on? The goggle's on OK?

Dinah: Yeah.

VR Demonstrator: All right. Now put your little headset on, and I'll take your lifeline and make sure you don't crash on any computers.

Dinah: Thank you.

VR Demonstrator: You're very welcome.

VR Sampler: Go for it.

[pause]

VR Sampler: I think I don't want to do it anymore. [laughs]

VR Demonstrator: Take it right off. It's all right. I only lasted a couple of minutes. It's OK.

Dinah: Wow.

VR Demonstrator: Yeah, it's...

Dinah: You're putting this on people and you only lasted a couple of minutes?

VR Demonstrator: Yeah, but I tell them. I'm honest about it.

VR Sampler: Yeah, it's OK.

Dinah: How would you describe it?

VR Sampler: I really enjoy the VR experience. I think it's wonderful. I can't wait till all the art museums are on VR and I can take my art history students through them. This subject matter is tough, and it's not for everyone. I think it's OK just to watch as much as you feel comfortable watching and then stop when you feel comfortable stopping.

Dinah: Tell me your name.

Brian Donnelly: Brian Donnelly.

Dinah: What did you see and hear?

Brian: I saw a story about a man who was put in incarceration and put into solitary confinement, and how it degraded him mentally. To cut a human off from other humans is, unless it's your choice, I don't think that should ever, ever be done to people. That's just not...We're more than we are, we are not just this body. We are our interactions.

Mary Kelly: I noticed that when you started to watch it your whole...the way you held your body changed, your shoulders started to slump, your whole body just seemed heavy watching it.

Brian: Yeah. It was hard. It weighs on your heart. I felt that way. It was there. I knew what I was getting into when they were talking of the subject matter, but it's like yeah. Then seeing the guy and really experience and not just a flat, screen person, it's like, "Poor guy, can someone give him a hug?"

Dinah: What is your reaction to going through it?

VR Sampler: It was upsetting. It just seemed like he was going right down a hole, it wasn't helping anybody, it wasn't helping him. At some point, he'd be getting out of jail and he'd be worse off. That's my first experience with a virtual reality. I've heard about it, but it was interesting to see how incredibly real it was.

Dinah: You've been wanting to do it and you've never done it before?

VR Sampler: Yeah, I've never tried virtual reality before but I keep seeing it around a lot, so that was an interesting first experience of it. It's just so weird to put on a headset and then some headphones and then you're completely in a different space. It feels like you are in a different space physically, but you're not.

Dinah: What was the worst part, do you think?

VR Sampler: The fact that the system is against the people that go in, and once they go in there's not really any coming back.

VR Sampler: No, solitary confinement, we try to ignore it. We try to push it under the rug because, who wants to think about it? I absolutely applaud Frontline's efforts in creating this film and I hope as many people as possible see it, and particularly our politicians. Private prisons have got to go.

Kenzie Audette: I am Kenzie Audette and I work for the VR department at Frontline. The piece we're showing here is a solitary cell and a solitary survivor, I guess you could say, and what it's done to him and how he lives his life going forward. One of the strengths of this device and VR in general is the ability for it to show you in three dimensions and put you in that space so that you can really feel how incredibly tiny the cell is and how cramped his conditions are, like you're with him in it as well. You can see very clearly immediately that if he were to lay horizontally in the cell, he would bump his head before he hit the other wall. This is not a very big space. I don't think that translates as well on a 2D screen. I would hope you would never want to go to solitary in real life, but I think there's always something to be learned by going there in this kind of environment, in this kind of setting. It's very safe. You get to make these choices, and you get to take it on and off how you see fit. It's about taking you to places you know that you wouldn't normally go. We do that in the documentary space as well. Like, we're taking you to Syria. You don't want to go to Syria, but you should experience these people's lives and know how they approach problems that they have. That's the same thing we're doing here with different stories.

Dinah: Thank you.

VR Sampler: [indecipherable 8:42] .

[pause]

Johanna Koljonen: In 1998, I was a year out of high school, and I had been a LARPer, a live role-player, for a few years. I'd organized eight or nine fantasy games myself, the kind where you dress up as fantasy heroes and run around in the forest.

I was asked to go to Turku, in Finland, to play a game. The game was called Ground Zero. This game was going to be set in Tulsa, Oklahoma, during the Cuban Missile Crisis, and we were going to play ordinary families.

Tulsa is one of those places we found out would be a target in the case of a nuclear war. A lot of quite ordinary American families had built fallout shelters in their back gardens and so on.

This game was set in a suburban area outside of Tulsa, where ordinary families, neighbors would go out down into their bomb shelter when the Cuban Missile Crisis was coming to a head.

Obviously, most of us were grownups, we thought then, which means at least 18 years old, but the youngest-looking player got to play a mentally disabled teenager, so we had one person to represent the innocence, I guess.

We played ordinary people. I was a school teacher, for instance. The guy who played my husband played a person who was a war veteran and afraid of loud noises.

We went down into this basement, and it was made to look like a bomb shelter. There was a whole wall of cardboard boxes containing the tinned food that we were going to eat in case of a nuclear war.

There was a radio that gave us the news and played patriotic music. We could hear the speeches of the president. Some of these were historical, of course, some of them were not.

Before the game, we were told that this might be a historical game, in which case, of course, the crisis is averted, a nuclear war doesn't happen, but it might also be a uchronic game, in which case an alternative history erupts. We thought, "Pfft, they're just saying this to give us an opportunity to be a little bit excited." That's what we thought.

We went down in there and we were following the radio, and the situation was getting worse and worse politically. Then we lost the radio because something was happening, the conflict was getting worse. Then we lost electricity and we put on the little candles. We were cooking our little beans on the little camper kitchen. Then the bomb fell on Tulsa.

[pause]

Johanna: What had had happened? They built a special effect. That whole wall of cardboard boxes that we hadn't opened because you don't open the cardboard boxes because it's scenography. You know they're going to be empty, but they weren't empty. They were filled with loudspeakers, the whole wall. [laughs]

They made a bomb sound. Actually, they blew out three of the speakers because the sound was too loud, but it was still pretty damn loud. We had just cooked our little bean meal and we just dropped all our plates, and screamed, and some people cried.

The guy who played my husband said later -- and he had trained himself to be afraid of loud noises -- and if he hadn't gone to the bathroom an hour before, he would have absolutely soiled himself in that situation.

It's funny now, but it's not actually that funny because we were really afraid. I remember a very bright light which couldn't possibly have been there. That's something that my mind has edited in after. We didn't die and the game didn't end, so clearly, we were survivors of a nuclear attack.

We kept playing for another 12 hours, knowing that the city wasn't there and probably that everybody we knew were dead. Then the game masters came in at the end and said, "The end." Then we came out and cried for a couple of days.

[pause]

Johanna: LARP is short for live action role-playing. We don't capitalize it in Europe anymore. It's a verb as well as a noun. It's a type of role-playing where you inhabit the character physically, so it's it's very close to improvise theater in that sense.

However, you don't have an audience. We talk about the first-person audience. The audience position is inside the body of the player because as you move through this world and you improvise and you improvise this narrative together within the frames that have been set by the designer or the player collective, everybody's story...

It's an interaction machine that will generate the interesting story outcomes. Everybody who moves through this story will, of course, be the main character of their own narrative. Everybody's story comes out different. Everybody gets to be the storyteller.

In the Nordic countries, Nordic LARP is very different from your typical American LARP in that it isn't competitive. It's collaborative, fundamentally. The characters can have different goals, but all of the players are on the same side, they have the same goal, which is to tell an awesome story for everybody.

The range of topics is much wider, and we are just as interested in tragedy as we are in entertainment.

I might go and play Hamlet for three days at actual Castle Elsinore, or I might play in the sense of experiencing a narrative designed by LGBTQ designers, which is about the birth of the AIDS epidemic in the United States, for instance, and about love and loss around that.

Or I can play a LARP in a theater black box, which is completely abstract visually, and none of the characters speak. You only express yourself through movement, and then you tell, in four hours, a very complex story about an expedition that dies on a mountain. It could be a bunch of different things.

Sometimes I'm going to go to a castle and pretend I'm a professor at Harry Potter University. That's also amazing. There's nothing wrong with that. I'm not opposed to having fun.

A lot of us who grew up with this mindset also start to notice how much we use that skill set in all of our careers, which could be digital games, or you could be a priest, or you could be...There's so many other things that you could be doing.

My name is Johanna Koljonen, and I am an experience designer, and writer, and a game designer, and a broadcaster, and a bunch of other things as well.

[pause]

Chip: That's our show for today. Thanks for listening.

Dinah: Thanks to Jason Reiner for taking the time to talk with us. You can find Jason and many of his immersive audio projects at earprint.com.

Chip: Thanks to Frontline and the Salem Film Fest, and to all of those who spoke with us about "After Solitary," the Frontline VR experience.

Dinah: Thanks to Johanna Koljonen for allowing us to share her story.

Chip: Find more content related to this episode on our website at pem.org/pemcast, and check out our new website. Producers for this episode are Whitney Van Dyke, Caryn Boehm, Melissa Woods, Dinah Cardin, Jennifer Juan, and me, Chip Van Dyke. Corbett Sparks is our audio engineer.

Dinah: Music for this episode was composed by Forrest James.

Chip: Tell your friends they can subscribe to the PEMcast on iTunes, SoundCloud, and pretty much anywhere you can find podcasts.

[music]