A maker space where art and technology merge

When Jack Wesson’s girlfriend was unable to visit her family over fall break last semester, he knew how to give her an enchanting glimpse of home.

Wesson brought her to the Center for Collaborative Arts & Media (CCAM) — an interdisciplinary research center at 149 York St. where traditional arts blend with computer science and technology. A Google Earth virtual-reality experience there enables users to stroll the streets anywhere on the globe.  

She donned a headset and was instantly transported to Malibu, her hometown. Wesson followed along on a monitor as she toured her neighborhood. She showed him the route she used to take to school. The experience made Wesson better appreciate the technology and the center that provides such easy access to it.

“It was emotional,” Wesson said. “I thought: ‘Wow, this is actually really powerful.’”

Wesson, a junior majoring in computing and the arts, has become a regular at the center where he is taking three classes this semester, pursuing an independent study project, and helping others use the center’s resources. He intends to pursue a career in virtual reality — making the hands-on learning experiences at the center time well spent, he noted.

“The center is invaluable to me,” he said. “I actually can’t imagine my life right now without it.” 

The center, formerly called the Digital Media Center for the Arts, re-opened in the fall of 2017 following a renovation and rebranding intended to establish it as a research hub where people from varied fields and backgrounds can collaborate on projects and tease out ideas using a broad range of media resources, including a range of cutting-edge digital tools.

The rebranded center is supported by an affiliated faculty that includes artists and experts in dance, drama, music, the visual arts, and computer science. A team of graduate fellows supports the programming and experimentation occurring at the center every day.

“One of the center’s virtues is that it brings together all of these really talented and innovative people into a single space,” said Johannes DeYoung, the center’s director and senior critic at Yale School of Art. “People bump into each other here and something really interesting and unique can serendipitously emerge from it. Where else does that happen?”

A sandbox space

The center is open to all Yale students, whether they are in Yale College or the graduate and professional schools. A Yale ID provides access to the facility’s amenities, many of which do not exist elsewhere on campus.

“We created some really smart spaces,” DeYoung said, standing in a large multi-faceted motion studio at the front of the facility — one of the renovation’s highlights — which is available for classes, events, and special projects.  

The studio is outfitted with a sprung floor, an eight-channel interactive projection system, and an integrated XR platform for working in virtual reality and augmented reality. A state-of-the-art motion capture system — the technology used to create Gollum in “The Lord of the Rings” films — is the studio’s crown jewel.  The system’s 20 cameras are arrayed along a metal grid suspended from the studio’s ceiling. They track the movements of an individual wearing a suit covered in reflective markers.  

“The cameras can track you even if you lie on the floor and the reflectors on your back are hidden. The system continue triangulating locations,” DeYoung said. “We’ve gone through a semester of getting it up and running. Now people are coming in with ideas about new work. It’s getting very exciting.”

The center also houses a video studio equipped with a green screen, a range of video and audio recording equipment, lighting, and another XR platform. A media lab is equipped with a large-format printer, small-scale 3-D printer, and a direct-to-substrate UV printer, which prints on three-dimensional surfaces. (For a recent book-fair fundraiser, the center’s staff printed an image on a nickel, which was sold for a dollar.) The lab is equipped with scanners, a laser cutter, and traditional bookbinding tools, whose presence underscores the center’s mix of low-tech and high-tech.

Students can reserve individual creative suites to work in private with a variety of equipment — drawing tablets, mixers and editing instruments, XR production tools, etc. — that they can access through the Student Technology Collaborative’s equipment lending program. The equipment checkout is located in the same hallway as the suites.

DeYoung pointed to two students who were huddled around a monitor in one of the suites.

“That’s a graduate student in design working with a student in computer science and architecture,” he said. “They’re developing a machine-learning algorithm to upscale image resolution. We’ll see how it goes.”

Minutes later, DeYoung described having encountered graduate students working on a project in a workspace off of the media lab.

“They had duct-taped some furniture-moving dollies together and were taking turns spinning each other while wearing VR headsets,” he said. “They were designing these funny experiences, but some of them were scaled up into interesting projects.”

This kind of experimentation gets to the core of the center’s mission, DeYoung said

“The center is kind of a sandbox space where people are free to try out new things or chase an idea,” he said. “If you get hooked on something worthy of more serious investigation, we can help channel you to the appropriate resources, whether on campus or elsewhere.”

Workshops and classes

A schedule of vibrant programming and undergraduate and graduate courses draw people into the center and inspire experimentation. The center hosts 24 classes during the academic year. This semester, DeYoung is teaching courses on digital animation and another on the moving image.

The center also hosts individual course sessions. The motion studio provided a venue last semester for final projects in “Laptop Ensembles,” a course taught by Konrad Kaczmarek, assistant professor in the Department of Music, in which students use computers and other technology to make music.

“We didn’t have an appropriate performance space,” Kaczmarek said. “My students were using projectors and multi-channel sound equipment, and the department doesn’t have a space to accommodate that. Johannes kindly offered the use of the motion studio. We had a lot of fun.”

Kaczmarek said he would like to integrate the motion studio into future iterations of the course.

“Johannes has been super generous with the time and space at the center, which makes sense because the center is going to be defined by how students and faculty use it,” he said. “They are really encouraging people to visit, use the hardware, and create cool stuff.”

The center’s programming underscores the breadth of disciplines and research interests that converge there. Weekly graduate-fellow workshops have focused on practical skills as diverse as macramé, using Photoshop, or scanning three-dimensional objects. Last semester, the center hosted workshops on the use of sound in video games, shadow puppets, and projection design, among other topics. The center’s Visiting Artist Program invites professional artists to discuss their work, craft, and ideas.

At a “Hackathon” event on Feb. 9-10, participants will work with virtual reality, augmented reality, motion-capture and other mixed-reality tools to build experiences related to global climate change.

Every Friday, students are welcome to attend an open workshop on motion-capture and virtual reality that is associated with “Blended Reality,” an applied research program that is based at the center.  

Chasing an idea

A partnership with Hewlett Packard, the Blended Reality initiative, which explores the capabilities of virtual reality and other emerging media, is supporting four research projects using virtual reality at the School of Medicine, the Center for Teaching & Learning, the Department of Music and School of Engineering, (a joint effort), and at the center.

DeYoung is Blended Reality’s principle investigator and the center supports all four projects.

Justin Berry, a critic at Yale School of Art and member of the center’s core faculty, leads the center’s project and has assembled an interdisciplinary team of students and faculty to test the limits of these new media and make them available to the student body.  

Understanding the capabilities of new media requires an inclusive approach, Berry said.

“There’s no best way to do this. We don’t have a road map,” Berry said. “Part of the idea here is that by bringing together people from different disciplines, we’re able to layer the strengths and visions of those disciplines to perhaps create a clearer picture of what will emerge out of these new technologies.”

Last academic year, Berry’s team produced several virtual-reality experiences and a collaborative work, called “The Sands,” that was recently installed at the B3 Biennial of the Moving Image — an international conference in Frankfurt, Germany.

This year, the team is focused on projects exploring how virtual reality affects people’s perceptions of light, sound, and body, Berry said.

The Friday workshops have attracted students not directly involved in the project and enticed them to consider the possibilities VR technology presents.

Monique Baltzer, a sophomore interested in majoring in art, is a regular at the weekly sessions.

“It’s super exciting,” Baltzer said of the opportunity to experiment with virtual reality. “It takes your imagination to new places. It’s a total different experience for your mind and body that I want to explore.”

Baltzer, who is taking both of DeYoung’s courses this semester as well as a course on 3D modeling taught by Berry, is considering a career in developing video games and she appreciates the chance to acquire the relevant skills.

“I’d like to have as many tools in my back pocket as possible,” she said.

Wesson praised the center’s staff and graduate fellows for their willingness to put aside their own projects and help others understand the technology.

“The staff is unbelievable,” he said. “They are experts at this field and are extremely good at bridging computing and the arts. At first, I would turn to them all the time.”

DeYoung and Berry are advising Wesson on his independent study project, which is based on “The Library of Babel,” Jorge Luis Borges’s short story set in a vast library of books containing every possible ordering of the letters of the alphabet, commas, periods, and spaces.

“I want to create a seemingly infinite library filled with texts that are randomly generated that people can engage with and get completely lost in a library of nonsense,” he said. “We’ll see if it pans out. I’m open to shifting gears and taking it in a weird direction. That’s what the center is about: having an idea, experimenting, and see where it takes you.”