Computing and the arts major melds science and creativity in visual arts, drama, music, and architecture

Computing and the Arts is a young major by Yale standards. Developed in 2008, it moved with the Department of Computer Science to the School of Engineering and Applied Science last spring. While other universities have digital arts majors, they often fall under the visual arts umbrella and require a small handful of computing classes. Yale is the only university to create an interdisciplinary program based in computer science in which students take an equally substantive track in one of Yale’s arts disciplines. During their senior year, computing and the arts majors work on a year-long project that synthesizes both disciplines.

Current tracks include art, history of art, music, theater studies, and, this term, architecture. YaleNews spoke with Julie Dorsey, professor and director of undergraduate studies in the Department of Computer Science, about the integration of computing and the arts, what careers are available to students, and why Yale is perfectly positioned to lead the field.

How did Yale’s computing and the arts major come to be?

It started with Paul Hudak, a professor of computer science, who passed away last year. He had a long-standing interest in music, both as an accomplished musician and as an expert in programming languages. When Paul was chair of the Department of Computer Science he recruited me and Holly Rushmeier in computer graphics.  The three of us, with our combined interest in computer science problems motivated by the arts, decided to build the program.  We then began to talk to our colleagues across the arts disciplines, who amazingly were interested in and enthusiastic about the program. Currently we work in the disciplines of art, history of art, theater studies, music, and architecture. 

What is your personal arts interest?

My interest is in architecture. As an undergraduate studying architecture, I stumbled into the Program of Computer Graphics at Cornell. I had been very interested in science and math in high school; suddenly I saw a way to bring my interests together. I went on to study at a graduate school for computer science, with a focus on visual computing. Today computing plays a major role in the discipline of architecture — from simulating light and air flow to modeling of architectural forms to the fabrication of realized structures.

What kind of problems do you tackle?

I’ve studied a wide range of problems in computer graphics, including sketch-based interfaces for early conceptual design, acceleration methods for real-time rendering of urban-scale scenes, and the creation of detailed photorealistic renderings. I’ve also worked on algorithms for lighting and acoustical design and visualization. I’ve been especially interested in modeling the visual appearance of materials. For example, my collaborators and I have developed techniques to represent the visual richness of irregular metal patinas and eroded stone. In all of these research thrusts, the challenges inherent in the design and visualization of the built environment — from the sheer scale of these scenes to their conception and representation — have inspired and informed my work, and opened up new areas of inquiry in the field of computer graphics more generally.

What are career options for computing and the arts majors?

These days many industries, where computing is important — film, television, architecture, games, and video — have a substantial arts core rather than a solely engineering or science core.  An arts focus is also needed in any industry that develops consumer products whose success depends on both functional and aesthetically pleasing design.

Students are interning and working for companies like Disney, Facebook, Microsoft, Apple, and startups of various types. Some become software engineers. Others are part of digital production teams at places like Pixar, working alongside technical engineers, designers, and art directors. Our students are well equipped to work on a team because they have this wide-ranging skill set and broad set of interests, and the ability to span disciplines that have different modes of working and thinking.

We also have students who go on to work in museums on cultural heritage projects. Artists interested in the next generation of tools learn how to program a computer, and ask and answer questions about what's possible. Donya Quick ’14 Ph.D., who now teaches Paul’s classes, developed a computer program that creates original music that sounds like Bach. There is also video game design. When we met with theater studies faculty, Joe Roach, the Sterling Professor of Theater and English and chair of the Theater Studies Advisory Committee, aptly noted that computer games are a new setting for theater, and designers will want the same well-developed characters and drama that you have in the physical theater.

What makes this major different from ones at other universities?

At the institutional level, the major is unique within the academic community. Many institutions have programs in digital media or new arts-inspired modes of human-computer interaction. But there is no comparable program with an integrated view of computing across art, art history, architecture, music, and drama. And there is no program that investigates computing broadly as a tool in analysis, design, and preservation, as well as artistic expression, in the arts. We require that students get an in-depth education in both computer science and the arts discipline that they choose.

At the disciplinary level, this initiative is unique within computer science. The integration of computer science education in the arts, with a view to expanding students’ communication and problem-solving skills, is new and will likely affect education in the field. The development of a major for students with talent in both computing and the arts is drawing top students who, in the past, may either have not considered computer science as a topic of study or Yale as a place to be. The major builds on four existing strengths: Yale’s leadership across all the multiple disciplines in the visual and performing arts, the Yale collections, the structure of the undergraduate curriculum in Yale college, and the global view of Yale.

Why Yale?

The major is ideally suited to Yale, where expertise and tradition in both computer science and the arts are strong. Our world-renowned colleagues in the arts are concerned with understanding and creating art in a deep sense, intellectually and aesthetically, which provides the major with an abundance of substantive intellectual challenges. Yale also as a long history of encouraging and embracing interdisciplinary studies. Indeed, collaborations between computer scientists, artists, art historians, architects, and musicians are already underway. I think the major is serving to solidify, embolden, and extend these collaborations in significant ways.

What are you working on right now?

I have a longstanding research project called Mental Canvas, which I’ve spun out as a start-up company. The research was funded by the National Science Foundation, first through traditional research grants and now under the auspices of the Small Business Innovation Research program.  This work explores the middle ground between sketching on paper and 3D modeling by computer. Another project involves material appearance in the built environment, which is important in computer graphics applications, such as historical reconstruction and architectural design. We are studying the measurement, modeling, and representation of material appearance at both small and large scales with an eye to developing a new perceptually based material appearance model. This will enable, among other things, the means to reduce the amount of data required for accurate simulations. I am also working on a new 3D modeling system and scene fabrication approach inspired by traditional papercrafting.