How AI can reveal new understandings of the past — and the future

The ancient city of Dura-Europos in present-day Syria has long fascinated archaeologists and historians with its wealth of cultural and linguistic diversity. But much of the valuable information about the city, which was founded in 300 B.C.E. and abandoned in the third century C.E., has been lost.

With the use of artificial intelligence (AI), though, Holly Rushmeier, the John C. Malone Professor of Computer Science at the Yale School of Engineering & Applied Science, is “reconstructing” city buildings as 3D models based on surviving evidence as well as helping to build an easily accessible body of knowledge about the ancient city.

In a completely different application, her lab is also using AI to better assess the state of land that has been damaged by Algerian forest fires — work that could benefit governments around the world in land recovery efforts.

In an interview, Rushmeier discusses her work on these two far-ranging projects. It is the latest in a series of conversations about AI with Yale researchers.

How is AI helping with your Dura-Europos research?

Holly Rushmeier: We’ve been experimenting with training networks to extract key contours from historic photos to use as a starting point for geometric modeling. There’s also a major undertaking of gathering facts about all of the artifacts at Dura-Europos and putting them into “linked open data” [a virtual data cloud] to be part of a knowledge graph to form something that can be efficiently used for question-answering in the future through Wikidata.

I think it’s really important to lay a foundation for future, better methods [of information gathering] that rely on specific, reliable data rather than just roaming around the Internet learning who knows what. So those are the two places I would say that AI, and in particular machine learning, intersect with that project.

Because right now, the information about Dura-Europos that’s available is all very scattered?

Rushmeier: Well, it’s in things like excavation reports and traditional books. But then the collections are fragmented. We have a certain number of artifacts here about Dura-Europos [at the Yale University Art Gallery]. Other museums have their data. The idea of putting everything into this linked open data — and Wikidata is one instance of linked open data — is a way that they can all be connected from across different sources.

Moving forward a couple millennia, in a separate project you’re also working on present-day forest fires.

Rushmeier: In northern Algeria and many other places, information about forest fires is difficult to get. But you can get satellite information from the European and U.S. satellites that cover the whole world.

Nadia Zikiou, a Ph.D. student from Algeria, is working on this to assess damage to natural resources. It’s very difficult to get a lot of on-the-ground information, but you can get hyperspectral data from satellites over time to track how things have been damaged, for example, by wildfires. In a project she’s looking at the best way to use that satellite data — which wavelength bands and what methods work best. You can take values that are detected at each pixel in a satellite image and combine them in a certain way to compute, for example, an index for the type of vegetation.

So the work she is doing compares convolutional neural networks (CNNs) and support vector machines (SVMs) — two kinds of machine learning — in how they predict the kind of damage from wildfires that you can get. It’s looking at what the damage is and how things are recovering.

And that could give the Algerian government a better way to plan the future of that land?

Rushmeier: Yes — to understand how to manage their resources.

How does machine learning figure into all of this?

Rushmeier: Say you’re using it to identify crops in a remote sensing image: you need lots and lots of examples of the things that might appear in these images labeled — “this is a certain kind of grass” and “this is a certain kind of tree,” for instance. If you have loads and loads of those labeled images, you can train the machine learning model so that when you get a new image, you can assign the labels with the model. It’s a great tool if you have enough and appropriate training data. So what this system will do, among other things, is enable people to train machine-learning models to understand this data.