The Met and Google are bringing the museum online

The baby Jesus in Italian sculptor Luca della Robbia’s “” is surprisingly…sassy. Wrapped in his placid mother’s arms, he even seems to be serving a “girl, please” side-eye. His whole figure is full of personality and detail; his baby hands and ears, though porcelain, look chubby enough to bite. Up close — very up close — he even has baby teeth.

On October 4, Joe Coscia, Jr., the Metropolitan Museum of Art‘s quiet but devoted chief photographer, undertook the task of capturing the Madonna, and her child with their scroll, digitally. 

“How often do you get to photograph a della Robbia?” Coscia said while he worked. “Maybe once in a lifetime — twice if you’re lucky.”

This is what Coscia and the imaging team do day in and day out: carefully stage, light, photograph, edit, and render the digital files of the Met’s 1.5 million artifacts, and send them off to curators, publishers, and, frequently, the digital department, for publishing online.

Since the creation of the digital division in 2009, the Met — like most cultural institutions — has been proactively wrestling with the question of what it means to be a museum in the digital age. How should the reach of a museum extend beyond its walls? 

Some museums choose to guard and curate their collections online just as they would in their buildings. Others fling open their digital doors, and let go of control over their collections in the name of reaching more people, and enabling further study and creation.

“Now that many people can access representations of museums and objects online, it’s forced museums to really think about what aspect of artwork they think is really special,” , an assistant professor of Information Studies at UCLA, said. “Every museum has to decide what its priorities are.”

For the most part, the Met has staked its flag on the side of open access; in 2017, it released 375,000 images of its public domain art objects on its website under . That means any person can download, use, and change these images however they see fit.

On Thursday, it went further. The Met has now released a public API connecting to over 200,000 open access pieces in its collection. 

An API, or Application Programming Interface, is a tool that allow computers to read and analyze a changing set of information. With the Met’s API, researchers, students, social media platforms, or anyone who can run code that interfaces with a digital database, will have access to information about — as the Met’s head of digital, , is fond of saying— “5,000 years of human history.” 

“In many ways we’ve been working towards this for a while, building on the launch of Open Access,” the Met’s director, Max Hollein, told Mashable. “We hope people will be creative and hands-on with our collection, emboldened to engage with it in new ways, and—through the data that is now available for every object, painting, sculpture in the public domain—we hope there will be a deep exploration of and fresh appreciation for the historical context, beauty, and resources that exist within this unparalleled collection.”

“We hope people will be creative and hands-on with our collection, emboldened to engage with it in new ways.”

The museum is launching the API in partnership with Google, which is using the API to pull these objects into the app and web archive. 

“Every month and every week with technologies advancing, I’m more convinced that technology can make art make a bigger impact in people’s lives,” , program manager for Google Arts & Culture in North America, said.

Although the Met’s collection has had an online presence for the last six years, Tallon and his department hope that the API will help the Met’s archive reach a farther and more diverse audience, whether through exposure on Google, Wikipedia, or even through social media platforms. They envision that it will enable the creation of creative research projects about the collection. Somewhat symbiotically, it could even serve as a resource that programmers can use to train A.I. in the development of image recognition programs.

“The museum is really trying to figure out what it means to open its doors in the digital age to make sure it can reach audiences around the world, to make sure it’s putting as few barriers as possible between people around the world and the objects that can inspire them,” Tallon said. “That really is the global aim here.”

Many cultural institutions are establishing their digital presences, whether through all-access APIs or highly curated digital exhibitions, and everything in between. A museum of the Met’s stature devoting its resources to digitization could provide a path forward for other institutions as they walk the tightrope between access and curation. And, together, define what it means to be a museum, online.

Above the Great Hall

Joe Coscia works in a matte black studio in the imaging department, a space directly above the Great Hall that has housed the department since its founding in 1906. Walking through requires navigating around 10-foot high white halves of spheres —  the backside of the museum’s famed domed ceiling. Photographers used to shoot using the natural light from the skylights of the domes, and develop the film on the rooftops of the Met above Fifth Avenue. 

Today, this is where imaging, working hand in hand with digital, help digitize the collection.

The process begins with curators who often request photographs of the objects. Every piece of physical art comes with its own metadata — the artist, the date, or any other descriptors. These are initially written by curators and put into the museum’s content management system, created and managed by Tallon’s digital team.

Then specialized art movers in the Met’s riggers department bring the piece into the studio, if it’s able to be moved. A photographer is assigned, based on their expertise (Joe Coscia loves shooting ceramics and bronze, and does a lot of porcelain). 

Photographers then stage the piece, making sure that the object stands out without getting lost in the shadows. Each surface, whether paint or bronze or marble, has its unique staging and lighting needs. Photographers then capture all the details requested by the curators, as well as whatever they notice on their own.

“Everything’s a challenge, because every single shot is different,” Coscia said.

Coscia works in his dark studio using a Hasselblad camera, shooting in 100 million megapixels that converts raw files into a 600 MB .TIF file. Coscia says the department has always had top of the line cameras, lights, and software, because “this collection absolutely needs the best equipment. The better the equipment, we can make more beautiful pictures of this incredible collection.” Only when zoomed in hundreds of percents on screen do people notice the baby Jesus’ teeth in the della Robbia.

“When you blow it up huge, sometimes you can see fingerprints, you can see all sorts of great things that the artists might have left,” Coscia said. “The curators love it.”

Once a photographer has secured the perfect shot, they send it to advanced post-production to get it ready for distribution. 

Heather Johnson is an imaging production assistant. She originally applied to be a museum security guard, but now she has a much different role: editing photographs of objects to make sure, as she says, that objects look in print or on screen just as they do in real life. In service to that mission, shadows are Heather’s nemesis.

“I think the thing that people would be surprised by is how hard it is to make something look like you would see it in real life,” Johnson said. “The first thing I learned here was how to make a shadow look real. Mostly because we’re so used to seeing shadows, that even if you have no sort of technical skills, you can look at an object and be like, something’s off there.”

Heather cleans up the enormous photo files pixel by pixel, which can be both meditative, or a pain. She also makes edits that photographers can’t in real life. The della Robbia came on what Coscia called an “unfortunate” wood pedestal that can’t be removed physically. But Heather can remove it digitally, so the creamy porcelain of Madonna and Child shines against the dramatic gray background, sans ugly wood.

Once employees like Heather and Joe finish their work, the head of imaging, Barbara Bridgers, hands the baton over to Loic Tallon’s digital team. Under Tallon, the division has 60 employees working on the website and building new digital tools and content. The team running point for the Met’s API is the collections squad, helmed by lead developer Spencer Kaiser. 

“We’re responsible for the collection online, the full stack all the way from the databases that the curators use to catalogue the objects,” Kaiser said. “What you see on the website is what we’ve produced.”

With work on building the API coming to a close, Kaiser’s team is now deep in a multitude of projects, including making the website “sexier,” and building an art timeline, to show what was happening in history during the production of various artworks. His team names their sprints based on constellations; at the time we spoke, they were currently finishing up Tucana.

“This collection absolutely needs the best equipment. The better the equipment, we can make more beautiful pictures of this incredible collection.”

With the API and the ongoing digitization process, Kaiser’s team receives digital files from the imaging team, as well as the metadata from curators. A big challenge for his team (and for digitization as a whole) has been making the format of the metadata consistent, since it comprises pieces that have been catalogued continuously over a century and a half. 

“Having talked to a lot of museum people about their data projects, we’re all super aware of how hard it is to get presentable data from this, and how much effort it takes to make this happen,” UCLA’s Dr. Posner said. 

Organizing databases and programming the API, Kaiser acknowledges that a lot of the technical work is not so different from what any developer making a content management system and API does. The difference is that his team does it at The Met — incidentally, in the same fifth floor space of the museum’s old slide library, where sepia-toned slides of greek statues or European oil paintings are still scattered around the office.

“This type of work, building APIs, can be a similar experience no matter where you are,” Kaiser said. “The real difference is that we get to work with such incredible artworks. The responsibility of getting that out into the world is what really makes a difference for us.”

The old slide library

Loic Tallon works from a standing desk in his office and when he speaks about his work, his words stream out while he simultaneously retrieves supporting documents, or looks up another burgeoning thought online. He recites the full of The Met at the drop of a hat, so quickly that it seems a talisman rather than a mere collection of words, because he says he is always thinking about the statement and how to best serve it. 

Tallon also works closely with institutions outside of the Met to make the collection easy to access wherever people already are online. The department has a “Wikipedian-in-residence,” who helps integrate the collection into Wikipedia articles. It also works closely with Google’s Arts & Culture platform, which serves as a digital portal to museum collections all over the world. And connecting with these platforms, in the case of the API, is really the next step in the process of digitization.

“It might not be sexy, but from a technical point of view, it’s a big step forward.”

“The Met is more than just a physical space—we share our content with the millions of people who follow us on social media and use our website, and digital platforms give us the ability to reach out even beyond these audiences,” museum director Max Hollein said. “This circles right back to the heart of the Museum’s mission—to connect people with art.”

Google’s own goal of organizing the world’s information works curiously well in tandem with the Met’s mission. That synchronicity is part of what’s made the Met partnership and the API a priority for Google Arts & Culture. “It ties back to the general mission statement of Google,” Delacroix said. “And that’s exactly what we’re doing, and doing it at a new scale, with the help of an API.”

Prior to the API launch, Google engineers manually uploaded the Met’s work onto its platform. But Delacroix says that that process is slow and painstaking. The API will enable the Google platform to ingest a huge amount of dynamic data at once. And while there were previously 2,000 Met works on Google Arts & Culture, the API swells that number to over 200,000.

“An API allows you to do that at scale in a painless way, because you have these two interfaces communicating, and doing the job for themselves,” Delacroix said. “It might not be sexy, but from a technical point of view, it’s a big step forward.”

“With the API coming out, we’re really assertively going down that route of trying to connect everyone in the world to the Met’s collection,” Tallon said. “Reducing the distance between people and the object that’s relevant to them — that’s the global goal.”

Museums log on

Disseminating art across the globe is not a mission the Met is undertaking alone — far from it. Many institutions, particularly internationally, have gone even further in efforts to digitize much larger collections. 

“It has been happening for some time, digitization,” , a digital research services librarian at the University of Nevada, Las Vegas, who studies how institutions can improve access for computational research, said. 

An API like the Met’s comes about when the museum decides that it wants to easily enable access and remix culture, or, the ability to create something new from something old. The Met is not the first to achieve this; The American and the European Union’s digital platform for cultural heritage, , are some of the institutions with an API. The Netherlands’ Rijksmuseum has been a notable leader in mass digitization, and has offered . 

“It’s a more recent development to enable access through APIs or bulk downloading,” Padilla said. “That’s a new chapter that extends and expands the various types of things that people can do with the product of all that digitization effort.”

An API is not the panacea of digitization for all institutions.

But an API is not the panacea of digitization for all institutions. The way that museums have gone about this challenge varies widely. Their approaches depend on the scope of the collection, the financial and personnel resources available, and the institution’s curatorial stance.

“I would never make a blanket statement like everybody has to digitize,” UCLA’s Dr. Posner said. “For some museums, it’s hard enough to keep the lights on.”

The in Los Angeles has a specialized collection of European, American, and contemporary art. Its project manager for digital initiatives is Philip Leers, who says that the museum approaches digitization in a similar way that it would approach creating a dynamic, context-filled exhibition, rather than digitizing its whole collection en-masse.

“We create digital resources that highlight parts of our collection that we think are important, or hidden, or that we have something interesting to say about,” Leers said. “It presents us with the opportunity to present the works with fairly rigorous context. We think of this as educational resources, and want to provide more than just images.”

The Hammer creates online versions of its popular and notable exhibitions as well. Leers explained that its 2011-2012 exhibition spotlighting 20th century African-American artists in Los Angeles, , was so popular that it preserved it and has expanded its digital archive in the years since. Popularity wasn’t the only factor that went into the online hub’s creation; the Hammer felt that it was important to elevate the visibility of art and a community that had been brushed over for too long.

The Smithsonian has a different challenge and approach altogether. Compared with the Met’s 1.5 million object collection, the Smithsonian has 155 million objects. Digitization has provided the Smithsonian with the ability to actually make that gargantuan archive accessible, so it has made it a priority. 

“I think it’s a great opportunity, an amazing opportunity,” Diane Zorich, director of Smithsonian’s digitization program office, said. “We have 155 million objects. Less than 1 percent can be exhibited at any time. This gives us an opportunity to make our collections so much more available to people in so many different ways.”

Still, Zorich said that the opportunity is also an obstacle in its own right. “We have a scale challenge that other museums don’t have,” she said. That’s where mass digitization comes in.

For art, design, and scientific sample collections, the Smithsonian has set up systems that involve a huge amount of preparation, but allow for the museum to capture images and create a digital archive much more quickly. It was able to digitize the Smithsonian’s design collection, housed at New York’s , by categorizing objects by size and shape (or, “envelopes”), and using the same production staging for all the items in a given envelope. It even digitized its using an actual conveyer belt.

The Smithsonian and Hammer museums’ approaches to digitization sit on opposite ends of a spectrum, both suited to each institution’s collection and perspective. The Hammer puts out a smaller amount of digital material, but presents it with the same curatorial context that it would in a physical exhibition. The Smithsonian’s mass digitization gives unprecedented access to its collection, but provides the information more as data, and less as “content.”

“Museums are used to doing things slowly and carefully, while the internet is fast-paced and messy, or it can be,” the Hammer’s Leers said. “Some museums are very free and open with their digital presence, and some are more painstaking. I think we’ve kind of erred towards that end of the spectrum.”

“The context versus access debate is a long standing debate that will probably go by the wayside as we move forward,” the Smithsonian’s Zorich said. “It will have to if museums want to stay relevant to their audiences.”

It’s difficult to achieve the best of both worlds, but that’s what the Met is trying to do. Employees like Joe Coscia, Heather Johnson, and Spencer Kaiser ensure that each photograph is beautiful, accurate, and customized; that the metadata is clean, and consistent with the wishes of a curator. 

“The quality of the images should match and equal the quality of the art.”

Of course, they are only able to take this kind of slow care in their work because they have the financial resources and institutional support to do so. The Open Access Initiative is specifically funded by Bloomberg Philanthropies.

“Institutions need money in order to do this stuff, and in order to sustain it, and especially in order to staff the initiatives,” Padilla said. “There is a fair amount of disparity in terms of having the financial resources to staff up and sustain effort in this space.”

Even though the Met went through in 2017, which resulted in a new CEO/President, and a new director, digital remained. While Tallon’s digital team employs more than 60 people, digital initiatives have continued to become even more deeply ingrained within multiple departments.

In balancing context and access, some of the Met’s objects online come with more curatorial and educational resources than others. The Met is not shooting for “mass digitization,” but it is going for a holistic digital presence that is reflective of the institution itself. 

“More than ever, we’re responsible for making sure that we’re sending the contents of this institution outward to the world,” the Met’s head of imaging Barbara Bridgers said. “We’ve just always felt that it was imperative that given the breadth and the depth of the collections in the Met, that the quality and the bar, the standard bar, that we use to capture works of art, should be at the very highest. The quality of the images should match and equal the quality of the art.” 

Buried treasure

This August, all 16 of its famous Van Gogh paintings — irises, wheat fields, self-portraits and all — in one gallery. These paintings are usually on loan at exhibitions around the world. 

But even before they reached the Met, they criss-crossed the world throughout the centuries. Thanks to digitization, that journey has been uncovered and visualized for anyone to learn from while they take in the paintings in the gallery.

“There are a lot of stories to be found in this data that you might not see in any exhibition,” Parsons professor Richard The told Mashable.

An API is all well and good, but what can it actually do? The Met’s undertaking is yielding, especially with the API, new ways to both access and understand the cultural legacy it contains. 

“There are a lot of stories to be found in this data that you might not see in any exhibition.”

Richard The leads a graduate course in data visualization at Parsons. Last year, his students used the Met’s collection data as the bases for their projects. One student made a dynamic, interactive map of where the Van Gogh paintings traveled before they arrived in New York; another plotted how many objects of different metals like gold or silver reside in the Met.

Dr. Posner’s UCLA students work with data from cultural institutions to find stories about the art and objects, yes, but also to reflect on how we as a culture have chosen what to canonize and memorialize about ourselves.

“There’s this whole other side of art that’s really all about information, and can be investigated by looking at trends, artists, nationalities, or genders,” Posner said. “So when you look at those aspects of a collection that can be expressed as data, you can see trends that turn out to be important, but which can’t necessarily be deduced just by looking at individual artworks.”

“You can’t really anticipate all of the types of uses that someone might want to make of a collection,” Padilla said. “Rather than trying to anticipate all of that, you can create an API, and that gives users the ability to remix collections, or even create new forms of access.”

That is exactly how the Met hopes people will come to the collection, now that it’s more accessible and machine-readable than before: with a fresh perspective that they might not have thought of themselves. 

Tallon also hopes the API and digitization as a whole enables art to become more seamlessly integrated with everyday life.

“There’s no rule for how you have to engage in this content,” Tallon said. “The dream scenario is every time someone goes online, they see an object from the Met’s collection, and they don’t even realize it’s an object from the Met’s collection. It’s the inspiration point somewhere.”

Tallon envisions .gif keyboards populated with della Robias or Rodins, Pinterest boards filled with patterns from ancient ceramics and fabrics. 

“I’m not kidding when I say everyone’s life would be that much better if you woke up and saw a beautiful image of something from around the world, and be able to serve up the image that best serves someone’s mood or personality at the time,” Tallon said. “I think we can do that. And then if people want to dive deeper, and get some more interpretive content, then great. If they just want to be inspired and just think differently for a fraction of their day, god bless too.”

A museum without walls

Barbara Bridgers’ office is warm in comparison to the concrete technical space of the rest of the imaging department. It has a wall covered in old motherboards, previously left on her desk by employees who figured she would know what to do with them (she did not); curiously, it comprises a sloping concrete ledge, which is really the remnants of a one-time museum wall. 

“This was a former external wall of the building,” Bridgers explained, gesturing. “When they were finishing up my office, they called me and said, well, you have a ledge.”

As technology has advanced, museums including the Met have had to decide what the scope of a museum should be. 

The choice is not absolute, but institutions like the Met, Hammer, and Smithsonian make judgments about and allocate precious resources toward whether to tear down an institution’s walls, or create a more dynamic space within. 

“Putting art on the walls is always going to be what museums do,” the Hammer’s Leers said. “But for a first time in a while, [digital] is putting a broad array of possibilities in front of us, and asking us to choose. Which can be uncomfortable, and scary, and exciting.”

“We’re more than a building now.” 

As with the original wall in Bridgers’ office, the Met has chosen to keep the foundation of the institution in tact, while expanding around the center, in order to ultimately transcend any physical space. The light-filled Greek and Roman sculpture gallery will always house figures from the West’s cultural roots. But the purposes of the former slide library and imaging department above will change and grow, and the people who dive deepest into the collection won’t necessarily work within the Met’s walls.

“I’m sure if you asked the people who founded the museum in the 1870s, their aim was to make the collection accessible by putting it on public display, quite literally,” Tallon said. “The technologies and opportunities, what it means to make something accessible, has changed so much. Even just the idea of what a museum is, what the Met is. We’re more than a building now.”