Predictions about artificial intelligence taking our jobs or, if claims about its potential to annihilate humanity prove correct, our lives, generate headlines. That A.I. might also help an art museum visitor better understand a painting is less likely to generate buzz but also more likely to be true in the short term. Some museums, intrigued by this latter possibility, are already using the technology, and not just administratively. The Crystal Bridges Museum of American Art in Bentonville, Arkansas, for example, launched a web-based “A.I. Art Companion” named Ask Rosie when it reopened in June following a 15-month expansion project.
“You take a photograph of the artwork and bring it into the app, and it will give you some didactics and learning right off the bat, such as who the artist is,” Shayne Hart, chief digital officer at Crystal Bridges, told Observer. “And then it may say, ‘what would you like to know about the artist?’ or, ‘what would you like to know about this painting?’ And then as the guest or the user is answering these questions, that person is essentially creating prompts.” Those prompts, which guide the A.I. in generating specific responses, produce a flow of information tailored to that visitor. (You can try it here.)
Ask Rosie, Hart explained, is “designed to meet a guest wherever they are in their art journey or experience,” reflecting back to the user “the tone that the user is engaging it in. So if I were having a very academic conversation with Ask Rosie, the app would tend to lean into more of an academic tone, more academic interest, data, information.” The tone can also shift midway through the exchange. A visitor might begin by using the art historical term “chiaroscuro” but later ask, “what in the hell is impasto?” At that point, according to Hart, the A.I. companion will shift accordingly “and reflect the tonality that you’re reflecting in your conversation with it. So, at any given moment, it’s following you cognitively and emotionally and tonally because it’s having a conversation with you. And that conversation is informed on Ask Rosie’s side by the tonality, language, all of the things that you’re saying to it.”
Stephen Reily, founding director of Remuseum, a nonprofit originally created within Crystal Bridges to promote innovation in American art museums, told Observer that “there was very strong resistance to the use of A.I. within the museum field for some understandable reasons,” but that “museum workers and the public are already using A.I. extensively.” Curators and museum policymakers, he argued, should get on board and take advantage of the innovations the technology affords.
Remuseum recently released a report, “Five Ways that A.I. Can Humanize Museums,” which noted that A.I. can help “each visitor build stronger, deeper and more personal connections to the art and the museum itself.” According to the report, the most immediate way museums are using artificial intelligence is to connect information on visitors, donors and members that is otherwise kept in “data silos” across different departments: “A development officer trying to understand a single supporter might have to move from a fundraising database to a collections database to a ticketing system to an email platform, assembling by hand a picture that ought to be available at a glance.”
Increasingly, though, institutions are also using the technology to improve the visitor experience. At the Toledo Museum of Art, A.I. collects information on visitors, including “where people are moving in the galleries, where they are lingering, what programs people are attending, are they taking advantage of the food service or retail, are they members or donors, are they visiting with another adult or with their kids,” Adam Levine, the museum’s director, told Observer. That might strike many as a bit unsettling, but he said the information “gives us some idea of where people linger, and we’ll then be able to use that data to create more personalized experiences.” For example, “if we know you like Impressionist paintings a lot, because you spend more time in that gallery than any other, we might invite you and the 25 other people who spend the most time in those galleries in a quarter to a special lecture round table with a curator.”
Matching visitors’ tastes with things they might also like may call to mind shopping at, say, L.L.Bean, where emails start arriving as soon as an order is placed, suggesting that people who bought one item might also be interested in others. But Levine clarified that “the difference is that we’re not trying to sell widgets. It’s about introducing people to programs, exhibitions and other art in the collection.” Ultimately, he added, “with our policies, safeguards and training, we do not believe there is any marginal risk in leveraging A.I. to deepen visitor engagement with art.”
Still, this kind of tracking-and-matching may give some museum visitors pause. Grace Yao is the founder of the San Francisco-area company Artlas, which describes itself as “a platform for cultural experiences” and is used by a number of museums in the U.S. and abroad. She said that “at the very first encounter of the app, we ask the visitors, are they kids or are they adults and what is their primary language and where they come from. This will give a big picture of the user. And then later, we create the first personalization. When they start using the app and look at different artworks, we know what they are interested in.”
The Artlas app is separate from museums’ internally created apps and carries a separate charge. As with Crystal Bridges’ Rosie A.I. app, Artlas users can photograph an artwork with their phones, and the app’s recognition feature can identify the work. Users then type or speak their questions about the art to an image of a stuffed cat called Metti, whose verbal responses come either in a male or female voice. All the information Artlas collects is “encrypted and in their profile and we are not using it for anything else and we are not sharing it with anyone else,” Yao said, then added, “not at this point.”
Perhaps because we, as Museum Voorlinden director Suzanne Swarts put it in a recent and related op-ed, “live in an economy designed to interrupt us,” art museums are increasingly looking for new ways to engage both visitors and potential visitors. “Traditionally, the experience that a museum offered for the visitor was one size fits all,” Reily said. “There might be a brochure with information. There is almost always wall text, but it’s provided in art historical terms that suit the regular museum visitor with higher education levels and more familiarity with art. I think the museums that are exploring these tools for personalization are acknowledging that every single visitor comes to the museum with a different background and a different set of interests.”
The Cleveland Museum of Art is another institution using technology to help visitors engage with artworks. “How can we help someone look more closely, ask better questions, find themselves in the art, have a more personal connection? And then we use technology to solve that problem,” Jane Alexander, the museum’s chief digital officer, told Observer, pointing out that the average museum visitor spends only two to nine seconds looking at an artwork. With the museum’s A.I. platform, ArtLens, “we’re having young people spend 10 minutes asking questions about these paintings.”
As one might expect, some visitors using ArtLens try to stump the technology. “People try to ask questions like ‘who won the World Series?’ and things like that,” Alexander said, but the A.I. isn’t easily thrown, “and it has really fun ways to bring you back to talking about the painting.”
The Artlas app, by contrast, isn’t limited to chatting about the works at hand or even to the works in a single museum; it can refer to works in other institutions and even answer questions unrelated to the art. “When the people are in the museum, they can ask, ‘where is the bathroom?’ or ‘what is the cafeteria serving for lunch?’ or ‘where can I do skateboarding?’ or ‘what is the best ice cream in the city?’ The app will go to the fundamental A.I. model, and it will answer you,” Yao explained. The biggest difference between Artlas and Ask Rosie is that Artlas can draw on data beyond what a museum supplies.
Not every museum is rushing to adopt A.I. technologies, however. Museum visitors who want more of a human touch can visit institutions like the Museum of the Moving Image in Astoria, Queens. Aziz Isham, the museum’s director, told Observer that while the museum does use A.I. for data management (collecting information about visitors and donors and storing it so administrators can access it quickly), it also employs more than 100 interns each year in its exhibition spaces to answer visitors’ questions. If a visitor looks befuddled, an intern will approach to offer an explanation. “The communal act of coming together is important,” Isham said. “We’re not trying to take the human element out of it.”