Mad Systems leverages AI to enhance visitor experiences with new Alice™ and Lory™ technologies
by Wendy Grant
Now that patent applications have been submitted, Maris Ensing, founder of the pioneering technology company Mad Systems, is ready to tell the world about his team’s work to harness artificial intelligence (AI) and deliver dynamic visitor experiences through its newest innovations, Alice™ and Lory™ (both patent pending). He forecasts that initial implementations of both technologies will be in venues within the next 18 months.
Alice and Lory will integrate smoothly with QuickSilver®, the unique wired/wireless Mad Systems AV platform based on non-proprietary hardware, but Mad will also generate non- QuickSilver® specific versions to be used with other systems.
Introducing Alice, the dynamic media generator
“Alice is a tool that allows us to hyper-personalize information and experiences,” Ensing says. “With Alice, you can create a curated Body Of Knowledge. Then that Body Of Knowledge is used as the basis from which personalized information is derived that meets visitors’ stated interests and needs.”
Ensing shared a hypothetical example of implementing Alice in an exhibition about Leonardo da Vinci. First, visitors register and share basic information such as their age, preferred language, and areas of interest; this information helps Alice determine how to personalize for each person. For a 42-year- old, French-speaking art aficionado, Alice will display information in French about da Vinci’s unique use of sfumato (a painting technique). For a 78-year-old, vision-impaired former engineer interested in the design and structure of da Vinci’s war machines, and whose first language is Spanish, Alice provides audio content in Spanish, along with high- contrast imagery and video exploring the artist’s creations. For a fourth-grade class on a field trip, Alice shares an introduction using images and audio that conform to the school system’s science standards for that grade.
Using facial recognition technology such as Mad Systems’ CheshireCat™, a radio-frequency identification (RFID) card, or even visitor’s smart devices, an ever-wider range of personalization is possible. Ensing says, “We’ve tested this out with the latest AI technology, and find that now we can tailor content for any age, for any area of interest and in any language, based on a curated Body Of Knowledge, even allowing for different presentation styles. We support hundreds of languages at this point. Incorporating AI technology provides us with an unprecedented level of flexibility.”
Alice will know if a person is visiting for their second or fourth or tenth time, and have the ability to provide new content with every visit. According to Greg Burks, Senior Software Development Engineer for Mad Systems, “Alice [and Lory, detailed below] make it possible to deliver variable content. Two people standing side by side could even have information presented simultaneously to meet the personalized preferences of each.”
Meet Lory, the personalized media delivery system
Ensing related that the creation of Lory, the next big Mad Systems innovation, was inspired by changes in the hearing aid industry. Due to a 2022 FDA ruling, hearing aids are becoming more affordable and accessible. In addition, their manufacturers are rapidly adopting Bluetooth technology in place of an inductive loop interface. This results in a new opportunity to expand and simplify the sharing of content for visitors using hearing aids. “We can stream information over WiFi to people’s smart devices, and from there it can be transferred to their hearing aids using Bluetooth,” Ensing says.
Multiple languages can be streamed at the same time. Moreover, the user interface is not restricted to hearing aids. It can go to other devices, such as a visitor’s own AirPods, or headphones supplied by the venue.
But what about sharing content with people who have no hearing at all, who use sign language? In seeking to answer this question, Ensing learned that there are multiple sign languages for different languages. His team is now testing Lory to support not just multiple languages, but multiple sign languages by streaming video-based information too. They’re exploring the use of AI-generated avatars to generate sign language video clips. And for vision-impaired visitors, the Mad Systems team is also working to enable Lory to communicate tactilely, ultimately outputting information to Braille devices.
Both Lory and Alice can also provide practical guidance in addition to exhibition content. Given that they know where the user is within a venue, the systems can provide location-based information, sharing nearby points of interest or services. If the visitor is near the theater, for instance, Lory can tell them about the next show. If the content is presented on a screen as well as through audio, Lory can share a trailer for a show. Visuals can be delivered to venue screens or to visitors’ smartphones.
Your favorite character can be your guide
Alice and Lory can generate avatars that allow each guest in a theme park, museum or other venue to choose the character that will be their personal guide.
Ensing explains, “We’re currently working with a venue that would like us to have five virtual people who meet you as you walk in. Each one provides a different viewpoint, and you can choose your preferred character to take you through the venue – or, if you do not want an avatar to show you around, you can ignore them entirely.”
They had considered filming five actors going all the way through the museum, stopping at each exhibit. But Alice allows for a new approach. Using the same technology that creates deepfake images, five images of people are all that is needed to set up the characters and their avatars. These avatars will lead visitors through the venue, with Alice delivering the content geared to age, interests and language.
“But this is where it really gets interesting,” says Ensing. “Imagine a teacher with 15 school kids arriving at a visitor center. The teacher registers, indicating that ‘These kids are seven years old, they are here to learn history and I want to lead them through the museum myself.’ We can take the teacher’s picture and turn the teacher into an avatar.” Now the teacher- based, AI-controlled avatar can lead them through the entire venue, speaking to the students at each exhibit, providing content appropriate for their age and learning needs – and even use different languages as necessary.
“Those are the kind of possibilities that we’re now looking at and working toward,” Ensing says. An avatar of the captain of a ship could guide a family on a cruise ship. A chef avatar can help you select your meal at a restaurant and wine to pair it with.
Ensing admits to having continual “wow moments” as he ponders the possibilities of what Alice can do. “Ultimately, think about smart cities,” he says. “You could have a personal concierge avatar who helps you get to the right bus or make appointments or a host of other things. Thinking about applications in the care industry, Alice can generate an avatar companion for an 85-year-old person who has memory loss. This is how this technology can lead us into the future.”
Curation and versatility
Concern about the quality of the content that Alice might provide is something that might come up for anyone who’s toyed with ChatGPT or other AI, and received unusable responses drawn from dubious sources. Limiting the AI’s working knowledge to a curated “Body Of Knowledge” from which the AI works is the key. This means that Alice’s responses are drawn from a written narrative, researched and written by curators, and delivered in the style of the original written document, ensuring quality control of the information delivered.
Alice can quickly search the documents in a collection or a curator’s research or the historical video footage an institution owns. Curators, subject matter experts or AI can metatag images, such as the photos in a historical society’s collection, video footage of a museum’s resident artists describing their creative process or a zoo’s keepers talking about the animals they care for. Then, when a visitor indicates they want to see what a historic site looked like when it was originally built, or how a sculptor created a particular artwork, or what it’s like to take care of tigers, Alice will be able to create a personalized narrative and a video that addresses the visitor’s interest, with information derived only from the curated information that has been supplied.
Alice can vary the length of the content to suit the visitor’s requirements. Someone who’s in a hurry might have 30 seconds, whereas a superfan might want to spend two-and-a- half minutes. And, again, it can be in virtually any language. In fact, multiple languages can be displayed at once. Says Ensing, “I walked into our lab the other day and we had Chinese, German, Arabic and English on the screen at the same time, displayed in such a way as to cause minimal distraction.” The different languages can be displayed on different areas of a screen, or on different screens, all at the same time, to support a multinational audience as necessary.
Alice can generate a variety of experiences beyond text and imagery. Sound effects, music, motion and even smells could help to transport visitors to the hanging gardens of Babylon or the edge of a volcano. In addition to projection and screens, audio-only is another possibility. “Alice uses text-to-speech very well,” Ensing says. “It’s feeling so close to having a real human talking to you that I think we’re close to being able to deploy that.”
The wide range of dynamic communication provided by these new patent-pending options takes Mad Systems’ advanced technology another step towards providing the experiences future audiences will require. Alice’s customized, curated experiences can be displayed on screens or projected surfaces, or be used as part of interactive experiences, and only use non-proprietary hardware. Alice can work with anything from standard screens to VR headsets to projection mapping onto 3D models. Staying in touch with client needs is, of course, just as much a priority as the march of technology. Ensing notes, “We want to use existing delivery methods and we want to be as agile as possible. We do not want to build equipment or design hardware. We want to create systems that deliver ultimate flexibility and longevity, that are as maintenance-free as possible, for our clients, based on non-proprietary hardware. We know that’s what they need.”
Teamwork makes the dream work
Collaborating with Mad Systems on various projects that will incorporate Alice and Lory is VoodooXR, a new company founded by Clayton Ferguson to develop mixed reality content for historical experiences and museums.
“We’re at the start of a new age right now,” says Ferguson. He previously worked with Mad Systems as exhibition and attraction producer on the Crayola IDEAworks traveling exhibition. This high-profile, successful traveling exhibition had its initial run in 2021 at the Franklin Institute and is still on the road. It utilized the Mad Systems QuickSilver AV system, which Ferguson praised for its capacity, form factor and versatility.
“QuickSilver truly is wireless, and the control part fits in a small, rolling server rack,” says Ferguson. “It connects all the pieces of an experience together and makes the components controllable. Every piece works individually, but also holistically. And with the AI technology that Mad Systems is patenting, it allows new levels of interaction and story-deepening ability, in creating experiences that can become alive, all around us. We’re working together to tell personalized, unique stories.”
Alice’s first incarnation may be semi-automatic, but it will become more and more automated over time, especially as capabilities change in the field of AI. Given that Alice and Lory are flexible technologies implemented using non-proprietary hardware, by implementing them now, venues will benefit by being on the leading edge and enjoy the technologies’ future growth and changes.
“Let’s face it,” Ensing says. “We’re not the only ones playing with AI. Our audience is, too. They expect personalized experiences. We need people to get fired up and tell their friends and family, ‘You have to come and see this!’ That’s how we get repeat visitors. That’s how we get an expanding audience.”
Ensing says the entire Mad Systems team is contributing to the development of the AI innovations.
Greg Burks, who previously held various roles for the Walt Disney Company, is primarily working on Alice. He says he’s excited to be working on a new system from the ground up. “We are coming up with ideas and giving them a try. And in the process, we’ll realize that there are four other things we could try,” he shares. “We get to play with AI every day here. This is stuff that has not been done before with software. It’s a lot of fun. And it will blow audiences’ minds.”
“I think that what we are creating is not evolutionary – it’s revolutionary,” says Ensing. “It needs to be fun. It needs to be entertaining or educational – or ideally, both. But it needs to be able to deliver personalized experiences – because that’s what people expect. If they don’t expect it today, I can guarantee you that they will expect it tomorrow.”
Mad Systems will attend the International Association of Amusement Parks and Attractions (IAAPA) Expo in Orlando and are available to take meetings there. To get in touch, or to arrange to see Alice and Lory and other Mad Systems technologies in the lab at Mad Systems headquarters in Orange, CA: [email protected]; www.madsystems.com. •
