Going “HAAM”

Do you know where your liver is? How about your spleen? Your kidneys?

Waving a hand in front of your mid-section and saying, “Somewhere around here,” isn’t enough, says biomedical artist and communicator Jon Bowen ’16 MS BVIS.

“When you go to a health practitioner, you need context to have a conversation,” he adds.

That’s why Bowen created the Human Augmented Anatomy Mirror, or HAAM, as the final project for his recently completed master’s degree.

HAAM is an interactive display application that shows viewers on their own bodies where the major organs are located, what they’re called and what they do. It’s designed to be easy to use, simple to understand, informative and entertaining. (Watch a video demonstration at amalgam.media/HAAM.html.)

Step up to the screen and images of your organs immediately appear, superimposed on your body. Turn or step to the side, and the organs move too. Unlike a drawing or model, which illustrate anatomy from an onlooker’s perspective, HAAM shows where the organs are in the user’s own body.

When you “touch” an organ, its name and brief information about its function appears on screen.

In game mode, the organs are arrayed to the side of the screen. The user can “grab” an organ and drag it to its proper place. Select the X-ray mode, and you’ll see the organs hidden behind the larger ones.

Up to six people can use HAAM at a time, which encourages conversation and makes it more fun, Bowen says.

“Jon did far more than just create a prototype. He rather successfully demonstrated that it could be a fully functional product,” says Kevin Brennan, clinical assistant professor of biomedical visualization and the committee chair for the project.

Bowen demonstrated a beta version at the 2016 Chicago Science Festival on May 14, sponsored by the Illinois Science Council at the Merchandise Mart. He’ll continue to refine HAAM based on user suggestions and responses at the event.

“There was a whole range of reactions,” Bowen says. “One girl ran away. Most people walked by, did a double take, then came up to it. Lots of people just started exploring and playing with it.”

“It was a great experience. It was really rewarding to see people react so positively.”

HAAM uses augmented reality, a technology that adds images and information to user surroundings (unlike virtual reality, which replaces the surroundings).

Before developing the program, Bowen delved into learning theory, focusing on how to present information without overwhelming the user.

“Augmented reality is great for that,” he says. “It’s not like a textbook, which is a wall of text. It presents an image that allows for the exploration of information. It’s a more natural way of learning.”

Bowen was inspired by video games like Sim City, which isn’t branded as an educational game but teaches users about topics like economics and urban planning.

“The goal is to educate; the hard part is making it fun,” he says.

Bowen also put a lot of thought into the physical aspects of the user interface.

“I did research on museums and found that people generally spend less than one minute per exhibit. If they have trouble understanding an exhibit, they walk away within seconds. That’s why I wanted the image to pop up immediately.

“Interacting with it, moving around, touching a place on your body (where the image of an organ appears) — that keeps people more engaged. I tried to keep it simple: point and select, or drag and drop.”

So far, he’s spent more than 150 hours developing HAAM.

He learned C Sharp, a programming language, and Unity, a game development platform. To create images of the organs, he used data from the National Library of Medicine’s Visible Human Project and the National Museum of Health + Medicine Chicago, creating three-dimensional images with Zbrush and 3ds Max digital modeling tools.

“I gained a lot of skills,” he says.

Now he’s looking to gain new ones, this time in business and entrepreneurship.

“This project absolutely has potential for future development,” Brennan says, including use in museums, classrooms, patient care and health professions training.

Inspired by science fiction authors like Isaac Asimov and Neal Stephenson, Bowen wants to continue using art, technology and imagination to communicate the complexity of science.

His ultimate goal: to run a small, serious games company (“serious” is the term used to describe games that have learning as their primary goal).

“There’s a lot of room out there for innovation,” he says.