A mixed reality museum concept for the 2017 Design Expo.
UX, IXD, & Motion
scroll down

Our team was challenged to create an “intentional design for positive cultural impact in mixed reality” for the 2017 Microsoft Design Expo. In doing so we wanted to demonstrate the value and differentiation of Mixed Reality (MR) to the environment, objects, and people participating in our solution and meet a need that is extensible to wider applications.

My Role

I was the IX & motion design lead on a team of 5. I collaborated with 3 team members to uncover research insights and provided creative vision to catalyze the concept development process. I partnered with 2 other designers to develop storyboard sketches into a hig-res visual keyframes to share the vision and led the production of 3D and motion graphics . I also supported research by conducting ethnographic observations and interviews during our contextual inquiry and contributed to our literature review.

01/2017 - 03/2017 (10 weeks)
Brian Orlando
Mihn-Anh Nguyen
Maureen McClennon
Michael Frampton


problem space

Museums are being redefined by technology in the digital age and the transformation promises to touch every aspect of what museums do, from how art and objects are presented and experienced to what is defined as art.NYT

As people live increasingly digital lives, museums have an important role to play in helping people understand the emerging hybrid culture. Our challenge was to explore the shifting identity of the museum and identify opportunities for digital content to strengthen public engagement and enhance learning outcomes. Key to this task is the need to integrate mixed reality technologies safely within the physical space of the museum without diminishing the artifacts and social interactions that make that place unique.



We began by investigating the historical context of the museum to gain a greater appreciation for its role in society. We then explored precedents of technology use within the museum environment and visited several museum environments in person to identify opportunities to enhance immersion and through mixed reality.



Literature review

We reviewed literature from a variety of academic and business sources to help contextualize the problem space. This led us to a greater appreciation for the historical importance of the museum. Armed with new insights, we were able to begin exploring the museum spaces in person.


Contextual Inquiry

We visited several museums to examine the exhibition environment and learned how and why people interact with the space. We learned that the social interaction among museum visitors shapes their engagement with the displays as much as displays shape their social interaction. We also noticed that current displays limit potential for interaction and immersion.


Ethnographic Interviews

We conducted non-directive ethnographic interviews with museum staff and visitors to solidify our understanding of the space, build empathy, and uncover unmet challenges. We learned how staff members engage visitors and the budgetary compromises they made during event planning.

Key Research Insights:

  1. The displacement of artifacts in protective displays prohibited interaction and obscured meaning.
  2. Museum exhibits often contained complex, text-heavy information that required long periods of passive engagement.
  3. Exhibit designs often devalued museum staff and missed opportunities for social engagement.

Text heavy exhibits are unapproachable to the average viewer.

Static exhibits miss opportunities for deeper contextual immersion.

Interactive exhibits create opportunities for social engagement.

Self-paced learning environments create opportunities for exploration.

context creates opportunities for novelty and immersion.



We needed to frame our solution quickly because our process time was extremely limited. We gathered our research observations and discussed them systematically to draw out insights. To do this, we expressed multiple viewpoints and focused on aligning the team to the interpretations that were most accurate and extensible to our goals. We chose the following principles to emphasize ways in which the technology could best enhance the museum experience for multiple types of museum-goers.

Key Design Principles:

  1. Create Immersive Contexts.

    Museums use historical assets as vehicles of communication but can sometimes fail to communicate the context in which these assets existed. Reconnecting artifacts with their context will supply addition meaning and opportunity for engagement.

  2. Create Dynamic and Engaging Content.

    Museum content is static and seldom updated on account of financial constraints. We can improve engagement by immersing the museum-goer in an interactive, multi-sensory experience.

  3. Emphasize Personalization.

    Museums make complex information accessible to people of all ages; however, this information does not take the person or group's existing knowledge base into account. Personal knowledge and interest can be engaged By leveraging dynamic content to create self-paced, explorative learning.

  4. Emphasize Social Interaction and Collaboration.

    Many museum-goers are seeking opportunities to learn and share knowledge with others: yet, static museum environments often discourage these exchanges. Therefore, museums should foster knowledge sharing amongst staff and museum-goers.

Problem Statement:

How might we use the immersive contexts of Mixed Reality to create a dynamic, yet personal learning experience within the museum?



We began exploring opportunities by sketching concepts individually and then discussing them as a group. This led us to a general concept. We then worked through several iterations of sketches to fill in the gaps, and refined them into storyboards, and eventually a high-fidelity interaction sequence to structure our experience around.



Concept Sketching

We explored many possible themes, but chose dinosaurs due to their iconographic representation of the museum experience. Above is a concept that offers satellite experiences throughout the region to drive engagement and funding for the museum.



Building on each others ideas, we soon had a clear direction. I then partnered with 1 other designer to develop concept sketches into storyboards to share our vision. We presented these storyboards to our sponsor for critical feedback and altered our story sequence in response to improve accessibility.

concept generation

Interaction Sequence

I collaborated with two other designers to translate systems-level interactions into a narrative sequence that illustrates their flow as a museum-goer would experience them during the exhibit.

Concept Statement:

Aura is Mixed Reality technology that creates a dynamic and immersive museum experience.



We refined our vision around our design principles and developed a video prototype to communicate our vision to stakeholders. This was a three week process requiring long hours of on-location videography, post-production, and time-intensive motion graphics work, and culminated in a multimedia presentation review.

Visual System

A diagram that shows the stylistic elements of the application



Since there are few established conventions for interacting within a mixed reality environment we focused on matching approaches to an optimal sensory context to create a diverse and engaging user journey. Through the process, we were able to utilize a variety of approaches from traditional 2D-interfaces translated to 3D-space, to less conventional approaches based on gaze, movement, proximity, and accessory hardware.

Immersive Context
The museum-goer enters the exhibit through a holographic fog that absorbs them into the context of the physical artifacts and holographic specimens they are viewing. I produced this scene by layering visual effects over several meticulously masked video clips.

Dynamic Content
As museum-goers move through different habitat rooms in the museum they will encounter novel moments that inspire wonder as they bring ancient environments to life. I composed this key scene by enhancing a 3D model with visual effects and layering it over green screen footage and a still background image.

Personalization & Collaboration
Aura brings info graphics to life allowing museum-goers to explore personal interests and share them with friends. Motion design by Brian Orlando.

Aura Concept Video


thoughts & lessons

As a medium for expression and communication, mixed reality is still in its infancy. As we learn how to use this technology we must not forget that anything of lasting value must first be rooted in human values. The most exciting elements are not those that take us the farthest from reality, but those that allow us to experience it the most deeply: to explore freely, to express our creativity, and to connect and share more deeply with the people around us.

Mixed reality is a promising technology and it was fun to explore its creative possibilities without being limited by business and production concerns. However, given more time, this project would have benefited from a greater depth of research. While we made an effort to gather perspectives from museum staff and visitors, we were not able to engage them deeply enough to learn what they truly desired from a mixed reality experience. As such, we tried to work to the strengths of the technology from a human-centered perspective, but missed some opportunities for social engagement and information integration that would have improved the value immensely. In fairness, we were given a technically demanding and widely-scoped task with little time to narrow it in systematic way. However, if I were to do this project again, I would focus on applying a smaller set of interactions to the environment and spend additional time on research to root them more deeply in human values.