The Whitney Museum of American Art Audio Guide
Dedicated to innovation and authentic experiences for visitors, the Whitney’s current Audio Guide features audio content sourced directly from artists and diverse voices described by one participant as “more like a conversation, than the dryness of a normal lecture [that] you hear in Audio Guides.” The Museum created a new prototype of this popular Audio Guide, and the usability study observed participants’ behaviors in using it to complete relevant tasks. By exploring the prototype’s understandability, the study resulted in findings and recommendations to further enrich visitor experiences with this digital product that helps to set The Whitney apart from other museums.
Case Study
A major intersection in the Whitney experience is between Audio Guide content and the galleries as physically experienced. The Whitney’s Audio Guide is a stand-out product that provides context for artworks through recorded primary sources (often the artists themselves). Listeners can better understand and connect the many artists’ visions and personalities. Developing a prototype supports the update to the Audio Guide that connects visitors to this rich content.
The following description highlights the user test’s design and analysis to support the development of this prototype. Participating Whitney visitors were the first to test and use the prototype, and the team observed their approaches to resolving specific tasks with it, which were recorded and analyzed. Based on direct feedback from this group of participants, this report presents recommendations to enhance the visitor experience of the Audio Guide.
Methodology
The purpose of this user test is to support the development of the Audio Guide prototype. Observing, recording, and analyzing the approaches participants took to perform the Task Scenarios made the primary goal of examining the performance and several features of the Audio Guide possible.
Strategy
The feedback that participants provided was analyzed to rate instances when participants relied on prior knowledge or more time to perform actions. These instances were accumulated by reviewing the observations, recorded conversations, surveys, and participant feedback.
User Test
This user test involved several methods and analysis techniques: pilot testing, random user sampling, task analysis, think-aloud protocol, probing questions, and rating scale. See Figure 2.
Figure 2.
A pilot test determined how effectively the tasks and questions in the study revealed visitor behaviors and preferences regarding the understandability of the Audio Guide prototype. The pilot test results informed the study’s redrafting and refinement into a final version for implementation.
User Testing and Tasks
The team chose eight visitors for the Audio Guide prototype user test, which took place on the seventh floor of the Whitney Museum. One team, consisting of a Tester and Moderator, performed the user test on 4/19/2018 with four participants. The following day, the second team, consisting of a Tester and Moderator duo, performed the user test with four participants.
After participants signed a consent form, the Testers followed the script to properly guide participants as they performed the tasks on an iPhone mobile device with iOS 11.3.1 from Apple Inc. Five of the eight participants with varying levels of familiarity with Whitney’s current audio guide system tested using the audio guide at the time of intercept. Moderators used a mobile device to record audio feedback, noted important points during scenarios, and observed the interactions that the participants followed to complete tasks. Participants completed a post-test questionnaire created with Google Forms.
The following are Task Scenarios:
- Using the Audio Guide, find any information you can about Gorky’s The Artist and His Mother. Go to the painting and listen to the audio provided (see Figure 3).
- Using the Audio Guide, find an exhibition you can view today (see Figure 4).
Figure 3.
Task Scenario 1: Using the Audio Guide or your mobile device: Find any information you can about Gorky’s The Artist and His Mother. Go to the painting and listen to the audio provided.
Figure 4.
Task Scenario 2: Find an exhibition that you can view today.
The moderators encouraged participants to explain their decisions and thoughts aloud throughout task resolution—specifically, the standard moderating techniques of Concurrent Think-Aloud (CTA) and Concurrent Probing (CP). Visual observation confirmed that participants had completed a Task Scenario. Afterward, participants judged whether they had accomplished the task and rated the difficulty they experienced in achieving it (see Figure 4).
Moderators assured Participants that their feedback was crucial, regardless of whether they completed tasks. Following the Task Scenarios, participants received reflective questions to gather more critical feedback. Demographic questions were an optional follow-up to the reflective questions and concluded the study's data collection. Figures 3 – 4 are detailed, step-by-step descriptions of the Task Scenarios.
Participants Profile
The post-test questionnaire provided the information to survey the participants’ profiles. Questions covered participants’ art and museum knowledge, how they gathered information about museum events, nationalities, device preferences, and past visits to the Whitney and other museums. Figures 5 – 11 present the analysis of this survey.
Figure 5.
Participants Profile: the number of visits the participants have made to museums in the past year.
Figure 6.
Participants Profile: how the participants find information about the exhibitions on display at the Whitney Museum.
Figure 7.
Participants Profile: participants' self-assessed knowledge of the Whitney Museum.
Figure 8.
Participants Profile: participants' self-assessed knowledge of the Whitney Museum.
Limitations of This Study
While it may seem that user tests are limitless in their benefits, researchers should be aware of certain tendencies that may cause errors in collecting critical information and making the correct observations. The team tested Drafts of the Task Scenarios before the test to avoid the researchers' mistakes. These test runs and edits also alleviated the possibility of incorrectly screening or inadvertently assisting participants.
Because of existing, inevitable social pressure, some responses to our questions may not be accurate. Mostly, this is due to participants aligning their answers with what they perceive to be more popularly accepted by society, regardless of the truth. Sometimes, participants may feel uncomfortable with or confide in the reliability of the Testers or Moderators. To reduce the significance of these limitations, participants received reassurance that all materials were confidential and that their skills were not a part of the test. Participants were also allowed to skip questions they felt uncomfortable answering and even end the test at their discretion.
Summary of Key Findings And Recommendations
Participants revealed a high level of interest in the Audio Guide's content. Some participants mentioned they expected the navigation links to perform differently than their actions. Participants mainly relied on matching artwork thumbnails with art in the gallery to relate the correct artwork stop number. They also relied on their memory to determine the current exhibitions.
Recommendations
Those findings and information led to this proposal of recommendations in support of the participants and their interaction with the Audio Guide:
- Increase legibility of artists' names and the titles of artworks [visual design and information organization] (see Figure 9).
- Refine shapes used in navigation and increase contrast to distinguish content [visual design and organization of navigation links] (see Figure 10).
- Incorporate filter by exhibition, filter by floor, and search by stop number functions [content structure and information organization] (see Figure 11).
Figure 9.
Recommendation 1.
Figure 10.
Recommendation 2.
Figure 11.
Recommendation 3.
Conclusion
Through the usability test process, we realized that the rigor and scope of the set test tasks may have led to the users' completion rates and ways of thinking. The Whitney Digital team must consider some of the findings, recommendations presented in this report and additional research methods. Further analysis is required by conducting different user-testing evaluations to deliver a seamless, digital experience. Due to the limitations of time and testing scope and some of the general, technological, and information biases inevitable in user experience testing, further analysis is required by conducting different user-testing evaluations to deliver a seamless digital experience.
Designing a reasonable task by scientific standards is the basis of the user experience test. Usability testing provides methodologies that make recruiting and testing participants on-site suitable. The entirety of the research process involved the team developing a comprehensive usability test for the Audio Guide, moderating the test, gathering feedback, finding and making comparisons in the test results, and complementing it with recommendations for the Whitney Museum. The solutions are only a few based on the visual design choices and the introduction of special functions to the Audio Guide. These recommendations further support the participants' desire to find specific content and search for particular artworks.
Selected Works
