A collaborative and research-driven project led the Libraries to overhaul and align their brand identity and website information architecture to redesign the site. The new Pratt Institute Libraries website delivers a space where the Directors’ visions can flourish while they meet library users’ needs.
Brand Strategy
Interaction Designer
Project Management
User Research
Visual Identity System
The following description highlights the user test’s design and analysis to support the development of this prototype. Participating Whitney visitors were the first to test and use the prototype, and the team observed their approaches to resolving specific tasks with it, which were recorded and analyzed. Based on direct feedback from this group of participants, this report presents recommendations to enhance the visitor experience of the Audio Guide.
The purpose of this user test is to support the development of the Audio Guide prototype. Observing, recording, and analyzing the approaches participants took to perform the Task Scenarios made the primary goal of examining the performance and several features of the Audio Guide possible.
The feedback that participants provided was analyzed to rate instances when participants relied on prior knowledge or more time to perform actions. These instances were accumulated by reviewing the observations, recorded conversations, surveys, and participant feedback.
This user test involved several methods and analysis techniques: pilot testing, random user sampling, task analysis, think-aloud protocol, probing questions, and rating scale. See Figure 2.
A pilot test determined how effectively the tasks and questions in the study revealed visitor behaviors and preferences regarding the understandability of the Audio Guide prototype. The pilot test results informed the study’s redrafting and refinement into a final version for implementation.
The team chose eight visitors for the Audio Guide prototype user test, which took place on the seventh floor of the Whitney Museum. One team, consisting of a Tester and Moderator, performed the user test on 4/19/2018 with four participants. The following day, the second team, consisting of a Tester and Moderator duo, performed the user test with four participants.
After participants signed a consent form, the Testers followed the script to properly guide participants as they performed the tasks on an iPhone mobile device with iOS 11.3.1 from Apple Inc. Five of the eight participants with varying levels of familiarity with Whitney’s current audio guide system tested using the audio guide at the time of intercept. Moderators used a mobile device to record audio feedback, noted important points during scenarios, and observed the interactions that the participants followed to complete tasks. Participants completed a post-test questionnaire created with Google Forms.
Task Scenario 1: Using the Audio Guide or your mobile device: Find any information you can about Gorky’s The Artist and His Mother. Go to the painting and listen to the audio provided.
Task Scenario 2: Find an exhibition that you can view today.
The moderators encouraged participants to explain their decisions and thoughts aloud throughout task resolution—specifically, the standard moderating techniques of Concurrent Think-Aloud (CTA) and Concurrent Probing (CP). Visual observation confirmed that participants had completed a Task Scenario. Afterward, participants judged whether they had accomplished the task and rated the difficulty they experienced in achieving it (see Figure 4).
Moderators assured Participants that their feedback was crucial, regardless of whether they completed tasks. Following the Task Scenarios, participants received reflective questions to gather more critical feedback. Demographic questions were an optional follow-up to the reflective questions and concluded the study's data collection. Figures 3 – 4 are detailed, step-by-step descriptions of the Task Scenarios.
The post-test questionnaire provided the information to survey the participants’ profiles. Questions covered participants’ art and museum knowledge, how they gathered information about museum events, nationalities, device preferences, and past visits to the Whitney and other museums. Figures 5 – 11 present the analysis of this survey.
Participants Profile: the number of visits the participants have made to museums in the past year.
Participants Profile: how the participants find information about the exhibitions on display at the Whitney Museum.
Participants Profile: participants' self-assessed knowledge of the Whitney Museum.
Participants Profile: participants' self-assessed knowledge of the Whitney Museum.
While it may seem that user tests are limitless in their benefits, researchers should be aware of certain tendencies that may cause errors in collecting critical information and making the correct observations. The team tested Drafts of the Task Scenarios before the test to avoid the researchers' mistakes. These test runs and edits also alleviated the possibility of incorrectly screening or inadvertently assisting participants.
Because of existing, inevitable social pressure, some responses to our questions may not be accurate. Mostly, this is due to participants aligning their answers with what they perceive to be more popularly accepted by society, regardless of the truth. Sometimes, participants may feel uncomfortable with or confide in the reliability of the Testers or Moderators. To reduce the significance of these limitations, participants received reassurance that all materials were confidential and that their skills were not a part of the test. Participants were also allowed to skip questions they felt uncomfortable answering and even end the test at their discretion.
Participants revealed a high level of interest in the Audio Guide's content. Some participants mentioned they expected the navigation links to perform differently than their actions. Participants mainly relied on matching artwork thumbnails with art in the gallery to relate the correct artwork stop number. They also relied on their memory to determine the current exhibitions.
Those findings and information led to this proposal of recommendations in support of the participants and their interaction with the Audio Guide:
Recommendation 1.
Recommendation 2.
Recommendation 3.
Through the usability test process, we realized that the rigor and scope of the set test tasks may have led to the users' completion rates and ways of thinking. The Whitney Digital team must consider some of the findings, recommendations presented in this report and additional research methods. Further analysis is required by conducting different user-testing evaluations to deliver a seamless, digital experience. Due to the limitations of time and testing scope and some of the general, technological, and information biases inevitable in user experience testing, further analysis is required by conducting different user-testing evaluations to deliver a seamless digital experience.
Designing a reasonable task by scientific standards is the basis of the user experience test. Usability testing provides methodologies that make recruiting and testing participants on-site suitable. The entirety of the research process involved the team developing a comprehensive usability test for the Audio Guide, moderating the test, gathering feedback, finding and making comparisons in the test results, and complementing it with recommendations for the Whitney Museum. The solutions are only a few based on the visual design choices and the introduction of special functions to the Audio Guide. These recommendations further support the participants' desire to find specific content and search for particular artworks.
Selected Works
AddtronicsWebsite Design
Apogee LightingWebsite & Print Design