Assessment of Perceived Usage of Dashboard

Fuchsia @ Google
Year: April 2023
Role: Lead UXR
Research Method(s): Literature Review, User Interviews, Google Analytics
Disclaimer: The visuals you see on this page are coming from my final UXR report. It is for showcasing my research reporting ability.
The initial reason why this dashboard came to be was that several people (mostly developers) came together and built this dashboard, with research not being considered. Later, research was done to see what target users to focus on, and how to design the experience for their workload, but what that research lacked was talking to other target users within the operating system, which would fundamentally change how you would design the experience.
Background and Goals
  • Investigate the “why” behind the perceived low usage of an internal dashboard that monitored smart displays metrics
  • Decide what the next step was: continue to improve the dashboard, maintain it, or get rid of it altogether to reassign the main engineer
  • Understand the impressions and perceptions of automated alerts from dashboards
Method
  • Literature Review, to understand how the dashboard became its current state and to see what groups of developers were talked to
  • Interviews, to talk to different groups of developers that were previously left out of the previous sample
    • Sample: Target users who has previously tried using the dashboard
    • Sample: Target users who had not tried it
  • Google Analytics, to see what target teams were using the dashboard, at what times in the day and during the past two quarters, and what parts of the dashboard was being interacted with
Crucial Insights
  • Target users did not know about the internal dashboard, pointing to a discoverability issue
    • Even when shown the dashboard, target users did not find any use out of it, pointing to a relevancy issue
  • The expectation that we began with, in how developers monitored metrics proactively was different from the reality, where metrics monitoring was dependent on the product release life cycle
  • There was significant interest in automated alerts, however there was hesitation as alerts, if not thought out properly, can easily lead to spam
Research Impact
  • Strategic Impact
    • Eliminated tech debt by relieving an engineer from maintaining a dashboard that no one used
  • Stakeholder collaboration Impact
    • Encouraged stakeholders to note-take, sit in on moderated interview sessions, debrief and co-analyze research findings through affinity mapping exercises
  • Product Impact
    • Provided clarity on the reasons why users didn't use the dashboard, leading stakeholders to unanimously agree to deprecate the dashboard
Reflections
  • ENGAGE THOSE STAKEHOLDERS! As you are trying to fine-tune your sampling strategy, check in with your stakeholders to see what their thoughts are on how you are defining your strategy and if it makes sense.
  • Taking into consideration the power dynamic of being a contractor and doing confidential research for full-time Google employees, it was intimidating to be the person to say “no one uses this, let’s not invest any resources towards this anymore”, but I didn’t have to explicitly suggest that the team should deprecate it for my stakeholders to axe the dashboard!