How I redesigned an critical dev ops tool to enable easier learning and efficiency.
💼 Role: Contracted UX Designer & Researcher
🗓️ Dates: Jan 2023 – May 2023
👥 Team: Product Manager, Engineering Manager & Front-end Engineer
🚀 Stage: Redesign of an established product, Discovery to Development
🏢 Client: SAAS Company with around 5k customers and 450 engineers
🔏 Note: I have received permission from the client to share the images on this page anonymously.
Opportunity:
Redesign Highly Utilized Dev Tool for Easier Navigation and Learnability
This client’s internal Build, Test & Release products are a highly utilized part of their dev team’s process (with ~1.1M page loads last year). The #1 request from their users is to improve the user experience and navigability. One user wrote in an October 2022 survey: “Need better navigation!! I'm constantly lost!! Please have UX/UI help with this!”
Improvements to these tools would help dev teams ensure:
customer facing issues can be rapidly diagnosed and fixed;
configurations are used that reduce repetitive manual effort and ensure compliance with the company’s security standards; and
newer employees can confidently learn and do best practices, reducing over-reliance on the few people with deep institutional knowledge.
Through this project I collaborated with the with the team’s Product Manager and Engineering leads to:
1) conduct discovery to get to the highest priority opportunities and workflows to improve
2) propose a redesign that solved the issues that could be developed and released iteratively.
Final Results
The proposed prototype earned a System Usability Score of 84/100 which was an incredible +41 point increase over the current product and +16 point increase over the web average.
Current Experience
✅ The current tool provides powerful value to users who learn it. Expert users described the system as “Reliable” and “Powerful” but also “Slow” and “Scary”
❌ System Usability Score of current application 43/100*
❌ 3 competing navigation patterns, with primary tasks spread across multiple pages.
❌ Heavy reliance on knowing where to go to do common tasks, which meant lots of dependence on existing experts to constantly solve issues.
❌ Inconsistent usage of terms which resulted in confusion and mixed conceptual models.
Proposed Redesign
✅ Organized the application around repos and teams which better matched user’s mental models and kept them focused on the processes they care most about.
✅ Made most common next step obvious and used wizards and smart defaults/suggestions for more complex tasks.
✅ Built clear concept model with the team to encourage use of consistent terms within the app.
How We Got There
Discovery Interviews
There was quite a bit of existing research that I was able to make use of immediately entering the project. However, there wasn’t sufficient detail in a few key areas which lead us to conduct interviews to answer the following:
❓ Research Question What are the most important problems to work on and how are those journeys/workflows currently accomplished?
💡 Hypotheses
We predict that frequent users of RMConsole have built up their own work-arounds and methods outside specific guidance or navigation within the tool.
We predict that the current navigation is difficult to understand, especially for newer users. </aside>
🎯 Objectives:
Identify the most frequent and most painful journeys & workflows that users are doing in RMConsole today.
Identify the steps users are actually taking to accomplish those journeys today. (With a focus on navigation to and within the tools)
Identify what’s working well and not working well in those workflows.
Identify additional tools they are using to help support their release process today.
👥 Participants: 12 users from varying roles and dev teams, with different levels of familiarity in the product
Through these interviews we were able to develop a clearer picture of the various roles users were playing in the product, the #1 key attribute that impacted their experience (level of familiarity and confidence), and the major workflows we wanted to target.
Problem Statements
Discovery interviews revealed the following priority problems to work on. They reference personas also developed as a part of those interviews.
1️⃣ Pipeline Details Page Release Novices & Release Familiars have difficulty learning how to interpret the pipeline page (esp. nested groups) and performing common workflows to remediate a stopped pipeline (Retry, Investigate Logs, Override, Archive).
Related Problems
Release Experts want to perform other common pipeline workflows (Reschedule, Override Deploy Window, Restart) with fewer clicks and fewer opportunities for mistakes.
Many time the reason for user intervention in a stopped pipeline is simply to manually retry a step, esp. on test/signals/build steps. How might we reduce the need for manual intervention?
2️⃣ CICD Navigation Release Novices & Release Familiars have difficulty navigating beyond the linked page from alerts or GitHub (most often the Pipeline Details Page) which impacts their ability to quickly find information they need to facilitate a release or remediate a support issue.
3️⃣ Repo Settings & Configuration Repo Configurators have difficulty understanding and configuring release settings which hinders utilization of best practices, increases friction of creating new repositories when needed, and makes troubleshooting issues or making updates difficult.
(Note: This priority was removed from the scope of my design explorations, but was saved for the team to explore in the future)
Ideation & Prototyping
Since this was an internal tool, there were plenty of users available and enthusiastic about the opportunity to co-create the future of this tool. We held an ideation session with the team and several subject matter experts, and iterated through various high level approaches before diving into more detailed prototypes.
I built two high-fidelity prototypes to take to user testing, centering their functionality around the top selected scenarios.
Usability Testing
We tested the two prototypes with 11 users (3 novices, 5 mid-level, 3 experts) on 5 of the most common scenarios. 6 users saw Prototype #1 first, 5 saw Prototype #2 first. Results were very consistent themes across all testers:
10/11 testers overall preferred Prototype #1, and thought either of the proposed prototypes were a vast improvement over the current product. (There was a hypothesis before testing that expert users who liked and were highly competent in the current experience, may still prefer what they already knew.)
Most users found the releases & pipelines view to be their favorite part of the prototype. They liked being able to do everything around their releases in one page and found it easy to understand what was going on quickly. This helped to prioritize which parts of the experience to build first.
There wasn’t much difference between novice, mid-level, and expert users ability to complete the scenarios.
We got valuable insights into ideas/features that users did not think were particularly valuable, i.e. things we could cut out of development scope.
The proposed prototype earned a System Usability Score of 84/100 which was an incredible +41 point increase over the current product.
Detailed Designs
Once we had determined the most usable design approach, it was time to get into the weeds. Working with the Engineers and Product Manager we broke the designs apart into chunks that could be developed and released iteratively, in order to learn quickly from real usage and adjust with feedback.
Mapping Workflows Beyond the Screen
One of the broken off chunks of work was making the Redeploy workflow easier. This was the flow diagram of the original process which extended beyond clicks in the app. Much of it also relied on users to “just know” where to go next.
We simplified this process into a 4-step wizard that provided everything they needed and gave novice users in our usability testing a lot of confidence that they would not need to ask expert users for help.
Designing All the States
As a highly configurable tool for engineers, there were also plenty of states and edge cases that were not covered in the original prototype or usability tests but needed to be figured out.
Impact of this Work
Development of the first phases of this work are still in progress. Early feedback has been overwhelmingly positive.
