June 3, 2023

Measuring success…

At the beginning of this series, I accidentally sent out a draft email to people who had joined on day 1. The email contained nothing but following question:

“How do you measure success?”

I had planned to write on the topic of measuring success earlier in the series, but decided it made more sense to at the end. The email wasn’t meant to go out, but some of you took it in stride and replied. Fellow list member and design system lead, Silviu Runceanu, shared the success measurements he uses:

Adoption - Process of prospective users becoming actual users of the design system

Task completion - Metric to measure how faster & confident the UX designer are working using the design system

Dev handoff - Measures the easiness of the handoff process, and if the productivity of the developers will be altered

Onboarding - The usefulness of an already system in case of a new hire onboarding

Scalability - Measure the productivity of the design system in maintaining & developing the product

Satisfaction - How happy are the clients that are using it? The developers? The designers? Does it make their job / “life” easier?

This is a good start… let’s dive a little deeper.

When you think about measuring success, it helps to revisit the product-mindset from earlier. What were the goals and objectives your team set out for the design system? What “villain” did you cast? What were the key performance indicators you defined at the start? After all, success is really just accomplishing a goal or purpose, so it will depend largely on what you set out to do at the start.

Here are some examples of common design system goals and how to measure against them:

• Are we making the teams more efficient? Measuring how long it takes to go from visual prototype to release in production; tracking average task completion time; user interviews with designers, engineers, and stakeholders;
• Can teams easily use and upgrade design system tools? Measuring how many teams are using which version of the design system (through automated tools or just check-ins), and how long it takes teams to update to new versions;
• Are we increasing the number of products adopting design system tools? Measuring the number of products that use or share design system resources (e.g. code libraries and design tool libraries) vs. the number that don’t, ideally relating that to the underling business value it represents;
• Are our products more functionally and visually consistent? Qualitative measurements, like printing out all product screen shots and looking at them at “10,000 feet,” though there are visual regression tools like BrowserStack and Chromatic that can help automate this process;
• Is the documentation helpful? Tracking documentation completion metrics like time on page vs. avg. read time, “was this page helplful” surveys, and user research sessions;
• Are the design tools useful for designers? A little harder to track than quantitative engineering measurements, but tracking component/symbol usage (some design tools offer this), number of detached components, and user feedback (# of comments, or participation in DS Slack channels);
• Are the code tools useful for engineers? Tracking quantitative measurements like analytics, how often a component is being used in the codebase;
• Is the code accessible and compliant? Manual and automated checks for accessibility and legal compliances (tools like AccessiBe or Evinced can help), accessibility score;
• Is the code performant and error-free? Performance metrics (load time, startup time, etc.) and quality metrics (issues, PRs, backlog, and other tech-debt related measurements);
• Have we reduced the complexity of our components? Tracking the total number of primary components across platforms, including the number of variations and props. As the number of components increase, complexity increases, so this can be a good incentive for standardizing your component structures across the org;
• Do teams feel supported? A NPS (net promoter score) survey for teams using the DS.

Before you deploy any big design system initiatives, take some baseline measurements for your existing products. The value of your design system may be obvious to design and engineering team who are working with it every day, but having some compelling measurements goes a long way toward getting support from leadership and other stakeholders.

So… how do you measure success? 🙃

—
Cheers,

Up Next: Some design systems practice…

← Full Archives