Mapping Your Journey with Metrics
Introduction
In this recording of “Insights from Experts” at the Componize 2022 Summit, Dawn Stevens of ComTech Services discusses a range of of commonly used content management and change management metrics. It is important to know what metrics matter most when it comes to measuring the introduction, use and effectiveness of your content. To help you decide what to measure, Dawn walks through each metric for change management in detail and shows you how they work together to form a complete picture of your content’s performance.
Takeaway: Metrics are essential for measuring the success of your content creation, management and delivery. In Mapping Your Journey with Metrics, Dawn explains which ones are worth tracking and how they can help you improve over time.
Metrics for change management are important for a few reasons
- They’re a way to measure progress. Measuring the success of your initiative helps you understand how effective it’s been, and that can inform future decisions about what to do next.
- They help you define success. Without metrics, it can be difficult to tell whether or not a change is actually making an impact.
- They provide insights into how people interact with your content.
A change in content strategy requires planning and monitoring
A content strategy is a plan that defines what content you need, who will create it and how to distribute it. It also includes how you will measure the success of your efforts.
Content strategy is not about creating more content or putting more information in your documentation. Instead, it’s about deciding which topics matter most for your users and making sure those topics are covered well in all channels.”
A good content strategy will help you
- Plan what content you need. This will help you create a content plan that is realistic, feasible and measurable.
- Know how that content fits into your overall content delivery plan/strategy.
Why have metrics for change management
Metrics are used to measure progress against goals, the impact of change and its success. They can also be used to measure the success of a change management initiative. At a basic level, metrics are useful because they help you identify gaps in how things are going and areas where improvements can be made. If you don’t know what is happening today, how will you be able to tell if your initiative was successful?
To justify the change
Think about the problem from a new perspective.
The first step in justifying change is to make sure you understand what you’re trying to solve, and why. If there’s no clear problem for your company or project, then it’s not worth solving. It’s important to get this right so that you can come up with an effective solution—and explain how that solution will benefit others as well as yourself.
What are the benefits of doing this? What are the costs? What are the risks? What alternatives do we have available? What are some possible consequences if we decide not to pursue this path at all?
To track progress of the changes being made
- Track progress against goals.
This is a good way to measure how well your project is going, and will help you determine whether or not it’s worth continuing. If the goals are clear and measurable, then you’ll be able to see the progress being made from one week to the next. This will also allow you to identify any problems that may arise along the way—and make adjustments accordingly. - Track progress against metrics.
Metrics are another way to track progress, as well as to see how effective the changes being made are. Metrics can be used to measure financial performance, customer satisfaction, productivity and more. For example, if one of your goals is to decrease field engineer callouts by 20%, then you’ll want to look at the those numbers every month or so in order to see whether they’re on track with that goal. - Track progress against standards.
Standards are another way to measure progress. Standards are a set of rules, guidelines or requirements that define the quality of something—and can include things such as safety standards and customer service standards. For instance, if one of your goals is to have no more than five callouts each month, then you’ll want to look at those numbers every month or so in order to see whether they’re on track with that goal. - Track progress against results.
In addition to tracking progress against goals, you should also track progress against results. Results are outcomes or effects that are produced by something—and can include things such as safety results or customer service results. - Track progress against expectations.
You should also track progress against expectations. Expectations are simply things that people expect to happen—and can include things such as safety standards or customer service standards. - Track progress against budget and expenditures.
You should also track progress against budget and expenditures. This can be a good way to see whether the project is on track with its costs and the money you’ve allocated for it. If you do this, make sure that there are clear definitions of what counts as progress against budget and expenditures—for example, whether progress means staying within budget or spending less than planned. - Track Progress Against Timelines, Milestones & Deadlines (if applicable).
Another way to track progress is by tracking it against timelines, milestones and deadlines. Timelines are specific periods of time, such as when you expect the project to be completed or what steps need to happen at which point in time. Milestones are key points in a project where one thing has been completed before moving onto another step; for example, creating converting word documents to XML format could be a milestone in a Structured Content conversion project. Deadlines are due dates that must be met.
To demonstrate achieving the goal
As you have most likely noticed, this process is cyclical. Metrics are collected during each iteration of your roadmap and used to assess performance and make adjustments. The adjustments you make should be based on feedback from your customers, employees, or other stakeholders in order to improve and manage the change. In addition to using metrics to gauge the success of each iteration of your roadmap, they can also be used as indicators that you have in fact achieved the goal set at the beginning of the change in content strategy or tool.
Change management metrics provide a complete picture of your initiative
Metrics are a critical part of any change management process. They’re also a key tool to help you understand how your content strategy is working, and whether your team is achieving the goals they’ve set out to achieve. But most importantly, metrics offer insight into what’s working—and what isn’t working—in your organization’s pursuit of its content strategy.
Create the right environment and behaviors to reach your goals
- Metrics help you to define the goal. They also help you to determine whether or not you’re on track to reach that goal. Metrics are an important part of any change management process, and they can help organizations better understand how their content strategy is working.
- Metrics help you to define the target. They help you to understand what success looks like, and they help your team know how they should be measuring their results. Metrics are also essential in helping organizations change the way they work. If you want your organization to focus on content strategy, then it’s important that everyone knows what that means—and how to measure success.
- Metrics help you to define the progress. Metrics are also an essential part of measuring success. When you’re rolling out a new content strategy, it can be hard to tell whether or not your team is making progress towards the goals that you’ve set for them. Using metrics helps organizations understand how well they’re performing and whether or not their content strategy is working as intended.
- Metrics help you to define the success criteria. When you’re rolling out a new content strategy, it can be hard to tell whether or not your team is making progress towards the goals that you’ve set for them. Metrics help organizations understand how well they’re performing and whether or not their content strategy is working as intended. If nothing else, metrics help organizations get a clearer picture of what’s working, what isn’t and why.
- Metrics help you to define the baseline, which is a historical data point that allows for comparison with current performance or future performance. The baseline should be created using historical data, not observed values from recent observations of your process. This can be very helpful in identifying whether something has changed (for example, if you start seeing more defects coming through after a process improvement effort).
Address the concerns of all stakeholders
- The concerns of all stakeholders should be considered, identified, understood and communicated
- A stakeholder is any person or group that can impact your project. Stakeholders include members of your team (internal and external), customers, partners and suppliers. They may also include regulatory bodies such as the Food & Drug Administration (FDA) or European Medicines Agency (EMA).
Types of indicators used to map change management metrics
The measurement types you will use depend on the purpose of your metrics. The following are some examples that you can use:
Quantitative data
These are numbers and measurements, for example meeting productivity goals. Quantitative data gives you a more precise picture of exactly how much is happening, but it doesn’t help explain why something might be happening.
Qualitative data
This is feedback from employees and customers that is not objective or measurable in numeric terms—for example, asking how satisfied they are with their experience when they use your documentation.
Qualitative data is descriptive and provides insight into what is happening in the process, but it doesn’t tell you how much of it is happening.
There’s a reason why these two types of data are referred to as “diametrically opposed” — they’re both useful for different things. In order for us to really understand what’s going on with our product, we need both types of information about our users’ experiences with it so that we can answer questions like “what are people doing wrong?” or “how can we improve this feature?”
Behavioral data
These are actions taken by employees or customers based on whether they are able to find content faster, or easier. This is the type of information that allows us to see what people are doing in the product and how they’re using it. For example, if we see someone searching for an answer and then contacting the online help center, we can infer that our search results aren’t helping them find what they need quickly enough.
Behavioral data helps us answer questions like “what are people doing wrong?” or “how can we improve this feature?”
The cost of measuring vs the cost of not knowing
The cost of measuring vs the cost of not knowing is small when compared to:
- The cost of making a mistake due to inaccurate assumptions about your data.
- The cost of not being able to predict future issues before they occur in the first place.
The cost of not understanding your data will always be greater than the cost of measuring it. If you don’t know what your users are doing, how can you help them? How much does it cost to measure vs not measure? The cost of making a bad decision that can turn away customers and hurt your business. The cost of not knowing if you are achieving your goals or how close you are to reaching them.
Key tips for mapping change management metrics
As you begin to collect metrics and measure the results of your efforts, you’ll find that they’re a critical tool for measuring and managing change. Metrics help you see the big picture and measure progress towards goals. They also help identify problems, if they exist at all; they can even identify trends that weren’t previously visible.
Metrics are not only useful in measuring the success of your efforts—they’re also important in helping others understand what’s been accomplished by their team or department. In addition, metrics can help inform future planning: if certain types of behaviors are associated with better outcomes on certain measures, then those behaviors become priorities for continued improvement over time (e.g., “We’ve seen an increase in our customer satisfaction scores; here’s why we think it happened”).
A balanced scorecard for measuring success
- Create Critical Success Factors: Your CSFs are the metrics that your team must focus on in order to achieve their goals. They’re the things you want to see get better over time, like customer satisfaction or employee retention rate. Write 1 to 3 critical success factors per domain.
- Identify measurable Key Performance Indicators (KPIs): KPIs are specific measurements of how well those CSFs are working out for your organization. Avoid ambiguity, ensure measurements reinforce the expected behavior, and consider all indicator types. Remember metrics that are useful and metrics that are easy to measure are not necessarily the same.
- Define success criteria for each KPI: These are benchmarks for how good an outcome is considered before it’s considered a win or a loss—if your team has no way of knowing whether they’ve accomplished anything yet!
- Define numerical ranges for quantitative measurements: Quantitative metrics such as profitability require objective values rather than just subjective opinions.
Key metrics to measure for successful change management
Once you have identified the business goals of your content strategy, it’s time to decide which metrics will help you measure its success. There are many reasons why this is an important step:
- It helps identify any gaps in your strategy or implementation plan.
- The data can be used for reporting purposes and/or used in future planning sessions.
- Metrics provide an objective way to measure progress against target objectives.
Measuring Customer Satisfaction
The Goal
Customer satisfaction and loyalty are the most important factors for a company’s long-term success. They lead to repeat customers, which means repeat revenue. Increase customer satisfaction with information products by making the information customers need clear and easily accessible.
Critical Success Factors (CSFs)
- Customers provide positive feedback on content
- Calls to technical support that are answered in the documentation decrease
- Time to locate needed information in content decreases
Key Performance Indicators (KPIs)
- Customer feedback
- Customer survey results
- Net promoter scores
- Technical support call content and numbers
- Web analytics – number of searches, bounce rates, etc.
Measuring Conformance to Business Process
The Goal
Content meets corporate standards and customer expectations for clarity, accuracy, usability, and findability.
Critical Success Factors (CSFs)
- Content sent through all required reviews and approvals
- Content conforms to corporate structure and style standards
- Content tagged for common metadata and search terms
Key Performance Indicators (KPIs)
- Number of errors found or reported in released content
- Time to locate required content
- Comprehension and readability score
Measuring Employee Learning and Growth
The goal
Employees are adequately trained and equipped to do their jobs well.
Critical Success Factors:
- What do people need to know in order for them to succeed?
- How do we measure that knowledge?
- Employees enjoy their work
- Training budget is available and used
- Employees stay informed of industry trends and best practices
- Tools and equipment are kept current
Key Performance Indicators
- Employee satisfaction surveys
- Employee turnover
- Cost per person spent on training
- Age of equipment and software licenses
Baseline your metrics before a change
To begin, you need to know what your baseline metrics are. If you don’t have anything to compare against, then it’s like trying to measure how fast a train is going by only looking at one track: you can’t really tell how much faster or slower the train has been traveling.
There are several ways that companies choose to baseline their metrics:
- Look at previous data from previous months/years and compare them with current data.
- Conduct surveys among customers/clients who have used products/services within a specified timeframe; then use these results as benchmarks for future comparison purposes under similar circumstances.
Measure change management performance using your metrics
Metrics are a tool to help you achieve your goals. They’re important, but they’re not the goal. The goal is to reach the destination—to improve your business in ways that matter to your customers and staff.
With this in mind, metrics are a way for you to measure progress as you work towards success.
Mistakes to avoid
If you get what you measure, then it’s important to be specific in what you are measuring. Avoid not seeing the wood for the trees and looking at the big picture. Additionally, avoid the temptation to improve metrics without first analyzing results.
- You get what you measure
- People adjust behavior based on what they are measured against
- Be specific in what you measure
- To change behavior, change what you measure
You’ve heard the saying “You get what you measure” many times. The idea is pretty simple: if something is measured, then people will adjust their behavior accordingly to improve the metric.
The key here is being specific about what gets measured and how often you do so; otherwise, there won’t be any incentive for employees or teams within your organization.
Avoid not seeing the wood for the trees, look at the big picture
To avoid not seeing the wood for the trees, look at the big picture. Don’t get bogged down in the details, metrics and processes. Don’t get bogged down by technology or tools.
Avoid the temptation to improve metrics without first analyzing the results
The following are some common pitfalls that you can avoid when thinking about what to measure and how to do it:
- It’s easy to get excited by those numbers that look good and make you feel confident in your work, but think carefully before making any big changes based on only a few months of data—or even worse, only one month of data! You might find yourself chasing trends instead of identifying patterns that matter for long-term success.
- Make sure you’re measuring what matters; don’t be tempted by vanity metrics. Don’t waste time tweaking something unrelated just because it looks good on paper.
- Don’t try too hard—and always keep an eye on how much time is required for each metric collection process versus how much information can actually be gleaned from its analysis; this will allow teams to focus their efforts efficiently.
Avoid defining too few or too many metrics
After you’ve identified the metrics you want to measure, it’s important to avoid the temptation of improving them without first analyzing and understanding their results. When you’re determining which metrics are most important for your business, what you ultimately want is a clear picture of how each one affects your bottom line and overall success. This will require some research on your part—you’ll need to identify which numbers matter most and why they do so in order to ensure that you’re making data-driven decisions when it comes time for evaluation.
When done right, this kind of analysis can help illuminate where improvements are needed while also providing an objective perspective on what’s working well within your business model or strategy. The end result is a much clearer picture of where your company stands and how close—or far—it has come from reaching its goals since implementing those changes in actionable plans going forward!
Conclusion
As we’ve seen, there are many types of metrics that can be used to measure and improve your content strategy. For example, you could track sales figures, customer satisfaction ratings or customer referrals—all valuable inputs for improving your tactics.
The key is to use metrics that are relevant to your business goals and objectives. It’s also important to choose metrics that can be easily tracked, measured and analyzed over time.
To get a full picture of how your change is going, you need to look at multiple metrics and use them in conjunction with each other. The key is being able to understand what each metric means and how it relates to your overall goals for that particular piece of content.
However, the key is to identify the metrics that are most important for your business, and then track them carefully. This will help you determine whether or not your content strategy is working—and if not, why not?
About the presenter
Dawn Stevens
Dawn Stevens is the President and owner of ComTech Services. In her 35-year career, Dawn has performed virtually every function within a documentation and training department, including project management, instructional design, writing, editing, and programming. Holding both engineering and technical communication degrees, Dawn combines a solid technical foundation with strong writing and design skills to identify and remove the challenges her clients face in producing usable content.
