Metrics that matter

Metrics are important.  These are the ones that you should be paying attention to.

In a rapidly evolving digital landscape, designing the perfect intranet is a challenge that many organisations face.  These days, aesthetics is a given, especially with the new Office 365 interface providing a rich and easy editing interface, so it’s easy to build a great-looking intranet.  Is it actually a good and useful experience for your users?

That is the right question to ask.  This article explains how I go about gathering these types of metrics and why.

Intranet metrics are very different from website metrics.

Website metrics reports generally contain overall traffic (which measure popularity), bounce rate (which measures single-page visits) and conversion rates (visitors who were converted to customers).

Intranet metrics are about the users and how effectively they can get their work done on the system, and how happy they are in using it.

Getting the right metrics

Based on my experience, I’ve outlined below what works when gathering feedback from users as it relates to an intranet.  I should mention the importance of “zeroing out” a group of people so everyone is in a similar situation is pivotal in gaining productive and relevant feedback.

Everyone’s day unfolds differently from everyone else’s.  Maybe they were just in a difficult meeting, or maybe they had to handle a difficult customer.  Zeroing out the room is a practice where you talk to people about why they are gathered and try to get them all on the same mental level to effectively get them in a better mindset than where they may have come from.

Jumping straight into a feedback meeting is not always advisable.  If users are completing this survey from their desks on their own time, a fun welcome screen can also help.

Separate out complaints from the Dream

Structure your user sessions very carefully to not muddy the waters with complaints.  A complaint from a single user can strike a chord with multiple others, which can easily cascade into agreements on how terrible the current system is perceived.

Instead, allow people to provide structured feedback on the current system.  Then in a separate event (or after a break), zero out the room and then have people dream about a brighter future.  Encourage them to speak in positives (it should… instead of it shouldn’t…) and keep steering discussions towards what users dream of having if they had to start from nothing.

Ensure your users’ dream is not just better by comparison but truly a bright future.

Ask the right questions in a digital feedback survey

Get some basic information about each user to allow follow-ups if necessary.  An email address is generally enough.

Understanding pain-points in the current system can be helpful, but will generally surface when you allow users to dream.  Ask them questions that do not have predictable answers, so you can focus time spent on the survey on productive questions.

• What content does this user access most often?
• Is the format of the content easy to digest and visually scannable?
• Is the content relevant?
• How often does this user use search?
• How often does this user read the corporate newsletter?

Weight your answers

You can also ask additional question to help weight other answers, e.g. if you want to know how often people read corporate news, you are interested in the answer from someone who is actually interested in reading it rather than someone who generally won’t follow news (since their answer is less relevant).  Think about the questions about relevancy of an answer that may arise if users feel strongly about something that may surprise stakeholders.

• How often does this user use social media?
• Does this user bring their mobile phone to access social media or the internet?
• How tech-savvy is this user?

You may also want to weight content answers on the department the user belongs to (someone in IT may handle a lot more system complaints than their peers in HR).

Make it fun

Ask users questions in a way that makes them smile and enjoy the process rather.  Keep the important answers clear and concise and have fun with the throwaway answers, e.g.:

How tech-savvy are you? (slider below from 5 – 1)

• 5: I have all the latest tech
• 4…
• 3…
• 2…
• 1: I still own a pager

Ensure answers are easy to process

Whether you’re gaining feedback in-person or via a digital survey, structure your questions to use multiple choice or sliders (1 – 5).  Providing a visual results chart is usually a requirement for stakeholders and having to manually alter answers to have them fit into measurable results can be time-consuming and difficult.

Additional survey resources

I like the Google HEART Framework, paired with The Goals-Signals-Metrics process, which measures:

• Happiness
• Engagement
• Adoption
• Retention
• Task success

And is then processed with

• Goals
• Signals
• Metrics

You can read more about it here:

Google HEART Framework, paired with The Goals-Signals-Metrics process
https://library.gv.com/how-to-choose-the-right-ux-metrics-for-your-product-5f46359ab5be

I also use the System Usability Scale, which was developed in 1986 and is still relevant today.  I alter the answers slightly on this to make it relevant to the project, but it’s important to keep the essence of the questions the same, since the data is processed using a very specific scoring mechanism.

You can read more about it here:

System Usability Scale (SUS)
https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html

In Summary

You can gain great metrics by asking the right questions to relevant users and getting users in the right mindset to answer them productively.