User testing and content management plan

User testing is a vital part of the development process. There are three key types of user testing:

1) Functional testing 

To ensure that the navigation, features and functions of the toolkit work as intended.  This is especially important to ensure that complex functionality such as Question and Answer tools and scoring tools are accurate and reliable.

2) Usability testing

To:

• Identify problems in the design of the toolkit
• Uncover opportunities to improve
• Learn about the target user’s behaviour and preferences

3) User acceptance testing

Testing the toolkit in the real world by its intended audience. 

Find out more

The Right Decision Service Implementation guide and standard operating procedure provide guidance and tools for each type of user testing.  These include the System Usability Scale (SUS) and Mobile app Rating Scale (MARS) for usability testing. You can find out more about usability testing at the UK Government usability testing website and at the Nielsen Norman user experience website.

Testing 'in development' toolkits 

You are required to comply with the standard operating procedure for 'in development' toolkits during the testing phase. This involves:

• Ensuring that "in development" or "in devt" is added in parentheses to the toolkit title.
• Hiding the toolkit from web and app as much as possible while it is under development. If it needs to be made visible for a short period for testing purposes, please apply password-protection. 
• Keeping any published time while in development and password protected to a minimum. Finding large numbers of 'in-development' password-protected toolkits that are not ready for use is frustrating for the many users not involved in the development process.

RDS toolkit request form part 2 must be completed before you progress to the next stage of requesting sign-off and go-live for your toolkit.

This governance form is based on the standard operating procedure for managing the content lifecycle for RDS toolkits. It aims to ensure that robust plans are in place for ongoing governance and updating of the toolkit content. 

Specifically, this form asks you to confirm responsibilities and processes for:

• Review and sign-off of content at time of initial publication
• Review and sign-off of initial delivery of the content via the Right Decision Service platform. This includes checking the accuracy of the content if transcribed from a written document into digital format, and also confirming acceptance of the toolkit structure, design and functionality.
• Proposing updates and signing off proposals for updates to the content after initial publication, out with the scheduled review dates.
• Review and sign-off of updated content via the RDS, out with the scheduled review dates.
• Determining the review date(s) for content.
• Receiving the automated alerts from the Right Decision Service when the content is due for review, and ensuring that the review takes place in a timely manner.

A key requirement for all RDS toolkits is to provide publication and review dates. A warning will appear on any toolkit page that exceeds the designated review date. In addition to the automated alerts sent as reminders that reviews are due, if these governance requirements are not met, the RDS team may contact you to ask you to update the content, or alternatively to provide documented confirmation that it is still safe to use. If necessary, the RDS team may ask that you remove the content or may do so directly in order to ensure compliance with RDS governance standards.

The Right Decision Service Implementation guide and standard operating procedure provide guidance and tools for the three types of user testing. 

Two key usability testing tools are: the System Usability Scale (SUS) and Mobile App Rating Scale (MARS)

You can find out more about usability testing at the UK Government usability testing website and at the Nielsen Norman user experience website.

RDS toolkit request form part 2 asks you to confirm your plans for content maintenance and updating. You should complete it in conjunction with reading the standard operating procedure for managing the content lifecycle for RDS toolkits.

Please share your experience in using any one of the three testing approaches outlined in the Implementation guide - Functional testing, Usability testing and User acceptance testing.  For example, you might try:

• Constructing and applying test cases for functional testing
• Using the System Usability Scoring tool or the Mobile App Rating Scale for usability testing.
• Asking end-users to work through test scenarios for user acceptance testing.

What was helpful about the process and the learning you gathered from the feedback?

What would you like to learn more about or gain more confidence in, and how could you go about this?

If you would like to be put in touch with an RDS toolkit manager or editor to share learning and ideas, please contact his.decisonsupport@nhs.scot.