By Nate Brown
In today’s age of remote user insights, the test script lies at the foundation of all research studies. The script drives everything in a study session regardless of the testing methodology. In a moderated research environment, the test script is a guide through the interview that keeps the moderator on track. However, it allows the researcher the ability to improvise and go off course in order to explore unplanned topics in the conversation. Unmoderated testing presents a whole different level of test script responsibility. Since there is no moderator to explain instructions or poach further for questions, the study participant relies solely on the test script to guide them through the research study.
In this article, we will explore the 3 main steps to creating a successful test script for any user research study.
Step 1 - Identify the Purpose
The first question one should ask themselves when creating a test script is, “what is the purpose of this research?” It is first and foremost important to understand why the research is being conducted. Maybe a designer needs some feedback on where to place a call to action button in a prototype, or whether the messaging of their onboarding process is confusing to consumers.
Once the researcher has identified the main goals and outcomes of the research, they will want to begin implementing these into their test script questions and activities. Sometimes researchers are afraid to bias the study, and for good reason. But if the questions or instructions do not target the main study goal, how are the participants supposed to know what kind of feedback the researcher is looking for? There is a balance between questions that are too general and ones that are too specific.
Bad Question Examples
Too General - “This is the website homepage, please give your thoughts & impressions.”
Too Specific - “What do you dislike most about the blue and white color scale of this homepage?”
In the first example, the range for available feedback is very wide. In a study of many participants, the responses will likely be hard to compile into trends that can be used as insights. General feedback, however, can still be valuable. This is especially true if the team is at an early stage in the design process where a more generative research approach is required. In the ladder example, the specificity of the question leads the participant to a negative response. Over specificity is almost always to be avoided unless the UX issue being tested is already well defined and previously test for.
Step 2 - Involve Key Stakeholders
Take time before creating a test script to digest how the research insights are going to be used. Understand what stakeholders will be reviewing the results, and make sure the questions and metrics being included in the test script will convey the insights most effectively. Stakeholders tend to land in two buckets, internal and external.
Internal Stakeholders - These are individuals within the organization. They include designers, product managers, and UX researchers. Whether the internal stakeholders are working directly in the research or simply have a hand in the creation of the asset, their thoughts and goals need to be considered. When forming the test script purpose in Step 1, include internal members as they likely had certain reasons for creating the asset the way they did. This is invaluable information to understand before creating a test script.
The other type of internal stakeholder that is often overlooked in study outcome formulation is business leadership. These are individuals in director or senior leadership positions. Their goals are often much broader and relate to the key performance indicators (KPIs) of the business. A major objective of user experience research is validating why it is useful in the first place. Make sure your main research goals include outcomes that are important to company leadership. At the end of the day, these are the individuals allocating budget to the research team.
External Stakeholders - This type of stakeholder is normally relevant to freelancers and agency researchers. Anyone who is creating assets for a client and needs to show value in the research being conducted. In this scenario, understand that the insights gained from the study need to be translated into layman’s terms so that the end client can understand what idea has been formulated. This is not to say a researcher should build the test script to fit only the client's goals, the focus should still be to make sure the research is geared toward identifying the key UX issues. However, it’s important to include a mix of qualitative and quantitative research in the study. This allows for a mix of visual data representation and the emotional qualitative feedback. A research platform like Userlytics allows researchers to collect both in the same study and represent them with data metrics and video-editing software built directly into the tool.
Step 3 - UX Your Test Script
Once the groundwork has been laid, it is time to finally start building the test script! It is interesting that many UX/UI designers are so geared toward usable assets, yet they build test scripts that are clunky and difficult to understand for the participants. This is more prevalent in unmoderated research because there is no moderator to guide the participant through the study. The user is solely reliant on the test script to understand what they are supposed to do and what kind of feedback they are encouraged to give.
The term “UX your test script” is an excellent way to conceptualize how to structure a test script. The easier it is to understand, the better likelihood the participants will give relevant observations. Here are a few key points to remember:
Be Human - Remember that the participant taking a study is coming in cold. Besides possibly the screener questions, the participant doesn’t have any background on the asset or the research being conducted. Build test script questions as if speaking directly taking to the participant. Be clear and concise, and use bullet points to break up text paragraphs so the information is easier to digest.
Start It Off Right - Begin your test script by informing the participant about the study and what to expect from the asset they will be testing. This is especially important with prototypes as many have specific click paths and don’t interact like normal websites. Avoid dropping the user into the asset as soon as the study begins. Link the asset to the participant when they are on the specific task that is requesting feedback. This can help them stay on track with where they are supposed to be on the asset in relation to the test script.
Ask Them to Speak Aloud - Instruct participants to give verbal feedback if the desired outcome is qualitative feedback. Do this early and often, especially at the start of any new testing flow. Participants who do usability testing often know this intuitively, but many participants (especially ones outside of platform panels) likely do not know what is expected of them in a research study.
Include Refocusing Tasks - Many times in a test script, all the instructions for the participant are loaded into the beginning of the study. For usability tests that are often thirty to sixty minutes long, it is unlikely the participant will remember the instructions from the beginning of the test script once fully immersed in the study. Try to break up the test script and have tasks built-in to recenter the participant on where they are supposed to be on the asset and what they should be doing. This is especially important if you have redirected the user to a new asset or webpage.
Conclusion
This article places a majority of the focus on the aspects of the test script that are outside of the actual building process. That is because there are many articles on the web that already describe good tips and tricks for building a script. However, many of them miss the subjective factors outside of the building process that determines the true success of a research project. Having a clear direction is incredibly important to making sure research goals are being attained. It is also invaluable to ensure those research goals align with the business goals of the organization. A company that is aligned with its research efforts can build customer empathy into all aspects of the business.
If you have any questions or would like to discuss research strategy together, please feel free to contact me at nbrown@userlytics.com.
Comments