Design & Product

How to Effectively Run Remote Usability Tests for Your Product

Dive into the steps for setting up tests, picking the right tools, and recruiting the best participants—all from the comfort of your own space!

Ben
Benjamin Tey

UX Researcher

How to Effectively Run Remote Usability Tests for Your Product

Remote usability testing can help you gather valuable insights about your product and target audience from different geographical locations without any limitations. They also cost just a fraction of traditional usability tests, and are more convenient for participants!

Not sure what remote usability tests are? They refer to tests that are taken while participants and researchers are spread across multiple geographical locations. This is different from in-person usability testing, where participants would be in the same room as researchers.

Discover What Your Users Really Think About Your Product

Elevate your product by tapping into genuine user experiences! Start using UXArmy for effective remote usability testing

Steps to Conduct Remote Usability Testing

With improvements in technology and innovation, as well as the availability of research platforms like UXArmy, remote usability testing can either be moderated by a researcher or self-reported (unmoderated usability testing).

Step 1: Choose Your Testing Approach

There are 2 forms of remote usability testing – moderated and unmoderated. 

Remote moderated usability testing has a similarity to an in-person study, where there is real-time interaction in a session involving a researcher and participants. The researcher and participants will be communicating with each other at the same time, but they are not in the same physical setting. Remote moderated usability testing sessions can involve more than one participant at a time, such as in a dyad, triad or focus group. 

Remote moderated usability testing is usually conducted via a video-conference tool. This form of testing works well when there is a higher budget or more time available, and when researchers need to ask participants follow-up questions in present time.

Scenario: You are a researcher living and working in the United States, and want to conduct a research interview with a participant from your target audience in India.

Remote unmoderated usability testing does not require a moderator to be present. The researcher will create tasks and instructions for participants via a remote-testing tool, while the participant attempts the test on their own at their convenience, while recording their screen and voice, whichever is deemed necessary by the researcher. 

Remote unmoderated usability tests involve only one participant at a time, although many individuals could be taking the test at the same time.

This form of testing brings about some advantages is useful when:

  • Results can be gathered quickly
  • Feedback is given in natural context and environment of use
  • Hawthorne effect (when individuals modify their behavior when they are being observed) can be eliminated

Scenario: You are a researcher based in Europe, who wants to test out your high-fidelity prototype of your app quickly, in many places around Southeast Asia where people are living in a different timezone.

A mix of the two methods can be used as well; it is not necessarily one or the other. Sometimes, a combination of the two approaches can complement findings and meet research objectives.

 Step 2: Pick the Right Tools for Your Usability Test

There are many options of remote usability tools to choose from, depending on your budget and research needs. We recommend choosing tools based on the functionalities and features you require for your project.

Here are some of the best tools for remote usability testing:

  • UXArmy
  • Maze
  • UserTesting
  • Optimal Workshop

When choosing a tool, pay attention to reviews, installation requirements, pricing plan, and features they offer.

 Why Should You Choose UXArmy?

UXArmy offers a whole suite of tools that collects qualitative and quantitative feedback in a quick and cost-effective manner.

The UXArmy platform offers a range of solutions:

UXArmy also offers recruitment services with a panel of tens of thousands of verified testers.

Step 3: Plan and Design Your Tasks

Before planning the remote usability test tasks or questions, ensure that:

  • You define a test objective, which allows you to narrow down the scope of the test and focus on the results you aim to achieve.
  • Tasks or questions lack bias and assumptions
  • Remaining neutral in tone
  • Avoid leading questions that might influence participants to answer in a certain way, that might skew results or compromise data integrity.
  • Example of a leading question could be: “How much did you enjoy our app?” This question assumes that participants enjoyed the app, and only encourages them to describe the enjoyment they have received; it does not allow them to mention that they did not enjoy the app.

Based on your target audience, it is important that remote unmoderated usability tests are prepared and catered to their level, especially since there will be no moderators to help or provide guidance. One way to ensure this is to create tests in the local language of your target audience. Also make sure simple language is used, instead of technical jargon.

Step 4: Conduct a Dry Run

A great way to conduct an effective remote usability test is to first perform internal dry runs (or trials/rehearsals) of the test script and tasks. Dry runs are done to ensure that tasks make sense to participants, all links or prototypes used in the test are working, and no questions are missed out. 

For remote moderated usability testing, test scripts should be rehearsed to make sure all questions and tasks can be completed within the planned time frame. Doing a dry run could help a researcher be more efficient after familiarizing themselves with a new tool, or prevent errors on the actual day after getting used to the usability test guide..

For remote unmoderated usability testing, doing dry runs before launch are crucial since participants will receive no help or guidance when they take on the tests at their own time. 

  • Go through each task as though you are the participant
  • Look out for any grammatical or spelling errors, and make sure there are no confusing task instructions or wrong ordering of questions
  • Ensure that all tasks in an unmoderated test can be performed on multiple devices
  • Completing the dry runs would provide you with more confidence to launch the test to real participants
  • Conduct a brief walkthrough of the tool or platform with participants, and provide a guide on how to take usability tests effectively, so that you can achieve quality results

 Step 5: Recruiting the Right Participants

To achieve your research objectives, it is key to recruit the right participants for your remote usability tests. 

  • Define concise eligibility criteria so that only a suitable profile of participants can attempt your test.
    For example, if you are looking for participants to take an unmoderated usability test of a food delivery app, “people who use food delivery apps” may not be concise enough. Instead, consider “people who have used food delivery apps more than once a week”.
  • Good screening questions are essential too. Ensure that the questions do not give away the purpose of the research, in the way that participants can game the system.
    For instance, a Yes/No question such as “Do you use social media?” might reveal that the research is about social media, hinting participants to choose the desirable answer of “Yes”. Instead, consider a multiple-choice question such as “What are your favorite activities?” with options “Playing Sports, Reading, Using Social Media, Cooking etc.”
  • Questions with exclusion criteria should be arranged at the start so participants who are not suitable can be filtered out early.

You can also utilize a trusted user panel to source for the right participants. UXArmy provides versatile and comprehensive recruitment services with quick turnaround, for both moderated and unmoderated user testing.

Step 6: Analyze and Act on the Results

Upon the completion of your usability test, you can proceed to analyze and act on the results.

  • Based on the test report, organize the results based on the usability issues identified (You can do so through UXArmy’s Analysis Space, where you can tag issues, create video clips and consolidate them into a highlight reel)
  • Arrange the usability issues based on their criticality and importance
  • Brainstorm on solutions with team members or stakeholders
  • Prepare solutions and test them out with users again for feedback and iterations to improve your app or website

Elevate Your UX Research with UXArmy

Join countless professionals in simplifying your user research process and delivering results that matter

Additional Tips

  • Add 15%-20% more participants than the number of responses you want to draw insights from, for unmoderated usability tests
  • Throughout a research project, you may realize that not all participants will provide valuable insights – adjust and tweak the screener to gather better participants

To ensure you are making the best use of time and resources, seek advice or support from trusted sources: UXArmy has a qualified team of highly experienced user researchers and usability experts that can assist you in deciding on methodologies for your research projects or provide advice for recruitment

Unlock Exclusive Insights

Subscribe for industry updates and expert analysis delivered straight to your inbox.