Can You Conduct Effective and Safe UX Research During a Pandemic?

During the pandemic, our clients need to carry on business as usual – delivering on projects, developing products, and enhancing their users' digital experiences. This cannot be done responsibly without proper research, yet the current situation means researchers have been faced with entirely new challenges.

How do we deal with these challenges at e-point? What can we offer to our clients in these trying circumstances as far as research is concerned?

Challenges

The challenges researchers encounter during the pandemic can be divided into two groups:

• Recruiting user testers, including signing contracts, accepting GDPR declarations, and settling fees.
• Conducting research remotely using respondents' devices – particularly mobile devices.

Recruiting Testers

To a certain degree, we’ve always done some remote recruiting of research participants. We’ve used our own network of contacts or requested some support from an agency that made appointments with respondents over the phone. The Covid-19 situation – which turned even such an everyday activity as signing recruiters' contracts into a challenge – encouraged us to look for new solutions.

Recruiting survey participants through online channels offers plenty of new opportunities that can be productively tapped even after we get back to normal. The ability to target advertising messages in social media allows our survey invitations to reach those people who most accurately meet the criteria (e.g. as to age, place of abode, family background, and interests). Combined with online forms, which enable us to better understand and screen potential respondents, this helps us be sure that we will talk to people who are really close to the project’s target user group.

So far, so good. But what about surveys’ legal requirements? One must sign a contract, collect GDPR, registration, and/or image use consent (if the survey is recorded) and finally compensate the tester for their time.

The easiest option is collecting such data through online forms or via e-mail; statements of intent submitted online are binding. With a view to our respondents' convenience and security, we’ve been examining the possibility of applying other tools (such as podpisz.to) or creating our own solution. Either one may certainly come in handy in the future.

Needs Analysis

Participants have been recruited. What happens next? As a matter of fact, conducting research such as individual in-depth interviews and focus groups remotely does not present any challenges. We use universally available and simple to apply solutions, including Google Hangouts or Skype (for interviews) and Zoom (an alternative to Google Hangouts for focus groups).

Before a specific solution is selected, it should be checked for the following qualities:

• Stability and Reliability. There is nothing worse than a connection that keeps breaking down.
• Ease of Use. Solutions should be widely recognized and comfortable for the testers to use.
• Presentation Options. A presentation may be the starting point for some questions, so the solution needs to support this feature.
• Ability to Record Sounds and Images. This makes further analysis much easier.

Naturally, we are not constrained to simple conversations while performing research. If we want to introduce more creative techniques, such as elements of brainstorming or project assessment, remote collaboration tools like Mural or Padlet can come in handy. Don’t forget, however, that each new tool adds more complications for those surveyed and calls for more effort on our part (to effectively moderate the session).

In the case of focus groups, there’s a need to brief each participant individually to prevent confusion while using the tools. A quick training session on the use of the tool and communication principles is recommended at the start of every survey; in time, it will become as common as introducing oneself.  It also seems reasonable to send emails containing information about the meeting as well as brief instructions. This builds security and confidence within the respondents, which is key to a successful study.

Best practices used offline work online, too. As long as it is practical, we should be running surveys in pairs. The moderator runs the survey and the observer (who is silent and invisible for most of the meeting) takes notes during the process. After each research session, it’s good to discuss key observations right away.

The diary survey is another type of needs analysis that can be conducted during the pandemic. These surveys are irreplaceable in testing customer journeys or other service design activities. It’s fairly problem-free; due to the nature of this method, it can be easily carried out remotely. Traditional, paper-based methods may be employed, accompanied by online questionnaires. Or additional tools such as ExperienceFellow may be chosen; these make the life of a researcher much easier.

In this case, an introductory or summary meeting may be held over the phone or video conference. What remains unclear, though, is the value of the collected data. Remote work, school lockdowns, reduced mobility – all of these put respondents in a very awkward situation. As long as we haven’t designed a tool to deal with crisis situations, diary surveys should be postponed until calmer times.

Usability Tests

Usability tests – both on prototypes and on implemented products – can be conducted on desktop devices using the screen sharing functions of popular video conferencing programs like Google Hangouts or Zoom. The situation gets a bit more complex if the survey is run on smartphones. We are currently testing tools that allow for easier integration with interactive prototypes and enable convenient testing on mobile devices, e.g. Lookback, UserTesting, and Maze.

No decision has been made yet about which tool is the most suitable. The following issues may occur:

• There may be no version of that software in the local language. 
• Testers may need to install and learn to operate the software (in addition to the test).
• Some demographic groups do not have access to desktop or laptop computers; some mobile research software requires the use of a combination of computer and smartphone.
• Specific requirements as to equipment parameters, Internet connection, and system version may exist for some tools.

All this means that recruitment criteria must be expanded for these elements, subsequently making the testing of some groups much more difficult.  

We should also resist the temptation of running surveys for applications and mobile services on desktop or laptop computers.  Despite the fact that InVision or even Google Chrome’s 'explore the site' mode allow users to simulate mobile views, this is an altogether different experience.  In short, never test 'tappable' solutions on 'clickable' devices!

Moving research to the Internet also presents an opportunity for befriending some slightly neglected research methods. Since the large-scale use of tree-testing methodology on the PZU.pl portal, I’ve become a great fan of the Optimal Workshop UX research tool. It’s also extremely useful for quick first-click tests or card sorting, and its options for sharing results and joint data analysis work well with Slack and Google Hangouts.

Remote UX Research Can Work

We have been expanding our UX research toolkit really quickly. We try to make sure the local and global situation does not affect the credibility and reliability of the research we conduct for our clients.

Has the pandemic made life more difficult? Certainly; it was particularly tough at the start. However, we view the new situation as an opportunity to develop innovative methodology that better accommodates the challenges that come with trying times.