OpenELIS

Project Details

Role

UX Research & Design

YEAR: 2021

Team

Beth Dunbar
Austin Kohnle
Jisu Park

Tools

Figma
Miro
Zoom
Google Suite

Usability Study Goals

Primary Goal

Identify user pain-points in

• ordering a test, and

• filling out test results

Secondary Goal

Overall perceptions in software usability

My Contributions

In terms of the scope of my work in this project, I contributed to all the aspects and milestones including study design, creating screeners and questionnaires, recruiting, usability test protocols and scripts, moderating user interviews, data collection, tabulation, and mixed methods data analysis. 

Study Design, and Recruiting

Collaborated with OpenELIS stakehoders and fellow researchers in building preliminary screeners and questionnaires for potential users. I also designed usability test protocols and wrote scripts.

Semi-structured User Interviews, Moderation, and Data Collection

Interviewed laboratory technicians and moderated physician usability tests. Collected data through video recordings, pre-and post task questionnaires.

Mixed Methods Data Analysis

Coding qualitative themes and key insights using Affinity Mapping. Analyzing test results using System Usability Scale (SUS), and Net Promoter Scores (NPS). Synthesized results, and prioritized usability issues.

Study Overview

The OpenELIS project team & developers requested a usability study to obtain formal user feedback for the first time. The stakeholders wanted to understand how to optimize software features, with specific interests in the two main areas of the software: lab ordering and results entry. Thus the usability was study aimed to:

1.  Identify user pain points in ordering a test and filling out test results,

2.  Assess the overall effectiveness of OpenELIS in supporting lab technicians and staff, including identifying bottlenecks and opportunities for software enhancement and optimization. 

Methods

Mixed Methods Process

•  Conducted virtual usability studies using Zoom (2/26/21 - 3/5/21)
  • with 1 moderator and 1 data collector for each session
•  Qualitative analysis
  • Semi-structured interviews
  • Quantitative analysis
• Quantitative analysis
  
  • SUS (System Usability Scale)
  • NPS (Net Promoter Score)
    

• Affinity diagrams to find key insights

• Sauro's severity scale for ratings

  • Critical: 4-5 participants reported it and led to task failure/leads to user irritation

  • Moderate: 2-3 participants, caused occasional failure delay or moderate irritation

  • Minor: 1 participant, caused some hesitation or slight irritation

  • Insight/Suggestion/Positive: Users mentioned idea or observation that does or could enhance the overall experience.

Participant Details

• Five international participants of medical professionals 
  
  • 1 physician
  • 4 laboratory professionals

Target Users

Primary, and Secondary Users

We interviewed, and tested the OpenELIS softwate with physicians and clinical laboratory techs in Seattle, and Cote d'Ivoire.

1      Clinical Staff

Physicians and licensed clinical laboratory managers are the primary target users who often oversee reference, and clinical labs. These are normally the super users who have more experience, with the capabilities of training a novice user.

2    Laboratory Technicians

Laboratory technicians who either retrieve or input lab results are the secondary users. These users have limited or no experience with laboratory information systems.

Navigation: Homepage / Entry- Point

Navigation: Test Request Landing Page

System Usability Scale (SUS), and Net Promoter Scores (NPS)

• Perceived usability of OpenELIS on the SUS scale was 67.4, this is 0.6 below the mean SUS score of 68. 
• Range values were (53 min - 85 max)
• Adjective rating was (OK). A good way to evaluate Adjective rating is comparing it to Acceptencence rating (Marginal). 
• There were three detectors (P1, P2 & P3). One passive participant (P4) this can be defined as uncertainty or on the fence of using a product or service. Lastly, there was one Participant (P5) was a promoter that can be defined as a person who would use a product or service as well as be likely to recommend it to friends or colleagues.

Source: Interpreting SUS Scores, MeasuringU by Jeff Sauro, Ph.D

Major Findings

From semi-structured interviews and post-test SUS surveys with five participants (3 new users / 2 existing users) from four countries, United States, Haiti, Malawi, and Cote d'Ivoire.

Areas of Strength

Software is functional (after a learning curve)
Users report moderate to high ease of use

Areas of Improvement

Navigation
Home Page / Entry Point (critical)
Test Request Landing Page (moderate)

Localization
Test Sample Site Name (critical)
Test Requester features (moderate)

Discoverability
Sample Type Selection (critical)
Search Buttons (critical) 

Reflection

What went well:

• Cooperation with client (targeted recruitment, natural test scenarios)
• 5 usability study sessions with international participants

What we would improve:

• Recruiting extra participants
• Time budget suitable for new users and less tech-savvy participants

Excitement from I-TECH DIGI team:

• Their first time hosting a usability study
• Eager for more HCDE student connections: practicum or portfolio projects

Raf Laus, Portfolio. https://rlaus.xyz. © 2024