Date of Project:
February 2021 - July 2021
Role: Recruiter &
User Researcher
Methods Used:
Human Factors Evaluation, Moderated Usability Testing, Task Analysis, Surveying, Qual & Quant Analysis
Deliverables:
Research Report for Aeragen and the FDA
About the Project
The COVID-19 global pandemic shook the world in 2020. People were infected at an alarming rate, hospitals were overrun with patients and many hospitals did not have sufficient PPE to protect their healthcare workers- many of which contracted the virus and/or developed the disease as a result of their work. This placed an additional burden on health systems as their patient counts grew and the number of available staff declined.
While some better-funded hospitals were fortunate enough to already have negative pressure rooms built, countless others were not as fortunate and were forced to be resourceful in rooming and treating their patients.
Parkview’s Director of Simulation and Innovation connected the UX Team with a startup working on a device to support negative airflow barriers for patients with respiratory infections when negative pressure rooms are unavailable. In order to submit a proposed medical device for Emergency Use Authorization with the FDA, a Human Factors Evaluation must be completed and the findings reported to the FDA.
The Problem
Negative air-flow hospital rooms were critical in the thick of the COVID-19 pandemic as they safely isolated infected patients from other non-infected patients while they were treated for the disease.
AerVENT is a mobile isolation tent that creates a localized volume of negative pressure around a patient, filtering viral particles that are expired by the patient and providing negative pressure space where this is typically unavailable. One example use case is instances of clinicians treating or transporting patients with suspected or confirmed cases of highly infectious respiratory illnesses and smaller scale or low-funded hospital settings where negative pressure rooms are limited or unavailable.
The COVID-19 pandemic was the catalyst for the invention of AerVent since negative pressure isolation rooms were few and many facilities struggled to convert their rooms or lacked the resources to do so. Although this idea was conceived during the COVID pandemic, it can be used for treating patients with other infectious respiratory diseases.
Goal & Objectives
As Human Factors Evaluation was a requirement for FDA Emergency Use Authorization, our goal was to evaluate the usability of the device in real-world settings. The study was conducted in a controlled lab setting at the Advanced Medical Simulation Lab.
We aimed to:
Evaluate interactions between users and the device
Uncover any usability issues and measure criticality (via task analysis, time on task, and task completion/ meeting our established success criteria)
Evaluate any potential hazards or use-related risks for intended users that could result in serious harm OR be mitigated through design modifications or user training.
For example, what, if anything, would hinder clinician workflow in high-stakes medical scenarios resulting in putting themselves or other patients at risk of exposure?
Report on the following per FDA guidelines:
Summary of known use issues/ potential hazards
“Critical tasks” to be performed using the device
Analysis of hazards and risks of using the device
Results of Human Factors Validation Testing
My Role
The team conducting this HFE was comprised of 2 UX Researchers, 1 Project Lead, and 5 Simulation Lab Technicians. As a Researcher, I was responsible for:
Facilitating client meetings for requirements gathering
Coordinating meetings with hospital clinicians to determine “critical use” scenarios for device testing
Recruitment and scheduling of participants
Pre- and post-test surveying
Qualitative and quantitative data analysis
Reporting per FDA guidelines
Methodology
Critical Task Identification
To identify critical tasks to be performed during the testing of this device, we consulted with advisors recommended by Aeragen along with Parkview co-workers with clinical expertise in various areas including Cardiology, Emergency Care, and Neurology.
The Emergency Department, Intensive Care Unit, and Acute Care Unit were identified as important use scenarios to test the critical tasks given the potential for harmful exposure to COVID-19 in these environments.
We opted to test three simulated scenarios for each of these clinical contexts and examined the participants performing these tasks with and without AerVENT for a total of 6 simulated test scenarios (for more, see the Usability Testing section).
Critical tasks were selected based on the likelihood to be used in emergencies or as a routine part of daily patient care and the likelihood to pose a risk for the patient, should AerVent interfere with the healthcare provider’s attempt to complete these life-saving procedures.
The critical tasks identified for our Human Factors Evaluation included the following:
Intubation
Extubation
Bi-level Positive Airway Pressure (BiPAP) Ventilation
Cardiopulmonary Resuscitation (CPR)
Patient Transport to and from different areas of the medical facility via hospital bed
Upper Body Assessment (eyes and chest)
Patient Bed Bath and Linen Change
Method Selection
A pre-test expert analysis of the hazards/ risks of using the device was conducted by myself and my research partner.
A Human Factors Evaluation was recommended in the FDA’s October 2020 webinar “Respirators and Other PPE for Health Care Personnel Use During COVID-19 Pandemic” to evaluate interactions between users and the proposed device.
We applied simulated use testing within our medical simulation lab utilizing high-fidelity medical manikins to control the critical tasks and provide a more standardized measure of usability issues and mitigate participant risk of exposure to COVID-19.
This design allowed us to capture time on task with and without AerVent and participants were given standard questionnaires following each of the 6 simulated scenarios (with and without the device) to evaluate perceived ease of use using a common scale.
Recruitment
Participants were recruited using snowball sampling based on specific inclusion criteria to provide a distribution of experience levels for each role across each session.
In addition to being 18 years of age or older and an actively practicing healthcare worker, there were specific inclusion criteria for each scenario (i.e. specific job roles were required to perform certain critical tasks).
Teams were comprised of 3 participants in roles consistent with a typical team formation in a realistic clinical setting (Physician, Nurse, and Patient Care Tech) across the Emergency, ICU, and ACU departments.
A total of 12 sessions were conducted with 36 participants.
Test Environment
The Medical Simulation Lab possessed all the features required for Human Factors testing. In the test area (simulation rooms modeled to scale after actual rooms at Parkview Regional Medical Center), the UX Facilitator led the participants through device training and a brief of the test scenarios.
Meanwhile, in the control room behind a one-way mirror, the participants’ interactions, reactions, and feedback were both recorded and observed in real-time by myself and our clinical subject-matter experts via a live AV feed (images to come).
Moderated Usability Testing & Questionnaire
The UX team conducted 4 simulated test sessions replicating real-world medical scenarios in 3 simulated clinical settings (ED, ICU, and Acute Care units).
Participant teams were comprised of 3 roles consistent with a typical team formation (ED DR, RT or Anesthesiologist, RN, and PCT).
Data from the testing sessions collected both in-situ and from recordings included qualitative observation notes and task times. Written responses to both open and close-ended questions were also collected from the participants following completion of the simulated scenario.
My team in the control room looked for the following:
Any observable user error, hesitation, confusion, or difficulty while performing the defined tasks in the scenario pertaining to the use of AerVent
Any communication between the participant teams relevant to the usability of AerVENT or task performance impacted by AerVent
Any action, maneuver, or handling of AerVent that may introduce risk to a person
Collected Time on Task for each critical task
Findings
Qualitative and quantitative data analysis of participant self-reports, Researcher observation notes, and Time on Task data revealed positive results for AerVent.
While most task completion times for scenarios utilizing AerVent were longer, as predicted, as it is an additional treatment applied to patients and an extra step for medical staff, it is likely that time will decrease with training and practiced use.
Through this research, the UX Research team uncovered, noted, and reported user-device interaction issues, hazards or risks to users introduced bt the device, device operation issues, deviations from the manufacturer’s recommended use, and clinical workflow changes due to the presence of AerVent.
Our evaluation showed no use errors or problems that could result in serious harm or that could not be mitigated through design modifications and user training.
Based on our observations and quantitative data, we delivered recommendations for user training, design updates and future testing and found AerVent to pose minimal use-related risk to users.
Outcome/ Impact
My final HFE report was delivered to our client, Aeragen for submission to the FDA.