Introduction

Background

Public safety officials perform tasks in various environments that severely limit their senses and ability to communicate. The aftermath of any emergency is filled with chaos, and the public safety personnel needs to coordinate and rescue the victims as quickly as possible effectively. They follow traditional and standardized procedures for logistics, assessment, triaging, etc. They are using regular radio communication, information from witnesses, and visuals from the site.

Problem Space

All emergencies require immediate action to rescue a victim and it is one of the utmost priorities. But there is a significant gap in action, response time, and lack of standardized system while responding to an emergency situation that is well-validated, reliable, and uniformly accepted.

Problem Statement

We converted the challenge description provided to us into a how might we statement to define our design direction and focus on the user’s most pressing needs. The problem statement we followed is:

Design Process

I collaborated with two designers, two developers, and a project manager for this project in researching to designing and developing a working prototype. We followed a combination of a Double diamond and Lean UX design processes for this project and a high-level process and cross-functional team involvement is represented in the diagram below.

Research

Public Safety Personnel

Public safety personnel are any law enforcement officer, fire department member, emergency medical response personnel, or other individuals responding in the aid of public safety or rescue. Their job is to rescue and attend to those injured, suppress fires, secure and police the disaster area and to begin the process of restoring order.

First Responders

are often the primary line of defense, responding to an evolving spectrum of natural and man-made threats.

Incident Commander

would help the agencies responding to a disaster to work together in a coordinated and systematic approach.

Operating Scenarios

There are four typical scenarios to explore the professions and the kinds of tasks they do and information that will be helpful for them to operate in an emergency.

Mass-Transit
Accident

Active
Shooter

Flood

Wildfire

Task Analysis

Firefighters respond to a variety of emergency situations, including fires, natural disasters, hazardous waste spills and medical emergencies. Below is a list of key tasks that the first responders and incident commanders are responsible in an emergency situation.

First Responder's Tasks

Locating and evacuating victims
Preparing for staging/triage/ treatment
Identifying and mitigating risks on-site
Mapping hazardous materials 
Maintaining safety of personnel and team
Identifying safe locations

Incident Commander's Tasks

Identify victims or assets at risk and assist
Track resources (e.g., personnel movement)
Advising/drawing feedback to inform responders
Establish geographic knowledge of area
Monitor evacuation routes
Detect injured individuals

Victim Triaging

Triage is the process of examining patients and assessing how urgently they need care, and determining who will get treated first. Traditionally, victims are triaged using the physical tags and color codes with manual entries of information.

START (Simple Triage and Rapid Treatment)

START helps responders rapidly assess victims’ ability to walk and their mental status, blood flow, and respiration. Based on the assessment, patients are placed in 1 of 4 color-coded categories:
IMMEDIATE: severely injured patients.
DELAYED: stable patients.
MINOR: patients with minor injuries.
DECEASED: victims who have been killed during the incident.

AR Platform

Spatial Computing enables the blending of digital contentand the physical environment

VIRTUAL REALITY

Digital environment isolated from the physical.

AUGMENTED REALITY

Digital content on top of the physical world

SPATIAL COMPUTING

Digital content interacts with the physical world

Input Methods

The input method for the device is fundamentally different from the way users are used to interact with digital content, especially compared to interactions within handheld augmented reality apps.

Fiducial Markers

Fiducial markers are visible objects placed in an imaging system (camera) as reference points or measuring units. In VR and AR, fiducial markers are used for positional tracking and overlaying digital objects in reference to the physical space.

Findings

Lack of centralized communication.

Fragmented tracking of triage statuses.

Difficult to track identity of victims.

Ideation

Personas

" I’d like to know the patient’s medical information while triaging to provide treatment."

" I need accurate information to determine severity and guide incident response activities."

Ryan Morris

Emergency Medical
Technician

Keenan Gill

Incident Commander
- Fire Dept.

Click on the persona’s picture to view the complete version

10x10 Sketches

We brainstormed different concepts and ideas using the 10x10 method with the personas in the context. We developed ideas, which are both practical and radical. These concept sketches gave us a direction towards converging feasible ideas into an appropriate solution.

Storyboard

We created two storyboards around scenarios that the first responder and incident command personas would operate. We explored how they shall use an AR solution to rescue and triage victims in an emergency.

First Responder’s Scenario

Paramedic’s response in a mass accident scenario, identifying, triaging, and taking victims to a hospital for treatment.

Incident Commander’s Scenario

Incident Commander’s response in a mass accident scenario, analyzing the situation and planning, and deploying resources.

Design

Product concepts

Traditionally, victims are triaged using the physical tags and color codes with manual entries of information. We propose to replace the tags with fiducial markers which can be scanned using the AR goggles and victim status can be tagged to it using the START system of triage algorithm.

Traditional
Triage Tag

LifeSavAR
Triage Tag

Victim Triaging Task Flow

‘LifeSavAR’ System Overview

Data is collected from the site sensors, cameras, and drones using the IoT sensors. This data is provided to the incident command to make necessary decisions while tackling the emergency situation.

Low-Fidelity Prototype

Using the storyboards and product concept sketches we designed low-fidelity prototypes to demonstrate the concept and the flow of things.

‘LifeSavAR’ Triage Marker

A triage tag with a fiducial marker is tagged to the victim. This is the first responder’s view through the AR headset.

Triaging Identified Victim

After scaning the tag, the first responder has 4 options to chooce from based on the victim’s status.

Adding Victim Details

The first responder can enter additional details about the victim and also mark the injuries on the body.

Victims' Status

The data entered is shared across all the other first responders. They would know the total number of victims on site and their status.

Incident Command

IC has all the information relayed from the first responders on site. The victim details and site information helps them manage the resouces.

High-Fidelity Prototype

We used Adobe XD to design the high-fidelity prototype. There were several iterations of the designs for the final screens bsed on the usability, accesability, and aestheics. Below are a few screens from the design.

Scenario 1: Mass Transit Accident

First Responder Interface - Assigning triage tag

First Responder Interface - Visualizing patient assessment

First Responder Interface - Triage tag expanded view

First Responder Interface - Triage status overview

Heads-up display interface for Emergency Doctor

Incident Commander Interface

Scenario 2: Flood

First Responder Interface

Incident Commander Interface

Scenario 3: Wildfire

First Responder Interface

Incident Commander Interface

Scenario 4: Active Shooter

First Responder Interface

Incident Commander Interface

Final Design

After multiple design iterations and testing with a few target users, we finalized our designs and collaborated with the developers to build our AR app. Keeping the time and budget constraints in mind, we prioritized a list of features that we would develop as an MVP. The final design iteration of the product is presented below.

First Responder Interface - Mass Transit Accident (AR view)

Hover to play the video

First Responder Interface - Mass Transit Accident (Prototype)

First Responder Interface - Active Shooter Scenario

First Responder Interface - Wildfire Scenario

First Responder Interface - Flood Scenario

Incident Commander Interface

Evaluation

User Testing

We conducted usability study sessions with two public safety personnel, one each from Indy EMS and Indy Fire to validate our designs and the interactions. Both the participants have served for more than 6 years in their respective fields of operations and well aware of all the typical tasks that comes with an emergency.

Results and Findings

The overall experience of the prototype was well received by both the public safety personnel. There were a few suggestions that were made by the participants to improve the experience.

For Mass-Transit Accident Scenarios

1. The system could display the location of victims that are tagged either by the drone or by the LifeSavAR system.

2. The resources section could also include things like tools, air bags, oxygen, doctor / surgeon.

For Wildfire Scenarios

1. It is ideal to have access to immediate information with a quick gesture instead of having it always present on the screen as it could be distracting.

2. Crossover communication between the fire department/ police/ EMS have to streamlined as it is difficult to use a radio with the AR device on.

This usability study confirmed that the basic usability aspects of the application are useful, usable, and functional. The evaluators felt that the system is consistent in terms of design which they liked. They also said that this system would definitely solve the problem of centralized communication and tracking of triage status.

Reflection

Next Iteration

We learned that AR technology combining with IoT sensor data can greatly improve the gap in current Triage process. If we had more time we would iterate our designs to accommodate:

1. voice recording and live interactions with the incident command.

2. Automating resource allocation using AI to make the system more efficient.

3. Assisted communication to make our architecture robust to use in real-time situation.

Key learnings

Collaborate & Empathize with devs

Apart from the design team, I got to collaborate with the Developers as well. I explained my thoughts regarding a particular concept followed by the designs and what we were expecting them to do. But somewhere I felt that there was a small communication gap in between, this taught me that you should be able to explain your views according to the other person.

Predict User Behavior

A product designer must understand user's behavior in such depth that they can predict their behavior. An in-depth understanding of the user’s workflow will assist in the ideation and execution of a seamless UX/UI design that the user will not even know exists!

Challenge

Project Specifics

Introduction

Research

First Responders

Incident Commander

First Responder's Tasks

Incident Commander's Tasks

VIRTUAL REALITY

AUGMENTED REALITY

SPATIAL COMPUTING

Ideation

Design

Evaluation

Reflection

Home

Next

Let's Connect!