The Face, Content, and Construct Validity Assessment of a Focused Assessment in Sonography for Trauma Simulator

doi:10.1016/j.jsurg.2015.04.003

Journal of Surgical Education

Volume 72, Issue 5, September–October 2015, Pages 1032-1038

https://doi.org/10.1016/j.jsurg.2015.04.003 Get rights and content

Objective

Hemorrhage identification in trauma care is a major priority. Focused assessment in sonography for trauma (FAST) offers a rapid, reliable means of detecting torso bleeding. The aims of this study were to conduct a face, content, and construct validity assessment of a FAST simulator and establish a rigorous assessment tool.

Design

Participants were requested to perform a FAST scan and state if any abnormality was found in each region. Metrics evaluated included time, errors, and missed targets. Accuracy of images obtained was assessed by 2 independent radiologists. Experts completed a face and content validity questionnaire at the end of the study.

Setting

The study took place in the simulation suite within the Academic Surgical Unit of the Department of Surgery and Cancer.

Participants

Novices had no prior experience with ultrasound, intermediates had less than 6 months experience with fewer than 50 FAST scans performed, and experts had more than 1 year of experience with greater than 100 FAST scans performed. There were 31 participants: 11 experts, 10 intermediates, and 10 novices.

Results

The face and content validity questionnaire scored high marks across all categories and achieved an overall median realism score of 8 ± 1.5 on a Likert scale. Experts performed the FAST scan faster with more accuracy and fewer errors than other cohorts (p < 0.001). Both the novices and intermediates were the slowest, least accurate, and either missed or made the most errors when scanning the lung bases and spleen.

Conclusions

This study has established the face, content, and construct validities of a FAST simulator, which could be used to accelerate training for novices. Additionally, it has demonstrated a rigorous method for FAST assessment, which has proven to be effective and in doing so addressed some of the criticisms leveled against it.

Introduction

In trauma care, swift identification of hemorrhage from the torso is a major priority.¹ The mortality of a patient with traumatic shock from intra-abdominal bleeding increases by approximately 1% for every 3 minutes that passes before definitive management.² Consequently, uncontrollable hemorrhage has accounted for greater than 90% of deaths in the military, the majority of which have occurred in the prehospital setting.³ Hence, rapid identification and control of bleeding, ideally in prehospital phase is of paramount importance. This remains a challenge and at present has not been overcome.

The focused assessment in sonography for trauma (FAST) protocol has been strongly advocated by several international societies as an effective diagnostic modality in the prehospital and hospital care settings for the rapid identification of bleeding.4, 5, 6, 7 The FAST scan examines for pericardial and intraperitoneal (perihepatic, perisplenic, and pelvis) fluid. In addition, the extended FAST examines for a pneumothorax/hemothorax.⁸ Advantages are that in experienced hands it is a swift, cheap, portable, and repeatable examination with no radiation exposure to the patient.⁸ The FAST scan has an essential role in patients with trauma presenting with hemorrhagic shock specifically to guide the need for immediate operative intervention where attempts to gain further imaging could adversely delay care.⁴ To this extent, FAST has demonstrated an improvement in prehospital diagnostic accuracy,⁹ reduction in trauma mortality,¹⁰ shortened time to operation, and a decrease in hospital stay and costs.11, 12 Despite this, there has been a limited uptake of FAST in the prehospital setting; reasons cited include cost of equipment, length of training to gain proficiency,¹³ and the lack of a standardized curriculum and assessment.¹⁴

The traditional apprenticeship model requires extensive practice on patients to gain the necessary competency. In the acute resuscitation phase of a critically ill patient with trauma, training novices in FAST may not be feasible or appropriate. On the contrary, simulation has been well adopted by the medical and surgical fraternity for trainees to accelerate up the learning curve in a safe supportive environment. A high-fidelity FAST simulator could bridge the training gap for prehospital and hospital clinicians to gain the necessary proficiency in this examination. According to consensus guidelines, such simulators should be validated before clinical teaching.¹⁵

Section snippets

Aims

With no previous evidence available and as a guide for future training systems, the aims of this study were to (1) perform a face, content, and construct validity assessment of the FAST simulator and (2) establish a rigorous FAST assessment tool.

Face, Content, and Construct Validity Assessment

Clinicians of varying levels of experience were invited to participate in this study by open e-mail invitation to the departments of emergency, radiology, and surgery. In light of no prior research available to provide a categorical separation of experience in FAST, expert opinion was gained from 8 radiologists working in the field at a level 1 trauma center. To enable a broad differentiation of experience, the following categories were chosen: novices had no prior experience with ultrasound,

Results

A total of 31 participants were included in the study, which consisted of 11 experts, 10 intermediates, and 10 novices. The expert cohort included 4 consultants from the radiology, emergency department(ED), and trauma specialties and 7 senior radiology registrars. The experts had a mean of 5.68 ± 4.8 years of ultrasound experience. The intermediate group consisted of 6 ED registrars and 4 general surgery registrars. The novice group included all core trainees in ED or general surgery. The

Discussion

The morbidity and mortality of hemorrhagic shock secondary to a traumatic insult remains high.¹ There is a need to streamline trauma workflow so that bleeding is identified rapidly, ideally by using FAST in prehospital care, resulting in a swifter management plan for hemorrhage control. An example of this was shown by Walcher et al.²⁰ who conducted a prospective multicentre study in 230 patients over a 12-month period. They performed prehospital FAST at a mean of 35 minutes before the ED team.

Conclusion

Hemorrhage identification in trauma care remains a high priority.1, 3 FAST offers a rapid practical means of detecting bleeding within the torso and is particularly suited for prehospital care. This study has addressed some of the criticisms of FAST such as the lack of a standardized assessment.13, 14 It has established the face, content, and construct validity of a FAST phantom, which has the potential to accelerate training for novices in a safe supportive environment. Additionally, it has

References (30)

J.C.R. Rippey et al.
Ultrasound in trauma
Best Pract Res Clin Anaesthesiol
(2009)
F. Lapostolle et al.
Usefulness of hand-held ultrasound devices in out-of-hospital diagnosis performed by emergency physicians
Am J Emerg Med
(2006)
L.A. Melniker et al.
Randomized controlled clinical trial of point-of-care, limited ultrasonography for trauma in the emergency department: the first sonography outcomes assessment program trial
Ann Emerg Med
(2006)
R. Aggarwal et al.
A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill
Am J Surg
(2006)
P. Duran-Gehring et al.
FAST wars: assessing competency and speed in a simulated trauma patient
Ann Emerg Med
(2013)
S.V. Soundappan et al.
Diagnostic accuracy of surgeon-performed focused abdominal sonography (FAST) in blunt paediatric trauma
Injury
(2005)
J.M. Hsu et al.
The accuracy of focused assessment with sonography in trauma (FAST) in blunt trauma patients: experience of an Australian major trauma service
Injury
(2007)
D.S. Kauvar et al.
Impact of hemorrhage on trauma outcome: an overview of epidemiology, clinical presentations, and therapeutic considerations
J Trauma
(2006)
J.R. Clarke et al.
Time to laparotomy for intra-abdominal bleeding from trauma does affect survival for delays up to 90 minutes
J Trauma
(2002)
B.J. Eastridge et al.
Death on the battlefield (2001-2011): implications for the future of combat casualty care
J Trauma Acute Care Surg
(2012)

T.M. Scalea et al.

Focused assessment with sonography for trauma (FAST): results from an international consensus conference

J Trauma

(1999)

L.J. Wherrett et al.

Hypotension after blunt abdominal trauma: the role of emergent abdominal sonography in surgical triage

J Trauma

(1996)

G.S. Rozycki et al.

A prospective study of surgeon-performed ultrasound as the primary adjuvant modality for injured patient assessment

J Trauma

(1995)

W.S. Hoff et al.

Practice management guidelines for the evaluation of blunt abdominal trauma: the East practice management guidelines work group

J Trauma

(2002)

C.G. Blood et al.

An assessment of the potential for reducing future combat deaths through medical technologies and training

J Trauma

(2002)

Cited by (5)

The surgeon-performed ultrasound: a curriculum to improve residents’ basic ultrasound knowledge
2017, Journal of Surgical Research
Citation Excerpt :
The different pathologies that can be diagnosed with an EFAST were taught showing representative images during the didactic sessions. Many studies have shown that the use of simulators is effective in teaching novices and thus can be used to teach residents about the pathologies in a nonstressful environment.27,28 Future work should continue to explore these methodologies.
Despite the development of ultrasound courses by the American College of Surgeons two decades ago, many residencies lack formal ultrasound training. The aim of this study was to assess the previous ultrasound experience of residents and the efficacy of a new ultrasound curriculum by comparing pre- and post-course tests.
A pre-course survey and test were sent to all residents at the University of Texas Southwestern Medical Center. Pre-interns and junior residents received a didactic lecture on ultrasound basics and the extended focused assessment with sonography for trauma and were given hands-on practice. Finally, a post-course test and survey were sent to the pre-interns and junior residents.
Only 11.3% of the residents reported having previous exposure to a formal ultrasound curriculum, and only 12.7% were taught by faculty. On the pre-course test, there was no difference in performance among senior residents, junior residents, and pre-interns (P = 0.114). After taking the course, the pre-interns improved their performance, and their average increased from 44.3% (standard deviation = 12.4%) to 66.1% (standard deviation = 12.2%; P < 0.001). The junior residents also had an improvement in their performance on the test after the course (P < 0.001). Junior residents performed better than pre-interns on the post-course test (P = 0.001).
The knowledge of surgical residents in ultrasound basics and extended focused assessment with sonography for trauma can be improved with the establishment of an ultrasound curriculum. We believe that such an educational endeavor should be encouraged by all surgical residencies.
Design and application of model for training ultrasound-guided vascular cannulation in pediatric patients
2016, Medicina Intensiva
Citation Excerpt :
The rate of success was 96%. Although whole-body ultrasound examination was mainly developed for adults, in pediatric patients it may help evaluating cardiac function and volemia, performing cardiopulmonary resuscitation (protocol FEER),7 monitoring and guiding endotracheal intubation (protocol TRUE),8 assessing pleural and pulmonary involvement, detecting free fluid in the peritoneum (protocol FAST)9 and guiding invasive procedures such as vascular access, thoracocentesis, pericardiocentesis, endotracheal intubation or lumbar puncture, in a safer and easier way at the bedside. Advances in vascular visualization lead to a more widespread use of vascular ultrasound as a helpful tool for central vascular access in critical pediatric patients.
Central vascular cannulation is not a risk-free procedure, especially in pediatric patients. Newborn and infants are small and low-weighted, their vascular structures have high mobility because of tissue laxity and their vessels are superficial and with small diameter. These characteristics, together with the natural anatomical variability and poor collaboration of small children, make this technique more difficult to apply. Therefore, ultrasound imaging is increasingly being used to locate vessels and guide vascular access in this population.
(a) To present a model that simulates the vascular system for training ultrasound-guided vascular access in pediatrics patients; (b) to ultrasound-guided vascular cannulation in the model.
The model consisted of two components: (a) muscular component: avian muscle, (b) vascular component: elastic tube-like structure filled with fluid. 864 ecoguided punctures was realized in the model at different vessel depth and gauge measures were simulated, for two medical operators with different degree of experience. The average depth and diameter of vessel cannulated were 1.16 (0.42) cm and 0.43 (0.1) cm, respectively. The average number of attempts was of 1.22 (0.62). The percentage of visualization of the needle was 74%. The most frequent maneuver used for the correct location, was the modification of the angle of the needle and the relocation of the guidewire in 24% of the cases. The average time for the correct cannulations was 41 (35.8) s. The more frequent complications were the vascular perforation (11.9%) and the correct vascular puncture without possibility of introducing the guidewire (1.2%). The rate of success was 96%.
The model simulates the anatomy (vascular and muscular structures) of a pediatric patient. It is cheap models, easily reproducible and a useful tool for training in ultrasound-guided puncture and cannulation.
La canalización vascular central es una técnica no exenta de riesgos sobretodo en la población de los pacientes pediátricos. El tamaño y peso de los pacientes más pequeños (recién nacidos y lactantes), la mayor movilidad de algunas de sus estructuras vasculares, la posición más superficial y diámetros más pequeños de sus vasos, la variabilidad anatómica que pueden presentar, asociado a la poca colaboración que presentan estos pacientes, hace más difícil la realización de esta técnica. A pesar de ello la ecografía está instaurándose para asistir a la realización de la punción vascular en dichos pacientes.
a) Diseñar un modelo experimental que permita la simulación vascular, la punción vascular ecodirigida y que sirva como método de aprendizaje y entrenamiento para la canalización ecoguiada de los vasos sanguíneos de los pacientes pediátricos. b) Realizar la punción ecoguiada en el modelo.
Se presenta un modelo compuesto por una porción muscular aviar a la que se introduce un sistema tubular elástico. Se simulan distintas profundidades del vaso así como diferentes diámetros de los mismos. Se realizaron 864 punciones ecoguiadas en el modelo con diferentes niveles de profundidad y de diámetro vascular por dos operadores con distinto grado de experiencia. La media de profundidad y diámetro de los vasos canalizados fue de 1,6 (0,42) cm y de 0,43 (0,1) cm respectivamente. El número medio de intentos fue de 1,22 (0,62). El porcentaje de visualización de la aguja fue del 74%. La maniobra más frecuentemente utilizada para la correcta canalización fue la recolocación de la aguja y la guía en el 24% de los casos. El tiempo medio hasta la correcta canalización fue de 41 (35,8) segundos. La complicación más frecuente fue la perforación vascular (11,9%) y la adecuada punción sin conseguir la introducción de la guía. La tasa de éxito fue del 96%.
El modelo presentado simula la anatomía (estructuras vasculares y musculares) del paciente pediátrico; es barato, fácilmente reproducible; permite la canalización y el aprendizaje de la técnica de la punción ecoguiada.
Assessing validity evidence for a serious game dedicated to patient clinical deterioration and communication
2020, Advances in Simulation
Determining procedures for simulation-based training in radiology: A nationwide needs assessment
2018, European Radiology
Collecting Validity Evidence for Simulation-Based Assessment of Point-of-Care Ultrasound Skills:
2017, Journal of Ultrasound in Medicine

View full text

Original ReportsThe Face, Content, and Construct Validity Assessment of a Focused Assessment in Sonography for Trauma Simulator

Objective

Design

Setting

Participants

Results

Conclusions

Introduction

Section snippets

Aims

Face, Content, and Construct Validity Assessment

Results

Discussion

Conclusion

Best Pract Res Clin Anaesthesiol

Am J Emerg Med

Ann Emerg Med

Am J Surg

Ann Emerg Med

Injury

Injury

Impact of hemorrhage on trauma outcome: an overview of epidemiology, clinical presentations, and therapeutic considerations

J Trauma

Time to laparotomy for intra-abdominal bleeding from trauma does affect survival for delays up to 90 minutes

J Trauma

Death on the battlefield (2001-2011): implications for the future of combat casualty care

J Trauma Acute Care Surg

Focused assessment with sonography for trauma (FAST): results from an international consensus conference

J Trauma

Hypotension after blunt abdominal trauma: the role of emergent abdominal sonography in surgical triage

J Trauma

A prospective study of surgeon-performed ultrasound as the primary adjuvant modality for injured patient assessment

J Trauma

Practice management guidelines for the evaluation of blunt abdominal trauma: the East practice management guidelines work group

J Trauma

An assessment of the potential for reducing future combat deaths through medical technologies and training

J Trauma

Original Reports
The Face, Content, and Construct Validity Assessment of a Focused Assessment in Sonography for Trauma Simulator