Variance between Radiology Resident Preliminary Reports and Attending Finalized Reports

Authors:

Richard E. Sharpe Jr., MD, MBA, Thomas Jefferson University; David Surrey; Levon N. Nazarian, MD; Richard J.T. Gorniak, MD; Raja Gali, MBA; Adam E. Flanders, MD

Hypothesis:

Attending radiologists make modifications to resident preliminary radiology reports as part of resident workflow. The frequency and significance of these changes is not known. We hypothesize that the proportion of radiology report edits will be related to the study modality and study complexity.

Introduction:

Radiology residents in many training programs draft preliminary reports that are sent to the Radiology Information System (RIS) and made available to clinicians using the Hospital Information System (HIS). Trainee preliminary reports are typically drafted after direct consultation and image review with the attending radiologist during daily readout/teaching sessions. However, other workflow models involve trainees submitting either full reports or brief comments without joint trainee and attending radiologist imaging review, such as during overnight call.

Subsequently, attending radiologists review these preliminary reports and may make edits. This attending finalized report becomes the official medical record in the HIS/RIS, and may overwrite the resident preliminary report. Since RIS and speech recognition software generally do not support versioning of reports, there is no reliable mechanism for radiology residents to efficiently track the changes attending physicians make to resident preliminary reports nor is it convenient to readily identify changes when they have been performed.

Increased volume of imaging studies, emphasis on efficiency, widespread access to remote PACS, increasing numbers of required administrative tasks for radiology residents and attendings, as well as resident work hours restrictions, presents a challenge to traditional radiology training structure. Resultant new workflow models may spatially and temporally separate resident and attending radiologists. Further, more radiology trainees may increasingly approve reports without ever having a joint image review with the attending radiologist.

Valuable attending radiologist time is required to make edits to radiology reports. Consequently, many attending radiologist edits are likely significant. Such changes may affect patient care by altering anatomic descriptors, or more accurately describing radiological findings. Moreover, gaining an understanding of the rationale for these report changes has tremendous potential educational value. Trainee awareness of these changes would likely affect the quality of future reports. Yet, there is no practical and efficient method to present these report corrections to the trainee, nor is there a means to measure trends in the frequency of these changes as the trainee progresses in their training.

Current radiology information and speech systems are adaptations to private practice workflow and, as such, do not readily accommodate to resident workflow. Functionality that could enhance the training experience is frequently lacking. Therefore, in most training programs there are no readily available and efficient mechanisms for residents to review attending edits and there are no standardized methods to track the frequency of report modifications for a given trainee. Manual tracking and logging of dictations is an intrinsically inefficient and inaccurate process.

The purpose of this study was to do an objective comparison between preliminary and final versions of radiology reports, across a representative cross section of a study types and personnel, to look for trends related to study type, complexity, and trainee/staff combinations. The goal is to determine whether metrics based upon frequency of report modifications can be used as an objective measure of training efficacy.

Methods:

To facilitate ease of resident/attending report comparison, a semi-automated report comparator application (RC) was created. A server-sided scripting daemon was constructed (ASP/ADO) to periodically query and extract preliminary and finalized reports at regular intervals from the RIS and store the reports in a database (mySQL) indexed by accession number, trainee, and attending. Character/word counts were automatically calculated for all reports and a percent difference was calculated and stored in the database.

Over a one-week interval, PGY-3 residents were asked to provide the RC preliminary-finalized radiology report pairs for analysis. Provided report pairs were analyzed by modality type and report character length. Whether any edits were made was noted, as well as the number of edits between preliminary and finalized reports were noted. Reports in modalities with fewer than 25 reports available were excluded.

Data analysis includes calculation of the total number of edits per report. Total number and percentage of reports that were not modified are also noted. Analyses of the average number of edits per report, as well as the average number of edits per edited report, were performed. The number of edits per average report words was also calculated. Subset analysis was performed by imaging modality.

Results:

Authors reviewed radiology resident preliminary report and attending radiologist finalized report pairs from 549 reports across 4 imaging modalities (approximately 5% of the reports issued by our department during the evaluated one-week period). Reports were provided by six of nine total PGY-3 residents, a 66% response rate.

There were 1,337 total edits to analyzed reports. Two hundred ninety-six (53.9%) reports were edited and 253 (46.1%) reports were not modified. There were 2.44 edits per report and 4.52 edits per edited report.

Reports were analyzed by imaging modality. Represented imaging modalities included, in order of decreasing frequency: plain radiography (371, 67.6%), interventional radiology (88, 16.0%), computerized tomography (82, 14.9%), and magnetic resonance imaging (8, 1.5%). MR reports were excluded from further analysis because there were only eight reports provided.

Of the 371 plain radiography reports, 134 (36%) reports were edited. Two hundred thirty seven (64%) were not edited. Among the edited plain radiography reports, there were 222 total changes made. This resulted in 0.598 (222/371) edits per plain radiography report and 1.62 (222/137) edits per edited plain radiography report. The average plain radiography report contained 85.1 words. There were 0.007 (0.598/85.1) edits per radiography report word.

Of the 88 interventional radiology (IR) reports, all 88 (100%) were edited. There were a total of 699 changes made, which corresponds to 7.94 edits per report. The average IR report contained 497.4 words. There were 0.015 (7.94/497.4) edits per IR report word.

Of 82 computed tomography (CT) reports, 70 (85.4%) were edited. Twelve (14.6%) CT reports were not edited. There were 357 total edits. This corresponds to 4.35 (357/82) edits per CT report, and 5.1 (357/70) edits per edited CT report. The average CT report contained 215.3 words. There were 0.020 (4.35/215.3) edits per CT report words.

Discussion:

Edits to radiology resident preliminary reports frequently occur by attendings during report finalization. Slightly more than half of all trainee preliminary reports were edited by attending radiologists.

This study documents that edits occur with different frequencies across imaging modalities. Most plain radiography reports were not edited. However, nearly all CT and IR reports were edited by attending radiologists. There were many more edits per CT or IR report than per plain radiography report.

Analysis of the number of edits per report word controls for the report word length differences across different imaging modalities. This metric compensates for additional likelihood of random factors, such as typographical errors, that could occur in these longer reports. On a per report word basis, there are approximately twice as many edits to more complex reports, such as CT and IR reports, than radiography reports. This suggests factors related to the imaging modality and/or report complexity may contribute to increased relative number of edits to CT and IR reports.

Limitations of this study include sample size and lack of precise edit characterization. While this study documents to frequencies of edits, it does not delineate their likely clinical or educational significance.

Future research is recommended to determine the frequency of clinically significant and educationally relevant edits to radiology reports, as well as the etiologies of these changes. With proper standardization, this could become an objective metric both for resident education and attending radiologist performance.

Conclusion:

Radiology preliminary reports are frequently edited by attending radiologists during report finalization. These edits are less common in radiography reports and more frequent in CT and IR reports. The frequency of these changes likely relates to the complexity of imaging modality and/or the radiology report. If standardized, report edits may become an objective metric both for resident education and attending radiologist performance.

Keywords:

Education
Radiology reports
Reporting