Every day, your OR teams use numerous real-world products and evidence-based techniques to better ensure every patient’s safety. However, when adverse events or near misses occur, or when process improvements are sought to improve patient safety, it’s not easy to generate usable data that can be analyzed and acted upon. Digital technologies like artificial intelligence (AI), virtual reality (VR) and augmented reality (AR) are helping facilities access and analyze much more data than ever before. It’s all part of an emerging environment in which digital technologies enable more precise, efficient and accurate surgeries with a higher degree of safety.
Informed Decisions Enhance Patient Safety
By: Joe Paone
Published: 4/24/2020
Artificial intelligence, virtual reality and augmented reality are helping surgeons leverage data to perform safer surgeries.
Artificial intelligence
The literature has produced a wealth of studies on AI’s potential to make procedures safer and more effective. AI can be used to assess data collected from past cases to predict where polyps may be present during screening colonoscopies. The information provides physicians with a better ability to detect and diagnose colorectal polyps they may have previously missed due to human error or blind spots (osmag.net/eB9HqK).
Orthopedic surgeon Kevin Stone, MD, of San Francisco’s Stone Clinic, uses AI to perfect surgical techniques. Dr. Stone uploads his medical records to an AI-driven data-mining system to perform outcome studies on procedures his clinic has developed for meniscus transplantation, Achilles tendon and cartilage repair, and comparison of allograft and autograft ACL reconstructions. Among other benefits, the resulting information gleaned from AI enhances patient safety. “We don’t think a doctor should do something to, or put something in, a patient that they don’t study outcomes for, and try to improve the process,” says Dr. Stone.
One AI-based system attracting attention is the OR Black Box, a medical data recorder that captures the patient’s physiological data, audio and video of the surgical team, and images of the surgical field, including video from endoscopic cameras. All that data is sent to a central server for analysis. That might sound a bit Big Brother, but the system’s creator says its real purpose is process improvement in the OR, especially in the wake of adverse events and near misses. The system can detect trends and provide insights on factors that might have gone unnoticed or aren’t fully understood or accounted for, like inefficiencies and distractions.
“The patient comes in for surgery and goes home and everything is fine, but there were periods of care in the operating room that could have resulted in a disaster,” says Teodor Grantcharov, MD, PhD, FACS, creator of the Black Box and a general surgeon at St. Michael Hospital in Toronto and a professor of surgery at the University of Toronto. “We provide very deep insights to those.”
The system lets surgical facilities be proactive rather than reactive, leveraging data to prevent future mishaps and difficulties. “The current model is that you wait for somebody to suffer a complication in order to analyze what happened and make improvements,” says Dr. Grantcharov. “We want to identify deviations from the perfect course of an operation, especially if there’s a pattern. We want to replace initiatives based on tradition, emotions or gut feeling, and make data-driven decisions.”
Twelve facilities in the U.S., Canada and Western Europe currently use the OR Black Box, with three more coming online soon. Dr. Grantcharov expects close to 30 institutions to be on board by year’s end. Rollout has been careful and deliberate.
Dr. Grantcharov has even turned down interested facilities he felt weren’t culturally prepared. “We’re focused on creating a network of institutions that share the same vision to make surgery safer, and to make the operating room safer, more efficient and a better working environment,” he says. “Unlike professionals in sports, oil, aviation or the nuclear industry, we in health care, especially in the OR, are not used to this level of transparency. We had to convince healthcare providers that this is something that is confidential and non-punitive and makes all of us better, something that makes our ORs safer and more efficient.”
Dr. Grantcharov is looking to partner with surgical facilities that are committed to improving quality and safety in a confidential and non-punitive fashion as opposed to facilities where every failure has to be associated with blame and shame or identifying individuals rather than system issues.
As such, the black box data analysis focuses on system factors like communication, teamwork, policy and the physical environment. Such analysis is traditionally assembled retrospectively based on recall, or using human observers, which consumes resources and can be distracting to staff. Aside from someone turning the black box on at the beginning of a procedure and off afterward, the surgical team doesn't see or interact with the system. The patient's identity, as well as those of everyone else in the room and the facility itself, is anonymized. But the information generated by analysis of the captured data can be used for educational and process-improvement purposes. The system, now on its second generation, requires capital equipment investment along with a subscription fee for the data analysis. The data transmitted to Dr. Grantcharov's team is analyzed by a mix of AI and human intelligence. Facilities access the analysis through an online portal. As more surgical facilities sign on, more data is collected and more trends are analyzed worldwide, providing richer analysis and insight than even the most data-aware facility could produce on its own.
Virtual and augmented reality

VR is also making inroads into surgical training and education. Wearing a VR headset, surgeons can learn how to safely use new techniques and tools, and prepare for procedures on specific patients. The surgeon is transported to a virtual, 3D world where they can better understand and interact with MRIs or CT scans of a patient and prepare for unique circumstances. Essentially, VR can enable the surgeon to practice the procedure and better navigate specific anatomy before they enter the OR. In addition, instrument and equipment vendors are increasingly providing VR training to surgeons to promote safer use and a better understanding of their products.
AR is enhancing surgical planning and enabling safer, more efficient execution in the OR through guides and overlays. Voluminous studies have been produced on AR-powered surgical applications. For example, one study used AR-generated, holographic 3D images to improve surgical approach planning in the treatment of severe intra-articular fractures of distal tibia in order to execute more precise and accurate placement of incisions and less subsequent tissue damage (osmag.net/DYw4Ar). Another successfully created prototypes of various AR-driven tools for safer robotic-assisted surgery, including virtual markers that can be attached to intraoperative images, computational tools for measuring distances, rehearsals of procedures, visual alarms that could alert staff to out-of-view instruments and viewing patient data in the OR (osmag.net/ QhBk6R). The surgeon interacts with the AR system through voice commands and by using the robotic instruments as cursors.
Because all this technology is digital, it can be shared for collaborative purposes. For example, imagine a doctor using a VR headset to perform a new surgical technique while hearing the voice of another surgeon halfway around the globe who is guiding and educating them. As IP networks become ever more robust and 5G data services that offer exponential bandwidth boosts, it's not difficult to imagine surgeons in far-flung places providing invaluable knowledge and insight, and assisting in live surgeries — perhaps even remotely controlling a robot in an OR in another country. It's all part of the dazzling potential for cutting-edge digital technologies to perfect techniques, processes, incisions, ablations and much more, all in the interest of patient safety and superior outcomes. OSM