Next generation intelligent surgical systems:

In the past 20 years, the robotics laboratory ALTAIR of the University of Verona has been carrying out research in applications of robotics to safety-critical situations, in particular, medicine and surgery. The focus has been on understanding the main constraints of the surgical process, from diagnosis to therapy, with the goal of increasing patient safety and comfort. It became evident that some form of intelligent supervision is necessary to ensure that information is properly exchanged and timed, and that actions are accurate and safe. Thus the concept of an autonomous robot has emerged as the research paradigm that can integrate and demonstrate the technologies of embedded AI. In surgery, autonomy is explored with respect to the execution of surgical tasks and bedside assistance during robotic surgery. In order to develop a safe autonomous system, realistic dynamic simulation of the environment where the robot will be acting is of critical importance. I will give an overview of the background justifying the need for autonomous functions in safety-critical applications and then I will describe some of our current research in the development of the building blocks of a cognitive robot for surgical applications.

The variety and complexity of modern medicine have been a motivation for many scientific developments. While medical imaging has become an integral part of today’s medical practice, new fields are emerging, such as robotics, simulation, augmented reality, or workflow analysis. This talk will highlight the increasing role of real-time numerical simulation in the fields of surgery and interventional radiology, with an impact in three major application areas: training, planning and computer-guided interventions.

Numerical simulations have been used for several decades to perform complex analyses of biomedical phenomena, with various levels of success. A specificity of our application context is the need for real-time simulations, adapted to the patient. To this end, we have developed dedicated numerical methods, which allow for real-time computation of finite element simulations. Adaptation to the patient’s anatomy is obtained by exploiting the image-rich context-specific to our application domain. While information about the organ shape, for instance, can be obtained pre-operatively, other patient-specific parameters can only be determined intra-operatively. To this end, we are developing data-driven simulations, which exploit information extracted from a stream of medical images.

The general principle consists in combining finite element approaches with Bayesian methods or deep learning techniques, in order to keep control over the underlying computational model while allowing for inputs from the real world. This has been applied to the modeling of liver biomechanics, its real-time simulation, and parametrization to achieve patient-specific augmented reality during surgery. It has also been used to perform 3D shape reconstruction of medical devices during endovascular procedures from single view fluoroscopic images. 

Applications of data-driven simulations based on these principles are plentiful, ranging from the modeling of cells to medical robotics. However, validation of the results remains complex and very time-consuming.

The goal of SARAS is to develop the next-generation of surgical robotic systems. These will allow a single surgeon to execute Robotic Minimally Invasive Surgery (R-MIS) without the need of an expert assistant surgeon, thereby increasing the social and economic efficiency of a hospital while guaranteeing the same level of safety for patients. The robotic system developed by the SARAS project will be called solo-surgery system and will consist of a pair of cooperating and autonomous robotic arms holding off-the-shelf surgical instruments. The arms are controlled by a cognitive architecture able to infer the status of the surgical procedure, make decision on the tasks to be executed to help the main surgeon over the procedure, and to plan and execute dual-arm collision-free trajectories in a dynamical environment as the patient abdomen is. The architecture will seamlessly integrate deep learning algorithms, supervisory control and model-predictive controllers.

Numerical simulation has become invaluable, namely for medical education, therapy planning, and robotics. However, simulation software remain expensive and require a high level of expertise to be used correctly and effectively.

SOFA (Software Open Framework Architecture) is an open-source framework primarily targeted at real-time simulation, with an emphasis on medical simulation. This innovative and collaborative tool is developed by a worldwide community of experts in physics simulation and gathers about 13 years of scientific research. Both academic and industrial developers create their own proprietary simulations based on SOFA, benefiting from its LGPL license. Many research centers actively work and publish using SOFA while six startups have been created for the last six years. This talk will present you SOFA and its international community.

Percutaneous medical procedures are among the least invasive approaches for accessing deep internal structures of organs without damaging surrounding tissues. Although they provide very good results, these interventions significantly increase the level of expertise required for practitioners.  Medical robotics has the potential to assist surgical gesture, overcome limitations due to human factors and increase the accuracy of tools positioning. However, the deformation of the organs remains an open problem limiting the development of these robots for wide use in the operating room.

We introduce new solutions for the control of medical robots interacting with soft tissues. The originality of our approach lies in the fact that we address the problem of deformations solving an inverse Finite Element problem directly in the control loop of the robot. This work is motivated by recent advances in the field of medical simulation. FE simulations are now widely used for training surgeons, and they recently reached a sufficient level of realism to assist surgeons during the operation thanks to augmented reality.

We show these FE models enable the possibility to control the motion of a robotic system accounting for deformations of the needle and the tissue during the insertion. This requires to solve important scientific obstacles including 1) complex modeling of needle/tissues interactions and numerical strategies to reach real-time computation 2) non-rigid registration since very few information can be extracted in real time from images during an intervention 3) numerical strategies to derive information provided by the simulations and compute input commands of the robot.