Next generation intelligent surgical systems:
Computer simulation meets robotics.
Verona, 13/11/2019 One day workshop, Free registration
Wednesday, 13 November 2019
Room Verde, Department of Computer Science, University of Verona
|Welcoming and registration
|Introduction to the workshop
|Dr. Diego Dall’Alba
|Embodied AI: towards autonomous functions in medicine, surgery and manufacturing
|Prof. Paolo Fiorini
|Data-driven real-time numerical simulation for computer-aided surgery
|Dr. Stéphane Cotin
|A cognitive architecture for semi-autonomous surgical robots: the EU SARAS project platform
|Prof. Riccardo Muradore
|SOFA: an open-source solution for collaborations, prototyping and innovation in simulation
|Robotic insertion of flexible needle in deformable structures using inverse finite element simulation
|Dr. Hadrien Courtecouisse
|ALTAIR Lab visit
Embodied AI: towards autonomous functions in medicine, surgery and manufacturing
In the past 20 years, the robotics laboratory ALTAIR of the University of Verona has been carrying out research in applications of robotics to safety-critical situations, in particular, medicine and surgery. The focus has been on understanding the main constraints of the surgical process, from diagnosis to therapy, with the goal of increasing patient safety and comfort. It became evident that some form of intelligent supervision is necessary to ensure that information is properly exchanged and timed, and that actions are accurate and safe. Thus the concept of an autonomous robot has emerged as the research paradigm that can integrate and demonstrate the technologies of embedded AI. In surgery, autonomy is explored with respect to the execution of surgical tasks and bedside assistance during robotic surgery. In order to develop a safe autonomous system, realistic dynamic simulation of the environment where the robot will be acting is of critical importance. I will give an overview of the background justifying the need for autonomous functions in safety-critical applications and then I will describe some of our current research in the development of the building blocks of a cognitive robot for surgical applications.
Paolo Fiorini, received the Laurea degree in Electronics Engineering from the University of Padova, (1976), the MSEE from the University of California at Irvine (1982), and the Ph.D. in ME from UCLA (1994). From 1977 to 1985 he worked for companies in Italy and in the US developing microprocessor-based controllers for domestic appliances, automotive systems, and hydraulic actuators. From 1985 to 2000, he was with NASA Jet Propulsion Laboratory, California Institute of Technology, where he worked on autonomous and teleoperated systems for space experiments and exploration. In 2001 he returned to the Department of Computer Science of the University of Verona (Italy) where is currently Full Professor. He founded the ALTAIR robotics laboratory to develop tele-robotic systems for space, medicine, and logistics, focusing on delicate tasks, such as robotic surgery. In the past 15 years he received several European and Italian grants in robotic surgery, addressing accuracy, safety and autonomous capabilities of surgical robots. He also founded four companies in robotic surgery. His activities were recognized by the Entrepreneurship Award at the European Robotics Forum, the Antonio D’Auria award for robotic aids for disabled people, the IEEE Fellow (2009), and several NASA Technical Awards.
Data-driven real-time numerical simulation for computer-aided surgery
The variety and complexity of modern medicine have been a motivation for many scientific developments. While medical imaging has become an integral part of today’s medical practice, new fields are emerging, such as robotics, simulation, augmented reality, or workflow analysis. This talk will highlight the increasing role of real-time numerical simulation in the fields of surgery and interventional radiology, with an impact in three major application areas: training, planning and computer-guided interventions.
Numerical simulations have been used for several decades to perform complex analyses of biomedical phenomena, with various levels of success. A specificity of our application context is the need for real-time simulations, adapted to the patient. To this end, we have developed dedicated numerical methods, which allow for real-time computation of finite element simulations. Adaptation to the patient’s anatomy is obtained by exploiting the image-rich context-specific to our application domain. While information about the organ shape, for instance, can be obtained pre-operatively, other patient-specific parameters can only be determined intra-operatively. To this end, we are developing data-driven simulations, which exploit information extracted from a stream of medical images.
The general principle consists in combining finite element approaches with Bayesian methods or deep learning techniques, in order to keep control over the underlying computational model while allowing for inputs from the real world. This has been applied to the modeling of liver biomechanics, its real-time simulation, and parametrization to achieve patient-specific augmented reality during surgery. It has also been used to perform 3D shape reconstruction of medical devices during endovascular procedures from single view fluoroscopic images.
Applications of data-driven simulations based on these principles are plentiful, ranging from the modeling of cells to medical robotics. However, validation of the results remains complex and very time-consuming.
Pr. Stephane Cotin is a Research Director at Inria and leader of the MIMESIS team. His main research interests are in physics-based simulation, real-time simulation of soft-tissue deformations, finite element modeling, augmented reality and medical applications of this research. In an effort to disseminate this work and to accelerate research in these fields, Pr. Cotin initiated the SOFA project (www.sofa-framework.org) which is now a reference Open Source solution for developing advanced simulations. He is author or co-author of more than 150 scientific articles and involved with several conferences, either as organizer or member of the board: International Symposium on Biomedical Simulation (ISBMS), International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), International Conference on Information Processing in Computer-Assisted Interventions (IPCAI).
Since 2012, he is also a co-founder and president of the scientific board of InSimo (www.insimo.fr). Before joining Inria, Pr. Cotin Research Lead for the Medical Simulation Group at CIMIT and an instructor at Harvard Medical School.
A cognitive architecture for semi-autonomous surgical robots: the EU SARAS project platform
The goal of SARAS is to develop the next-generation of surgical robotic systems. These will allow a single surgeon to execute Robotic Minimally Invasive Surgery (R-MIS) without the need of an expert assistant surgeon, thereby increasing the social and economic efficiency of a hospital while guaranteeing the same level of safety for patients. The robotic system developed by the SARAS project will be called solo-surgery system and will consist of a pair of cooperating and autonomous robotic arms holding off-the-shelf surgical instruments. The arms are controlled by a cognitive architecture able to infer the status of the surgical procedure, make decision on the tasks to be executed to help the main surgeon over the procedure, and to plan and execute dual-arm collision-free trajectories in a dynamical environment as the patient abdomen is. The architecture will seamlessly integrate deep learning algorithms, supervisory control and model-predictive controllers.
Riccardo Muradore (S’99-M’04) received the Laurea degree in information engineering in 1999 and the Ph.D. degree in electronic and information engineering in 2003 from the University of Padova, Padova, Italy. He was a Postdoctoral Fellow with the Department of Chemical Engineering, University of Padova, from 2003 to 2005. He then spent three years at the European Southern Observatory, Munich, Germany, as a Control Engineer, working on adaptive optics systems. In 2008, he joined the ALTAIR Robotics Laboratory, University of Verona, Verona, Italy, where, since 2013, he has been an Assistant Professor. His research interests include robust control, robotics, teleoperation, networked control systems, and adaptive optics.
SOFA: an open-source solution for collaborations, prototyping and innovation in simulation
Numerical simulation has become invaluable, namely for medical education, therapy planning, and robotics. However, simulation software remain expensive and require a high level of expertise to be used correctly and effectively.
SOFA (Software Open Framework Architecture) is an open-source framework primarily targeted at real-time simulation, with an emphasis on medical simulation. This innovative and collaborative tool is developed by a worldwide community of experts in physics simulation and gathers about 13 years of scientific research. Both academic and industrial developers create their own proprietary simulations based on SOFA, benefiting from its LGPL license. Many research centers actively work and publish using SOFA while six startups have been created for the last six years. This talk will present you SOFA and its international community.
Dr. Hugo Talbot originally studied mechanical engineering, and graduated in 2010 from both Karlsruhe Institute of Technology (Germany) and INSA Lyon (France). He defended his PhD in medical simulation at Inria (France) in July 2014. His work focused on the real-time simulation of the electrical activity of the human heart. From 2014 until late 2015, Hugo worked on the simulation of cryoablation and cardiac electrophysiology as research engineer.
Since January 2016, Hugo Talbot is the coordinator of the open-source project SOFA. SOFA is an open-source framework for multi-physics simulation and is being developed for more than 12 years. Today, SOFA benefits from a large international community made up of research centers and companies. The objectives of Hugo are to develop the community, to ensure the technical evolution of the software, to foster research collaborations and technology transfers.
Robotic insertion of flexible needle in deformable structures using inverse finite element simulation
Percutaneous medical procedures are among the least invasive approaches for accessing deep internal structures of organs without damaging surrounding tissues. Although they provide very good results, these interventions significantly increase the level of expertise required for practitioners. Medical robotics has the potential to assist surgical gesture, overcome limitations due to human factors and increase the accuracy of tools positioning. However, the deformation of the organs remains an open problem limiting the development of these robots for wide use in the operating room.
We introduce new solutions for the control of medical robots interacting with soft tissues. The originality of our approach lies in the fact that we address the problem of deformations solving an inverse Finite Element problem directly in the control loop of the robot. This work is motivated by recent advances in the field of medical simulation. FE simulations are now widely used for training surgeons, and they recently reached a sufficient level of realism to assist surgeons during the operation thanks to augmented reality.
We show these FE models enable the possibility to control the motion of a robotic system accounting for deformations of the needle and the tissue during the insertion. This requires to solve important scientific obstacles including 1) complex modeling of needle/tissues interactions and numerical strategies to reach real-time computation 2) non-rigid registration since very few information can be extracted in real time from images during an intervention 3) numerical strategies to derive information provided by the simulations and compute input commands of the robot.
Dr. Hadrien Courtecuisse is a research Scientist (Chargé de recherche) in the team AVR/ICube at CNRS in Strasbourg (since November 2013). He obtained his Ph.D thesis in the SHAMAN team at Inria in 2011 (New parallel architectures for interactive medical simulations). In 2012 he did a postdoc as research-associate in the Institute of Mechanics and Advanced Materials (IMAM) at Cardiff University with Stéphane Bordas. In 2013 he moved to Strasbourg to work as a research engineer for the Institut Hospitalo-Universitaire (IHU). His major contributions are related to sparse linear algebra, collision detection, simulation of contact response, with a particular interest to real-time simulations and parallel architectures such as GPU and parallel co-processor. In 2013, he also joined the Inria Mimesis project lead by Stéphane Cotin and mainly focused his research activity on the development of original methods to control medical robots interacting with deformable structure using inverse real-time FE models.
Department of Computer Science, University of Verona (Italy)
Strada le Grazie 15, 37134 Verona