PURPOSE: Understanding the three-dimensional anatomy of percutaneous intervention in prostate cancer is essential to avoid complications. Recently, attempts have been made to use machine learning to automate the segmentation of functional structures such as the prostate gland, rectum, and bladder. However, a paucity of material is available to segment extracapsular structures that are known to cause needle deflection during percutaneous interventions. This research aims to explore the feasibility of the automatic segmentation of prostate and extracapsular structures to predict needle deflection. METHODS: Using pelvic magnetic resonance imagings (MRIs), 3D U-Net was trained and optimized for the prostate and extracapsular structures (bladder, rectum, pubic bone, pelvic diaphragm muscle, bulbospongiosus muscle, bull of the penis, ischiocavernosus muscle, crus of the penis, transverse perineal muscle, obturator internus muscle, and seminal vesicle). The segmentation accuracy was validated by putting intra-procedural MRIs into the 3D U-Net to segment the prostate and extracapsular structures in the image. Then, the segmented structures were used to predict deflected needle path in in-bore MRI-guided biopsy using a model-based approach. RESULTS: The 3D U-Net yielded Dice scores to parenchymal organs (0.61-0.83), such as prostate, bladder, rectum, bulb of the penis, crus of the penis, but lower in muscle structures (0.03-0.31), except and obturator internus muscle (0.71). The 3D U-Net showed higher Dice scores for functional structures ([Formula: see text]0.001) and complication-related structures ([Formula: see text]0.001). The segmentation of extracapsular anatomies helped to predict the deflected needle path in MRI-guided prostate interventions of the prostate with the accuracy of 0.9 to 4.9 mm. CONCLUSION: Our segmentation method using 3D U-Net provided an accurate anatomical understanding of the prostate and extracapsular structures. In addition, our method was suitable for segmenting functional and complication-related structures. Finally, 3D images of the prostate and extracapsular structures could simulate the needle pathway to predict needle deflections.

PURPOSE: The bronchoscopist's ability to locate the lesion with the bronchoscope is critical for a transbronchial biopsy. However, much less study has been done on the transbronchial biopsy route. This study aims to determine whether the geometrical attributes of the bronchial route can predict the difficulty of reaching tumors in bronchoscopic intervention. METHODS: This study included patients who underwent bronchoscopic diagnosis of lung tumors using electromagnetic navigation. The biopsy instrument was considered "reached" and recorded as such if the tip of the tracked bronchoscope or extended working channel was in the tumors. Four geometrical indices were defined: Local curvature (LC), plane rotation (PR), radius, and global relative angle. A Mann-Whitney U test and logistic regression analysis were performed to analyze the difference in geometrical indices between the reachable and unreachable groups. Receiver operating characteristic analysis (ROC) was performed to evaluate the geometrical indices to predict reachability. RESULTS: Of the 41 patients enrolled in the study, 16 patients were assigned to the unreachable group and 25 patients to the reachable group. LC, PR, and radius have significantly higher values in unreachable cases than in reachable cases ([Formula: see text], [Formula: see text], [Formula: see text]). The logistic regression analysis showed that LC and PR were significantly associated with reachability ([Formula: see text], [Formula: see text]). The areas under the curve with ROC analysis of the LC and PR index were 0.903 and 0.618. The LC's cut-off value was 578.25. CONCLUSION: We investigated whether the geometrical attributes of the bronchial route to the lesion can predict the difficulty of reaching the lesions in the bronchoscopic biopsy. LC, PR, and radius have significantly higher values in unreachable cases than in reachable cases. LC and PR index can be potentially used to predict the navigational success of the bronchoscope.

When preprocedural images are overlaid on intraprocedural images, interventional procedures benefit in that more structures are revealed in intraprocedural imaging. However, image artifacts, respiratory motion, and challenging scenarios could limit the accuracy of multimodality image registration necessary before image overlay. Ensuring the accuracy of registration during interventional procedures is therefore critically important. The goal of this study was to develop a novel framework that has the ability to assess the quality (i.e., accuracy) of nonrigid multimodality image registration accurately in near real time. We constructed a solution using registration quality metrics that can be computed rapidly and combined to form a single binary assessment of image registration quality as either successful or poor. Based on expert-generated quality metrics as ground truth, we used a supervised learning method to train and test this system on existing clinical data. Using the trained quality classifier, the proposed framework identified successful image registration cases with an accuracy of 81.5%. The current implementation produced the classification result in 5.5 s, fast enough for typical interventional radiology procedures. Using supervised learning, we have shown that the described framework could enable a clinician to obtain confirmation or caution of registration results during clinical procedures.

This study aims to validate the advantage of the new engineering method to maneuver multi-section robotic bronchoscope with first person view control in transbronchial biopsy. Six physician operators were recruited and tasked to operate a manual and a robotic bronchoscope to the peripheral area placed in patient-derived lung phantoms. The metrics collected were the furthest generation count of the airway the bronchoscope reached, force incurred to the phantoms, and NASA-Task Load Index. The furthest generation count of the airway the physicians reached using the manual and the robotic bronchoscopes were 6.6 +/- 1.2th and 6.7 +/- 0.8th. Robotic bronchoscopes successfully reached the 5th generation count into the peripheral area of the airway, while the manual bronchoscope typically failed earlier in the 3rd generation. More force was incurred to the airway when the manual bronchoscope was used (0.24 +/- 0.20 [N]) than the robotic bronchoscope was applied (0.18 +/- 0.22 [N], p<0.05). The manual bronchoscope imposed more physical demand than the robotic bronchoscope by NASA-TLX score (55 +/- 24 vs 19 +/- 16, p<0.05). These results indicate that a robotic bronchoscope facilitates the advancement of the bronchoscope to the peripheral area with less physical demand to physician operators. The metrics collected in this study would expect to be used as a benchmark for the future development of robotic bronchoscopes.

OBJECTIVE: The purpose of this article is to report the translational process of an implantable microdevice platform with an emphasis on the technical and engineering adaptations for patient use, regulatory advances, and successful integration into clinical workflow. METHODS: We developed design adaptations for implantation and retrieval, established ongoing monitoring and testing, and facilitated regulatory advances that enabled the administration and examination of a large set of cancer therapies simultaneously in individual patients. RESULTS: Six applications for oncology studies have successfully proceeded to patient trials, with future applications in progress. CONCLUSION: First-in-human translation required engineering design changes to enable implantation and retrieval that fit with existing clinical workflows, a regulatory strategy that enabled both delivery and response measurement of up to 20 agents in a single patient, and establishment of novel testing and quality control processes for a drug/device combination product without clear precedents. SIGNIFICANCE: This manuscript provides a real-world account and roadmap on how to advance from animal proof-of-concept into the clinic, confronting the question of how to use research to benefit patients.

[Background] Electromagnetically Navigated Bronchoscopy (ENB) is currently the state-of-the art diagnostic and interventional bronchoscopy. CT-to-body divergence is a critical hurdle in ENB, causing navigation error and ultimately limiting the clinical efficacy of diagnosis and treatment. In this study, Visually Navigated Bronchoscopy (VNB) is proposed to address the aforementioned issue of CT-to-body divergence. [Materials and Methods] We extended and validated an unsupervised learning method to generate a depth map directly from bronchoscopic images using a Three Cycle-Consistent Generative Adversarial Network (3cGAN) and registering the depth map to preprocedural CTs. We tested the working hypothesis that the proposed VNB can be integrated to the navigated bronchoscopic system based on 3D Slicer, and accurately register bronchoscopic images to pre-procedural CTs to navigate transbronchial biopsies. The quantitative metrics to asses the hypothesis we set was Absolute Tracking Error (ATE) of the tracking and the Target Registration Error (TRE) of the total navigation system. We validated our method on phantoms produced from the pre-procedural CTs of five patients who underwent ENB and on two ex-vivo pig lung specimens. [Results] The ATE using 3cGAN was 6.2 +/- 2.9 [mm]. The ATE of 3cGAN was statistically significantly lower than that of cGAN, particularly in the trachea and lobar bronchus (p < 0.001). The TRE of the proposed method had a range of 11.7 to 40.5 [mm]. The TRE computed by 3cGAN was statistically significantly smaller than those computed by cGAN in two of the five cases enrolled (p < 0.05). [Conclusion] VNB, using 3cGAN to generate the depth maps was technically and clinically feasible. While the accuracy of tracking by cGAN was acceptable, the TRE warrants further investigation and improvement.

Navigated bronchoscopy for the lung biopsy using an electro-magnetic (EM) sensor is often inaccurate due to patient breathing movement during procedures. The objective of this study is to evaluate whether registration of neural network- generated depth images can localize the bronchoscope in navigated bronchoscopy negating the need for EM sensor and error caused by breathing motion. [Methods] Dual CNN-generated depth images followed chained ICP registration were validated in the study. Accuracy was measured by the error between the location after registration and the location of the standard electromagnetic sensor. Difference in accuracy between regions that the neural networks had trained on (seen regions) and regions the networks had never encountered (unseen regions) was validated. [Results] The data collected points to the success of the bronchoscopic localization. Overall mean error of accuracy was 8.75 mm and the overall standard deviation was 4.76mm. For the seen region, the mean error was 6.10mm and the standard deviation was 2.65mm. For the unseen region, the mean error was 11.6mm and the standard deviation was 4.87mm. The results of the two-sample t-test shows that there is a statistically significant difference between the unseen and the seen region. [Conclusion] The results for registration demonstrate that this technique has potential to be implemented in navigational bronchoscopy. The technique produced less error than the electromagnetic sensor in practice, especially accounting for the estimated practical error due to experimental setup.

BACKGROUND: In a combined endoscopic third ventriculostomy (ETV) and endoscopic tumor biopsy (ETB) procedure, an optimal tool trajectory is mandatory to minimize trauma to surrounding cerebral tissue. OBJECTIVE: This paper presents wire-driven multi-section robot with push-pull wire. The robot is tested to attain follow-the-leader (FTL) motion to place surgical instruments through narrow passages while minimizing the trauma to tissues. METHODS: A wire-driven continuum robot with six sub-sections was developed and its kinematic model was proposed to achieve FTL motion. An accuracy test to assess the robot's ability to attain FTL motion along a set of elementary curved trajectory was performed. We also used hydrocephalus ventricular model created from human subject data to generate five ETV/ETB trajectories and conducted a study assessing the accuracy of the FTL motion along these clinically desirable trajectories. RESULTS: In the test with elementary curved paths, the maximal deviation of the robot was increased from 0.47 mm at 30 turn to 1.78 mm at 180 in a simple C-shaped curve. S-shaped FTL motion had lesser deviation ranging from 0.16 to 0.18 mm. In the phantom study, the greatest tip deviation was 1.45 mm, and the greatest path deviation was 1.23 mm. CONCLUSION: We present the application of a continuum robot with FTL motion to perform a combined ETV/ETB procedure. The validation study using human subject data indicated that the accuracy of FTL motion is relatively high. The study indicated that FTL motion may be useful tool for combined ETV and ETB.

Current standard workflows of ultrasound (US)-guided needle insertion require physicians to use their both hands: holding the US probe to locate interested areas with the non-dominant hand and the needle with the dominant hand. This is due to the separation of functionalities for localization and needle insertion. This requirement does not only make the procedure cumbersome, but also limits the reliability of guidance given that the positional relationship between the needle and US images is unknown and interpreted with their experience and assumption. Although the US-guided needle insertion may be assisted through navigation systems, recovery of the positional relationship between the needle and US images requires the usage of external tracking systems and image-based tracking algorisms that may involve the registration inaccuracy. Therefore, there is an unmet need for the solution that provides a simple and intuitive needle localization and insertion to improve the conventional US-guided procedure. In this work, we propose a new device concept solution based on the ring-arrayed forward-viewing (RAF) ultrasound imaging system. The proposed system is comprised with ring-arrayed transducers and an open whole inside the ring where the needle can be inserted. The ring array provides forward-viewing US images, where the needle path is always maintained at the center of the reconstructed image without requiring any registration. As the proof of concept, we designed single-circle ring-arrayed configurations with different radiuses and visualized point targets using the forward-viewing US imaging through simulations and phantom experiments. The results demonstrated the successful target visualization and indicates the ring-arrayed US imaging has a potential to improve the US-guided needle insertion procedure to be simpler and more intuitive.

PURPOSE: Magnetic resonance imaging (MRI)-guided percutaneous cryotherapy of abdominal lesions, an established procedure, uses MRI to guide and monitor the cryoablation of lesions. Methods to precisely guide cryotherapy probes with a minimum amount of trial-and-error are yet to be established. To aid physicians in attaining precise probe alignment without trial-and-error, a body-mounted motorized cryotherapy-probe alignment device (BMCPAD) with motion compensation was clinically tested in this study. The study also compared the contribution of body motion and organ motion compensation to the guidance accuracy of a body-mounted probe alignment device. METHODS: The accuracy of guidance using the BMCPAD was prospectively measured during MRI-guided percutaneous cryotherapies before insertion of the probes. Clinical parameters including patient age, types of anesthesia, depths of the target, and organ sites of target were collected. By using MR images of the target organs and fiducial markers embedded in the BMCPAD, we retrospectively simulated the guidance accuracy with body motion compensation, organ motion compensation, and no compensation. The collected data were analyzed to test the impact of motion compensation on the guidance accuracy. RESULTS: Thirty-seven physical guidance of probes were prospectively recorded for sixteen completed cases. The accuracy of physical guidance using the BMCPAD was 13.4 ± 11.1 mm. The simulated accuracy of guidance with body motion compensation, organ motion compensation, and no compensation was 2.4 ± 2.9 mm, 2.2 ± 1.6 mm, and 3.5 ± 2.9 mm, respectively. Data analysis revealed that the body motion compensation and organ motion compensation individually impacted the improvement in the accuracy of simulated guidance. Moreover, the difference in the accuracy of guidance either by body motion compensation or organ motion compensation was not statistically significant. The major clinical parameters impacting the accuracy of guidance were the body and organ motions. Patient age, types of anesthesia, depths of the target, and organ sites of target did not influence the accuracy of guidance using BMCPAD. The magnitude of body surface movement and organ movement exhibited mutual statistical correlation. CONCLUSIONS: The BMCPAD demonstrated guidance accuracy comparable to that of previously reported devices for CT-guided procedures. The analysis using simulated motion compensation revealed that body motion compensation and organ motion compensation individually impact the improvement in the accuracy of device-guided cryotherapy probe alignment. Considering the correlation between body and organ movements, we also determined that body motion compensation using the ring fiducial markers in the BMCPAD can be solely used to address both body and organ motions in MRI-guided cryotherapy.

This paper presents the development, preclinical evaluation, and preliminary clinical study of a robotic system for targeted transperineal prostate biopsy under direct interventional magnetic resonance imaging (MRI) guidance. The clinically integrated robotic system is developed based on a modular design approach, comprised of surgical navigation application, robot control software, MRI robot controller hardware, and robotic needle placement manipulator. The system provides enabling technologies for MRI-guided procedures. It can be easily transported and setup for supporting the clinical workflow of interventional procedures, and the system is readily extensible and reconfigurable to other clinical applications. Preclinical evaluation of the system is performed with phantom studies in a 3 Tesla MRI scanner, rehearsing the proposed clinical workflow, and demonstrating an in-plane targeting error of 1.5mm. The robotic system has been approved by the institutional review board (IRB) for clinical trials. A preliminary clinical study is conducted with the patient consent, demonstrating the targeting errors at two biopsy target sites to be 4.0 and 3.7, which is sufficient to target a clinically significant tumor foci. First-in-human trials to evaluate the system's effectiveness and accuracy for MR image-guide prostate biopsy are underway.

PURPOSE: Current manual catheters for transbronchial biopsy in the lung lack a steering ability, which hampers a physician's ability to reach nodules in the peripheral lung. The objective of this paper is to design and build a multisection robot with a follow-the-leader motion and compare the performance of the conventional catheter and our robotic catheter in the right main and right segmental lobar bronchus. METHODS: A three-section continuum robot with an outer diameter of 3 mm was developed. Each section includes one anchored wire and two driving wires made of stainless steel. Follow-the-leader control is implemented using a joystick for a physician to control the distal section of the robot, while the subsequent two sections follow the controlled distal section. RESULTS: The robotic catheter deviated from the preplanned approach path by less than the manual catheter did (robotic: [Formula: see text] mm and manual: [Formula: see text] mm), with [Formula: see text]. The average force applied to the wall, producing potential trauma to the wall, was less for the robotic catheter ([Formula: see text] N) than for the manual catheter ([Formula: see text] N), [Formula: see text]. CONCLUSION: This study demonstrated an improvement in the maneuverability for the robotic catheter. In addition to a greater aptitude for reaching a peripheral area of the lung, these findings suggest that the designated target in a peripheral area can be reached with less trauma to the bronchi wall.

Intra-operative imaging is sometimes available to assist needle biopsy, but typical open-loop insertion does not account for unmodeled needle deflection or target shift. Closed-loop image-guided compensation for deviation from an initial straight-line trajectory through rotational control of an asymmetric tip can reduce targeting error. Incorporating robotic closed-loop control often reduces physician interaction with the patient, but by pairing closed-loop trajectory compensation with hands-on cooperatively controlled insertion, a physician's control of the procedure can be maintained while incorporating benefits of robotic accuracy. A series of needle insertions were performed with a typical 18G needle using closed-loop active compensation under both fully autonomous and user-directed cooperative control. We demonstrated equivalent improvement in accuracy while maintaining physician-in-the-loop control with no statistically significant difference (p > 0.05) in the targeting accuracy between any pair of autonomous or individual cooperative sets, with average targeting accuracy of 3.56 mm. With cooperatively controlled insertions and target shift between 1 and 10 mm introduced upon needle contact, the system was able to effectively compensate up to the point where error approached a maximum curvature governed by bending mechanics. These results show closed-loop active compensation can enhance targeting accuracy, and that the improvement can be maintained under user directed cooperative insertion.

While the interaction between a needle and the surrounding tissue is known to cause a significant targeting error in prostate biopsy leading to false-negative results, few studies have demonstrated how it impacts in the actual procedure. We performed a pilot study on robot-assisted MRI-guided prostate biopsy with an emphasis on the in-depth analysis of the needle-tissue interaction in vivo. The data were acquired during in-bore transperineal prostate biopsies in patients using a 4 degrees-of-freedom (DoF) MRI-compatible robot. The anatomical structures in the pelvic area and the needle path were reconstructed from MR images, and quantitatively analyzed. We analyzed each structure individually and also proposed a mathematical model to investigate the influence of those structures in the targeting error using the mixed-model regression. The median targeting error in 188 insertions (27 patients) was 6.3 mm. Both the individual anatomical structure analysis and the mixed-model analysis showed that the deviation resulted from the contact between the needle and the skin as the main source of error. On contrary, needle bending inside the tissue (expressed as needle curvature) did not vary among insertions with targeting errors above and below the average. The analysis indicated that insertions crossing the bulbospongiosus presented a targeting error lower than the average. The mixed-model analysis demonstrated that the distance between the needle guide and the patient skin, the deviation at the entry point, and the path length inside the pelvic diaphragm had a statistically significant contribution to the targeting error (p  <  0.05). Our results indicate that the errors associated with the elastic contact between the needle and the skin were more prominent than the needle bending along the insertion. Our findings will help to improve the preoperative planning of transperineal prostate biopsies.

Patient-mounted needle guide devices for percutaneous ablation are vulnerable to patient motion. The objective of this study is to develop and evaluate a software system for an MRI-compatible patient-mounted needle guide device that can adaptively compensate for displacement of the device due to patient motion using a novel image-based automatic device-to-image registration technique. We have developed a software system for an MRI-compatible patient-mounted needle guide device for percutaneous ablation. It features fully-automated image-based device-to-image registration to track the device position, and a device controller to adjust the needle trajectory to compensate for the displacement of the device. We performed: (a) a phantom study using a clinical MR scanner to evaluate registration performance; (b) simulations using intraoperative time-series MR data acquired in 20 clinical cases of MRI-guided renal cryoablations to assess its impact on motion compensation; and (c) a pilot clinical study in three patients to test its feasibility during the clinical procedure. FRE, TRE, and success rate of device-to-image registration were 2.71 ± 2.29 mm, 1.74 ± 1.13 mm, and 98.3% for the phantom images. The simulation study showed that the motion compensation reduced the targeting error for needle placement from 8.2 mm to 5.4 mm (p  <  0.0005) in patients under general anesthesia (GA), and from 14.4 mm to 10.0 mm (p < 1.0 × 10(−5)) in patients under monitored anesthesia care (MAC). The pilot study showed that the software registered the device successfully in a clinical setting. Our simulation study demonstrated that the software system could significantly improve targeting accuracy in patients treated under both MAC and GA. Intraprocedural image-based device-to-image registration was feasible.

Robots have been found to be a useful tool in magnetic resonance imaging (MRI)-guided intervention. The utility of robots in MRI-guided therapy ranges from aid for precision targeting to high-dexterity surgical tools to improve or even enable new MRI-guided therapy options. The objective of this article is to review the technical aspects of robotics in MRI-guided interventions, highlight the role of MRI robots in prostate interventions, and finally discuss the future contribution of emerging robotics technology useful in MRI-guided intervention.

PURPOSE: To develop and evaluate an approach to estimate the respiratory-induced motion of lesions in the chest and abdomen. MATERIALS AND METHODS: The proposed approach uses the motion of an initial reference needle inserted into a moving organ to estimate the lesion (target) displacement that is caused by respiration. The needles position is measured using an inertial measurement unit (IMU) sensor externally attached to the hub of an initially placed reference needle. Data obtained from the IMU sensor and the target motion are used to train a learning-based approach to estimate the position of the moving target. An experimental platform was designed to mimic respiratory motion of the liver. Liver motion profiles of human subjects provided inputs to the experimental platform. Variables including the insertion angle, target depth, target motion velocity and target proximity to the reference needle were evaluated by measuring the error of the estimated target position and processing time. RESULTS: The mean error of estimation of the target position ranged between 0.86 and 1.29 mm. The processing maximum training and testing time was 5 ms which is suitable for real-time target motion estimation using the needle position sensor. CONCLUSION: The external motion of an initially placed reference needle inserted into a moving organ can be used as a surrogate, measurable and accessible signal to estimate in real-time the position of a moving target caused by respiration; this technique could then be used to guide the placement of subsequently inserted needles directly into the target.

Magnetic Resonance Imaging (MRI) provides both anatomical imaging with excellent soft tissue contrast and functional MRI imaging (fMRI) of physiological parameters. The last two decades have witnessed the manifestation of increased interest in MRI-guided minimally invasive intervention procedures and fMRI for rehabilitation and neuroscience research. Accompanying the aspiration to utilize MRI to provide imaging feedback during interventions and brain activity for neuroscience study, there is an accumulated effort to utilize force sensors compatible with the MRI environment to meet the growing demand of these procedures, with the goal of enhanced interventional safety and accuracy, improved efficacy and rehabilitation outcome. This paper summarizes the fundamental principles, the state of the art development and challenges of fiber optic force sensors for MRI-guided interventions and rehabilitation. It provides an overview of MRI-compatible fiber optic force sensors based on different sensing principles, including light intensity modulation, wavelength modulation, and phase modulation. Extensive design prototypes are reviewed to illustrate the detailed implementation of these principles. Advantages and disadvantages of the sensor designs are compared and analyzed. A perspective on the future development of fiber optic sensors is also presented which may have additional broad clinical applications. Future surgical interventions or rehabilitation will rely on intelligent force sensors to provide situational awareness to augment or complement human perception in these procedures.

The regulatory, ethical, and legal barriers imposed on medical robots necessitate careful consideration of different levels of autonomy, as well as the context for use.