Abstract
Purpose:
NousNav is a low-cost, open-source neuronavigation platform built to address the high costs and resource limitations that hinder access to advanced neurosurgical technologies in low-resource settings. The low-cost and accessibility of the system is made possible using consumer-grade optical tracking and open-source software packages. This study aims to assess the performance of these core enabling technologies by quantifying their spatial accuracy and comparing it to a commercial gold standard.
Methods:
A series of experiments were conducted to evaluate the capabilities of the selected hardware and registration infrastructure utilized in NousNav. Each component was tested both in a simulated bench-top environment and clinically across four brain tumor resection cases.
Results:
The Optitrack Duo tracker used by NousNav was found to have a mean localization error of 0.8mm (SD 0.4mm). In bench-top phantom testing, NousNav had an average target registration error of 5.0mm (SD 2.3mm) following patient registration. Clinical evaluations revealed a mean distance of 4.2mm (SD 1.5mm) between points reported by NousNav versus those obtained using a commercial neuronavigation system.
Conclusion:
These experiments highlight the role of baseline camera tracking performance, tracked instrument calibration, and patient positioning on the spatial performance of NousNav. They also provide an essential benchmark assessment of the system to help inform future clinical use-cases and direct ongoing system development.
Keywords: Neuronavigation, open-source, low-cost, benchmarking
1. Introduction
Global inequities in neurosurgical care are profound, both in terms of disease burden and access to neurosurgical care. Although low- and middle-income countries (LMICs) account for an estimated 82% of patients deemed to require neurosurgical intervention, they host only 56% of the neurosurgical workforce [1]. This disparity is particularly acute on the African continent, where LMICs represent 15% of the global neurosurgical demand but have access to less than 1% of neurosurgeons [2]. Widespread access to neurosurgical care has been increasingly recognized as a critical area of development to address global surgical inequities [3]. While technological and clinical innovations continue to improve neurosurgical outcomes, the benefits of this progress are often not available to patients in LMICs [4]. These medical advances tend to rely on expensive technology – an inherent barrier to access for a large percentage of the population.
Computer-assisted neuronavigation is one such technology that affords significant advantages for clinicians and their patients yet remains inaccessible to many. These systems are typically used to fuse pre-operative imaging with intraoperative instrument tracking to allow the surgeon to localize critical anatomy. Widely used commercial neuronavigation systems include Brainlab (Brainlab, Munich, Germany), Medtronic’s StealthStation (Medtronic, Dublin, Ireland), and the Stryker Nav3i (Stryker, Kalamazoo, Michigan, USA). These systems typically range in price from $250,000 to $750,000 USD [5]. This high upfront cost is reflective of the equipment, software licenses, instruments, customer support, sales, and other costs related to commercial clinical systems in high-resource settings. In addition, there are expensive disposable components that must be replaced for each procedure. All of these considerations drastically limit the healthcare settings where such a system can be sustainably deployed [6].
The NousNav project is an ongoing research effort to build an inexpensive neuronavigation system that reduces these upfront costs by over 95% while working with local clinicians and engineers to establish capacity on-site for operation and maintenance [7]. The NousNav system is comprised of an accessible infrared optical tracker, inexpensive reflective markers to track surgical instruments, and an open-source software package to support pre-operative planning, patient registration, and intraoperative guidance. Generally, NousNav follows the basic paradigm of a frame-less, optically-tracked neuronavigation platform while introducing two core innovations: 1. Inexpensive, consumer-grade equipment, and 2. The use of well-established open-source software to support essential neurosurgical workflows.
While these innovations do achieve the goal of reducing the upfront cost of the system, it is important to understand the impact of using these technologies on system performance. A commonly used metric for evaluating the overall spatial accuracy of a navigation system is target registration error (TRE). This metric can be used to quantify the discrepancy between the true position of target anatomy and the position predicted in the navigation display [8]. To contextualize the range of TRE values that commercial optically-tracked neuronavigation systems tend to achieve, a recent meta analysis reported a mean TRE of 3.96mm (SD 1.98mm) across 1431 measurements on patients and 2.49mm (SD 1.62mm) across 7627 phantom-based measurements [9]. While minimizing this spatial error is generally the objective of surgical navigation systems, the upper cut-off value for TRE varies depending on the procedure being performed.
There are certain procedures that have a more exacting standard for accuracy. In a procedure like deep brain stimulation, for example, electrode placement within 2mm of the intended target can be considered sufficient to not require repositioning [10]. This level of accuracy generally requires the use of a sophisticated neuronavigation system. In contrast, there are a number of neurosurgical procedures that are regularly carried out using anatomical landmarks and hand-held rulers in lower-resource contexts yet would benefit from some form of real-time navigation. Some examples include planning the location of a craniotomy, entry point selection for twist drill procedures, bed-side ventriculostomy, abscess drainage, and subdural hematoma evacuations [11]. These procedures represent the ideal candidates for the use of a low-cost neuronavigation platform, since they can tolerate the localization error of existing low-cost navigation techniques but tend to have improved outcomes when neuronavigation systems are introduced. Craniotomy is one such example of a surgical procedure that can be performed in the absence of neuronavigation, but introducing image-guidance leads to smaller, more precise craniotomies with reduced blood loss and surgical time [12].
The TRE of a navigation system is a complex aggregate that reflects both the inherent hardware capabilities and the cumulative errors introduced at each stage of the surgical workflow. The design decisions that enable NousNav to emulate a commercial neuronavigation solution at a fraction of the cost may affect the achievable spatial performance of the system. Quantifying this impact in a controlled setting is imperative to accurately convey the performance and limitations of the system to potential users, as well as understanding how the system could be improved. It can be especially challenging to benchmark surgical navigation systems in a clinical setting where a ground truth anatomical target position may be unknown. For this reason, bench-top unit testing can be combined with clinical evaluation of individual components to help infer the expected performance of a surgical navigation platform. It also serves as a reference measurement against which system improvements can be quantitatively assessed for their positive or negative impact on the overall performance.
Therefore, in this paper, we evaluate the performance of NousNav in both bench-top and clinical settings to assess the system’s overall spatial accuracy. We also aim to understand which elements of the system contribute most to error in order to determine what should be improved in future design iterations. More specifically, we quantify the impact of the low-cost tracking camera and the patient registration workflow on spatial measurement using an anatomical phantom and within an OR environment. We use the Brainlab Curve neuronavigation system as a reference for all experiments. Through these experiments, we aim to establish a baseline performance measurement for NousNav, contextualize these results against a commercial counterpart, and identify directions through which future work can improve the system.
2. Methods
2.1. System overview
The NousNav system is comprised of an optical tracking camera, a computer to run the NousNav software application, a patient reference array, hereafter referred to as a static reference body (SRB), and a tracked pointer. While the tracking camera and computer can be swapped to accommodate different budgets and hardware availability, this study focuses on using an Optitrack Duo infrared tracking platform (NaturalPoint, Inc., Corvallis, Oregon, USA) and a Dell XPS 15 laptop (Dell Technologies, Round Rock, Texas, USA). The cost of the whole system in this embodiment is less than $5000 USD, and can be seen in Figure 1.
Fig. 1.
NousNav system overview including the tracking camera, pointer, static reference body, a skull phantom in place of the patient, and a demonstration of the imaging that would be shown in the software for real-time navigation. Note that a different pointer and static reference body than those shown here are used during the experiments.
The software for NousNav is based on 3D Slicer, a freely available, open-source medical image visualization and analysis platform with an established history of use in health-equity settings [13]. With over 1.5 million downloads and an active international community of users and developers, 3D Slicer has proven itself as an enabling technology for accessible image-guided therapy (IGT) solutions in limited-resource healthcare contexts [14]. Furthermore, numerous image-guided surgery projects have demonstrated that the open-source workflows implemented in 3D Slicer to support registration and calibration are sufficiently precise for surgical applications [15],[16], [17]. NousNav’s custom 3D Slicer application is integrated with the Optitrak Duo through the Public Library for Ultrasound (PLUS) Toolkit [18] and OpenIGTLink [19], simplifying the development and deployment process.
2.2. NousNav workflow
The NousNav workflow can be broken down into three steps: 1. Pre-operative planning, 2. Registration, and 3. Navigation. The overview and sub-steps for each part of the process can be seen in Figure 2.
Fig. 2.
Left to right: Step 1: Pre-operative planning involving skin and target segmentation, trajectory planning, and landmark selection. Step 2: Registration steps involving pointer calibration, landmark registration and dense registration. Step 3: Navigation screen that shows real-time navigation and surgical guidance relative to pre-operative plans.
The workflow starts with an initial planning phase. During the planning phase, users load 3D imaging of the patient and are guided through several steps to process the 3D imaging data. The first step involves segmenting the patient’s skin. The user can control the segmentation by manipulating a slider to control the thresholding level and smoothness of the resulting model. NousNav uses 3D Slicer’s built in segmentation tools to threshold, smooth, bridge gaps, and generate a convex hull of the skin. A model of the surgical target is then produced through user segmentation, as well as a planned trajectory along which to access the target. The plan is completed by the user identifying at least 3 registration landmarks on the surface of the skin segmentation. The anatomical landmarks typically used for registration are the right outer canthus, left outer canthus, and nasion, however the user has the option of selecting up to 9. The full list of possible registration landmarks includes the tragus and inner canthus on both sides, as well as the inion and acanthion.
The next phase is registration, which is facilitated using well-established tools in SlicerIGT, an open-source package for image-guided interventions in 3D Slicer. This phase begins with the user calibrating an optically-tracked pointer to establish where its tip is located relative to reflective markers. The user is also instructed to secure an SRB rigidly to the patient’s skull fixation hardware to enable direct tracking of the patient position. The patient registration is then started by using the tracked pointer to digitize the landmarks identified on the patient during the planning phase. This serves to establish an initial alignment between the physical patient and the imaging data through a paired-point registration. This registration is then refined by tracing the tracked pointer along the surface of the patient’s skin to collect up to 200 points for ICP-based registration[20]. The user is able to view the points that have been collected in real time, and can add more points or redo the point collection as needed.
The user confirms registration accuracy by comparing surface landmarks on the patient’s head to the imaging dataset. Following this verification step, the user can proceed to the navigation phase. This final phase helps the user visualize the patient, surgical target, intended trajectory and tracked pointer in a similar manner to commercial neuronavigation systems, assisting with pre-operative craniotomy planning and intraoperative spatial reasoning. The spatial accuracy with which the visualization can reflect the physical location of instruments relative to the real patient is dictated by the quality of the previously described workflow. As outlined, decisions and actions made by both the developers and end-users at each step of the process can contribute to the overall system error.
2.3. Experiments
For each of the subsequent experiments, we use the equipment (pointers, DRBs, SRBs, and camera) and software included in the Brainlab Curve neuronavigation system as a baseline. We chose to use Brainlab because it is a widely adopted and well established platform for neurnavigation, and the Brainlab Curve was readily available at our clinical center. The following sections describe these experiments and comparisons in more detail.
2.3.1. Bench-top tracked pointer evaluation
The baseline performance of the Optitrack Duo tracking system was assessed and compared to the commercial navigation system’s tracker using two Optitrack calibration bars as phantoms. Each bar has four precisely spaced divots, with distances measured to 1-micron tolerance by the manufacturer. Since the trackers cannot measure divots directly, we used the commercial system’s tracked pointers to measure divot locations. These pointers each use three or more retroreflective spheres manufactured by NDI to facilitate tracking (Northern Digital Inc., Waterloo, Canada). The first pointer, referred to as Pointer A, comes pre-calibrated in the commercial navigation system. The second pointer, Pointer B, consists principally of a commercial cluster of retroreflective markers, hereafter referred to as a dynamic reference body (DRB), which is designed to be rigidly attached to surgical instruments. For all experiments presented in this work, the commercial DRB is securely clamped to a stainless steel surgical probe and requires manual pivot calibration to determine the tool tip’s location. Both pointers are shown on the right side of Figure 3.
Fig. 3.
The experimental setup used during the tracked pointer experiment, showing the location of the cameras relative to the calibration bars and tracked pointers. The red box highlights the arrangement of the two trackers relative to one another. On the right side of the figure, Pointer A and Pointer B are shown to scale relative to one another.
Although NousNav includes its own pointer and SRB, as seen in Figure 1, all experiments in this study were conducted using commercial SRBs and pointers. This approach was intended to eliminate confounding variables, such as differences in pointer geometry or length, allowing the impact of the tracking system and workflow differences to be assessed in isolation. To this end, Pointers A and B were pivot calibrated within NousNav to facilitate their use in measuring divot locations. For both neuronavigation systems, the pointers were tracked relative to a commercial SRB secured to a table. The calibration bars were mounted orthogonally on the table, as shown in Figure 3.
To control for the distance of the trackers from the calibration bars, both trackers were positioned parallel to the table and within 20 cm of each other. Pivot calibration for each of the pointers was carried out five times, with the root mean square error (RMSE) reported by the least-squares minimization algorithm recorded for each calibration. Since the RMSE can be used as an indication of estimated pointer tip precision, the calibration that yielded the lowest RMSE was then selected for use within the experiment. The table was placed equidistant from the two trackers at one of three clinically realistic distances: 1 meter, 1.5 meters, and 2 meters. All measurements were made in the Advanced Multimodality Image Guided Operating (AMIGO) suite at Brigham and Women’s Hospital under typical OR lighting conditions. At each table location, for each pointer, all 8 divots were measured 3 separate times. During each digitization, the pointer tip was placed within the divot and held in a single static pose visible to both trackers. The pointer tip location was recorded using NousNav and the commercial system simultaneously. This yielded 288 separate divot localizations, or 216 inter-divot distances. Following outlier removal using the interquartile range method, 202 inter-divot distances remained and were used for subsequent analysis.
2.3.2. Bench-top workflow evaluation
Patient-to-image registration is a critical component of the NousNav software. This registration impacts all downstream tasks within the system, limiting the achievable accuracy of NousNav and influencing the clinical interpretation of spatial information. We therefore designed an experiment to compare the impact of using open-source software libraries to perform patient registration versus proprietary commercial libraries. Again, as a baseline comparison, we use the BrainLab Curve system.
The experiment was performed using an anatomical phantom skull in the AMIGO suite. A total of 21 radio-opaque fiducial spheres were adhered to the surface of the skull. This included 8 fiducials on the exterior of the skull and 13 fiducials on the cranial floor of the phantom. These fiducial locations were selected to correspond with the intended use-cases of NousNav. The exterior fiducials assess the performance of NousNav during more superficial neurosurgical procedures, including craniotomy planning, hematoma evacuation or abscess drainage. The fiducials adhered to the cranial floor simulate the deepest locations within the skull that might need to be accessed and measured after a craniotomy is performed. A CT scan of the skull was acquired with 0.4mm slice thickness and the fiducials were localized in the imaging using a simple thresholding approach. These spherical fiducials were then replaced with divots provided by the commercial system to correspond with the center of each spherical fiducial. The skull was pinned using a Mayfield head clamp, and a standard commercial cranial navigation SRB was secured to the head clamp. The commercial system and NousNav tracking cameras were both placed 1.3 m away from the phantom. To control for differences in tool geometries between NousNav and the commercial system, both systems used the same pointers for all phases of the experiment.
An expert user started by performing the skin segmentation and registration fiducial placement in both the commercial system and in NousNav. The patient registration was then computed in both systems by using Pointer A to localize a set of landmarks on the surface of the phantom. Dense registrations were performed separately for the commercial system and NousNav. The standard registration approach for the commercial system uses approximately 50 points placed individually whereas NousNav relies on roughly 200 points obtained by tracing the pointer along the skin surface. Following registration, a tracked pointer was used to localize each of the 21 divots. The pointer tip location was obtained in both systems simultaneously to control for the impact of pointer angle and position within the divot. The TRE was determined by measuring the Euclidian distance from the fiducials localized in the CT imaging and their observed position within each navigation system. This workflow is illustrated in Figure 4.
Fig. 4.
Workflow steps for registration and testing on phantom skull. Left to right: Step 1 - Sparse registration with key landmarks, Step 2 - Dense registration with surface tracing, Step 3 - Removing the top of the skull, Step 4 - Target tracing inside the skull.
Since the patient registration workflows of NousNav and the commercial system differ, it was important to consider additional variables that might contribute to their difference in performance. One such variable was the use of pre-calibrated versus manually calibrated pointers, as was explored in Section 2.3.1. The same two pointers were used in this experiment, meaning that both pointers required manual calibration within NousNav but only Pointer B required calibration for use with the commercial system. The two navigation systems also have different options for the number of sparse points to capture during initialization of the patient registration. NousNav requires a minimum of 3 landmark points and supports up to 9 landmarks points being used for sparse registration. Two separate trials were performed to test whether increasing the number of sparse registration points from the minimum of 3 to the maximum allowable number of 9 had any impact on the quality of the resulting registration. A total of 295 measurements were performed, with 13 outliers omitted following interquartile range-based outlier removal.
2.3.3. Clinical workflow evaluation
Finally, to better understand the differences between NousNav and a commercial neuronavigation system in a clinical setting, the systems were compared during four brain tumor resections in the AMIGO suite at Brigham and Women’s Hospital. For these clinical comparisons the appropriate ethics approvals were obtained and informed patient consent was documented by a clinical fellow before each case. To minimize disruption to the clinical workflow, a commercial pointer and SRB was used by both navigation systems. This allowed NousNav and the commercial system to simultaneously report the position of the same physical pointer, reducing the impact of intra-operative testing on overall procedure time.
For each procedure, the NousNav camera and computer were brought into the operating room prior to surgery and set up in a similar position and orientation to the commercial system. The commercial SRB and pointer geometries were integrated into NousNav so that both systems could track the same hardware simultaneously and obtain points during the procedure. The same patient imaging used by the commercial system was loaded for registration and measurement in NousNav. A neurosurgeon supervised various pre-operative steps within NousNav, including skin segmentation and annotation of anatomical landmarks for use during registration. A pivot calibration and axis calibration of the pointers was performed pre-operatively. Following registration of the commercial system to the patient, Pointer A was used with the NousNav system to obtain landmark positions for an initial alignment with the imaging data. The dense registration was then obtained by tracing along the patient skin, with 200 points obtained across the face and forehead.
Following patient registration and qualitative validation of the registration by the clinician, a series of points were simultaneously obtained using the NousNav system and the commercial system. This included a mix of points along the skin surface, captured using Pointer A, as well as points along the skull prior to craniotomy and within the tumor resection cavity, captured using a sterile version of Pointer B. The positions reported by both systems were directly compared, with the commercial system’s positions serving as the ground truth for evaluation. This approach was necessary because tissue deformation and the difficulty of precisely locating anatomical structures in the body during surgery make it challenging to establish an exact ground truth. Furthermore, many of the superficial landmarks that can be most easily localized are used through the process of generating and validation patient registration, excluding them from use as targets to compare the two systems.
3. Results & Discussion
3.1. Bench-top camera evaluation
Across the data captured by Pointers A and B, the commercial system had a mean inter-divot localization error of 0.2mm (SD 0.2mm) while the NousNav system had a mean error of 0.8mm (SD 0.4mm). An ANOVA test confirms that this was a significant difference in the error observed using each system (p <0.0001). There was no significant difference in localization error between Pointer A and Pointer B using either system (p = 0.6471).
Divots were localized at three different distances from the tracking cameras: 1, 1.5, and 2 meters. These results are illustrated in Figure 5, where the inter-divot error is shown separately for each cart position and tracking system. The distance was found to have a significant impact on the tracking performance for both the NousNav system (p <0.0001) and the commercial system (p = 0.0168). Post hoc analysis indicated that position 1 had a significantly lower error than positions 2 (p <0.0001) and 3 (p <0.0001) using NousNav. For the commercial system, this relationship was only significant between positions 1 and 3 (p = 0.004).
Fig. 5.
Boxplots showing the inter-divot localization error in mm at distances of 1 meter, 1.5 meters and 2 meters from the trackers. The NousNav and commercial trackers are illustrated separately. The positions in the workspace for each evaluation are also highlighted on the left.
Given the intended use-cases of these systems, with the commercial system able to support frameless stereotactic biopsy and other high precision procedures, and NousNav intended to outperform navigation-free neurosurgery in lower-resources settings, these differences are expected. However, as the distance from the trackers increased, the average localization error changed significantly. This is a known phenomenon among stereo optical tracking systems where the fiducial localization error (FLE) is anisotropic, or notably higher orthogonal to the plane of the camera’s sensor than in the sensor plane [21, 22]. This results in greater error as the object being tracked is further away from the camera.
It is important to note that the manufacturer of the Optitrack Duo reports the range of the camera as being up to 4.5 meters, so the range in this experiment falls within the operating parameters of the tracker [23]. This same data sheet lists the expected fiducial localization error (FLE) as being ”sub-millimeter” without specifying an exact value. Despite the increased complexity of tracking a tool rather than a single retroreflective marker, the Duo was still able to perform this experimental task with sub-millimeter accuracy. The Brainlab Curve system uses the NDI Polaris Vega (Northern Digital Inc., Waterloo, Canada) tracking platform to perform localization, which has a range of up to 3 meters [24]. A recent study assessing the NDI Polaris found a static localization error of roughly 0.1mm when measuring the distance between two retroreflective spheres, which aligns with our findings in this experiment [25].
In the case of the commercial system, this increase in localization error is gradual, as seen in Figure 5, with the average error increasing from 0.2mm to 0.3mm as the distance from the tracker increased from 1 meter to 2 meters. The anisotropic error distribution is more pronounced in NousNav, with the localization error increasing by 0.5mm when the experiment is conducted 2 meters away from the tracker compared to 1 meter. The lower baseline accuracy of the Optitrack Duo versus the Brainlab Curve’s camera, combined with the Duo’s smaller distance between the stereo tracking sensors, likely both contribute to this more pronounced anisotropy. This is valuable information to share with users of NousNav as it could lead to improved spatial tracking performance strictly through careful positioning of the system. It would also be possible to characterize exactly what the optimal workspace would be for a given camera configuration by leveraging the models of stereoscopic tracking described by Danilchenko and Fitzpatrick [26] and Ma et al. [22]. Future selection of new candidate cameras for the system, or construction of custom tracking setups from off-the-shelf cameras, could benefit from information about how their theoretical workspace matches the intended clinical applications of the system. Positioning within the optimal range of the camera could also be enforced through NousNav’s software, with the tracker’s field of view artificially restricted to distances that yield lower error.
3.2. Bench-top workflow evaluation
Across all measurements in the bench-top workflow experiment, the mean TRE obtained using NousNav (n=155) was 5.0mm (SD 2.3mm), while the mean TRE using the commercial system (n=126) was 2.2mm (SD 1.1mm). This data is illustrated by the boxplot in Figure 6. A one-way ANOVA test found a statistically significant difference in TRE between the NousNav and the commercial system (p <0.001).
Fig. 6.
Left: A close up of the targets that were evaluated and what the reported distances represent. Right: Average euclidean distances for the target in NousNav and a commercial system.
Controlling for the number of sparse landmarks used for registration, the average TRE when 3 sparse points are used (n=166) is 3.6mm (SD 2.1mm), while the TRE when 9 sparse points are used (n=115) is 4.0mm (SD 2.8mm). Calculating the TRE obtained using each distinct pointer, Pointer A had a mean TRE of 3.4mm (SD 2.6mm) across 142 measurements, whereas Pointer B had a TRE of 4.2mm (SD 2.6mm) over 139 measurements. A one-way ANOVA test indicated a significant difference in TRE between the two pointers (p = 0.003).
Given the baseline difference in tracking error established in the first experiment, the overall difference in system error is expected. Beyond the difference in camera between the commercial system and NousNav, there are other distinctions in their respective registration workflows that could contribute to this higher TRE using NousNav. One such distinction is the use of pre-calibrated pointers in the commercial system versus user-calibrated pointers in NousNav. For measurements using the commercial system, we found that the pre-calibrated Pointer A had significantly lower error than Pointer B which required user calibration (p <0.01). It is worth noting that there are other distinctions between Pointer A and Pointer B which could explain this significant difference in performance. These include the geometry of their retroreflectors, the position of the DRB relative to the pointer tip, the shape of the pointer tip, and the shape of the divot used to perform pivot calibration [27, 28]. Figure 3 highlights some of these differences, including the use of a longer pointer shaft and sharper tip in Pointer A.
Despite these clear differences in geometry, when calibrated manually using NousNav there was no longer a significant difference in their localization performance. This suggests that the user calibration step may contribute to the variation seen in pointer performance with the commercial system, and highlights the role that user calibration could play in contributing to the higher TRE of NousNav. The improved tracking performance of pre-calibrated pointers must nonetheless be balanced against their increased complexity to manufacture and the risks of mechanical deformation leading to error, both of which make pre-calibrated pointers less suitable for a low-cost system. Future work related to this topic will focus on quantifying the impact of using a pre-calibrated pointer with NousNav and exploring options for user verification of pre-calibrated instruments in lieu of user-initiated tip calibration.
Despite their differences, both systems do share some similar features to support patient registration. For example, both systems allow the user to customize their selection of sparse registration landmarks for pair-point registration. The impact of this selection on the subsequent spatial performance was evaluated in both systems by alternating between using 3 landmarks for registration and using 9 landmarks. While the commercial system benefited from the use of 9 landmarks rather than just 3 landmarks, increasing the number of landmarks did not have a notable effect on the accuracy of NousNav’s localization. This may be due to fundamental differences in their paired point registration algorithms, such as outlier removal techniques, that make Brainlab better suited to incorporating additional points. The higher FLE of NousNav’s Optitrack Duo compared to the Brainlab tracking camera, as established in Section 3.1, may have also limited the utility of these added landmark points in reducing the overall registration error and downstream TRE.
3.3. Clinical workflow evaluation
In total, 98 points were captured over 4 surgical procedures, with a mean distance of 4.2mm (SD 1.5mm) between the points from Brainlab and those from NousNav. The mean distances in each case are summarized in Table 1. During case 2, 43 points were captured across 4 distinct locations. This included on the skin surface, the skull surface, the surface of the brain, and within the tumor resection cavity. These measurements are summarized in Table 2. The points captured on the brain surface and within the resection cavity are visualized in Figure 7, with points captured using NousNav visible in blue and those from the commercial navigation system in green.
Table 1.
Mean distances and standard deviations (SD) between points captured by NousNav and the commercial system across all cases.
| Case | Points Captured (n) | Mean Distance (mm) | SD (mm) |
|---|---|---|---|
| Case 1 | 11 | 4.0 | 2.3 |
| Case 2 | 43 | 5.0 | 0.8 |
| Case 3 | 21 | 5.2 | 1.6 |
| Case 4 | 23 | 3.2 | 0.7 |
Table 2.
Breakdown of mean distances and standard deviations (SD) for points captured in Case 2.
| Point location | Points Captured (n) | Mean Distance (mm) | SD (mm) |
|---|---|---|---|
| Skin Surface | 14 | 4.2 | 1.4 |
| Skull Surface | 7 | 3.0 | 0.4 |
| Brain Surface | 3 | 5.1 | 0.3 |
| Tumor Cavity | 21 | 5.0 | 0.8 |
Fig. 7.
Visualization of the inter-point distances within the cranial resection cavity and a close-up of what they represent when comparing a commercial navigation system to NousNav.
It is important to clarify that this experiment does not directly measure the TRE of either system. Rather, by using the commercial system as our comparator, these clinical tests give an estimate for how closely NousNav aligns with a clinical gold standard. The clinical gold standard is still subject to its own error including random error introduced by the commercial system’s tracking camera and systematic error added during the patient registration phase. This can impact the spatial accuracy of pointer localization in unpredictable ways, including an underestimation or overestimation of the performance of either system. Despite these limitations, this experiment is still useful as an initial assessment of how well NousNav’s measurements correspond with an existing system in a clinical setting. Even assuming the disparity between the commercial system and NousNav points is strictly due to errors within the NousNav system, a mean localization error of 4.5mm is sufficient for some of the initial use-cases of NousNav where navigation can provide some benefit. These include training clinicians in the use of neuronavigation, planning craniotomy extent and location, choosing a trajectory towards a lesion not visible at the brain surface, and burr hole placement for subdural hematoma evacuation.
There are a number of related open-source projects that aim to support image-guided neuronavigation. These projects range from low-level packages aimed at developers of image-guide solutions to fully integrated platforms that provide high level support to users. An example of the latter is the Intraoperative Brain Imaging System (IBIS), which provides a platform to facilitate the use of various neuronavigation techniques such as intraoperative ultrasound and augmented reality through a surgical microscope [29]. The IBIS platform uses many of its own low-level libraries to perform calibration and registration tasks, with the exception of using the Public Library for Ultrasound (PLUS) Toolkit and OpenIGTLink protocol to communicate with various hardware components [17, 19]. The system reports a fiducial registration error of 3.72mm (SD 1.27mm), which is a different metric than is reported here for NousNav and is therefore difficult to compare, however a median TRE of 2.54mm is possible following use of intraoperative ultrasound [29]. While IBIS is well-suited for numerous research and clinical contexts, the lack of integration with existing open-source platforms like 3D Slicer and focus on higher-cost instruments including surgical microscopes and VR limit its applications in lower-resource settings. The InVesalius Navigator platform is another example of an open-source ecosystem that supports end-to-end image-guided neurosurgical solutions [30]. The platform reports a TRE of approximately 1.5mm, however these values were achieved on an acrylic grid phantom rather than an anatomical phantom. The platform also uses a paired-point registration technique based on anatomical landmarks without a dense registration step which may impact robustness in a clinical setting.
There are some limitations of these experiments that should be considered for future work. The first is that all experiments were performed using commercial pointers and SRBs rather than NousNav’s custom pointers and SRBs. This was done intentionally to enable direct comparison of tracking cameras and workflow while controlling for tool geometry, as well as assess the advantages of using pre-calibrated tools. Future system tests will include assessing the impact of using NousNav’s custom tools on overall system performance. Another limitation is that the anthropromorphic phantom experiment uses a simplified phantom that does not simulate skin motion or tissue deformation. While this likely has the effect of inflating the observed spatial performance of both systems, this additional complexity is captured during the clinical experiments which illustrate similar magnitudes of error. Finally, working to improve the accuracy of the NousNav system will continue to be a priority. These experiments give a useful baseline against which future changes can be measured, and stress the role of supporting users through system positioning, tool calibration and patient registration as a means of optimizing system performance.
4. Conclusion
This paper reports a quantitative evaluation of the performance of a novel low-cost computer-integrated neuronavigation system, NousNav, in order to inform system design decisions and characterize expected clinical accuracy. The use of low-cost hardware, custom neuronavigation software based on open-source packages, and accessible hardware tools has a measurable but tolerable impact on the spatial accuracy of the overall system. In the context of global neurosurgery, where the benefits of neuronavigation are currently only available to a small fraction of all practicing neurosurgeons, NousNav represents an important step towards reducing financial barriers and facilitating equitable access. Further testing of the system by clinical and research collaborators will be paramount to fully understanding practical system performance and ensuring sustainable use in clinical and research settings is feasible.
Acknowledgments
We thank Dr. Frédéric Racicot, MD, MSc, and Dr. Charles Couturier, MD, PhD, for their assistance with data collection and their valuable feedback throughout the NousNav project.
Funding:
Colton Barr is supported by a National Science and Engineering Research Council (NSERC) Canada Graduate Scholarship - Doctoral (CGS-D) award, a Vector Institute Student Research Grant, and a Walter C. Sumner Memorial Fellowship. This research was undertaken in part thanks to funding from the Canada Research Chairs Program. This research was also supported in part by the National Institutes of Health through the Advanced Technologies - National Center for Image Guided Therapy (AT-NCIGT) under grant number P41EB028741. We thank the Jennifer Oppenheimer Cancer Research Initiative for their support. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Footnotes
Statements and Declarations
Competing Interests: We have no conflicts of interest to declare.
Ethics and informed consent: All clinical data included in this study was collected in compliance with a protocol approved by the Institutional Review Board at the Brigham and Women’s Hospital. All participants provided informed verbal and written consent.
References
- [1].Dewan MC, Rattani A, Fieggen G, Arraez MA, Servadei F, Boop FA, Johnson WD, Warf BC, Park KB: Global neurosurgery: the current capacity and deficit in the provision of essential neurosurgical care. executive summary of the global neurosurgery initiative at the program in global surgery and social change. Journal of neurosurgery 130(4), 1055–1064 (2018) [DOI] [PubMed] [Google Scholar]
- [2].Fuller AT, Barkley A, Du R, Elahi C, Tafreshi AR, Von Isenburg M, Haglund MM, et al. Global neurosurgery: a scoping review detailing the current state of international neurosurgical outreach. Journal of neurosurgery 134(4), 1316–1324 (2020) [DOI] [PubMed] [Google Scholar]
- [3].Park KB, Johnson WD, Dempsey RJ: Global neurosurgery: the unmet need. World neurosurgery 88, 32–35 (2016) [DOI] [PubMed] [Google Scholar]
- [4].Haglund MM, Fuller AT: Global neurosurgery: innovators, strategies, and the way forward: Jnspg 75th anniversary invited review article. Journal of neurosurgery 131(4), 993–999 (2019) [DOI] [PubMed] [Google Scholar]
- [5].Rossi VJ, Wells-Quinn TA, Malham GM: Negotiating for new technologies: guidelines for the procurement of assistive technologies in spinal surgery: a narrative review. Journal of Spine Surgery 8(2), 254 (2022) [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Charles U, Anthony A, Evaristus N, Morayo S, Paul J, Chinedu A: Computer-assisted brain surgery (neuronavigation) in abuja, north central nigeria: a 3-year retrospective review and practical challenges. Nigerian Postgraduate Medical Journal 26(3), 174–181 (2019) [DOI] [PubMed] [Google Scholar]
- [7].Léger É, Horvath S, Fillion-Robin J-C, Allemang D, Gerber S, Juvekar P, Torio E, Kapur T, Pieper S, Pujol S, et al. Nousnav: a low-cost neuronavigation system for deployment in lower-resource settings. International Journal of Computer Assisted Radiology and Surgery 17(9), 1745–1750 (2022) [DOI] [PubMed] [Google Scholar]
- [8].Fitzpatrick JM, West JB, Maurer CR: Predicting error in rigid-body point-based registration. IEEE transactions on medical imaging 17(5), 694–702 (1998) [DOI] [PubMed] [Google Scholar]
- [9].Fick T, Doormaal JA, Hoving EW, Willems PW, Doormaal TP: Current accuracy of augmented reality neuronavigation systems: systematic review and meta-analysis. World neurosurgery 146, 179–188 (2021) [DOI] [PubMed] [Google Scholar]
- [10].Kremer NI, Laar T, Lange SF, Muller SS, La Bastide-van Gemert S, Oterdoom DM, Drost G, Dijk JMC: Stn-dbs electrode placement accuracy and motor improvement in parkinson’s disease: systematic review and individual patient meta-analysis. Journal of Neurology, Neurosurgery & Psychiatry 94(3), 236–244 (2023) [DOI] [PubMed] [Google Scholar]
- [11].Rae E, Lasso A, Holden MS, Morin E, Levy R, Fichtinger G: Neurosurgical burr hole placement using the microsoft hololens. In: Medical Imaging 2018: Image-guided Procedures, Robotic Interventions, and Modeling, vol. 10576, pp. 190–197 (2018). SPIE [Google Scholar]
- [12].Upadhyay UM, Golby AJ: Role of pre-and intraoperative imaging and neuronavigation in neurosurgery. Expert Review of Medical Devices 5(1), 65–73 (2008) [DOI] [PubMed] [Google Scholar]
- [13].Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, Fillion-Robin J-C, Pujol S, Bauer C, Jennings D, Fennessy F, Sonka M, et al. 3d slicer as an image computing platform for the quantitative imaging network. Magnetic resonance imaging 30(9), 1323–1341 (2012) [DOI] [PMC free article] [PubMed] [Google Scholar]
- [14].Ruiz-Alzola J, Skareb A, Idriss AM, Diao B, Ferreira JDRDP, Rodrigues MAF, Rivero LL, Kikinis R: Train the trainers: medical technology for the sustainable development of africa. In: 2018 IEEE Global Humanitarian Technology Conference (GHTC), pp. 1–8 (2018). IEEE [Google Scholar]
- [15].Ungi T, Gauvin G, Lasso A, Yeo CT, Pezeshki P, Vaughan T, Carter K, Rudan J, Engel CJ, Fichtinger G: Navigated breast tumor excision using electromagnetically tracked ultrasound and surgical instruments. IEEE Transactions on Biomedical Engineering 63(3), 600–606 (2015) [DOI] [PubMed] [Google Scholar]
- [16].Chen X, Xu L, Wang H, Wang F, Wang Q, Kikinis R: Development of a surgical navigation system based on 3d slicer for intraoperative implant placement surgery. Medical engineering & physics 41, 81–89 (2017) [DOI] [PMC free article] [PubMed] [Google Scholar]
- [17].Ungi T, Lasso A, Fichtinger G: Open-source platforms for navigated image-guided interventions. Medical image analysis 33, 181–186 (2016) [DOI] [PubMed] [Google Scholar]
- [18].Lasso A, Heffter T, Rankin A, Pinter C, Ungi T, Fichtinger G: Plus: open-source toolkit for ultrasound-guided intervention systems. IEEE transactions on biomedical engineering 61(10), 2527–2537 (2014) [DOI] [PMC free article] [PubMed] [Google Scholar]
- [19].Tokuda J, Fischer GS, Papademetris X, Yaniv Z, Ibanez L, Cheng P, Liu H, Blevins J, Arata J, Golby AJ, et al. Openigtlink: an open network protocol for image-guided therapy environment. The International Journal of Medical Robotics and Computer Assisted Surgery 5(4), 423–434 (2009) [DOI] [PMC free article] [PubMed] [Google Scholar]
- [20].Besl PJ, McKay ND: Method for registration of 3-d shapes. In: Sensor Fusion IV: Control Paradigms and Data Structures, vol. 1611, pp. 586–606 (1992). Spie [Google Scholar]
- [21].West JB, Maurer CR: Designing optically tracked instruments for image-guided surgery. IEEE transactions on medical imaging 23(5), 533–545 (2004) [DOI] [PubMed] [Google Scholar]
- [22].Ma B, Moghari MH, Ellis RE, Abolmaesumi P: Estimation of optimal fiducial target registration error in the presence of heteroscedastic noise. IEEE transactions on medical imaging 29(3), 708–723 (2010) [DOI] [PubMed] [Google Scholar]
- [23].OptiTrack: V120:Duo Camera Specifications. Accessed: 2025-01-07 (2025). https://optitrack.com/cameras/v120-duo/specs.html [Google Scholar]
- [24].Northern Digital Inc. (NDI): Polaris Optical-EM Tracking System Specifications. Accessed: 2025-01-07 (2019). https://www.ndieurope.com/wp-content/uploads/2019/11/10005447-Optical-EM-Polaris.pdf [Google Scholar]
- [25].Fattori G, Lomax AJ, Weber DC, Safai S: Technical assessment of the ndi polaris vega optical tracking system. Radiation oncology 16(1), 87 (2021) [DOI] [PMC free article] [PubMed] [Google Scholar]
- [26].Danilchenko A, Fitzpatrick JM: General approach to first-order error prediction in rigid point registration. IEEE transactions on medical imaging 30(3), 679–693 (2010) [DOI] [PMC free article] [PubMed] [Google Scholar]
- [27].Ma B, Banihaveb N, Choi J, Chen EC, Simpson AL: Is pose-based pivot calibration superior to sphere fitting? In: Medical Imaging 2017: Image-guided Procedures, Robotic Interventions, and Modeling, vol. 10135, pp. 476–483 (2017). SPIE [Google Scholar]
- [28].Yaniv Z: Which pivot calibration? In: Medical Imaging 2015: Image-guided Procedures, Robotic Interventions, and Modeling, vol. 9415, pp. 542–550 (2015). SPIE [Google Scholar]
- [29].Drouin S, Kochanowska A, Kersten-Oertel M, Gerard IJ, Zelmann R, De Nigris D, Bériault S, Arbel T, Sirhan D, Sadikot AF, et al. Ibis: an or ready open-source platform for image-guided neurosurgery. International journal of computer assisted radiology and surgery 12, 363–378 (2017) [DOI] [PubMed] [Google Scholar]
- [30].Souza V, Matsuda R, Peres A, Amorim P, Moraes T, Silva J, Baffa O: Invesalius navigator, a free and open-source software for navigated transcranial magnetic stimulation. Brain Stimulation: Basic, Translational, and Clinical Research in Neuromodulation 12(2), 571 (2019) [Google Scholar]