Over the course of this summer, I have been contributing to the InVesalius organization as part of Google Summer of Code 2024.

GSoC

Google Summer of Code is an amazing initiative from Google that helps people onboard into open source projects. New contributors submit proposals to several organizations, and during the course of the project (can be 6 weeks or 12 weeks) they develop some new feature or work in a bug or usability improvement for the software.

InVesalius

InVesalius is a medical imaging and neuronavigation software that uses Computed Tomography and Magnetic Resonance Imaging to reconstruct a 3D model of the body part. Among the features present in the software there is image segmentation, that allows to isolate regions from the rest of the scan. Other useful features include measuring distances and densities within the scans, exporing meshes from 3D printing and adittionally, brain mapping routines.

This software is used by researches worldwide to plan and analyze data from experiments. It can be also used to develop prosthetics and do surgical planning.

I selected InVesalius as my organization for GSoC because it is an initiative that started in Brazil and I wanted to contribute with my knowledge to the technology developed in my home country.

Project Proposal

The project proposal was to increase the capabilities of the software and make it easier for users to edit the scans. CT and MRI scans deliver a voxel model with different values at each voxel. Segmentation can then be performed in order to extract, for instance, a particular type of tissue like compact bone. Thresholding segmentation is not always perfect because of noise in the measurements, for instance, although complementary segmentation are being developed and put to use. Even though the need for manually editing a mask by a specialist is required, and so far this had been done slice by slice, in a tedious way.

NOTE: In InVesalius, a mask is a 3D voxel model based on the scan. It does not have to be a subset of the original scan, but for most of the use cases it contains only part of the original information.

The proposal of the project was to accelerate this mask edition by performing the mask editing using the “Volume Viewer” window and applying the changes in 3D across the entire mask.

Comments on the code

First and foremost, this is the pull request that contained all the changes to develop the project.

UI Controls in WXPython

Commits: 2, 6

For UI controls, the software uses WXPython. I created more controls to configure the operations and to trigger the edit.

Interaction Styles in VTK

Commits: 3

The other end for the UI plus the volumetric manipulation resides in the VTK interface of the project. In VTK, in order to interact with the viewers, an interaction style is used for binding user inputs in the viewer (mostly with the mouse) to actions such as changing the camera state. For editing the masks, an interactor style was created, capable of drawing a polygon in the volume viewer.

Cython code

Commits: 4, 7

The algorithm that performs the mask cut was partly implemented using Cython, which is an extension to python that allows it to be compiled in C++ and called from normal python code. The advantages are that it runs natively instead of using a virtual machine, making the execution much faster. The drawbacks are of course a limitation to what you can do with the language. Additionally, the use of prange makes the for loop parallel, speeding up even more the operation.

One of the most exciting parts for me was seeing how fast it can be to use cython when operating on a large dataset. The runtime fell from more than 10 minutes to just a few seconds.

Demo

Some screenshots of the new features:

1. New buttons in the “Manual Edition” menu

2. Polygon selection in the volume viewer:

3. Different modes for mask cut:

   Mode	Before Edit	After Edit
   Include inside polygon
   Exclude inside polygon
   Include inside polygon within depth
   Exclude inside polygon within depth

Future Work

Some improvements to the features have already been discussed and mapped by the end of the project. I personally would like to contribute with the following additions:

• More user friendly depth values: Instead of using the range [0,1], the depth value could be a spatial value (the real distance of the scan) so it is easier to interpret.
• Using 3D shapes to perform the mask edit: Instead of solely drawing polygons on the screen to create the edit, some shapes like ellipsoids and boxes could be drawn in 3D to create the edit as well. This would make the edit even more granullar and potentially faster.