Realtime Visualization of the Connectome in the Browser using WebGL

Introduction

Recent advances in browser technology along with the widespread proliferation of Graphics Processing Units (GPUs) have enabled new visualization capabilities in the browser. Analyzing the connectome requires intensive preprocessing along with realtime visualization for results exploration. The introduction of WebGL [1] has made exploration of such data possible in the browser, enabling visualizations across a range of devices without any client-side software requirements other than a WebGL-enabled browser. This poster introduces several visualization techniques that were developed for exploration of connectome data.

Methods

The publicly available Connectome Mapper [2] provides a pipeline to automatically generate structural networks from raw diffusion MRI data for the human brain. In the segmentation stage, T1 MPRAGE MRI is processed by Freesurfer [3] producing gray / white matter segmentations. The Diffusion Toolkit [4] is used for reconstruction, and a deterministic streamline algorithm [2] is used for tractography, generating fiber tracts of the same subject. A parcellation is generated for cortical and subcortical regions-of-interest. These datasets are then coregistered, and a network is generated weighting the connectivity between regions based on the fiber tracts.

In our project, the results of the Connectome Mapper are directly loaded in the browser using WebGL and JavaScript. The FreeSurfer cortical surface reconstruction binary files are loaded and processed in JavaScript and converted to WebGL vertex buffer objects for rendering. The surfaces are overlaid with per-vertex curvature values computed during the FreeSurfer processing stream. The tractography data is likewise parsed in the JavaScript code and rendered as line primitives colored based on direction. Finally, the structural network itself is converted to JSON (JavaScript Object Notation) as an offline preprocess and loaded into the browser using JavaScript. The networks are visualized in 3D along with the fiber tracts and surfaces enabling exploration of connectivity information in realtime.

Results

Figure 1 shows a screenshot of the WebGL-based visualization. The visualization is composed of the pial surface colored with the per-vertex mean curvature, fiber tracts colored by direction, regions-of-interest rendered with cubes, and lines between regions denoting connectivity. The connectivity lines are colored based on the mean fiber tract length connecting the two regions-of-interest. The visualization can be tuned interactively and contains user interface elements that allow for display of pial or white matter surface, selection of curvature overlay, restriction of fiber tract length, and selection of connectivity regions-of-interest.

Conclusions

We presented a realtime visual exploration of the connectome data including FreeSurfer structural reconstruction, tractography, and network data all within the browser. This project establishes the capability of WebGL as a viable visualization platform for structural connectivity data and provides a means for rapidly exploring and validating the results of Connectome Mapper pipeline. WebGL is underpinned by OpenGL ES 2.0, a graphics standard already supported by a wide range of GPUs on handheld devices including the iPhone, iPad, and numerous Android phones. As WebGL proliferates to such devices, techniques such as those demonstrated in our poster will be enabled across a range of devices.

References

[1] WebGL Specification, http://www.khronos.org/registry/webgl/specs/latest/

[2] Connectome Mapping Toolkit, http://www.cmtk.org/

[3] Freesurfer, http://surfer.nmr.mgh.harvard.edu/

[4] Diffusion Toolkit, http://trackvis.org/dtk/