In the previous weeks / month I have finished my Literature study and started my actual MSc thesis project. The outcome of my literature study was that illustrative visualization will play a major role in medical visualization systems. In addition, the user should be in control of the system, and the system should be comprehensible. This requires comprehensible specification of illustrative styles, and a natural way of interaction which is customizable by the user.
However, even more important is usability of the system. Merely having these techniques available to the user is not enough. The best example is found in transfer function specification. Transfer function specification is a complex and cumbersome process, even with state-of-the-art methods. The idea to focus on comprehensible specification of illustrative styles is supported by the semantic layers [Rautek 07], and style transfer functions [Bruckner 07]. Both focus on comprehensible specification, semantic layers go as far as using semantic values in the natural language of the domain of the user.
Considering interaction with the application, most important is that interaction should feel natural, and should be customizable by the user. Two-handed interaction might be a solution to interacting with the data in a natural way. It is already natural to use both of our hands in a lot of situations in our every day life, so extending this to interaction with virtual data would seem logical to us. Considering customizability of interaction, this relates to our idea that the user should be the one controlling the system. One part of this control is specification of illustrative styles, the other part would be specification of the way of interaction. Again, our idea is supported by Rautek et al. in their work on interaction-dependent semantic layers [Rautek 08b].
The envisioned system
Our envisioned visualization system for anatomy education will make use of a high-resolution histological dataset acquired from the Leids Universitair Medisch Centrum (LUMC). In addition, a lower resolution MRI dataset is also present. In time, important objects can be segmented by a team of anatomists of the LUMC. This segmentation can then be used to apply different visual properties to focus objects, near-focus objects and context objects. Until then, test segmentations can be used for this purpose.
The distinctive feature of our system is example-based appearance specification (EbAS). The users will be able to specify the appearance of physical objects in the dataset using pre-generated, static (2D) illustrative examples, as opposed to the natural language of the domain of the technique, which they are unfamiliar with. In addition, our system will use semantic layers [Rautek 07] for specification of mappings not involving physical objects, and style transfer functions ( [Bruckner 07a]) for general transfer function specification.
The user interacts with the visualization by use of (GUI) widgets and gpu-based clipping techniques. In order to maximize ease of interaction, the use of two-handed interaction techniques will be researched. Two-handed interaction techniques match with our strong bimanual motor skills and might make interaction with the visualization more straightforward. Furthermore, interaction-dependent semantic layers ( [Rautek 08b]) will enable the user to customize interaction with the data according to their requirements.
Finally, surgical simulation techniques such as simulation of cutting, volume deformation, collision detection, and (haptic) interaction, will be applied to match the currently used approach; cadaver dissection. This way, the switch between the current method and our system will feel more natural to the users.
Current status
As I just recently started working on this project, there isn’t very much to show as of yet. I am working with Voreen which, up till now, seems to be a good choice. I have created an RGB volume from my frozen cadaver dissections (2052 RGB TIF slices) and loaded the data in Voreen. I have customized the standard GLSLRaycaster to get familiar with Voreen and have my own Raycasting module to do whatever I like with it. It enables me to edit the fragment shader as well as the vertex shader in VoreenVE (the application, at runtime).
My next step will be to register the matching MRI dataset with the RGB dataset created from the TIF slices. Currently, I have been using MITK to segment some objects in my dataset. However, MITK seems only capable of saving in the STL format, which, by the best of my knowledge, cannot directly be loaded into voreen. I have ITK in DeViDE coderunner to convert the STL to a .hdr and .img dataset which can be loaded into Voreen. However, there seems to be a dimension and translation mismatch which I still need to figure out.
Concluding
Two pictures to show the idea 
.
- Photographic dataset generated from the 2052 RGB TIF slices and visualized using Voreen and a GLSLRaycaster.
- Photographic data + MRI data, unfortunately I still have to register the datasets and find a decent transfer function for the MRI data.
[Rautek 07] Peter Rautek, Stefan Bruckner & Eduard Gröller. Semantic layers for illustrative volume rendering. IEEE transactions on visualization and computer graphics, vol. 13, no. 6, pages 1336–43, January 2007.
[Bruckner 07a] S Bruckner & M E Gröller. Style Transfer Functions for Illustrative Volume Rendering. Computer Graphics Forum, vol. 26, no. 3, pages 715–724, 2007.
[Rautek 08b] Peter Rautek, Stefan Bruckner & M. Eduard Gröller. Interaction-Dependent Semantics for Illustrative Volume Rendering. Computer Graphics Forum, vol. 27, no. 3, pages 847–854, May 2008.
Popularity: 50%
Goed bezig Bastijn. Ik hoop dat de registratie met MRI een beetje gaat lukken. Mijn kleine teen zegt dat dat een HELE grote uitdaging is
.
Bedankt
. Tien van mijn tenen zijn het met je eens, dus tips en tricks zijn altijd welkom!