ENV 2006 CS4.1

Envisioning Information: Case Study 4

Focus and Context for Volume Visualization

ENV 2006 CS4.2

Focus + Context for Volume Rendering

• Marcelo Cohen has studied the application of focus and context ideas for volume rendering

• Implemented as texture-based volume rendering

– Distortion carried out on the GPU

• Case study from neurosurgery

ENV 2006 CS4.3

• Aneurysm– A dilation of a blood vessel caused by weakening of its wall– Can rupture and cause death by haemorrhage

Motivation

ENV 2006 CS4.4

• Diagnosis– Angiography– Magnetic Resonance Angiography (MRA)– Computerized Tomography Angiography (CTA)

Motivation

ENV 2006 CS4.5

Motivation

• Our problem

– Traditional imaging methods are limited

• Difficult to see the entire structure• Can have multiple viewpoints, but still 2D

– A solution is to use volume visualization

• Source: CTA dataset• Collaborating with Nick Phillips - a neurosurgeon from Leeds

General Infirmary

ENV 2006 CS4.6

Volume Visualization

• Volume– Stack of “slices” (images)

• Slice– Usually is a grey-scale image– For each voxel, the grey level

indicates the corresponding tissue density

• How to display it ?

ENV 2006 CS4.7

Direct Volume Rendering

• Two-step procedure

1) Classification: assign opacities and colours to

different tissue types

• Produces a transfer function

ENV 2006 CS4.8

Direct Volume Rendering

2) Rendering: many different ways

A fast method: using 3D texture mapping in

hardware

– Volume is sliced into parallel planes perpendicular to the scene observer

– Planes are then drawn back to front

Kruger & Westermann 2003

ENV 2006 CS4.9

Back to Motivation

• Colours and opacities were chosen to show the brain arteries

• Surgeon needs to see aneurysm in detail

• But is also interested in exploring the surrounding vessels

• How to enhance the visualization ?

ENV 2006 CS4.10

• A powerful visualization technique

• Assumptions– Identify focus in data (most important part)– The remainder is called context

• Main idea– Magnify the focus– Compress the context– Preserve the same screen space

Focus and Context

ENV 2006 CS4.11

Demo: Bifocal Display

ENV 2006 CS4.12

Bifocal Display: How Does it work?

• Transformation function – shown just for x-axis

• From texture (xt) to object (xo) space: xo = f(xt)

ENV 2006

• Applying the F+C ideas in 3D ?• In this case, focus is a region around the aneurysm• Again, extension of the 2D method

– 3D cartesian bifocal– 9 regions in 2D (32) = 27 regions in 3D (33)

Application: 3D Medical Data

ENV 2006 CS4.14

Using the CPU to calculate distortion effect

preprocessing rendering

ENV 2006 CS4.15

CPU-based distortion: results

• Surgeon pleased with rendering results – 20 frames per second• But distortion does not happen in real-time – surgeon wants this

to be interactive

ENV 2006

Exploiting the programmability of the modern graphics processor (GPU)

– It's possible to alter the way that image data is handled by the GPU– Fragment shader is a programme that runs on the GPU

• Can be written in a high-level language such as OpenGL Shading Language

• Executed for every fragment (pixel) drawn on screen

Hardware-accelerated Distortion

.. But the transformationneeds to be calculatedfor every pixel, every slice… so rendering is slow:2 frames per second

ENV 2006 CS4.17

Balancing load on CPU and GPU

• Best solution achieved by combination of CPU and GPU• Fragment shaders are able to access texture maps created by CPU• So… pre-compute distortion transformation using the CPU and store in

a texture array

• Look up the texture array within the fragment shader on the GPU

eg to encode f(xt) 1D texture array

preprocessingrendering

ENV 2006 CS4.18

Different Mapping Effects

ENV 2006 CS4.19

Extending the Effects

• A texture has three components: R, G, B and A• So we can store further information – for example, in the x

texture, a highlighting factor (hx), or an opacity factor (x)

Then the highlighting for a voxel is computed from hxhyhz

The opacity factors combine similarly to give an atternuation factor at a voxel.

ENV 2006 CS4.20

Example - Highlighting

ENV 2006 CS4.21

Example - Attenuation

ENV 2006 CS4.22

Example – User interface

ENV 2006 CS4.23

Example – Other datasets

ENV 2006 CS4.24

The Movie