|
Taking the Visible Human Interactive Atlas Real-Time
REAL-TIME DISPLAY OF ARBITRARY OBLIQUE SECTIONS
 |
|
The Visible Human Interactive Atlas (VHIA) was originally conceived as an aid to learning anatomy from the perspective of clinical endoscopic ultrasound. The user selects and builds a series of organs and structures from a menu, which we refer to as “models." The user then passes a plane through the models using a mouse, and when the plane stops moving, the resulting anatomy contained within the plane is shown in a window next to the model (Figure 1; click on the link below Figure 1A to manipulate the this model). The user-chosen polygonal models, combined with the display of a moveable flat plate, give a simple, interactive ability to input the position and orientation of an oblique image relative to the three-dimensional anatomy of the Visible Human Male. With the web version of the VHIA, the resulting image is created on a server located at the University of Colorado's Center for Human Simulation (CHS) and sent to the user’s computer for display. Receiving the updated image takes from three to four seconds over a high-speed internet connection.
The CHS server has enough Random Access Memory (RAM) to hold the entire Visible Human Male (VHM), allowing the user to interact with any part of the body. The web version of the Visible Human Interactive Atlas is a little like having a multi-million page atlas with an easy way to jump to any desired page. However, the database is still better than a textbook, since the resulting oblique images are interactive, allowing the user to browse the names of the structures once the image has arrived. This is accomplished by simultaneously sampling an identification volume in which each volume element of the Red, Green, and Blue (RGB) data is labeled by structure. The same data is used to produce simulated ultrasound from a sector of the chosen plate, a topic for a future technical update.
The VHIA uses all integer arithmetic to quickly produce the RGB oblique images. It is the transmission time of the resulting image that causes the delay. Today, personal computers can be purchased with enough RAM to hold large sections of the VHM, and Gigahertz or faster processors are the norm. That means that you can take 50 million operations to create an image and still maintain a respectable 20 frames per second display rate. Thus, it is now possible to create a VHIA that is completely real-time.
SO, WHAT DOES REAL-TIME GET ME?
Great care was taken to make moving the navigation plate of the VHIA as intuitive and easy as possible. Utilizing combinations of three mouse buttons, the user can rapidly achieve any desired position and orientation. [See the VHJOE Technical Update in Volume 2, Issue 2.] However, the cursor is fated to being a two-dimensional input in a three-dimensional world. Thus, the user generally makes position and orientation moves with one degree of freedom fixed. The resulting state of the plate accurately reflects a desired position and orientation, but the path that the plate took to get there does not resemble the “sweeping” motion of the ultrasound probe. This is not an issue when the result is a single image as supplied by the web version of the VHIA. The real-time VHIA, however, brings the potential to experience clinically relevant motion.
To address this, Dr. David Rubinstein, of the CHS, added the ability to input "spline paths." The user can construct paths by recording states along the desired path, much like a series of bus stops along a desired transportation route. When the spline path is “played back,” the navigation plate follows the path and the corresponding oblique images are displayed in real-time. The spline paths can be saved to disk and read in when desired. Spline paths can be created corresponding to any desired movement, including motions that occur during an endoscopic examination. The resulting playback then resembles a clinically relevant fly-through of planar images. Video Clip 1 below shows the real-time VHIA in action.
|
|
Video Clip
1: View of the real-time VHIA in action.
|
A STEP TOWARDS SIMULATION
 |
Figure 2 |
Real-time display of oblique images takes us a step closer to creating a simulator for developing Endoscopic UltraSound (EUS) skills. We recently attached a Polhemus Patriot tracking sensor [Polhemus: Colchester, VT] to the end of a flexible scope. This allows us to determine the position and orientation of the tip of the scope for real-time control of the navigation plate. We used the VHIA to produce a spline representing the position and orientation that the probe might take through the esophagus, stomach, and duodenum if the flexible scope were inserted in a relaxed state--i.e. no knobs twisted and no torsion applied along the flexible shaft. We then created a flexible tunnel for the real scope to follow. Then, as the flexible scope was advanced through the tunnel, we used the position and orientation of the tip to estimate the depth of insertion as well as the change in orientation produced by twisting the knobs and applying torsion along the flexible shaft. The result was a scope-driven Visible Human Interactive Atlas, complete with simulated ultrasound. This technology was demonstrated at the recent 2004 UEGW meeting in Prague (Figure 2).
THE NEXT STEP Because there is an assumption that the probe follows a predefined path, the above input device gives is no method for the probe to take largely variant paths, as would be possible in the stomach. There are two distinctly different methods that we are considering to address this shortcoming. The first method would be to create a device resembling a flexible EUS scope, but having encoders attached to the knobs and a method for measuring the depth of insertion as well as the twist along the flexible shaft. From these measurements, we could estimate the position and orientation of the virtual probe based on simple physics coupled with the anatomic constraints of the VHM. Motors could potentially be concatenated with the encoders to give haptic feedback representing constraints imposed upon the scope by the anatomy. This method would give us maximum flexibility as the entire environment would be virtual.
The second, simpler, approach would be to create a flexible cavity in the shape of the VHM alimentary canal. Once again, the Polhemus could be used to track the position and orientation of the EUS probe. But now, instead of being constrained to a predefined path, the scope would have the same basic freedom of movement as it would in a real alimentary canal. Presumably the feel of twisting the knobs and applying torsion along the flexible shaft would also be similar to their corresponding feels in a real body. This method lacks the flexibility of a fully VR environment. But at this time, we are constrained to showing the oblique images and corresponding simulated ultrasound of the VHM, so that is not a significant constraint. We hope to try out the later method in the next couple of months.
If this system continues to evolve, it would be nice to include variant anatomy. However, our near-term goal is to create a simulator for teaching normal anatomy as seen by EUS along with EUS probe manipulation skills.
|
