1.0 COURSE DESCRIPTION:

This course will introduce the essential topics of haptics programming and touch feedback in virtual reality. The intention is to provide students with the skills necessary to aid in the design, development, deployment, and the evaluation of new virtual reality haptic simulators and applications. The hardware that will be used is the SensableTechnologies' PHANToM haptic devices, the "Reachin Display" unit to provide the user with ergonomic feel, Crystal Eyes stereo glasses, and the Immersion Virtual Laparoscopic Interface. The  graphics programming environment is EAI/Sense8's WorldToolkit API of OpenGL calls. The haptics programming environment is Sensable Technologies' GHOST API of physics computational calls. The control computers are dual Pentium processor workstations with a Wildcat graphics OpenGl accelerator running Windows NT.  The course includes discussions on: Force calculations, Newton's law of motion F=ma, Hooke's Law of spring forces F=-kx, mass-springs-damper physics, real-time VR programming, 3D graphics, programming force feedback devices,  haptic sensing and control, physical modeling, collision detection, surface deformation, physical constraints, deformation of virtual objects at the 3D mesh/vertex level, military applications, entertainment applications, programming medical devices for surgical simulation.

Students will design and write real-time haptic applications using the 'C', C++ programming  languages with WorldToolkit and GHOST. Potential applications include: surgical simulators, hand/glove applications, haptic VR force feedback games, haptic VR art sculpting, haptic design of objects.

Course Credits:  4 s. h.

Prerequisites:  CS362 and CS375 Graphics and Virtual Reality or permission from the Instructor. The intent of the course is to provide a high-level seminar for students wishing to pursue graduate studies or research oriented careers in this area of computer science.

1.1 RATIONALE/BACKGROUND.

The holy grail of virtual reality and simulation since its inception has been ""To create the illusion so well that you feel you are actually doing it". While this goal is still actively being pursued,  the past ten years have shown a steady evolution in virtual reality (VR) technologies .  VR technology is now being used for many things, for example: air traffic control simulations,  architectural design, aircraft design, acoustical evaluation (sound proofing and room acoustics), computer aided design, education (virtual science laboratories, cost effective access to sophisticated laboratory environments), entertainment (a wide range of immersive games),  legal/police (re-enactment of accidents and crimes), medical applications such as virtual surgery,  scientific visualization (aerodynamic simulations, computational fluid dynamics), telepresence and robotics,  and flight simulation.

Until recently, the one major component lacking in VR simulations has been the sense of  touch  (haptics). The user could reach out and touch a virtual object, but would place his/her hand right through the object. This was very disconcerting for the user. Incorporating haptics into VR environments now has an added new dimension, that of force feedback. While haptic technology promises to provide the missing link in VR simulators, it arrives with many unresolved problems and complexities.

The components of such a system include: a 3D computer graphics workstation with OpenGL hardware acceleration, a 3 DOF haptic device, VR/3D graphics simulation software, haptic software, and device interfaces such as a laparoscopic instrument . Today's 3D computer graphics workstations (PC's) can render polygonal models of moderate complexity, on the order of tens of thousands of polygons, at interactive update rates of 30 Hz or more. This means that some algorithms that were not previously practical are now feasible.  Haptic interfaces with force feedback are now commercially available. Most notably the Phantom device from Sensable Technologies (www.sensable.com).  Most operate using three degrees of freedom (DOF), although six DOF devices have recently been introduced.

1.2 SUGGESTED TEXT.

Grigore Burdeau,  "Haptics and Touch Feedback in Virtual Reality ", John Wiley & Sons, Inc. New York, N.Y. 1996.
 

1.3 BIBLIOGRAPHY

[1] Richard M. Satava, "Cybersurgery: Advanced Technologies for Surgical Practice", John Wiley & Sons, Inc. New York, N.Y. 1998.

[2] Dimitris Metaxas,  "Modelling for Medical Applications", Course Notes #20 in the Proceedings of the Annual ACM SIGGRAPH 2000 Conference, New Orleans, La, August 2000.

[3] Grigore Burdeau, M. Srinivasan,  "Haptics in Virtual Environments Workshop", Workshop Notes of the Annual IEEE Virtual Reality '2000  Conference, Rutgers University, New Brunswick, N.J., March 2000.

[4] Grigore Burdeau,  "Haptics and Touch Feedback in Virtual Reality ", John Wiley & Sons, Inc. New York, N.Y. 1996.

[5] D. Terzopoulos, J.C. Platt, H. Barr, K. Fleischer, "Elastically Deformable Models", Computer Graphics Proceedings of the Annual ACM SIGGRAPH '87 Conference, Vol. 21. pp. 205-214, July 1987.

[6] D. Terzopoulos, K. Fleischer, "Modelling InElastic Deformation: Vicoelasticity, Plasticity, Fracture", Computer Graphics, Proceedings of the
  Annual ACM SIGGRAPH '88, Vol. 22. pp. 269-278, August 1988.

[7] D. Metaxas, D. Terzopoulos, "Dynamic Deformation of Solid Primitives with Constraints", Computer Graphics, Proceedings of the Annual ACM SIGGRAPH '92, Vol. 26. pp. 309-312, July 1992.

[8] David Baraff and A. Witkin, "Large Steps in Cloth Animation", in Proceedings of the Annual ACM SIGGRAPH '98 Conference, Vol. 33, pps. 43-54, July 1998 .

[5] Nadia Thalmann, " Cloth Animation", in Proceedings of the Annual ACM SIGGRAPH '98 Conference, Vol. 33, pps. 43-54, July 1998 .

[6] Suvranu De, M. Srinivasan, "Thin Walled Models for Haptic and Graphical Rendering of Soft Tissues in Surgical Simulations", Proceedings of Medicine Meets Virtual Reality (MMVR '99), IOS Press, San Francisco, CA. January 1999, pps. 94-99.

[7] C. Basdogan, M. Srinivasan, S. Small, S. Dawson, "Force Interactions in Laparoscopic Simulations: Haptic Rendering of Soft Tissues", Proceedings of Medicine Meets Virtual Reality (MMVR 6), IOS Press, San Diego, CA. January 1996, pps. 19-22.

[8] Jeffrey Berkley, P. Oppenheimer, S. Weghorst, D. Berg, G. Raugi, D. Haynor, M. Ganter, C. Brooking, G. Turkiyyah, "Creating Finite Element Models from Medical Images", Proceedings of Medicine Meets Virtual Reality (MMVR '2000), Newport Beach, CA. January 27-30, 2000, IOS Press,  pps. 26-32.

 [9] G. Debunne, M. Desbrun, A. Barr, M, Cani, "Interative Multi-Resolution Animation of Deformable Models", Computer Graphics,
 Proceedings of the Annual ACM SIGGRAPH, August 1999.

[10] "General Haptic Open Software Toolkit GHOST Programmer's Guide", Sensable Technologies Inc., June 1999.

[11] Morten Bro-Nielsen, D. Helfrick, B. Glass, X. Zeng, and H. Connacher, "VR Simulation of Abdominal Trauma Surgery", Medicine Meets Virtual Reality 6 (MMVR-6), San Diego, California, 1998, IOS Press,  pps. 117-123.

[12] Mark Ottensmeyer, E. Ben-Ur, K. Salisbury, "Input and Output for Surgical Simulation: Devices to Measure Tissue Properties in vivo and a Haptic Interface for Laparoscopic Simulators", Proceedings of Medicine Meets Virtual Reality (MMVR '2000), Newport Beach, CA. January 27-30, 2000, IOS Press,  pps. 236-242.

[13] Morten Bro-Nielsen, "Fast Finite Elements For Surgery Simulation", Proceedings of Medicine Meets Virtual Reality 5 (MMVR '97), 1997, pps. 395-400.

[14] Karl Renig, V. Spitzer, H. Pelster, T. Johnson, T. Mahalik, "More Real-Time Visual and Haptic Interaction with Anatomical Data", Proceedings of Medicine Meets Virtual Reality 5 (MMVR '97), 1997, pps. 155-158.

[15] Paul Gorman, T. Krummel, R. Webster, M. Smith, D. Hutchens,, "A Prototype Haptic Lumbar Puncture Simulator", Proceedings of Medicine Meets Virtual Reality (MMVR '2000), Newport Beach, CA. January 27-30, 2000, IOS Press,  pps. 106-109.

[16] Tobias Salb, S. Ghanai, O. Burgert, R. Dillmann,, "Interactive Simulation of Tooth Cleaning with an Interdental Brush", Proceedings of Medicine Meets Virtual Reality (MMVR '2000), Newport Beach, CA. January 27-30, 2000, IOS Press,  pps. 295-301.

[17] O'Toole RV, Playter RR, Krummel TM, Blank WC, Cornelius NH, Roberts WR, Bell WJ, Raibert M:  Measuring and developing suturing technique with a virtual reality surgical simulator.  J Am Coll Surg 1999;189:114-27. .

[18] Jacob Rosen, M. MacFarlane, C. Richards, B. Hannaford, M. Sinanan, "Surgeon-Tool Force/Torque Signatures - Evaluation of Surgical Skills in Minimally Invasive Surgery", Proceedings of Medicine Meets Virtual Reality (MMVR '99), San Francisco, CA, January 1999, IOS Press,  pps. 290-296.

[19] Kevin Cleary, C. Lathan. "Surgical Simulation: Research Review, Computing Challenges, and Spine Biopsy Simulator", Parallel Computing, 1997, pps. 1-15.

[20] Sunil Singh, M. Bostrom, D. Popa, C. Wiley, "Design of an Interactive Lumbar Puncture Simulator with Tactile Feedback", Proceedings of Robot and Human Communication, ROMAN '93, pps. 156-159.

[21] Sunil Singh, M. Bostrum, D. Popa, C. Wiley, "Design of an Interactive Lumbar Puncture Simulator with Tactile Feedback", Proceedings of IEEE International Conference on Robotics and Automation, 1994, pps.1734-1752.

[22] Paul Gorman, J. Lieser, W. Murray, R. Haluck, T. Krummel. "Assessment and Validation of a Force Feedback Virtual Reality Based Surgical Simulator", In the Proceedings of the Third PHANToM Users Group Workshop, Cambridge, Massachusetts, 1998.

[23] Morten Bro-Nielsen, J. Tasto, R. Cunningham, G. Merril, "PREOP Endoscopic Simulator: A PC-Based Immersive Training System for Bronchoscopy", Medicine Meets Virtual Reality 7, San Francisco, CA, January 20-23,1999,  IOS Press, pps. 76-82.

[24] Markus Kukuk, B. Geiger, "Registration of real and virtual endoscopy - a model and image based approach", Medicine Meets Virtual Reality 2000, Newport Beach, CA, January 20-24, 2000, IOS Press, pps. 168-174.

[25] L.M. Auer,  "Virtual endoscopy for planning and simulation of minimally invasive neurosurgery", In CVRMed-MRCAS '97, Lecture Notes in Computer Science - 1205, pps. 315-318, March 1997.

[26] Joseph Tasto, K. Verstreken, J. M. Brown, J. Bauer, "PreOp Endoscopy Simulator: From Bronchoscopy to Ureteroscopy", Medicine Meets Virtual Reality 2000, Newport Beach, CA, January 20-24, 2000, IOS Press, pps. 344-349.

[27] D.P. Jang, M. Han, S. Kim, "Virtual Endoscopy using Surface Rendering and Perspective Volume Rendering", Medicine Meets Virtual Reality 7, San Francisco, CA, January 20-23,1999,  IOS Press, pps. 161-166.

[28] C. Baur, D. Guzzoni, O. Georg, "VIRGY: A Virtual Reality and Force Feedback Based  Endoscopic Surgery Simulator", Medicine Meets Virtual Reality 6, San Diego, CA, January 28-31,1998,  IOS Press, pps. 110-116.

[29] Christopher Sutton, R. McCloy, A. Middlebrook, P. Chater, M. Wilson, R. Stone, "MIST - A Laparoscopic Surgery Procedures Trainer and Evaluator", Medicine Meets Virtual Reality 1997, IOS Press, 1997, pps. 598-607.
 
 

                                                                                     Last Updated 1/19/2001