Virtual touch screen explained

A virtual touch screen (VTS) is a user interface system that augments virtual objects into reality either through a projector or optical display using sensors to track a person's interaction with the object. For instance, using a display and a rear projector system a person could create images that look three-dimensional and appear to float in midair.[1] Some systems utilize an optical head-mounted display to augment the virtual objects onto the transparent display utilizing sensors to determine visual and physical interactions with the virtual objects projected.[2]

History

Development

During a 2009 study at the Korea Electronics Technology Institute, a virtual touch screen system was built using the touch point tracking technique based on a time of flight camera with a Kalman filter.[7] To address the widespread problem in conventional virtual touch screen systems (i.e. responding sensitively to even the slightest hand movement and failing to recognize the touch point accurately), the study had applied the Kalman filter tracking technique to predict and track the touch point consecutively. Being forecast and recognized, the predicted touch point was bound to exist all the time, but an abrupt movement of touch point was likely to be recognized only insensitively. A commercial version based on this technology was never implemented.

Usage

Currently, virtual touch screens (VTS) are used primarily for entertainment and research purposes. A popular virtual touch screen system is the Kinect system made commercially available by Microsoft. The Kinect system is primarily used for consumer entertainment such as playing games and using the menus found on the Xbox system.[8] [9] However, in recent years the Kinect has been used for research projects in the field of robotics and 3-D interactions outside of the field of games such as browsing the web, and medical practices.[10] [11] [12]

There is a potential for the technology where the 3D visualization and physical interaction with the objects is necessary, such as teaching students introductory calculus.[13]

Virtual touch screen technology can also be applied to turn non-touch displays into interactive, touch-capable surfaces using a Kinect sensor in combination with a projector.[14] This type of application allows virtual touch technology to enhance personal display usage as well as better communication and presentation of data to be used in education or business settings.In medical practices, virtual touch interaction has been applied to allow users to view and manipulate digital data such as 3D images used in medical scans, allowing medical staff to interact with the data without any physical contact, thus avoiding the need for re-sterilization.[15] Additionally, Microsoft's Kinect sensor has also employed the use of virtual touch screen technology to help stroke patients recover and improve motor function in their limbs through a game-like system involving interpretation of gestures combined with adjustable difficulty levels based on the user's performance.[16]

Human factors and ergonomics

One of the key issues with virtual environments that utilize touch are the haptics. For example, utilizing virtual touch screen for medical training could cut costs down for training medical staff for touch delicate operations such as surgery. However, the haptics of such training does not provide sufficient physical touch feedback necessary for a virtually trained practitioner to perform a real surgery. Incorporating sufficient haptic feedback to virtual touch screen interactions is an active area of research.[17]

See also

References

External links

Notes and References

  1. Book: Gary B. Shelly, Thomas J. Cashman, Misty E. Vermaat, Jeffrey J. Quasney . Discovering computers 2008 : introductory. 2008. Thomson/Course Technology. Boston, MA. 978-1423912040. etal.
  2. Koh. Eunjin. Won. Jongho. Bae. Changseok. Vision-based Virtual Touch Screen Interface. 2008 Digest of Technical Papers-International Conference on Consumer Electronics. 2008. 1–2.
  3. News: Displair company from Astrakhan demonstrated interactive screenless display. 2012-04-17 . kiosks.ru . 2012-06-08.
  4. Web site: "Project Natal" 101 . Microsoft . June 1, 2009 . June 2, 2009 . https://web.archive.org/web/20120121223600/http://download.microsoft.com/download/A/4/A/A4A457B3-DF5D-4BF2-AD4E-963454BA0BCC/ProjectNatalFactSheetMay09.zip . January 21, 2012 . dead . mdy-all .
  5. Web site: Kain. Erik. Microsoft Augments Reality With Windows Holographic. Forbes. 21 January 2015. 21 January 2015.
  6. Web site: Myerson. Terry. The next generation of Windows: Windows 10. Blogging Windows. Microsoft. 21 January 2015. 21 January 2015.
  7. AHN. YANG-KEUN. KIM. MIN-WOOK. PARK. YOUNG-CHOONG. CHOI. KWANG-SOON. PARK. WOO-CHOOL. SEO. HAE-MOON. JUNG. KWANG-MO. Implementation of 3D Virtual Touch Screen. Korea Electronics Technology Institute. 2009.
  8. https://www.engadget.com/2013/05/21/microsofts-new-kinect-is-official/ Microsoft's new Kinect is official: larger field of view, HD camera, wake with voice
  9. News: Kinect is always listening on Xbox One, but privacy is a 'top priority' for Microsoft . Dave Tach . Polygon . May 21, 2013 . May 22, 2013.
  10. News: With Kinect Controller, Hackers Take Liberties . Jenna . Wortham . November 21, 2010 . The New York Times . November 25, 2010.
  11. Web site: Medical practice finds use for Kinect hack. QuickJump Gaming Network. December 27, 2010. May 23, 2011.
  12. Web site: radRounds. Xbox Kinect-based radiology & medical image exploration. December 4, 2011.
  13. Boggess. Jason. Harding. Chris. Improving Introductory Calculus Education with 3-D Visualization and Virtual Touch (Haptics). Virtual Reality Applications Center, Iowa State University, Ames, Iowa. 2007.
  14. Web site: Ubi Interactive. 2015-04-21. Make the World Your Touchscreen.
  15. Web site: GestSure. 2015-04-21. Take control of your operating room..
  16. Web site: Microsoft Research. 2015. 2015-04-21. Stroke Recovery With Kinect. Chang. Eric.
  17. The importance of the sense of touch in virtual and real environments . IEEE MultiMedia. 2006 . Robles-De-La-Torre . Gabriel . 13 . 3 . 24–30 . 1070-986X . 10.1109/MMUL.2006.69 . 16153497.