Article

SmartCanvas: a gesture-driven intelligent drawing desk system

Authors:

Zhenyao Mo,

J. P. Lewis,

Ulrich NeumannAuthors Info & Claims

IUI '05: Proceedings of the 10th international conference on Intelligent user interfaces

Pages 239 - 243

https://doi.org/10.1145/1040830.1040881

Published: 10 January 2005 Publication History

Get Access

Abstract

This paper describes SmartCanvas, an intelligent desk system that allows a user to perform freehand drawing on a desk or similar surface with gestures. Our system requires one camera and no touch sensors. The key underlying technique is a vision-based method that distinguishes drawing gestures and transitional gestures in real time, avoiding the need for "artificial" gestures to mark the beginning and end of a drawing stroke. The method achieves an average classification accuracy of 92.17%. Pie-shaped menus and a "rotate-to-and-select" approach eliminate the need for a fixed menu display, resulting in an "invisible" interface.

References

[1]

L. Bretzner, I. Laptev, and T. Lindeberg. Hand gesture recognition using multi-scale color features, hierarchical models and particle filtering. In Proc. of Face and Gesture 2002, pages 423--428, 2002. Demo video is available at http://www.nada.kth.se/cvap/gvmdi/DrawBoard2.mpg.

Digital Library

Google Scholar

[2]

J. Callahan, D. Hopkins, M. Weiser, and B. Shneiderman. An empirical comparison of pie vs. linear menus. In Proc. of the ACM Conf. on Human Factors in Computing System (CHI 88), pages 95--100, 1988.

Digital Library

Google Scholar

[3]

X. Chen, H. Koike, Y. Nakanishi, K. Oka, and Y. Sato. Two-handed drawing on augmented desk system. In Proc. of 2002 International Conference on Advanced Visual Interfaces (AVI 2002), May 2002.

Digital Library

Google Scholar

[4]

J. Davis and M. Shah. Visual gesture recognition. Vision, Image and Signal Processing, 141(2), pages 101--106, 1994.

Crossref

Google Scholar

[5]

D. Hall, C. L. Gal, J. Martin, O. Chomat, T. Kapuscinski, and J. L. Crowley. Magicboard: a contribution to an intelligent office environment. In Proc. of the International Symposium on Intelligent Robotic Systems (SIRS 99), pages 215--223, 1999.

Google Scholar

[6]

D. Heckenberg and B. C. Lovell. Mime: a gesture-driven computer interface. In Proc. of SPIE vol. 4067, Visual Communications and Image Processing, pages 261--268, 2000.

Crossref

Google Scholar

[7]

M. J. Jones and J. M. Rehg. Statistical color models with application to skin detection. In Tech. Rep. CRL 98/11, Compaq Cambridge Research Lab, 1998.

Google Scholar

[8]

J. Martin and J.-B. Durand. Automatic handwriting gestures recognition using hidden markov models. In Proc. of Face and Gesture 2000, pages 403--409, 2000.

Digital Library

Google Scholar

[9]

J. B. Martinkauppi, M. N. Soriano, and M. H. Laaksonen. Behavior of skin color under varying illumination seen by different cameras at different color spaces. In Proc. of SPIE vol. 4301, Machine Vision Applications in Industrial Inspection IX, pages 102--113, 2001.

Crossref

Google Scholar

[10]

M. R. Mine, F. P. Brooks Jr., and C. H. Sequin. Moving objects in space: exploiting proprioception in virtual-environment interaction. In Proc. of Siggraph 97, pages 19--26, 1997.

Digital Library

Google Scholar

[11]

K. Oka, Y. Sato, and H. Koike. Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems. In Proc. of Face and Gesture 2002, 2002.

Digital Library

Google Scholar

[12]

G. Welch and G. Bishop. An introduction to the kalman filter. In Siggraph 2001 course material, 2001. Available at http://www.cs.unc.edu/char126welch/kalman/.

Google Scholar

[13]

P. Wellner. Interacting with paper on the digitaldesk. Communications of the ACM, 36(7), pages 86--97, 1993.

Digital Library

Google Scholar

Cited By

View all

Choi OSon YLim HAhn S(2019)Co-Recognition of Multiple Fingertips for Tabletop Human–Projector InteractionIEEE Transactions on Multimedia10.1109/TMM.2018.288060821:6(1487-1498)Online publication date: Jun-2019
https://doi.org/10.1109/TMM.2018.2880608
Chaman SGhadi AKetkar N(2018)Sixth Sense Teaching Aid2018 International Conference on Advanced Computation and Telecommunication (ICACAT)10.1109/ICACAT.2018.8933732(1-5)Online publication date: Dec-2018
https://doi.org/10.1109/ICACAT.2018.8933732
Liang JLin KChen HWang W(2018)Turn Any Display Into a Touch Screen Using Infrared Optical TechniqueIEEE Access10.1109/ACCESS.2018.28127566(13033-13040)Online publication date: 2018
https://doi.org/10.1109/ACCESS.2018.2812756
Show More Cited By

Index Terms

SmartCanvas: a gesture-driven intelligent drawing desk system
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
    2. Interaction paradigms

Recommendations

Enabling Finger-Gesture Interaction with Kinect
VINCI '15: Proceedings of the 8th International Symposium on Visual Information Communication and Interaction

A large number of tracking and gesture recognition algorithms and technologies have been developed in the field of human-computer interactions thanks to the introduction of cameras with depth sensors such as Microsoft's Kinect. Most of the techniques ...
Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware

Human-Computer Interaction (HCI) exists ubiquitously in our daily lives. It is usually achieved by using a physical controller such as a mouse, keyboard or touch screen. It hinders Natural User Interface (NUI) as there is a strong barrier between the ...
Novel Haar features for real-time hand gesture recognition using SVM

Due to the effect of lighting and complex background, most visual hand gesture recognition systems work only under restricted environments. Here, we propose a robust system which consists of three modules: digital zoom, adaptive skin detection, and hand ...

Comments

Information & Contributors

Information

Published In

IUI '05: Proceedings of the 10th international conference on Intelligent user interfaces

January 2005

344 pages

ISBN:1581138946

DOI:10.1145/1040830

General Chair:
Rob St. Amant
North Carolina State University, USA
,
Program Chairs:
John Riedl
University of Minnesota, USA
,
Anthony Jameson
DFKI and International University in Germany

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 January 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

IUI05

Sponsor:

IUI05: Tenth International Conference on Intelligent User Interfaces

January 10 - 13, 2005

California, San Diego, USA

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
695
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Choi OSon YLim HAhn S(2019)Co-Recognition of Multiple Fingertips for Tabletop Human–Projector InteractionIEEE Transactions on Multimedia10.1109/TMM.2018.288060821:6(1487-1498)Online publication date: Jun-2019
https://doi.org/10.1109/TMM.2018.2880608
Chaman SGhadi AKetkar N(2018)Sixth Sense Teaching Aid2018 International Conference on Advanced Computation and Telecommunication (ICACAT)10.1109/ICACAT.2018.8933732(1-5)Online publication date: Dec-2018
https://doi.org/10.1109/ICACAT.2018.8933732
Liang JLin KChen HWang W(2018)Turn Any Display Into a Touch Screen Using Infrared Optical TechniqueIEEE Access10.1109/ACCESS.2018.28127566(13033-13040)Online publication date: 2018
https://doi.org/10.1109/ACCESS.2018.2812756
Sagayam KHemanth D(2017)Hand posture and gesture recognition techniques for virtual reality applicationsVirtual Reality10.1007/s10055-016-0301-021:2(91-107)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.1007/s10055-016-0301-0
Wu HWang JZhang X(2017)Combining hidden Markov model and fuzzy neural network for continuous recognition of complex dynamic gesturesThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-015-1147-233:10(1265-1278)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1007/s00371-015-1147-2
Karray MKrid MSellami D(2016)A survey of AR systems and a case study of virtual keyboard based camera projector system2016 International Image Processing, Applications and Systems (IPAS)10.1109/IPAS.2016.7880150(1-8)Online publication date: Nov-2016
https://doi.org/10.1109/IPAS.2016.7880150
Wu HWang JZhang X(2016)User-centered gesture development in TV viewing environmentMultimedia Tools and Applications10.1007/s11042-014-2323-575:2(733-760)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1007/s11042-014-2323-5
Wu HWang J(2016)A visual attention-based method to address the midas touch problem existing in gesture-based interactionThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-014-1060-032:1(123-136)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1007/s00371-014-1060-0
Xie JShen X(2015)Hand Posture Recognition Using Kinect2015 International Conference on Virtual Reality and Visualization (ICVRV)10.1109/ICVRV.2015.9(89-92)Online publication date: Oct-2015
https://doi.org/10.1109/ICVRV.2015.9
Yao YFu Y(2014)Contour Model-Based Hand-Gesture Recognition Using the Kinect SensorIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2014.230253824:11(1935-1944)Online publication date: Nov-2014
https://doi.org/10.1109/TCSVT.2014.2302538
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Enabling Finger-Gesture Interaction with Kinect

Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware

Novel Haar features for real-time hand gesture recognition using SVM