Article

Layered Representations for Human Activity Recognition

Authors:

Ashutosh GargAuthors Info & Claims

ICMI '02: Proceedings of the 4th IEEE International Conference on Multimodal Interfaces

Page 3

https://doi.org/10.1109/ICMI.2002.1166960

Published: 14 October 2002 Publication History

Abstract

We present the use of layered probabilistic representations using Hidden Markov Models for performing sensing, learning, and inference at multiple levels of temporal granularity. We describe the use of the representation in a system that diagnoses states of a user's activity based on real-time streams of evidence from video, acoustic, and computer interactions. We review the representation, present an implementation, and report on experiments with the layered representation in an office-awareness application.

References

[1]

M. Brand and V. Kettnaker. Discovery and segmentation of activities in video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 2000.

Digital Library

[2]

M. Brand, N. Oliver, and A. Pentland. Coupled hidden markov models for complex action recognition. In Proc. of CVPR97, pages 994-999, 1996.

Digital Library

[3]

M. Brandstein and H. Silverman. A practical methodology for speech source localization with microphone arrays. 11(2):91-126, 1997.

[4]

H. Buxton and S. Gong. Advanced Visual Surveillance using Bayesian Networks. In International Conference on Computer Vision, pages 111-123, Cambridge, Massachusetts, June 1995.

[5]

B. Clarkson and A. Pentland. Unsupervised clustering of ambulatory audio and video. In International Conference on Acoustics, Speech and Signal Processing, ICASSP'99, volume VI, pages 3037-3040, 1999.

Digital Library

[6]

J. Fernyhough, A. Cohn, and D. Hogg. Building qualitative event models automatically from visual input. In ICCV'98, pages 350-355, 1998.

Digital Library

[7]

J. Forbes, T. Huang, K. Kanazawa, and S. Russell. The batmobile: Towards a bayesian automated taxi. In Proc. Fourteenth International Joint Conference on Artificial Intelligence, IJCAI'95, 1995.

Digital Library

[8]

A. Galata, N. Johnson, and D. Hogg. Learning variable length markov models of behaviour. International Journal on Computer Vision, IJCV, pages 398-413, 2001.

Digital Library

[9]

J. Hoey. Hierarchical unsupervised learning of event categories, Unpublished Manuscript, 2001.

[10]

E. Horvitz, A. Jacobs, and D. Hovel. Attention-sensitive alerting. In Proc. of Conf. on Uncertainty in Artificial Intelligence, UAI'99, pages 305-313, 1999.

Digital Library

[11]

S. S. Intille and A. F. Bobick. A framework for recognizing multi-agent action from visual evidence. In AAAI/IAAI'99, pages 518-525, 1999.

Digital Library

[12]

Y. Ivanov and A. Bobick. Recognition of visual activities and interactions by stochastic parsing. IEEE Trans. on Pattern Analysis and Machine Intelligence, TPAMI, 22(8):852- 872, 2000.

Digital Library

[13]

B. Johnson and S. Greenberg. Judging people's availability for interaction from video snapshots. In Proc. of the IEEE Hawaii International Conference on System Sciences, HICS'99, 1999.

Digital Library

[14]

S. Li, X. Zou, Y. Hu, Z. Zhang, S. Yan, X. Peng, L. Huang, and H. Zhang. Real-time multi-view face detection, tracking, pose estimation, alignment, and recognition, 2001.

[15]

A. Madabhushi and J. Aggarwal. A bayesian approach to human activity recognition. In Proc. of the 2nd International Workshop on Visual Surveillance, pages 25-30, 1999.

Digital Library

[16]

L. Rabiner and B. Huang. Fundamentals of Speech Recognition. 1993.

Digital Library

[17]

F. B. S. Hongeng and R. Nevatia. Representation and optimal recognition of human activities. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR'00, 2000.

[18]

T. Starner and A. Pentland. Real-time american sign language recognition from video using hidden markov models. In Proceed. of SCV'95, pages 265-270, 1995.

Digital Library

[19]

A. Wilson and A. Bobick. Recognition and interpretation of parametric gesture. In Proc. of International Conference on Computer Vision, ICCV'98, pages 329-336, 1998.

Digital Library

[20]

J. Zacks and B. Tversky. Event structure in perception and cognition. Psychological Bulletin, 127(1):3-21, 2001.

Cited By

Jeyakumar JSarker AGarcia LSrivastava M(2023)X-CHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808047:1(1-28)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580804
Wei BHu WYang MChou C(2019)From Real to ComplexACM Transactions on Sensor Networks10.1145/333802615:3(1-32)Online publication date: 9-Aug-2019
https://dl.acm.org/doi/10.1145/3338026
Hu TZhu XGuo WWang SZhu J(2019)Human action recognition based on scene semanticsMultimedia Tools and Applications10.1007/s11042-017-5496-x78:20(28515-28536)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1007/s11042-017-5496-x
Show More Cited By

Recommendations

Incorporating uncertainty in a layered HMM architecture for human activity recognition
J-HGBU '11: Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding

In this study, conditioned HMM (CHMM), which inherit the structure from the latent-dynamic conditional random field(LDCRF) proposed by Morency et al. but is also based on a Bayesian network [1, 2]. Within the model a sequence of class labels is ...
Layered abduction for speech recognition from articulation
Combining Spectral Representations for Large-Vocabulary Continuous Speech Recognition

In this paper, we investigate the combination of complementary acoustic feature streams in large-vocabulary continuous speech recognition (LVCSR). We have explored the use of acoustic features obtained using a pitch-synchronous analysis, Straight, in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '02: Proceedings of the 4th IEEE International Conference on Multimodal Interfaces

October 2002

526 pages

ISBN:0769518346

Copyright � Copyright (c) 2002 Institute of Electrical and Electronics Engineers, Inc. All rights reserved.

Sponsors

Publisher

IEEE Computer Society

United States

Publication History

Published: 14 October 2002

Check for updates

Qualifiers

Article

Acceptance Rates

ICMI '02 Paper Acceptance Rate 87 of 165 submissions, 53%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

63
Total Citations
View Citations
337
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jeyakumar JSarker AGarcia LSrivastava M(2023)X-CHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808047:1(1-28)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580804
Wei BHu WYang MChou C(2019)From Real to ComplexACM Transactions on Sensor Networks10.1145/333802615:3(1-32)Online publication date: 9-Aug-2019
https://dl.acm.org/doi/10.1145/3338026
Hu TZhu XGuo WWang SZhu J(2019)Human action recognition based on scene semanticsMultimedia Tools and Applications10.1007/s11042-017-5496-x78:20(28515-28536)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1007/s11042-017-5496-x
Helmer SPersia F(2016)ISEQL, an Interval-based Surveillance Event Query LanguageInternational Journal of Multimedia Data Engineering & Management10.4018/IJMDEM.20161001017:4(1-21)Online publication date: 1-Oct-2016
https://dl.acm.org/doi/10.4018/IJMDEM.2016100101
Coppola CKrajn�k TDuckett TBellotto N(2016)Learning temporal context for activity recognitionProceedings of the Twenty-second European Conference on Artificial Intelligence10.3233/978-1-61499-672-9-107(107-115)Online publication date: 29-Aug-2016
https://dl.acm.org/doi/10.3233/978-1-61499-672-9-107
Zhang TYang ZJia WYang BYang JHe X(2016)A new method for violence detection in surveillance scenesMultimedia Tools and Applications10.1007/s11042-015-2648-875:12(7327-7349)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.1007/s11042-015-2648-8
Bambach SCrandall DYu CZhang ZCohen PBohus DHoraud RMeng H(2015)Viewpoint Integration for Hand-Based Recognition of Social Interactions from a First-Person ViewProceedings of the 2015 ACM on International Conference on Multimodal Interaction10.1145/2818346.2820771(351-354)Online publication date: 9-Nov-2015
https://dl.acm.org/doi/10.1145/2818346.2820771
Wei BHu WYang MChou CNath SKrishnamachari BRowe A(2015)Radio-based device-free activity recognition with radio frequency interferenceProceedings of the 14th International Conference on Information Processing in Sensor Networks10.1145/2737095.2737117(154-165)Online publication date: 13-Apr-2015
https://dl.acm.org/doi/10.1145/2737095.2737117
Ahouandjinou AMotamed CEzin E(2015)A temporal belief-based hidden markov model for human action recognition in medical videosPattern Recognition and Image Analysis10.1134/S105466181503002525:3(389-401)Online publication date: 1-Jul-2015
https://dl.acm.org/doi/10.1134/S1054661815030025
Cheng GHuang YWan YBuckles B(2015)Exploring Temporal Structure of Trajectory Components for Action RecognitionInternational Journal of Intelligent Systems10.1002/int.2169030:2(99-119)Online publication date: 1-Feb-2015
https://dl.acm.org/doi/10.1002/int.21690
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents