short-paper

MirAIProjection: Real-time Projection onto High-speed Objects by Predicting Their 3D Position and Pose using DNNs

Authors:

Kosuke Maeda,

Hideki KoikeAuthors Info & Claims

AVI '20: Proceedings of the 2020 International Conference on Advanced Visual Interfaces

Article No.: 59, Pages 1 - 5

https://doi.org/10.1145/3399715.3399839

Published: 02 October 2020 Publication History

Get Access

Abstract

Allowing projections on moving objects is associated with a problem that a projection might shift due to the delay between tracking and projection. In the present paper, we proposed a new prediction model based on deep neural networks that can be used to predict both pose and position of the target object. As a result, we developed a real-time tracking and projection system named"MirAIProjection that employs motion-capture cameras and common projectors. We conducted several experiments to evaluate the effectiveness of the proposed system and demonstrated that the proposed system could reduce the slipping and increase the accuracy and robustness of the projection.

References

[1]

S. Kagami and K. Hashimoto. 2019. Animated Stickies: Fast Video Projection Mapping onto a Markerless Plane through a Direct Closed-Loop Alignment. IEEE Transactions on Visualization and Computer Graphics 25, 11 (2019), 3094--3104.

Crossref

Google Scholar

[2]

Jarrod Knibbe, Hrvoje Benko, and Andrew D Wilson. 2015. Juggling the effects of latency: Software approaches to minimizing latency in dynamic projector-camera systems. In Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. ACM, 93--94.

Digital Library

Google Scholar

[3]

Hideki Koike and Hiroaki Yamaguchi. 2015. LumoSpheres: real-time tracking of flying objects and image projection for a volumetric display. In Proceedings of the 6th Augmented Human International Conference. ACM, 93--96.

Digital Library

Google Scholar

[4]

Ishikawa Group Laboratory. 2018. DynaFlash v2 and Post Reality. http://ishikawa-vision.org/vision/dynaflashv2/index-e.html.

Google Scholar

[5]

Yuri Mikawa, Tomohiro Sueishi, Yoshihiro Watanabe, and Masatoshi Ishikawa. 2018. Variolight: Hybrid dynamic projection mapping using high-speed projector and optical axis controller. In SIGGRAPH Asia 2018 Emerging Technologies. ACM, 17.

Digital Library

Google Scholar

[6]

Shio Miyafuji, Masato Sugasaki, and Hideki Koike. 2016. Ballumiere: Real-time tracking and spherical projection for high-speed moving balls. In Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces. ACM, 33--37.

Digital Library

Google Scholar

[7]

Ramesh Raskar, Greg Welch, Kok-Lim Low, and Deepak Bandyopadhyay. 2001. Shader lamps: Animating real objects with image-based illumination. In Rendering Techniques 2001. Springer, 89--102.

Digital Library

Google Scholar

[8]

Shuya Suda, Yasutoshi Makino, and Hiroyuki Shinoda. 2019. Prediction of Volleyball Trajectory Using Skeletal Motions of Setter Player (AH2019). Association for Computing Machinery, New York, NY, USA, Article 16, 8 pages. https://doi.org/10.1145/3311823.3311844

Digital Library

Google Scholar

[9]

Yoshihiro Watanabe, Gaku Narita, Sho Tatsuno, Takeshi Yuasa, Kiwamu Sumino, and Masatoshi Ishikawa. 2015. High-speed 8-bit image projector at 1,000 fps with 3 ms delay. In The International Display Workshops. 1064--1065.

Google Scholar

[10]

Erwin Wu and Hideki Koike. 2018. Real-Time Human Motion Forecasting Using a RGB Camera (VRST '18). Association for Computing Machinery, New York, NY, USA, Article 118, 2 pages. https://doi.org/10.1145/3281505.3281598

Digital Library

Google Scholar

Cited By

View all

Yoshimura KHashimoto N(2024)Real-Time Radiometric Compensation on Deforming Non-Rigid Surfaces using Motion Prediction in Projection Mappingプロジェクションマッピングにおける動き予測を用いた変形する非剛体への実時間光学的補正The Journal of The Institute of Image Information and Television Engineers10.3169/itej.78.34878:3(348-354)Online publication date: 2024
https://doi.org/10.3169/itej.78.348
IWAI D(2024)Projection mapping technologies: A review of current trends and future directionsProceedings of the Japan Academy, Series B10.2183/pjab.100.012100:3(234-251)Online publication date: 11-Mar-2024
https://doi.org/10.2183/pjab.100.012
Caron GBelghiti MDessaux A(2022)3D Model-Based Tracking of Puppet in Depth Images for the Dynamic Video-Mapping of Its SuitOptoelectronic Devices in Robotic Systems10.1007/978-3-031-09791-1_1(1-21)Online publication date: 15-Jun-2022
https://doi.org/10.1007/978-3-031-09791-1_1

Index Terms

MirAIProjection: Real-time Projection onto High-speed Objects by Predicting Their 3D Position and Pose using DNNs
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
2. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

Ballumiere: Real-Time Tracking and Spherical Projection for High-Speed Moving Balls
ISS '16: Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces

Projections on moving objects have a problem in that the projection may slip because of the delay between tracking and projection. Here, we propose a new prediction method combining a Kalman filter and a three-frame feedback model that switches between ...
OmniEyeball: Spherical Display Embedded With Omnidirectional Camera Using Dynamic Spherical Mapping
UIST '16 Adjunct: Adjunct Proceedings of the 29th Annual ACM Symposium on User Interface Software and Technology

Recently, 360-degree panorama and spherical displays have received more and more attention due to their unique panoramic properties. Compared with existing works, we plan to utilize omnidirectional cameras in our spherical display system to enable ...
Dynamic projection mapping onto a deformable object with occlusion based on high-speed tracking of dot marker array
VRST '15: Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology

In recent years, projection mapping has attracted much attention in a variety of fields. Generally, however, the objects in projection mapping are limited to rigid and static or quasistatic objects. Dynamic projection mapping onto a deformable object ...

Comments

Information & Contributors

Information

Published In

AVI '20: Proceedings of the 2020 International Conference on Advanced Visual Interfaces

September 2020

613 pages

ISBN:9781450375351

DOI:10.1145/3399715

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

AVI '20

AVI '20: International Conference on Advanced Visual Interfaces

September 28 - October 2, 2020

Salerno, Italy

Acceptance Rates

AVI '20 Paper Acceptance Rate 36 of 123 submissions, 29%;

Overall Acceptance Rate 128 of 490 submissions, 26%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
124
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yoshimura KHashimoto N(2024)Real-Time Radiometric Compensation on Deforming Non-Rigid Surfaces using Motion Prediction in Projection Mappingプロジェクションマッピングにおける動き予測を用いた変形する非剛体への実時間光学的補正The Journal of The Institute of Image Information and Television Engineers10.3169/itej.78.34878:3(348-354)Online publication date: 2024
https://doi.org/10.3169/itej.78.348
IWAI D(2024)Projection mapping technologies: A review of current trends and future directionsProceedings of the Japan Academy, Series B10.2183/pjab.100.012100:3(234-251)Online publication date: 11-Mar-2024
https://doi.org/10.2183/pjab.100.012
Caron GBelghiti MDessaux A(2022)3D Model-Based Tracking of Puppet in Depth Images for the Dynamic Video-Mapping of Its SuitOptoelectronic Devices in Robotic Systems10.1007/978-3-031-09791-1_1(1-21)Online publication date: 15-Jun-2022
https://doi.org/10.1007/978-3-031-09791-1_1

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Ballumiere: Real-Time Tracking and Spherical Projection for High-Speed Moving Balls

OmniEyeball: Spherical Display Embedded With Omnidirectional Camera Using Dynamic Spherical Mapping

Dynamic projection mapping onto a deformable object with occlusion based on high-speed tracking of dot marker array

Comments

Published In

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Ballumiere: Real-Time Tracking and Spherical Projection for High-Speed Moving Balls

OmniEyeball: Spherical Display Embedded With Omnidirectional Camera Using Dynamic Spherical Mapping

Dynamic projection mapping onto a deformable object with occlusion based on high-speed tracking of dot marker array

Comments

Information

Published In

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations