research-article

Intuitive virtual production tools for set and light editing

Authors:

Jonas Trottnow,

Stefan Seibert,

Simon Spielmann,

Farshad Einabadi,

Clemens K. H. Sielaff,

Oliver GrauAuthors Info & Claims

CVMP '15: Proceedings of the 12th European Conference on Visual Media Production

Article No.: 6, Pages 1 - 8

https://doi.org/10.1145/2824840.2824851

Published: 24 November 2015 Publication History

Abstract

This contribution describes a set of newly developed tools for virtual production. Virtual production aims to bring together the creative production aspects into one real-time environment, to overcome the bottlenecks of offline processing in digital content production. This paper introduces tools and an architecture to edit set assets and adjust the lighting set-up. A set of tools was designed, implemented and tested on tablet PCs and one augmented reality and a virtual reality device. These tools are designed to be used on a movie set by staff not necessarily familiar with 3D software. Further, an approach to harmonize light set-ups in virtual and real scenes is introduced. This approach uses an automated image-based light capture process, which models the dominant lights as discrete light sources with fall-off characteristics to give required fine details for close range light set-ups and overcomes limitations of traditional image-based light probes. The paper describes initial results of a user evaluation using the developed tools in production-like environments.

References

[1]

ARRI ALEXA. URL arri.com/camera/alexa.

[2]

Enttec Open DMX USB. URL enttec.com/opendmxusb.php.

[3]

Leap Motion Controller. URL leapmotion.com.

[4]

Meta Spaceglasses. URL getameta.com.

[5]

Ncam Technologies. URL ncam-tech.com.

[6]

Oculus VR DK2. URL oculus.com/en-us/dk2.

[7]

OptiTrack. URL optitrack.com.

[8]

Project Dreamspace. URL dreamspaceproject.eu.

[9]

Robust 3D reconstruction from visual data (R3DR). URL resources.mpi-inf.mpg.de/R3DR/dlm.

[10]

The Foundry. URL thefoundry.co.uk.

[11]

Unity Technologies. URL unity3d.com.

[12]

ZeroMQ. URL zeromq.org.

[13]

Zeus Scout. URL itunes.apple.com/us/app/zeus-scout/id588588196.

[14]

S. Agarwal, R. Ramamoorthi, S. Belongie, and H. W. Jensen. Structured importance sampling of environment maps. In Proc. ACM SIGGRAPH, pages 605--612, 2003.

Digital Library

[15]

G. Balakrishnan. SmartVCS. URL girishbalakrishnan.com/smartvcs.

[16]

P. Debevec. Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Proc. ACM SIGGRAPH, page 32, 1998.

Digital Library

[17]

P. Debevec. A median cut algorithm for light probe sampling. In Proc. ACM SIGGRAPH, page 33, 2008.

Digital Library

[18]

F. Einabadi and O. Grau. Discrete light source estimation from light probes for photorealistic rendering. In Proc. BMVC, 2015.

[19]

O. Grau. A 3D production pipeline for special effects in TV and film. In Proc. Mirage 2005, Computer Vision/Computer Graphics Collaboration Techniques and Applications, 2005.

[20]

V. Helzle, O. Grau, and T. Knop. Virtual production. In M. A. Magnor, O. Grau, O. Sorkine-Hornung, and C. Theobalt, editors, Digital Representations of the Real World, pages 347--358. CRC Press, 2015.

[21]

T. Kollig and A. Keller. Efficient illumination by high dynamic range images. In Proc. Eurographics Rendering, pages 45--51, 2003.

Digital Library

[22]

M. Nitsche and F. Kirschner. Hybrid interface design for distinct creative practices in real-time 3D filmmaking. In Proc. Tangible, Embedded, and Embodied Interaction, pages 303--306, 2013.

Digital Library

[23]

L. Northam, J. Istead, and C. S. Kaplan. RTFX: On-set previs with UnrealEngine3. In Proc. Entertainment Computing, pages 432--435, 2011.

Digital Library

[24]

I. Wald, S. Woop, C. Benthin, G. S. Johnson, and M. Ernst. Embree: a kernel framework for efficient CPU ray tracing. ACM Trans. Graphics, 33(4):143:1--143:8, 2014.

Digital Library

Cited By

Aguilar ISementille ASanches S(2019)ARStudioMultimedia Tools and Applications10.1007/s11042-019-08064-4Online publication date: 20-Aug-2019
https://doi.org/10.1007/s11042-019-08064-4
Grau OHelzle VJoris EKnop TMichoud BSlusallek PBekaert PStarck J(2017)Dreamspace: A Platform and Tools for Collaborative Virtual ProductionSMPTE Motion Imaging Journal10.5594/JMI.2017.2712358126:6(29-36)Online publication date: Aug-2017
https://doi.org/10.5594/JMI.2017.2712358
Spielmann SSchuster AG�tz KHelzle VKopf JFu C(2016)VPETSIGGRAPH ASIA 2016 Technical Briefs10.1145/3005358.3005370(1-4)Online publication date: 28-Nov-2016
https://dl.acm.org/doi/10.1145/3005358.3005370

Index Terms

Intuitive virtual production tools for set and light editing
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Mixed / augmented reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Collaborative interaction
      2. Graphical user interfaces
    2. Interaction techniques
      1. Gestural input

Recommendations

XRStudio: A Virtual Production and Live Streaming System for Immersive Instructional Experiences
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

There is increased interest in using virtual reality in education, but it often remains an isolated experience that is difficult to integrate into current instructional experiences. In this work, we adapt virtual production techniques from filmmaking to ...
VPET: a toolset for collaborative virtual filmmaking
SA '16: SIGGRAPH ASIA 2016 Technical Briefs

Over the last decades the process of filmmaking has been subject to constant virtualization. Empty green screen stages leave the entire on-set crew clueless as real props are often replaced with virtual elements in later stages of production. With the ...
Virtual Production in Action: A Creative Implementation of Expanded Cinematography and Narratives
SIGGRAPH '22: ACM SIGGRAPH 2022 Posters

Virtual Production fulfills George Lucas's early dream of having an engulfing “space-opera in the sky” (1). Epic Games’ focus on realistic interactive 3D game environments using Unreal Engine, revolutionized the field of film-making, by replacing rear ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CVMP '15: Proceedings of the 12th European Conference on Visual Media Production

November 2015

121 pages

ISBN:9781450335607

DOI:10.1145/2824840

Conference Chair:
John Collomosse
University of Surrey, UK

Copyright � 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

BMVA: British Machine Vision Association and Society for Pattern Recognition
Google Inc.
NVIDIA
CDE: Centre for Digital Entertainment
YouTube: YouTube
The Foundry: The Foundry Visionmongers Ltd.
Autodesk: Autodesk

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 November 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

FP7

Conference

CVMP 2015

Sponsor:

BMVA
CDE
YouTube
The Foundry
Autodesk

CVMP 2015: 12th International Conference on Visual Media Production

November 24 - 25, 2015

London, United Kingdom

Acceptance Rates

Overall Acceptance Rate 40 of 67 submissions, 60%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
337
Total Downloads

Downloads (Last 12 months)54
Downloads (Last 6 weeks)6

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Aguilar ISementille ASanches S(2019)ARStudioMultimedia Tools and Applications10.1007/s11042-019-08064-4Online publication date: 20-Aug-2019
https://doi.org/10.1007/s11042-019-08064-4
Grau OHelzle VJoris EKnop TMichoud BSlusallek PBekaert PStarck J(2017)Dreamspace: A Platform and Tools for Collaborative Virtual ProductionSMPTE Motion Imaging Journal10.5594/JMI.2017.2712358126:6(29-36)Online publication date: Aug-2017
https://doi.org/10.5594/JMI.2017.2712358
Spielmann SSchuster AG�tz KHelzle VKopf JFu C(2016)VPETSIGGRAPH ASIA 2016 Technical Briefs10.1145/3005358.3005370(1-4)Online publication date: 28-Nov-2016
https://dl.acm.org/doi/10.1145/3005358.3005370

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents