Coherent Rendering of Virtual Smile Previews with Fast Neural Style Transfer
Title | Coherent Rendering of Virtual Smile Previews with Fast Neural Style Transfer |
Publication Type | Conference Paper |
Year of Publication | 2019 |
Authors | Vasiliu, V., Sörös, G. |
Conference Name | 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) |
Date Published | Oct. 2019 |
Publisher | IEEE |
ISBN Number | 978-1-7281-0987-9 |
Keywords | artistic style transfer research, augmented reality, autoencoder, Autoencoder Neural Network, Cameras, coherent rendering, convolutional neural network, dental virtual try-on application, Dentistry, fast neural style transfer, Image color analysis, Metrics, neural nets, neural style transfer, original frame, post-processing method, pubcrawl, rendered frame, rendered overlays, rendering (computer graphics), resilience, Resiliency, Scalability, style transfer, Task Analysis, Training, virtual content, virtual rendering process, virtual smile previews |
Abstract | Coherent rendering in augmented reality deals with synthesizing virtual content that seamlessly blends in with the real content. Unfortunately, capturing or modeling every real aspect in the virtual rendering process is often unfeasible or too expensive. We present a post-processing method that improves the look of rendered overlays in a dental virtual try-on application. We combine the original frame and the default rendered frame in an autoencoder neural network in order to obtain a more natural output, inspired by artistic style transfer research. Specifically, we apply the original frame as style on the rendered frame as content, repeating the process with each new pair of frames. Our method requires only a single forward pass, our shallow architecture ensures fast execution, and our internal feedback loop inherently enforces temporal consistency. |
URL | https://ieeexplore.ieee.org/document/8943754 |
DOI | 10.1109/ISMAR.2019.00-25 |
Citation Key | vasiliu_coherent_2019 |
- original frame
- virtual smile previews
- virtual rendering process
- virtual content
- Training
- Task Analysis
- style transfer
- Scalability
- Resiliency
- resilience
- rendering (computer graphics)
- rendered overlays
- rendered frame
- pubcrawl
- post-processing method
- artistic style transfer research
- neural style transfer
- neural nets
- Metrics
- Image color analysis
- fast neural style transfer
- Dentistry
- dental virtual try-on application
- convolutional neural network
- coherent rendering
- Cameras
- Autoencoder Neural Network
- autoencoder
- augmented reality