Visible to the public Coherent Rendering of Virtual Smile Previews with Fast Neural Style Transfer

TitleCoherent Rendering of Virtual Smile Previews with Fast Neural Style Transfer
Publication TypeConference Paper
Year of Publication2019
AuthorsVasiliu, V., Sörös, G.
Conference Name2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
Date PublishedOct. 2019
PublisherIEEE
ISBN Number978-1-7281-0987-9
Keywordsartistic style transfer research, augmented reality, autoencoder, Autoencoder Neural Network, Cameras, coherent rendering, convolutional neural network, dental virtual try-on application, Dentistry, fast neural style transfer, Image color analysis, Metrics, neural nets, neural style transfer, original frame, post-processing method, pubcrawl, rendered frame, rendered overlays, rendering (computer graphics), resilience, Resiliency, Scalability, style transfer, Task Analysis, Training, virtual content, virtual rendering process, virtual smile previews
Abstract

Coherent rendering in augmented reality deals with synthesizing virtual content that seamlessly blends in with the real content. Unfortunately, capturing or modeling every real aspect in the virtual rendering process is often unfeasible or too expensive. We present a post-processing method that improves the look of rendered overlays in a dental virtual try-on application. We combine the original frame and the default rendered frame in an autoencoder neural network in order to obtain a more natural output, inspired by artistic style transfer research. Specifically, we apply the original frame as style on the rendered frame as content, repeating the process with each new pair of frames. Our method requires only a single forward pass, our shallow architecture ensures fast execution, and our internal feedback loop inherently enforces temporal consistency.

URLhttps://ieeexplore.ieee.org/document/8943754
DOI10.1109/ISMAR.2019.00-25
Citation Keyvasiliu_coherent_2019