Biblio

In this paper, the layer choices of the image style transfer method using the VGG-19 neural network are studied. The VGG-19 network is used to extract the feature maps which have their implicit meaning as a learning basis. If the layers for stylistic learning are not suitably chosen, the quality of style transferred image may not look good. After making experiments, it can be observed that the color information is concentrated on lower layers from conv1-1 to conv2-2, and texture information is concentrated on the middle layers from conv3-1 to conv4-4. As to the higher layers from conv5-1 to conv5-4, they seem to be able to depict image content well. Based on these observations, the methods of color transfer, texture transfer and style transfer are presented and make comparisons with conventional methods.

In today's society, even though the technology is so developed, the coloring of computer images has remained at the manual stage. As a carrier of human culture and art, film has existed in our history for hundred years. With the development of science and technology, movies have developed from the simple black-and-white film era to the current digital age. There is a very complicated process for coloring old movies. Aside from the traditional hand-painting techniques, the most common method is to use post-processing software for coloring movie frames. This kind of operation requires extraordinary skills, patience and aesthetics, which is a great test for the operator. In recent years, the extensive use of machine learning and neural networks has made it possible for computers to intelligently process images. Since 2016, various types of generative adversarial networks models have been proposed to make deep learning shine in the fields of image style transfer, image coloring, and image style change. In this case, the experiment uses the generative adversarial networks principle to process pictures and videos to realize the automatic rendering of old documentary movies.

Recent research endeavors have shown the potential of using feed-forward convolutional neural networks to accomplish fast style transfer for images. In this work, we take one step further to explore the possibility of exploiting a feed-forward network to perform style transfer for videos and simultaneously maintain temporal consistency among stylized video frames. Our feed-forward network is trained by enforcing the outputs of consecutive frames to be both well stylized and temporally consistent. More specifically, a hybrid loss is proposed to capitalize on the content information of input frames, the style information of a given style image, and the temporal information of consecutive frames. To calculate the temporal loss during the training stage, a novel two-frame synergic training mechanism is proposed. Compared with directly applying an existing image style transfer method to videos, our proposed method employs the trained network to yield temporally consistent stylized videos which are much more visually pleasant. In contrast to the prior video style transfer method which relies on time-consuming optimization on the fly, our method runs in real time while generating competitive visual results.

We propose a method for transferring an arbitrary style to only a specific object in an image. Style transfer is the process of combining the content of an image and the style of another image into a new image. Our results show that the proposed method can realize style transfer to specific object.