Visual Computing

University of Konstanz
IEEE Transactions on Image Processing

Data-Driven Synthesis of Cartoon Faces Using Different Styles

Y. Zhang, W. Dong, C. Ma, X. Mei, K. Li, F. Huang, B. Hu, O. Deussen

Abstract

This paper presents a data-driven approach for automatically generating cartoon faces in different styles from a given portrait image. Our stylization pipeline consists of two steps: an offline analysis step to learn about how to select and compose facial components from the databases; a runtime synthesis step to generate the cartoon face by assembling parts from a database of stylized facial components. We propose an optimization framework that, for a given artistic style, simultaneously considers the desired image-cartoon relationships of the facial components and a proper adjustment of the image composition. We measure the similarity between facial components of the input image and our cartoon database via image feature matching, and introduce a probabilistic framework for modeling the relationships between cartoon facial components. We incorporate prior knowledge about image-cartoon relationships and the optimal composition of facial components extracted from a set of cartoon faces to maintain a natural, consistent, and attractive look of the results. We demonstrate generality and robustness of our approach by applying it to a variety of portrait images and compare our output with stylized results created by artists via a comprehensive user study.

BibTeX

@article{Zhang2017DataDrivenSynthesis,
  author     = {Y. Zhang and W. Dong and C. Ma and X. Mei and K. Li and F. Huang and B. Hu and O. Deussen},
  doi        = {10.1109/TIP.2016.2628581},
  issn       = {1057-7149},
  journal    = {IEEE Transactions on Image Processing},
  keywords   = {face recognition;feature extraction;feature selection;image enhancement;image matching;artistic style;cartoon face automatic generation;data-driven synthesis;eye region extraction;facial component selection;image composition adjustment;image feature matching;image-cartoon relationships;offline analysis;portrait image;runtime synthesis;stylization pipeline;stylized facial components database;Bayes methods;Databases;Electronic mail;Facial features;Optimization;Shape;Visualization;Cartoon face;component-based modeling;data-driven synthesis;face stylization},
  month      = {jan},
  number     = {1},
  pages      = {464-478},
  title      = {Data-Driven Synthesis of Cartoon Faces Using Different Styles},
  volume     = {26},
  year       = {2017},
}

Supplemental Material

Paper (.pdf, 6.2 MB)