Few-Shot Knowledge Transfer for Fine-Grained Cartoon Face Generation

Abstract

Cartoon face generation is a task that aims to convert real-face photos to exquisite cartoons. In this paper, we are interested in generating fine-grained cartoon faces for various groups, such as women, men, kids and the elderly. Although the cartoon faces of these groups share similar style, the appearances in various groups could still have some specific characteristics. In our setting, we assume that one of these groups consists of sufficient training data while the others only contain few samples, thus we can study how to transfer knowledge among groups and learn group-specific characteristics with only few samples. To tackle this problem, we propose a multi-branch translation model architecture and a two-stage training process. First, a basic translation model for the common group (which consists of sufficient data) is trained. Then, given new samples of other groups, we extend the basic model by creating group-specific branches for each new group. Group-specific branches are updated directly to capture specific appearances for each group while the remaining group-shared parameters are updated indirectly to maintain the distribution of intermediate feature space. To validate the effectiveness of our method, we further collect a cartoon-face dataset consisting of four groups with carefully designed characteristics. Experiments show that our approach is capable to generate high-quality cartoon faces for various groups. Codes and datasets are available at https://github.com/payne53/few_shot_face2cartoon.