MSEmbGAN : Multi-Stitch Embroidery Synthesis via Region-Aware Texture Generation Network

Xinrong Hu, Chen Yang, Jin Huang, Lei Zhu, Ping Li, Bin Sheng, Tong-Yee Lee, Senior Member, IEEE

0 Abstract

Although convolutional neural networks (CNNs) have been widely used in embroidery synthesis, these methods tend to fail in predicting diverse stitch types, which are often contained in real embroidery images. The reason behind is that these network architectures do not have an independent module to capture the stitch features of an embroidery image, and lack a multi-stitch embroidery dataset that is labeled with the stitch type. In this paper, we propose a multi-stitch embroidery generative adversarial network (MSEmbGAN) by considering the influences of stitch types on predicting embroidery image synthesis. To the best of our knowledge, our work is first convolutional neural network for synthesizing multi-stitch embroidery images. To achieve this, we devise the region-aware texture generation network. It detects multiple regions of the input image using a stitch classifier and generates stitch texture for each region based on its shape features. In addition, a structure colorization network with a structure discriminator is developed to further optimize the whole image by maintaining the structure consistency between input images and the output embroidery image with multiple stitches. Meanwhile, we present a multi-stitch embroidery dataset which is labeled with multiple stitch types (three single-stitch types and a multi-stitch type). To the best of our knowledge, this dataset is also the largest embroidery dataset currently: more than 30K high-quality multi-stitch embroidery images (over 13K aligned content-embroidery images and over 17K unaligned images). Experimental results including a user study show that our MSEmbGAN  clearly outperforms both the current state-of-the-art traditional method on embroidery synthesis and the popular state-of-the-art style transfer methods.

1 Introduction Video

2 Comparision

2.1 Related works comparison

2.2 Quantitative evaluation

2.3 User study

3 Dataset

4 Stitch Distributions