Handwritten Text Generation with Character-specific Encoding for Style Imitation

We propose a novel method for handwritten text generation that uses a style encoder based on a vision transformer network. The style encoder encodes handwriting style from reference images and allows the generator to imitate it. The encoder learns to disentangle style information from the content by learning to recognize who wrote the text. The self-attention mechanism in the encoder allows us to produce character-specific encodings by using characters in the target sequence as queries. Our method can generate handwritten text images in random styles by sampling random latent vectors. In comparison with existing methods, our method outperforms them for handwritten text generation in terms of the quality of generated images and their fidelity with respect to the distribution of real images. Our method also achieves significantly better performance at imitating handwriting styles defined by reference images. Furthermore, the model generalizes well to unseen data and can generate handwritten images of words and character sequences as well as imitate handwriting styles not included in the training data.