TL
TL;DR
This work reports cross-entropy and bits-per-character (BPC) results of the trained models, and discusses interesting points on some examples of the generated $\LaTeX{}$ code.
Abstract
In this work, we tackle the problem of structured text generation, specifically academic paper generation in $\LaTeX{}$, inspired by the surprisingly good results of basic character-level language models. Our motivation is using more recent and advanced methods of language modeling on a more complex dataset of $\LaTeX{}$ source files to generate realistic academic papers. Our first contribution is preparing a dataset with $\LaTeX{}$ source files on recent open-source computer vision papers. Our second contribution is experimenting with recent methods of language modeling and text generation such as Transformer and Transformer-XL to generate consistent $\LaTeX{}$ code. We report cross-entropy and bits-per-character (BPC) results of the trained models, and we also discuss interesting points on some examples of the generated $\LaTeX{}$ code.
References25 items
1
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context
2
Topic-to-Essay Generation with Neural Networks
3
Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context
4
Tensor2Tensor for Neural Machine Translation
5