End-to-end Document Recognition and Understanding with Dessurt (2022-03-30T00:00:00.000000Z)

TL;DR

Dessurt is a relatively simple document understanding transformer capable of being fine-tuned on a greater variety of document tasks than prior methods and is able to handle a variety ofdocument domains and tasks.

Abstract

We introduce Dessurt, a relatively simple document understanding transformer capable of being fine-tuned on a greater variety of document tasks than prior methods. It receives a document image and task string as input and generates arbitrary text autoregressively as output. Because Dessurt is an end-to-end architecture that performs text recognition in addition to the document understanding, it does not require an external recognition model as prior methods do. Dessurt is a more flexible model than prior methods and is able to handle a variety of document domains and tasks. We show that this model is effective at 9 different dataset-task combinations.

Authors

Brian L. Davis

4 papers

B. Morse

4 papers

Brian L. Price

16 papers

TL;DR

Abstract

Authors

References40 items

Recognition-free Question Answering on Handwritten Document Collections

BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents

Asking questions on handwritten document collections

DocFormer: End-to-End Transformer for Document Understanding

Visual FUDGE: Form Understanding via Dynamic Graph Editing

DocReader: Bounding-Box Free Training of a Document Information Extraction Model

Cost-effective End-to-end Information Extraction for Semi-structured Document Images

Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer

LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding

End-to-End Handwritten Paragraph Text Recognition Using a Vertical Attention Network

Text and Style Conditioned GAN for the Generation of Offline-Handwriting Lines

DocVQA: A Dataset for VQA on Document Images

OrigamiNet: Weakly-Supervised, Segmentation-Free, One-Step, Full Page Text Recognition by learning to unfold

LayoutLM: Pre-training of Text and Layout for Document Image Understanding

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Deep Visual Template-Free Form Parsing

A Computationally Efficient Pipeline Approach to Full Page Offline Handwritten Text Recognition

FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents

Information extraction from historical handwritten document images with a context-aware neural model

Start, Follow, Read: End-to-End Full-Page Handwriting Recognition

Decoupled Weight Decay Regularization

Data Augmentation for Recognition of Handwritten Words and Lines Using a CNN-LSTM Network

Attention is All you Need

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition

End To End

Distilling the Knowledge in a Neural Network

Evaluation of deep convolutional nets for document image classification and retrieval

Building a test collection for complex document information processing

The IAM-database: an English sentence database for offline handwriting recognition

Donut: Document Understanding Transformer without OCR

Are End-to-End Systems Really Necessary for NER on Handwritten Document Images?

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Language Models are Unsupervised Multitask Learners

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

POS Tagging and Named Entity Recognition on Handwritten Documents

– (4.42%) Cell: The query contains the texts of a row and a column header and the model must read the corresponding cell

Wikimedia downloads

– (4.02%) Link Above: Same as the above tasks, but only read text of linked entities up the hierarchy

– (4.42%) Column Header: Same as the above task by reading the column header

Field of Study