Conceptual Capt… (2018-07-01T00:00:00.000000Z)

Abstract

We present a new dataset of image caption annotations, Conceptual Captions, which contains an order of magnitude more images than the MS-COCO dataset (Lin et al., 2014) and represents a wider variety of both images and image caption styles. We achieve this by extracting and filtering image caption annotations from billions of webpages. We also present quantitative evaluations of a number of image captioning models and show that a model architecture based on Inception-ResNetv2 (Szegedy et al., 2016) for image-feature extraction and Transformer (Vaswani et al., 2017) for sequence modeling achieves the best performance when trained on the Conceptual Captions dataset.

Authors

Radu Soricut

17 papers

Piyush Sharma

3 papers

Nan Ding

6 papers

Abstract

Authors

References33 items

Cold-Start Reinforcement Learning with Softmax Policy Gradient

Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures (Extended Abstract)

Attention is All you Need

Improved Image Captioning via Policy Gradient optimization of SPIDEr

Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors

Training and Evaluating Multimodal Word Embeddings with Large-scale Web Annotated Images

SPICE: Semantic Propositional Image Caption Evaluation

Review Networks for Caption Generation

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures

Sequence Level Training with Recurrent Neural Networks

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

Deep visual-semantic alignments for generating image descriptions

CIDEr: Consensus-based image description evaluation

From captions to visual concepts and back

Long-term recurrent convolutional networks for visual recognition and description

Show and tell: A neural image caption generator

Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models

Sequence to Sequence Learning with Neural Networks

Neural Machine Translation by Jointly Learning to Align and Translate

Microsoft COCO: Common Objects in Context

From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions

Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics

ImageNet classification with deep convolutional neural networks

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

FlumeJava: easy, efficient data-parallel pipelines

ImageNet: A large-scale hierarchical image database

METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments

Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics

Long Short-Term Memory

Learning Deep Architectures for AI

Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization

Field of Study

Venue Information

Name

Type

URL

Alternate Names