Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024-02-29T00:00:00.000000Z)

TL;DR

This paper comprehensively review existing efforts that integrate RAG technique into AIGC scenarios, and introduces the benchmarks for RAG, discuss the limitations of current RAG systems, and suggest potential directions for future research.

Abstract

Advancements in model algorithms, the growth of foundational models, and access to high-quality datasets have propelled the evolution of Artificial Intelligence Generated Content (AIGC). Despite its notable successes, AIGC still faces hurdles such as updating knowledge, handling long-tail data, mitigating data leakage, and managing high training and inference costs. Retrieval-Augmented Generation (RAG) has recently emerged as a paradigm to address such challenges. In particular, RAG introduces the information retrieval process, which enhances the generation process by retrieving relevant objects from available data stores, leading to higher accuracy and better robustness. In this paper, we comprehensively review existing efforts that integrate RAG technique into AIGC scenarios. We first classify RAG foundations according to how the retriever augments the generator, distilling the fundamental abstractions of the augmentation methodologies for various retrievers and generators. This unified perspective encompasses all RAG scenarios, illuminating advancements and pivotal technologies that help with potential future progress. We also summarize additional enhancements methods for RAG, facilitating effective engineering and implementation of RAG systems. Then from another view, we survey on practical applications of RAG across different modalities and tasks, offering valuable references for researchers and practitioners. Furthermore, we introduce the benchmarks for RAG, discuss the limitations of current RAG systems, and suggest potential directions for future research. Github: https://github.com/PKU-DAIR/RAG-Survey.

Authors

Penghao Zhao

1 papers

Hailin Zhang

1 papers

Qinhan Yu

3 papers

TL;DR

Abstract

Authors

References401 items

From RAG to QA-RAG: Integrating Generative AI for Pharmaceutical Regulatory Compliance Process

WorldSimBench: Towards Video Generation Models as World Simulators

Cross-Modal Retrieval-enhanced code Summarization based on joint learning for retrieval and generation

Multi-Head RAG: Solving Multi-Aspect Problems with LLMs

GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning

M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions

HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

RAG-RLRC-LaySum at BioLaySumm: Integrating Retrieval-Augmented Generation and Readability Control for Layman Summarization of Biomedical Texts

Question-Based Retrieval using Atomic Units for Enterprise RAG

ERATTA: Extreme RAG for Table To Answers with Large Language Models

Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximization

RAG-based Explainable Prediction of Road Users Behaviors for Automated Driving using Knowledge Graphs and Large Language Models

Telco-RAG: Navigating the Challenges of Retrieval Augmented Language Models for Telecommunications

Code Search Is All You Need? Improving Code Suggestions with Code Search

RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrievers

Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity

Dynamic Contexts for Generating Suggestion Questions in RAG Based Conversational Systems

Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases

You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrieval

PipeRAG: Fast Retrieval-Augmented Generation via Algorithm-System Co-design

RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation

Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering

WIKIGENBENCH: Exploring Full-length Wikipedia Generation under Real-World Scenario

Structure-Guided Adversarial Training of Diffusion Models

Retrieval is Accurate Generation

Contextualized Diffusion Models for Text-Guided Image and Video Generation

Two-stage Generative Question Answering on Temporal Knowledge Graph Using Large Language Models

A fine-tuning enhanced RAG system with quantized influence measure as AI judge

CodeS: Towards Building Open-source Language Models for Text-to-SQL

Citation-Enhanced Generation for LLM-based Chatbots

Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Models

REPOFUSE: Repository-Level Code Completion with Fused Dual Context

Benchmarking Retrieval-Augmented Generation for Medicine

Exploring the Impact of Table-to-Text Methods on Augmenting LLM-based Question Answering with Domain Hybrid Data

EvoR: Evolving Retrieval for Code Generation

What Evidence Do Language Models Find Convincing?

MURRE: Multi-Hop Table Retrieval with Removal for Open-Domain Text-to-SQL

Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMs

In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss

RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model

Retrieval Augmented Thought Process for Private Data Handling in Healthcare.

G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering

UniLog: Automatic Logging via LLM and In-Context Learning

LESS: Selecting Influential Data for Targeted Instruction Tuning

Retrieval-Augmented Score Distillation for Text-to-3D Generation

M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation

C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models

RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval

CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models

Corrective Retrieval Augmented Generation

MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries

The Power of Noise: Redefining Retrieval for RAG Systems

UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems

Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs

Prompt-RAG: Pioneering Vector Embedding-Free Retrieval-Augmented Generation in Niche Domains, Exemplified by Korean Medicine

The Chronicles of RAG: The Retriever, the Chunk and the Generator

CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges

Seven Failure Points When Engineering a Retrieval Augmented Generation System

How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering

Reinforcement Learning for Optimizing RAG for Domain Chatbots

Improving Diffusion-Based Image Synthesis with Context Prediction

Cross-Modal Retrieval and Semantic Refinement for Remote Sensing Image Captioning

Retrieval-Augmented Egocentric Video Captioning

RRGcode: Deep hierarchical search-based code generation

keqing: knowledge-based question answering is a nature chain-of-thought mentor of LLM

KnowledgeNavigator: leveraging large language models for enhanced reasoning over knowledge graph

AMD: Anatomical Motion Diffusion with Interpretable Motion Decomposition and Fusion

Retrieval-Augmented Generation for Large Language Models: A Survey

RIGHT: Retrieval-augmented Generation for Mainstream Hashtag Recommendation

E&V: Prompting Large Language Models to Perform Static Analysis by Pseudo-code Execution and Verification

Lessons from Building StackSpot Al: A Contextualized AI Coding Assistant

RTLFixer: Automatically Fixing RTL Syntax Errors with Large Language Model

Experimental Analysis of Large-scale Learnable Vector Storage Compression

LM-Cocktail: Resilient Tuning of Language Models via Model Merging

De-fine: Decomposing and Refining Visual Programs with Auto-Feedback