Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) (2022-03-24T00:00:00.000000Z)

TL;DR

A flexible and unified text-to-text paradigm called “Pretrain, Personalized Prompt, and Predict Paradigm” (P5) for recommendation, which unifies various recommendation tasks in a shared framework and will revolutionize the technical form of recommender systems towards universal recommendation engine.

Abstract

For a long time, different recommendation tasks require designing task-specific architectures and training objectives. As a result, it is hard to transfer the knowledge and representations from one task to another, thus restricting the generalization ability of existing recommendation approaches. To deal with such issues, considering that language can describe almost anything and language grounding is a powerful medium to represent various problems or tasks, we present a flexible and unified text-to-text paradigm called “Pretrain, Personalized Prompt, and Predict Paradigm” (P5) for recommendation, which unifies various recommendation tasks in a shared framework. In P5, all data such as user-item interactions, user descriptions, item metadata, and user reviews are converted to a common format — natural language sequences. The rich information from natural language assists P5 to capture deeper semantics for personalization and recommendation. Specifically, P5 learns different tasks with the same language modeling objective during pretraining. Thus, it serves as the foundation model for various downstream recommendation tasks, allows easy integration with other modalities, and enables instruction-based recommendation. P5 advances recommender systems from shallow model to deep model to big model, and will revolutionize the technical form of recommender systems towards universal recommendation engine. With adaptive personalized prompt for different users, P5 is able to make predictions in a zero-shot or few-shot manner and largely reduces the necessity for extensive fine-tuning. On several benchmarks, we conduct experiments to show the effectiveness of P5. To help advance future research on Recommendation as Language Processing (RLP), Personalized Foundation Models (PFM), and Universal Recommendation Engine (URE), we release the source code, dataset, prompts, and pretrained P5 model at https://github.com/jeykigung/P5.

References86 items

Causal Factorization Machine for Robust Recommendation

OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning

Scaling Law for Recommendation Models: Towards General-purpose User Representations

Popcorn: Human-in-the-loop Popularity Debiasing in Conversational Recommender Systems

Multitask Prompted Training Enables Zero-Shot Task Generalization

SimpleX: A Simple and Strong Baseline for Collaborative Filtering

Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation

PPT: Pre-trained Prompt Tuning for Few-shot Learning

Finetuned Language Models Are Zero-Shot Learners

Learning to Prompt for Vision-Language Models

Counterfactual Explainable Recommendation

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

HOOPS: Human-in-the-Loop Graph Reasoning for Conversational Recommendation

Personalized Transformer for Explainable Recommendation

One4all User Representation for Recommender Systems in E-commerce

Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

The Power of Scale for Parameter-Efficient Prompt Tuning

Learning How to Ask: Querying LMs with Mixtures of Soft Prompts

GPT Understands, Too

Cross-Domain Recommendation: Challenges, Progress, and Prospects

Unifying Vision-and-Language Tasks via Text Generation

What Makes Good In-Context Examples for GPT-3?

Making Pre-trained Language Models Better Few-shot Learners

Learning from Task Descriptions

The Turking Test: Can Language Models Understand Instructions?

Generate Neural Template Explanations for Recommendation

One Person, One Model, One World: Learning Continual User Representation without Forgetting

Tutorial on Conversational Recommendation Systems

S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization

Language Models are Few-Shot Learners

Neural Collaborative Filtering vs. Matrix Factorization Revisited

Neural Collaborative Reasoning

A Survey on Conversational Recommender Systems

How Can We Know What Language Models Know?

A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research

Adaptive Feature Sampling for Recommendation with Missing Content Feature Values

Neural News Recommendation with Multi-Head Self-Attention

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Feature-level Deeper Self-Attention Network for Sequential Recommendation

MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation

Personalized Fashion Recommendation with Visual Explanations based on Multimodal Attention Network: Towards Visually Explainable Recommendation

Are we really making much progress? A worrying analysis of recent neural recommendation approaches

Hierarchical Gating Networks for Sequential Recommendation

From Zero-Shot Learning to Cold-Start Recommendation

Reinforcement Knowledge Graph Reasoning for Explainable Recommendation

BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer

Towards Conversational Search and Recommendation: System Ask, User Respond

Self-Attentive Sequential Recommendation

Conversational Recommender System

Explainable Recommendation: A Survey and New Perspectives

Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding

Decoupled Weight Decay Regularization

Joint Representation Learning for Top-N Recommendation with Heterogeneous Information Sources

Cross-Domain Recommendation: An Embedding and Mapping Approach

Neural Rating Regression with Abstractive Tips Generation for Recommendation

Attention is All you Need

Learning to Generate Product Reviews from Attributes

DeepFM: A Factorization-Machine based Neural Network for CTR Prediction

Joint Deep Modeling of Users and Items Using Reviews for Recommendation

Towards Conversational Recommender Systems

Wide & Deep Learning for Recommender Systems

Neural Machine Translation of Rare Words with Subword Units

Practical Lessons from Predicting Clicks on Ads at Facebook

Do users rate or review?: boost phrase-level sentiment labeling with review-level sentiment classification

Explicit factor models for explainable recommendation based on phrase-level sentiment analysis

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Recurrent neural networks

Learning Attribute-to-Feature Mappings for Cold-Start Recommendations

Factorization Machines

Matrix Factorization Techniques for Recommender Systems

BPR: Bayesian Personalized Ranking from Implicit Feedback

Relational learning via collective matrix factorization

Addressing cold-start problem in recommendation systems

Item-based collaborative filtering recommendation algorithms

Improving Personalized Explanation Generation through Visualization

Prefix-Tuning: Optimizing Continuous Prompts for Generation

Crossing the Format Boundary of Text and Boxes: Towards Unified Vision-Language Modeling

Autoprompt:Elicitingknowledgefromlanguagemodelswithautomatically generatedprompts

Language Models are Unsupervised Multitask Learners

A Meta-Learning Perspective on Cold-Start Recommendations for Items

Content-Based Recommendation Systems

Industry Report: Amazon.com Recommendations: Item-to-Item Collaborative Filtering

Categories and Subject Descriptors

An open architecture for collaborative filtering of netnews

RecSys ’22, September 18–23, 2022, Seattle, WA, USA