MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask (2021-02-09T00:00:00.000000Z)

TL;DR

Results on three real-world datasets demonstrate that the proposed MaskNet models outperform state-of-the-art models such as DeepFM and xDeepFM significantly, which implies MaskBlock is an effective basic building unit for composing new high performance ranking systems.

Abstract

Click-Through Rate(CTR) estimation has become one of the most fundamental tasks in many real-world applications and it's important for ranking models to effectively capture complex high-order features. Shallow feed-forward network is widely used in many state-of-the-art DNN models such as FNN, DeepFM and xDeepFM to implicitly capture high-order feature interactions. However, some research has proved that addictive feature interaction, particular feed-forward neural networks, is inefficient in capturing common feature interaction. To resolve this problem, we introduce specific multiplicative operation into DNN ranking system by proposing instance-guided mask which performs element-wise product both on the feature embedding and feed-forward layers guided by input instance. We also turn the feed-forward layer in DNN model into a mixture of addictive and multiplicative feature interactions by proposing MaskBlock in this paper. MaskBlock combines the layer normalization, instance-guided mask, and feed-forward layer and it is a basic building block to be used to design new ranking model under various configurations. The model consisting of MaskBlock is called MaskNet in this paper and two new MaskNet models are proposed to show the effectiveness of MaskBlock as basic building block for composing high performance ranking systems. The experiment results on three real-world datasets demonstrate that our proposed MaskNet models outperform state-of-the-art models such as DeepFM and xDeepFM significantly, which implies MaskBlock is an effective basic building unit for composing new high performance ranking systems.

Authors

Zhiqiang Wang

2 papers

Junlin Zhang

4 papers

Qingyun She

1 papers

MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask

TL;DR

Abstract

Authors

References25 items

MaCAR: Urban Traffic Light Control via Active Multi-agent Communication and Action Rectification

Neural Collaborative Filtering vs. Matrix Factorization Revisited

Hierarchical Gating Networks for Sequential Recommendation

FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction

AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks

Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts

xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems

Latent Cross: Making Use of Context in Recurrent Recommender Systems

Squeeze-and-Excitation Networks

Deep & Cross Network for Ad Click Predictions

Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks

DeepFM: A Factorization-Machine based Neural Network for CTR Prediction

Language Modeling with Gated Convolutional Networks

Field-aware Factorization Machines for CTR Prediction

Layer Normalization

Wide & Deep Learning for Recommender Systems

Deep Learning over Multi-field Categorical Data - - A Case Study on User Response Prediction

Highway Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Practical Lessons from Predicting Clicks on Ads at Facebook

Ad click prediction: a view from the trenches

Factorization Machines

Web-Scale Bayesian Click-Through rate Prediction for Sponsored Search Advertising in Microsoft's Bing Search Engine

Matrix Factorization Techniques for Recommender Systems

DLP-KDD

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names