Core Concepts
GraphFM is a novel approach that leverages the strengths of Factorization Machines and Graph Neural Networks to explicitly model beneficial feature interactions in an interpretable manner.
Abstract
The paper proposes a novel model called Graph Factorization Machine (GraphFM) that combines the strengths of Factorization Machines (FM) and Graph Neural Networks (GNN) to address the limitations of each approach in modeling feature interactions.
Key highlights:
- FM can only model pairwise (second-order) feature interactions, while higher-order interactions lead to combinatorial explosion. GNNs can model higher-order interactions but rely on the assumption that neighboring nodes share similar features, which may not hold for feature interaction modeling.
- GraphFM treats features as nodes and their interactions as edges in a graph. It first selects the beneficial feature interactions using a metric function, then aggregates these selected interactions using an attentional mechanism to update the feature representations.
- By stacking multiple layers, GraphFM can model feature interactions of increasing orders, with the highest order determined by the layer depth. This allows it to capture higher-order interactions in an explicit and interpretable manner.
- Experiments on CTR prediction and recommender system datasets show that GraphFM outperforms state-of-the-art methods, and the visualization of the learned interaction graphs provides insights into the model's decision-making process.
Stats
The number of feature fields in the Criteo, Avazu, and MovieLens-1M datasets are 39, 23, and 7 respectively.
Quotes
"To solve the problems, we propose a novel approach, Graph Factorization Machine (GraphFM), by naturally representing features in the graph structure."
"By treating features as nodes and their pairwise feature interactions as edges, we bridge the gap between GNN and FM, and make it feasible to leverage the strength of GNN to solve the problem of FM."
"Extensive experiments are conducted on CTR benchmark and recommender system datasets to evaluate the effectiveness and interpretability of our proposed method."