[Paper Review] Multi-Interest Recommendation: A Survey

• Metadata
• TL;DR
• Introduction
• Key Contributions
• Background
• What Aspects Does It Model?
  • User-oriented aspects
  • Item-oriented aspects
• Method / Architecture
• Extractor (1): Dynamic Routing
• Extractor (2): Attention-based
• Extractor (3): Non-linear Transformation
• Aggregator
  • Training Objective: Diversity Regularization
    • Representation regularization
    • Distribution regularization
  • Applications and Public Datasets
  • Challenges and Future Directions
  • Limitations and Critical Assessment
  • Takeaways
  • References
  • Further Reading

Metadata

TL;DR

• Representing a user as a single embedding vector can’t capture their multifaceted, time-varying preferences. Multi-interest recommendation explicitly extracts multiple interest vectors $\mathbf{H}_u = {\mathbf{h}_u^1, …, \mathbf{h}_u^K}$ from a user’s historical interactions, enabling fine-grained preference modeling.
• This survey organizes the field around three progressive questions — why multi-interest matters, what aspects it models, and how it is built — and decomposes every method into two core modules: an interest extractor and an interest aggregator. It is the first comprehensive survey dedicated to multi-interest modeling.
• Extractors fall into dynamic routing (CapsNet), attention and its variants, and non-linear transformation; aggregators split into representation aggregation and recommendation aggregation. The survey also covers diversity regularization against representation collapse, application scenarios and public datasets, and open directions: adaptive interest count, efficiency, denoising, explainability, long-tail/cold-start, and LLM/diffusion frontiers.

Introduction

A recommendation system filters information from a user’s historical interactions to help them make decisions efficiently. Over the past two decades it has succeeded across e-commerce, social media, news, and entertainment. Yet real-world user preferences and item attributes are inherently diverse and uncertain. Take movie recommendation: a single user’s history mixes romance, fiction, and comedy, and a single movie (e.g., Life is Beautiful) belongs to romance, comedy, and war at once. Conventional methods that represent users and items as single vectors struggle to capture these intricate interactions and multifaceted item themes.

Multi-interest recommendation (also called multi-faceted preference recommendation) addresses this. Instead of one embedding per user $u$, it learns a set of $K$ latent interest vectors $\mathbf{H}_u = {\mathbf{h}_u^1, \mathbf{h}_u^2, …, \mathbf{h}_u^K}$ and uses them jointly for candidate recommendation. Decomposing preference into multiple components yields more accurate, fine-grained, and diverse recommendations — especially when behavior spans many categories, topics, or contexts — and improves explainability, since the system can identify which intent maps to which item facet.

The survey answers three progressive research questions: (1) why is multi-interest modeling important for recommendation? (2) what aspects does it focus on? and (3) how is it applied, down to the technical details of representative modules? Existing recommendation surveys either dive into specific tasks/applications or focus on single-representation methods, overlooking diverse user preferences and the multifaceted nature of items. This work fills that gap.

Key Contributions

• The first comprehensive synthesis of multi-interest modeling. It systematically reviews user multi-interest modeling research and presents its significance to the broader community.
• An innovative taxonomy. It classifies methods along orthogonal primary axes — recommendation task, modeling aspect, and research concern — giving a unified framework for analyzing the literature structurally.
• Identification of challenges and future directions. It surfaces unresolved limitations and outlines prospective research directions worth deep investigation.
• (Reviewer’s note) The real payoff is reducing every method to a 2-component framework — extractor + aggregator. Fix those two axes (dynamic routing vs. attention; fuse at representation vs. recommendation stage) and dozens of papers line up into a single table (Table 3).

Background

Two canonical tasks underpin multi-interest recommendation.

CTR prediction. Given an item set $\mathcal{I}$, a user set $\mathcal{U}$, and an interaction matrix $\mathbf{R} = [r_{i,j}]$ (with $r_{i,j}=1$ if user $u_i$ interacted with item $i_j$), estimate the probability that user $u$ clicks item $i$ when it is displayed.

Sequential recommendation. Unlike CTR, this models preference evolution and temporal patterns (long- vs. short-term). Interactions are ordered chronologically into $\mathcal{S}_u = {i_1, i_2, …, i_t}$, and the goal is to predict the item at step $t+1$.

From single to multiple representations. Under representation learning, both tasks learn a user representation $\mathbf{h}_u \in \mathbb{R}^{1 \times d}$ and an item representation $\mathbf{x}_i \in \mathbb{R}^{1 \times d}$, with the predicted score given by their inner product.

Because of uncertain user intentions, complex behavior patterns, and ambiguous item themes, a single vector is insufficient. Multi-interest recommendation therefore expands the user representation from a single vector into a list $\mathbf{H}_u = [\mathbf{h}_u^1, …, \mathbf{h}_u^K]$ (or the item into $\mathbf{X}_i = [\mathbf{x}_i^1, …, \mathbf{x}_i^K]$), where $K$ is the number of interests (or item aspects). This in turn requires an aggregation step, which comes in two flavors.

• Representation aggregation. Fuse the multiple interest representations into one vector before prediction.

where $\phi_u(\cdot)$, $\phi_i(\cdot)$ can be pooling, concat, attention, or neural networks.

• Recommendation aggregation. Compute a per-interest score first, then combine the scores via a strategy such as a max operation.

The history of the field traces back to Li et al. (2005), who first proposed that “users have many completely different interests” for e-commerce. But the field took off in 2019 when MIND (Li et al., 2019) used dynamic routing to represent multiple interests and proved its value on Mobile Tmall’s online traffic. Influential follow-ups — ComiRec, SINE, MINER — followed. Per DBLP, the count of related papers grew from 27 before 2021 to 53 in 2024, reaching 172 cumulatively by March 2025.

What Aspects Does It Model?

Modeling aspects are split by the source of external information into user-oriented and item-oriented.

User-oriented aspects

• Spatial / location. Users’ travel, accommodation, and activity behaviors vary with location — shopping malls and museums in a modern metropolis, natural scenery and monuments in a historical city. Crucial for travel and lifestyle platforms.
• Temporal & periodic. Interests shift over time and exhibit periodic trends — surfing in summer, skiing in winter. Long- and short-term preference evolution are modeled jointly.
• Social group. Users of similar social standing make similar decisions (a herding effect); preference similarity within a group enriches multi-interest profiles.
• Behaviors. Search, click, comment, add-to-cart, favorite, and purchase each reveal different latent interests and preference strengths. Modeling interests across behavioral dimensions improves both explainability and accuracy.

Item-oriented aspects

• Attributes. Items carry categories, tags, brands, and knowledge entities. Gym-goers prioritize nutritional content; fashion elites prefer designer brands. Hypergraph neural networks and knowledge graphs inject this structured knowledge into representation learning.
• Reviews. Item reviews, written by consumers, are more persuasive than merchant descriptions and reveal true preferences — useful for both user and item modeling.
• Multi-modality. Identifying items by a unique ID alone ignores the rich numeric/categorical attributes, images, videos, and text. Riding the success of large CV and NLP models, this is now the hottest sub-area.
• Domain. Users act across domains and platforms. Cross-domain information boosts target-domain performance and alleviates cold-start.

A second important distinction: explicit multi-interest modeling requires external side information to reveal preference strength per aspect, whereas implicit modeling learns multi-interest representations from historical interactions alone. Methods like DMIN, ComiRec, and TimiRec predefine a fixed interest count and then model the vector set with dynamic routing or attention — a simple yet effective paradigm that is now mainstream.

Method / Architecture

A multi-interest framework has two core components. The interest extractor learns multiple interest representations from interacted items and side information; the interest aggregator fuses these representations (or combines the per-interest recommendation results) into the final output.

Extractor (1): Dynamic Routing

CapsNet’s (Sabour et al., 2017) dynamic routing specializes in hierarchical structure learning. A capsule is a group of neurons responding to properties of an object; the activation vector’s length encodes the probability a property is present and its orientation encodes the property’s strength. For multi-interest modeling we define a set of interest capsules $\mathbf{H} = [\mathbf{h}_1, …, \mathbf{h}_K]$, updated by a non-linear squash function.

Here $\Vert \mathbf{s}_j\Vert $ is the Euclidean ($L^2$) norm. squash acts as an activation that suppresses short vectors toward zero and compresses long vectors toward unit length. $\mathbf{s}_j$ is computed as:

$\mathbf{W}_{ij}$ is a learnable transformation matrix and $c_{ij}$ is the coupling coefficient — the probability that item $i$ couples to capsule $j$, generated by iterative routing and initialized to zero. The full procedure is below.

Algorithm 1: Dynamic Routing
Input:  iteration rounds R; item representations [x_1, ..., x_t]; routing logits b_ij = 0
Output: multi-interest representations [h_1, ..., h_K]
for r = 1, 2, ..., R do
  for each low-level capsule (item) i:      c_ij = softmax(b_ij)
  for each high-level capsule (interest) j: s_j = Σ_i c_ij W_ij x_i
  for each high-level capsule j:            h_j = squash(s_j)
  update:                                   b_ij = b_ij + h_j (W_ij x_i)^T
end for
return [h_1, ..., h_K]

Extractor (2): Attention-based

Using attention instead of dynamic routing is the most widely adopted approach. Against learnable interest embeddings $\mathbf{E} = [\mathbf{e}_1, …, \mathbf{e}_K]$, it extracts $K$ interest representations from the interacted items $[\mathbf{x}_1, …, \mathbf{x}_t]$.

$\mathbf{W}_j$, $\mathbf{b}_j$ are per-interest parameters, $\sigma(\cdot)$ is an activation, and $\tau$ is a temperature controlling sharpness. Unlike routing, this finishes in one forward pass. Variants abound: some raise $\mathbf{e}_j\sigma(\mathbf{W}_j\mathbf{x}_i + \mathbf{b}_j)^\top$ to a power $\gamma$ to reshape the distribution — larger $\gamma$ concentrates attention on salient items, $\gamma=0$ degenerates to average pooling, and $\gamma \to \infty$ becomes hard attention (pick the single most relevant item). Others swap the inner product for cosine similarity, or combine Gumbel-softmax with hard attention.

Extractor (3): Non-linear Transformation

A third approach derives interests directly from the user representation $\mathbf{u}$ via a non-linear transformation, rather than from each interacted item.

where $\mathbf{u}$ is a neural-network user representation and $\mathbf{W}_j$, $\mathbf{b}_j$ are the per-interest transformation matrix and bias.

Extractor trade-offs (authors’ take). Dynamic routing clusters behaviors into discrete interest slots via iterative assignment, giving clear separation and interpretability — but adds compute from iteration and struggles when interests subtly overlap. Attention produces soft, content-based mixtures: flexible, strong on long-range dependencies, easy to implement — but suffers redundancy among interest vectors, high memory cost on long sequences, and weaker explicit separation. The authors favor routing when interpretability/diversity/separation matter (e.g., retrieval), attention for fine-grained interactions and sequential contexts, and note that hybrids (routing followed by attention refinement, or attention with diversity constraints) often hit the best efficiency-performance trade-off.

Aggregator

The aggregator merges the multiple interest representations into the final recommendation. Here are the two paradigms at the module level.

Representation aggregation — concat / pooling. The simplest approach: concat or mean/max pooling followed by an MLP.

The MLP handles both dimensional transfer and mapping interests from different aspects into a uniform latent space.

Representation aggregation — attention. This view splits preference into a “basic interest” and “diverse interests.” It correlates the basic interest $\mathbf{h}_b$ (from historical interactions) with each interest $\mathbf{h}_i$ and takes a weighted sum.

with $\mathbf{W}$ learnable. To boost diversity and ease user fatigue, some variants measure similarity via projection distance instead.

Representation aggregation — RL selector. Not all interests contribute to the current item, so some works dynamically select the most relevant interest via reinforcement learning. A policy $\pi_\theta(s, a)$ maps state $s$ (the user’s interest w.r.t. the current item) to an action distribution (whether an interest is selected), with recommendation performance as the reward. A dueling Q-network variant follows the Bellman equation:

Recommendation aggregation — mean/max pooling. Score item $i$ per interest $j$ as $y_i^j$, then pool.

Recommendation aggregation — attention. Weight each score by the relevance between target item $\mathbf{x}_i$ and interest $\mathbf{h}_j$.

Aggregator trade-offs. Representation aggregation builds interest vectors before ranking, letting each drive candidate generation — but raises training/inference cost, depends on fusion modules, and may waste compute on redundant/noisy representations. Recommendation aggregation merges results after generation, so it slots into existing pipelines and offers flexible diversity control — but its cost explodes when candidates are huge, and merging can introduce redundancy/conflicts that hurt the final result.

Training Objective: Diversity Regularization

Interest representations from routing or attention risk representation collapse: the model learns a trivial solution where all interests become indistinguishable and shrink to a narrow point, gutting representational power. The fix is a disagreement regularization term that pushes interest representations apart. There are two families.

Representation regularization

Enlarge pairwise distance among interests in the semantic hyperspace.

Cosine similarity. Minimize cosine similarity across all interest pairs to increase distinctiveness.

Contrastive learning. Use an InfoNCE loss as the regularizer, pulling instances toward positives and away from negatives.

The positive $\mathbf{h}_i^+$ comes from noise perturbation/dropout on the original; negatives $\mathbf{h}_j^-$ are sampled from other interest representations.

Distribution regularization

Instead of the representations themselves, increase the variation of each interest’s distribution.

Covariance regularization. Constrain the diagonal (variances) of the covariance matrix of the item-interest routing matrix $\mathbf{C} \in \mathbb{R}^{t \times K}$.

$\bar{\mathbf{C}}$ is the mean of $\mathbf{C}$ along the first axis, and $\Vert \cdot\Vert _F$ is the Frobenius norm.

Element-wise regularization. More directly, compare two interests’ attention distribution matrices element-wise.

with $\mathbf{W}_i$, $\mathbf{W}_j$ the attention distribution matrices of interests $i$, $j$ and $\odot$ the Hadamard product. The authors note: orthogonality/separation penalties (representation reg.) boost diversity cheaply but hurt accuracy if too strong; distributional constraints ensure coverage in clustering-based models but may misalign with real behavior; contrastive learning catches subtly overlapping interests but is sensitive to negative-pair design. They recommend hybrids such as regularization plus contrastive learning.

Applications and Public Datasets

Multi-interest modeling is applied both to general scenarios (online shopping, entertainment) and to specialized domains (healthcare, education). The survey’s catalog of scenario datasets:

Industrial deployments include MIND (Tmall); ComiRec, PIMI, UMI, ULIM (Alibaba online recommendation); Trinity (unifying multi-/long-term/long-tail interests on Douyin and Douyin Lite); and LongRetriever (multi-interest retrieval over ultra-long sequences). ULIM, for example, tackles latency on Taobao by partitioning long behavior sequences into category-aware subsequences and using a pointer-generator network to predict top-K interest categories.

Challenges and Future Directions

The survey closes with six directions, each motivated by why it is the current bottleneck.

• Adaptive interest extraction (5.1). Most methods fix the interest count $K$. But news usually centers on a single topic while movies are inherently ambiguous and multifaceted, so a fixed $K$ is unrealistic and blocks cross-domain knowledge transfer. Proposed remedies: sparse interest-capsule activation, density-based clustering (DBSCAN), hierarchical clustering dendrograms, and silhouette score / gap statistic to choose $K$ dynamically.
• Efficiency (5.2). Multi-interest modeling costs far more than general recommendation. The number of interests, dynamic routing’s iteration (learnable matrices between capsules grow quadratically with capsule count), external side-information encoding, and the aggregation strategy are all bottlenecks.
• Extraction for denoising (5.3). Not every item benefits the user’s intention (accidental clicks, noisy text). Dynamic routing inherently soft-filters noise — signals that don’t align with any coherent interest cluster are down-weighted — offering a principled path to interest-level denoising. Yet how to reduce noise within/across interest capsules is barely explored.
• Explainability (5.4). In a multi-interest setting we must explain not just why an item is recommended, but which interest among multiple preferences drove it. Uncovering the fine-grained latent mapping between items and interest representations remains open.
• Long-tail & cold-start (5.5). Data sparsity is the root cause of both. Multi-interest modeling has a structural edge: heterogeneous auxiliary signals can be explicitly injected into interest-vector augmentation. Multi-modal pretraining, cross-domain interest prototypes, and spatial-temporal/group signals reinforce cold-item representations.
• Frontier methodology (5.6). (i) RL — frame interactions as real-time decisions and model interest selection as a policy. (ii) LLM/VLM — news content encoding, item context modeling, DPO (direct preference optimization) for multi-preference alignment, multi-modal pretraining. (iii) Diffusion — the reverse process can refine multi-interest representations by mitigating noisy interactions, but recovering “true” multifaceted interests is hard given preference ambiguity and exposure bias.

Limitations and Critical Assessment

• No unified empirical benchmark. As a synthesis it’s valuable, but it offers no table reproducing methods under one dataset and protocol. Which extractor/aggregator combo actually wins, and by how much, in which scenario is left for the reader to chase across original papers.
• The hybrid recommendation lacks quantitative backing. “Routing + attention hybrids are best” is intuitive but presented without quantitative evidence (e.g., an efficiency-performance trade-off curve), so it remains a qualitative claim.
• Fuzzy explicit/implicit boundary. The explicit-vs-implicit split is useful, but where methods that use side information minimally fall is not sharply defined.
• (Author-acknowledged limits.) Fixed interest count, high compute cost, immature interest-level denoising and explainability, and the theory-practice gap in diffusion models are candidly flagged as open problems.
• No standardized metrics. There is no agreed metric measuring diversity, accuracy, and explainability jointly, making cross-paper “diversity improvement” claims hard to compare — a structural weakness of the whole sub-field.

Takeaways

• Every multi-interest method reduces to “extractor + aggregator.” When reading a new paper, ask three questions: (1) is extraction dynamic routing, attention, or non-linear transformation? (2) does aggregation happen at the representation or recommendation stage? (3) how is diversity enforced? Table 3 is that map.
• Representation collapse is the Achilles’ heel. Extracting $K$ vectors that all collapse to one point is no better than a single representation. Which regularizer you pick — cosine / contrastive / covariance / element-wise — governs the model’s actual diversity.
• Fixed $K$ is a stopgap. The single fact that news is single-topic while movies are multifaceted already shows that adaptively choosing the interest count is the next-generation challenge.
• Multi-interest is a natural fit for long-tail and cold-start. Its structure injects heterogeneous side information directly into interest vectors, giving it an inherent edge over single representations on sparsity — backed by industrial deployments (Tmall, Alibaba, Douyin).
• LLMs and diffusion are the next frontier. Folding multi-modal and world knowledge into interest extraction has started, but recovering “true” multifaceted interests via diffusion must overcome the fundamental obstacle of exposure bias.

References

• Paper: Multi-Interest Recommendation: A Survey (DOI: 10.1145/3789510)
• Code / curated repo: WHUIR/Multi-Interest-Recommendation-A-Survey
• Venue: ACM Transactions on Information Systems, Vol. 44, No. 4, Article 78 (April 2026)

Further Reading

• Multi-Interest Network with Dynamic Routing for Recommendation at Tmall (Li et al., CIKM 2019) — MIND, which kicked off multi-interest recommendation by representing user interests with dynamic routing.
• Controllable Multi-Interest Framework for Recommendation (Cen et al., KDD 2020) — ComiRec, pairing multi-interest extraction with an aggregation module that controls accuracy vs. diversity.
• Sparse-Interest Network for Sequential Recommendation (Tan et al., WSDM 2021) — SINE, adaptively inferring a sparse set of interests per user from a large concept pool.
• Exploring Periodicity and Interactivity in Multi-Interest Framework for Sequential Recommendation (Chai et al., 2021) — PIMI, integrating sequence periodicity and time intervals into multi-interest modeling.
• Dynamic Routing Between Capsules (Sabour et al., NeurIPS 2017) — the original CapsNet paper from which multi-interest dynamic routing is borrowed.