JoWUA

Abstract

There are four interconnected weaknesses of sequential recommender systems when dealing with distribution shift scenarios. First, merging the intrasession transitions into a point vector creates an information bottleneck, which is inadequate for modeling user behaviors precisely. Second, deterministic session representation cannot account for the fast-changing nature of user interests. Third, making a closed-world assumption from the interactions between users and items disregards all external semantic and contextual signals, making the system less generalizable. Lastly, latent variables such as popularity bias and temporal drift are poorly dealt with by traditional models and negatively impact performance under distribution shifts. In order to overcome these shortcomings, introduce HIDRA-Rec, a framework combining two-level item and session graph encoders and an entropy-driven Diffusion-Based Interest Modeling (DBIM) method in conjunction with a cascaded knowledge distillation pipeline of LLMs. The proposed model exploits LLM-driven marker attention mechanism, closed-form Gaussian-KL distribution alignment, and entropy-driven adaptive memory to facilitate streamwise update of LLM-free recommendations. The proposed HIDRA-Rec model is tested on MovieLens-100k, MovieLens-1M, Amazon-Book, and Yelp using standardized popularity-shift and temporal-shift settings which correspond to the most powerful baseline CURE. System-wise, the HIDRA-Rec model was developed as a web-scale ubiquitous intelligent recommendation framework which is capable of processing the stream of users' interactions in a distributed digital environment in order to construct sessions, adaptively learn the context and perform fast inference for use in web application domains such as e-commerce and social networking sites.