<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by AI-Advance on Medium]]></title>
        <description><![CDATA[Stories by AI-Advance on Medium]]></description>
        <link>https://medium.com/@lifengyi_6964?source=rss-398e39ee75a0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dmbNkD5D-u45r44go_cf0g.png</url>
            <title>Stories by AI-Advance on Medium</title>
            <link>https://medium.com/@lifengyi_6964?source=rss-398e39ee75a0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 06:27:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@lifengyi_6964/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[TLDR: This work releases NineRec, a large-scale multi-modal dataset suite for transferable…]]></title>
            <link>https://medium.com/@lifengyi_6964/tldr-this-work-releases-ninerec-a-large-scale-multi-modal-dataset-suite-for-transferable-c6f695bc8efb?source=rss-398e39ee75a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6f695bc8efb</guid>
            <dc:creator><![CDATA[AI-Advance]]></dc:creator>
            <pubDate>Wed, 17 Apr 2024 02:11:03 GMT</pubDate>
            <atom:updated>2024-04-17T02:11:03.189Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cross-domain &amp; Cross-platform Recommendation Systems with Multimodal Features: Dataset and Benchmark</h3><p><strong>TLDR:</strong> This work releases NineRec, a large-scale multi-modal dataset suite for transferable recommendation, which comprises a large-scale source domain recommendation dataset and nine diverse target domain recommendation datasets from short-video platforms. The whole dataset suite contains over 30 million user-short interactions, 2 million users, and 400,000 items with raw textual descriptions and cover images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xrqlaWcsC_YfYg4ovikuhQ.png" /></figure><p>In short videos and multimedia platforms, where user behavior is guided by visual and textual signals, the effective utilization of these two modal features to construct transferable recommendation models has drawn attention from the community. To address the issues of small-scale and lack of raw modal information in the current recommendation datasets, as well as their inapplicability to transferable recommendation research, this work has released a large-scale dataset under short video scenario that includes raw images and descriptive texts. Additionally, it provides rich baseline implementations for the dataset, which comprehensively covers various modern recommendation architectures and advanced modern image/text encoders.</p><p><strong>Technical Report:</strong> <a href="https://arxiv.org/pdf/2309.07705.pdf">https://arxiv.org/pdf/2309.07705.pdf</a></p><p><strong>Code:</strong> <a href="https://github.com/westlake-repl/NineRec">https://github.com/westlake-repl/NineRec</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M3XAe0QYYdthlWdc6Zvt8A.png" /></figure><p>NineRec is a large-scale multi-modal data suite collected from public video platforms, including user comments/bullet screen/like behaviors, covers, titles and other video metadata on platforms. These curial attributes were gathered carefully under extremely strict collection rules. Each item is represented by a high-resolution cover image (e.g. the average resolution of 1920x1080 in Bili series) and an original descriptive text (originally in Chinese and translated in English by DeepL).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BXreDxf5V54l1SS_t5LU0Q.png" /></figure><p>In detail, the NineRec dataset has the following features:</p><p>- Provides original modal content: NineRec allows researchers to directly utilize large-scale high-resolution images and semantically rich natural languages to construct recommendation models, which is expected to lead to better recommendation performance.</p><p>- High-quality, diverse, and complex visual features. NineRec encompasses multiple downstream datasets with different topics, including Food, Dance, Movie, Cartoon, Music, and datasets from other platforms, like DY, KU, TN, QB. The types of cover images are diverse, which, unlike the product display images in the Amazon dataset, often contain complex scene and entity information.</p><p>- General cross-domain and cross-platform scenario. NineRec have no overlapped items or user-item interactions between source and target datasets, and between any of target datasets. This setup makes NineRec a novel dataset compared to the traditional transferable recommendation research relying on shared ID information.</p><p>- Large-scale dataset: NineRec is one of the largest multi-modal datasets for transferable recommendation and is expected to become a valuable dataset for training multi-modal recommendation foundation models.</p><p>- Content-driven scenario: The behavioral data of NineRec is collected from content-delivery platforms aimed for entertainment/news. User behaviors are driven by content, especially the content of covers and title texts. Therefore, NineRec supports in-depth research and development of recommendation models that emphasize content/visual contents.</p><p>The data collection process lasted approximately 10 months from September 2021 to July 2022. Taking Bili source as an example, short videos from more than 20 channels (including main channels with various categories) were collected as the follow procedures:</p><p>- Collected about 1000–2000 videos per channel by frequently requesting the webpage.</p><p>- Went to the pages of all these videos, which often contained many links to other videos. In each page, 3–5 videos were selected randomly.</p><p>- Repeated the above stage many times. Merged all videos and removed duplicates.</p><p>- Collected the text, cover image, and user-video interactions (public comments, including the bullet-screen comment) for each video, and ensured that each video interacted at most 3,500. Only the latest pair of user-video interactions were recorded, even though the user might comment on a video multiple times.</p><p>- A group of students was assigned to check the quality of images and text manually and remove about 1% of items that were poor quality (e.g. images with just black background, image-text mismatches, overly sensational text descriptions, etc.).</p><p>The figure below displays the statistical characteristics of the partial data. The top displays the item distributions of all datasets typically follow the long-tail distribution in most recommendation systems. The middle shows that the number of user interactions are mainly in the range of [5,100], where [5,20) is the majority, and the bottom represents the main occurring time of user-item interactions from 2017 to 2022.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VFyb4DnFxAYdujl_h1DYzg.png" /></figure><p>This paper establishes comprehensive benchmark results. These results provide the first systematic verification of the possibility of implementing transferable recommendation models based on pure modal information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JCzf2DKhus0nritUWKnUVw.png" /></figure><p>More information about this dataset can be found in the technical report.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6f695bc8efb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multimodal Multi-domain Recommendation System DataSet]]></title>
            <link>https://medium.com/@lifengyi_6964/multimodal-multi-domain-recommendation-system-dataset-e814afcdc68a?source=rss-398e39ee75a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/e814afcdc68a</guid>
            <dc:creator><![CDATA[AI-Advance]]></dc:creator>
            <pubDate>Wed, 17 Apr 2024 02:10:35 GMT</pubDate>
            <atom:updated>2024-04-17T02:10:35.268Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>TLDR:</strong> This work releases NineRec, a large-scale multi-modal dataset suite for transferable recommendation, which comprises a large-scale source domain recommendation dataset and nine diverse target domain recommendation datasets from short-video platforms. The whole dataset suite contains over 30 million user-short interactions, 2 million users, and 400,000 items with raw textual descriptions and cover images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xrqlaWcsC_YfYg4ovikuhQ.png" /></figure><p>In short videos and multimedia platforms, where user behavior is guided by visual and textual signals, the effective utilization of these two modal features to construct transferable recommendation models has drawn attention from the community. To address the issues of small-scale and lack of raw modal information in the current recommendation datasets, as well as their inapplicability to transferable recommendation research, this work has released a large-scale dataset under short video scenario that includes raw images and descriptive texts. Additionally, it provides rich baseline implementations for the dataset, which comprehensively covers various modern recommendation architectures and advanced modern image/text encoders.</p><p><strong>Technical Report:</strong> <a href="https://arxiv.org/pdf/2309.07705.pdf">https://arxiv.org/pdf/2309.07705.pdf</a></p><p><strong>Code:</strong> <a href="https://github.com/westlake-repl/NineRec">https://github.com/westlake-repl/NineRec</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M3XAe0QYYdthlWdc6Zvt8A.png" /></figure><p>NineRec is a large-scale multi-modal data suite collected from public video platforms, including user comments/bullet screen/like behaviors, covers, titles and other video metadata on platforms. These curial attributes were gathered carefully under extremely strict collection rules. Each item is represented by a high-resolution cover image (e.g. the average resolution of 1920x1080 in Bili series) and an original descriptive text (originally in Chinese and translated in English by DeepL).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BXreDxf5V54l1SS_t5LU0Q.png" /></figure><p>In detail, the NineRec dataset has the following features:</p><p>- Provides original modal content: NineRec allows researchers to directly utilize large-scale high-resolution images and semantically rich natural languages to construct recommendation models, which is expected to lead to better recommendation performance.</p><p>- High-quality, diverse, and complex visual features. NineRec encompasses multiple downstream datasets with different topics, including Food, Dance, Movie, Cartoon, Music, and datasets from other platforms, like DY, KU, TN, QB. The types of cover images are diverse, which, unlike the product display images in the Amazon dataset, often contain complex scene and entity information.</p><p>- General cross-domain and cross-platform scenario. NineRec have no overlapped items or user-item interactions between source and target datasets, and between any of target datasets. This setup makes NineRec a novel dataset compared to the traditional transferable recommendation research relying on shared ID information.</p><p>- Large-scale dataset: NineRec is one of the largest multi-modal datasets for transferable recommendation and is expected to become a valuable dataset for training multi-modal recommendation foundation models.</p><p>- Content-driven scenario: The behavioral data of NineRec is collected from content-delivery platforms aimed for entertainment/news. User behaviors are driven by content, especially the content of covers and title texts. Therefore, NineRec supports in-depth research and development of recommendation models that emphasize content/visual contents.</p><p>The data collection process lasted approximately 10 months from September 2021 to July 2022. Taking Bili source as an example, short videos from more than 20 channels (including main channels with various categories) were collected as the follow procedures:</p><p>- Collected about 1000–2000 videos per channel by frequently requesting the webpage.</p><p>- Went to the pages of all these videos, which often contained many links to other videos. In each page, 3–5 videos were selected randomly.</p><p>- Repeated the above stage many times. Merged all videos and removed duplicates.</p><p>- Collected the text, cover image, and user-video interactions (public comments, including the bullet-screen comment) for each video, and ensured that each video interacted at most 3,500. Only the latest pair of user-video interactions were recorded, even though the user might comment on a video multiple times.</p><p>- A group of students was assigned to check the quality of images and text manually and remove about 1% of items that were poor quality (e.g. images with just black background, image-text mismatches, overly sensational text descriptions, etc.).</p><p>The figure below displays the statistical characteristics of the partial data. The top displays the item distributions of all datasets typically follow the long-tail distribution in most recommendation systems. The middle shows that the number of user interactions are mainly in the range of [5,100], where [5,20) is the majority, and the bottom represents the main occurring time of user-item interactions from 2017 to 2022.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VFyb4DnFxAYdujl_h1DYzg.png" /></figure><p>This paper establishes comprehensive benchmark results. These results provide the first systematic verification of the possibility of implementing transferable recommendation models based on pure modal information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JCzf2DKhus0nritUWKnUVw.png" /></figure><p>More information about this dataset can be found in the technical report.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e814afcdc68a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Paper Title: Protein language models-assisted optimization of a uracil-N-glycosylase variant…]]></title>
            <link>https://medium.com/@lifengyi_6964/paper-title-protein-language-models-assisted-optimization-of-a-uracil-n-glycosylase-variant-221e80080a19?source=rss-398e39ee75a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/221e80080a19</guid>
            <dc:creator><![CDATA[AI-Advance]]></dc:creator>
            <pubDate>Tue, 16 Apr 2024 02:18:37 GMT</pubDate>
            <atom:updated>2024-04-22T00:55:35.821Z</atom:updated>
            <content:encoded><![CDATA[<h3>Protein Language Model to Accelerate Enzyme Direct Evolution</h3><p><em>Paper Title: Protein language models-assisted optimization of a uracil-N-glycosylase variant enables programmable T-to-G and T-to-C base editing</em></p><blockquote>Paper: <a href="https://www.cell.com/molecular-cell/pdf/S1097-2765(24)00088-1.pdf">https://www.cell.com/molecular-cell/pdf/S1097-2765(24)00088-1.pdf</a></blockquote><blockquote><em>Code: </em><a href="https://github.com/westlake-repl/ProteinLM-TDG2-Mutation"><em>https://github.com/westlake-repl/ProteinLM-TDG2-Mutation</em></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/0*dAqyUzB1lrQmIKie.png" /></figure><p>In the field of genome editing, single-base editors achieve precise modifications of specific bases in the genome by fusing programmable DNA-binding proteins with base-modifying enzymes, without causing double-strand breaks in the DNA. Although relying on cytosine base editors (CBEs) or adenine base editors (ABEs) mediated by deamination reactions, these editors can achieve mutations from C to T or A to G, they still have limitations in inducing all types of point mutations, especially in terms of transversion mutations.</p><p>On February 19, 2024, the team led by Xing Chang from the School of Medicine and the team led by Fajie Yuan from the School of Engineering at Westlake University published a research article titled “Protein language models-assisted optimization of a Uracil-N glycosylase variant enables programmable T-to-G and T-to-C base editing” in <em>Molecular Cell</em>. This study is the first to use a protein language model (PLM) to modify uracil-N-glycosylase (UNG) and develop a highly active variant that can recognize normal thymine. Based on this, the researchers constructed a novel base editor, TSBE, which can effectively achieve T-to-G or T-to-C editing.</p><p>he research team first obtained cytosine DNA glycosylase (CDG) that specifically recognizes cytosine and thymine DNA glycosylase (TDG) that specifically recognizes thymine by adjusting the substrate specificity of uracil DNA glycosylase (UDG). Under the precise guidance of nCas9 and sgRNA, these enzymes can cleave Cor T bases, achieving the creation of abasic sites in one step, leading to more specific and efficient transversion mutations in the subsequent DNA repair process (Figure 1).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/441/0*JBrJS0VkqdFLst-T.png" /></figure><p>Figure 1 Development of a novel base editor based on uracil DNA glycosylase variant</p><p>The research team first fused the UDG variants CDG and TDG at the N-terminus of SpCas9n, resulting in N-CDG with only low editing ability for C bases and N-TDG with similarly weak editing effects for T bases. By inserting CDG into the middle of SpCas9n, CE-CDG was obtained, which significantly improved the efficiency of editing C bases, with C &gt; G conversions being predominant. However, inserting TDG into the middle of SpCas9n to obtain CE-TDG (TSBE1) still yielded unsatisfactory results for editing T bases (Figure 2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/454/0*a4VYS5c2MsjqFwxD.png" /></figure><p>Figure 2 Different fusion strategies of TDG with SpCas9n and editing efficiency</p><p>In order to further optimize TSBE, researchers used a protein language model (PLM) to predict and design high-activity mutants of TDG (Figure 3), a strategy that significantly improved the protein evolution efficiency of TSBE. The protein language model draws on the concept of “language models” in natural language processing (NLP) to predict the structure and function of proteins by analyzing the amino acid sequences of proteins. Although the application of protein language models in the optimization of specific enzymes and molecular evolution is still in its infancy, its potential has been validated in this study.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/470/0*HA5_53Uo-Ayym2Ud.png" /></figure><p>Figure 3 Process of predicting TDG mutants using a protein language model and experimental validation</p><p>By designing a scoring and ranking scheme using the language model, researchers selected TDG variants that were likely to enhance enzyme activity. Through experimental validation of these mutants, it was found that out of 48 predicted TDG mutants tested, over 50% exhibited enzyme activity enhancements ranging from 1.5 to 11-fold, a success rate far exceeding random mutagenesis. The optimized TSBE could efficiently induce T-to-G or T-to-C base substitutions in human cell lines and precisely correct pathogenic mutations in db/db mouse embryos (Figure 4).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/0*oSuuPO8zynLWPTlK.png" /></figure><p>Figure 4 Efficient induction of base conversion in mouse embryos by TSBE</p><p>It is worth noting that recently, through random mutagenesis and screening of UNG mutants, researchers have also successfully developed a transversion editor targeting C and T bases1. Some mutations that enhance TDG function are consistent with the results predicted by the protein language model. This suggests a potential synergistic effect between protein language models and traditional random mutagenesis and screening methods in the field of protein-directed evolution.</p><p>In summary, the findings of this study not only establish a new strategy for developing novel base editors but also demonstrate the tremendous potential of using protein language models for enzyme molecular evolution without the need for large amounts of specific task training data or experimental validation. These findings will pave the way for future developments in genome editing technology.</p><ol><li>Ye, L., Zhao, D., Li, J., Wang, Y., Li, B., Yang, Y., Hou, X., Wang, H., Wei, Z., Liu, X., et al. (2024). Glycosylase-based base editors for efficient T-to-G and C-to-G editing in mammalian cells. Nature Biotechnology. 10.1038/s41587–023–02050-w.</li></ol><p>Reprint source：<a href="https://zhuanlan.zhihu.com/p/683240317">https://zhuanlan.zhihu.com/p/683240317</a></p><p>Code： <a href="https://github.com/westlake-repl/ProteinLM-TDG2-Mutation">https://github.com/westlake-repl/ProteinLM-TDG2-Mutation</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=221e80080a19" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Protein Language Modeling Using Structural Alphabet]]></title>
            <link>https://medium.com/@lifengyi_6964/awesome-protein-language-modeling-using-structural-alphabet-07f7e070d588?source=rss-398e39ee75a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/07f7e070d588</guid>
            <dc:creator><![CDATA[AI-Advance]]></dc:creator>
            <pubDate>Tue, 16 Apr 2024 02:16:29 GMT</pubDate>
            <atom:updated>2024-04-16T06:42:41.005Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>SaProt: Protein Language Modeling with Structure-aware Vocabulary (ICLR2024 Spotlight)</blockquote><blockquote><a href="https://www.biorxiv.org/content/10.1101/2023.10.01.560349v1"><em>URL: </em></a><a href="https://www.biorxiv.org/content/10.1101/2023.10.01.560349v1">https://www.biorxiv.org/content/10.1101/2023.10.01.560349v1</a></blockquote><blockquote>Code: <a href="https://github.com/westlake-repl/SaProt">https://github.com/westlake-repl/SaProt</a></blockquote><p>Protein structures are often considered more informative compared to sequences because they directly determine the function of the protein. With the great breakthrough brought by AlphaFold2 (AF2), we have a large number of predicted structures available. How can we best use these structures? By training the next generation of protein language models! Like in Natural Language Processing (NLP) or Computer Vision (CV), a powerful base model could be deployed for various downstream tasks, e.g. mutational effect prediction or protein-protein interaction prediction in the protein field.</p><p>In our paper, we use Foldseek to process protein structures. Foldseek is a VQ-VAE model that encodes a protein structure into a structural sequence (the same length as residue sequence). Since it was trained to recover the distances or angles from discrete tokens, the structural sequence could represent the 3D structure well.</p><p>By encoding structures into one-dimensional discrete tokens, we combine them with amino acids (there are 20 amino acids in nature) to form a Structure-aware Vocabulary as a way of embedding structural information into model inputs and enhancing the model’s representational capabilities. Our pre-training model uses the largest number of protein structures currently available (about 40 million), and was trained on 64 A100s for 3 months, resulting in an open-source model SaProt with a size of 650 M parameters. Experimental results show that our model outperforms previous sequence and structure models on various protein tasks.</p><h3>How and why to combine structure and amino acid sequences</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/511/0*49qj8xDY7-RVPAJ_" /></figure><p>As mentioned earlier, we first encoded the protein (using Foldseek’s structure vocabulary, where each token represents a different local structure) to generate a one-dimensional structural sequence , such that the structural sequence is of equal length to the amino acid sequence. We used a simple but effective approach: we computed the Cartesian product (i.e., a two-by-two combination) of the structure vocabulary and the amino acid vocabulary to form a new structure-aware vocabulary.</p><p>This way, for each site of the protein, its residue type and corresponding local structure can be combined into an element of the new vocabulary, thus achieving our goal: allowing the model to take into account both sequence and structure information of the protein.</p><p>This is followed by Masked Language Modeling (MLM) pre-training using the Bert architecture in a classical way (see our paper for more details on training).</p><h3>Pre-training with structure-aware vocabulary decreases overfitting</h3><p>Why do we need to encode structures in this way? Let’s look at plots of the results of pre-training using different ways of encoding the structure:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*ejGSARi4Jqbt-gU0" /></figure><p>On the left and in the middle of the figure are two classical ways of modeling protein structures, i.e., encoding the structural information into bias and then adding it to the attention map of the transformer (e.g., Evoformer, Uni-Mol) or modeling the spatial relationships of proteins using graph neural networks (e.g., MIF, GearNet, etc.). From the loss plot, you can see that when the above two modeling approaches are pre-trained on the AF2 structure using the MLM loss function, the model overfits very quickly (as shown by the fact that the predicted loss is very good on AF2 structures, but the loss doesn’t decrease or even rise on real PDB structures).</p><p>We think this is due to the fact that the protein structure predicted by AF2 carries some hidden patterns, and since the first two approaches are directly modeling the 3D coordinates of the protein, these hidden patterns may be easily recognized by the model, which results in information leakage, allowing the model to easily complete the training goal without actually understanding the evolutionary relationships. In contrast, our structure-aware vocabulary ignores coordinate values by encoding protein structures into one-dimensional structural sequences while preserving structure information as much as possible, so the model is able to efficiently use the information without being affected by the hidden patterns.</p><h3>Awareness Of Protein Structure</h3><p>Our model gained strong representation capabilities by training on 40 million protein structures. A possible question is: how can we determine that our model learned more structure information rather than being trained better? We therefore tested SaProt and ESM-2 on the contact prediction task. We froze the backbone of the model and trained only a classification head. The results are as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*II_JqStcnCNNc1gq" /></figure><p>SaProt greatly outperforms ESM-2, which indicates that SaProt contains very rich structure information, enabling it to obtain very excellent results on structure prediction tasks. Meanwhile, we visualized alpha helix proteins and beta sheet proteins on the SCOPe database with the following results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/757/0*zy_8HH3PYa-G32ts" /></figure><p>The SaProt visualization clearly distinguishes alpha proteins from beta proteins, whereas the ESM-2 visualization mixes the two proteins, which demonstrates the strong ability of our model to perceive changes in the structures.</p><p>Conclusion</p><p>In our work, we developed a model that uses a structure-aware vocabulary and does well in a range of tests while avoiding overfitting. We believe that expanding the size of the structure vocabulary and/or scaling the model could further improve our results, but I hope I’ve shown you how this approach is useful and powerful!</p><p>Check out our <a href="https://www.biorxiv.org/content/10.1101/2023.10.01.560349v2">paper</a> for more details, and please reach out if you have any questions!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=07f7e070d588" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Autoregressive Generative Recommendation with Temporal CNNs]]></title>
            <link>https://medium.com/@lifengyi_6964/autoregressive-generative-recommendation-with-temporal-cnns-5403705140e5?source=rss-398e39ee75a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/5403705140e5</guid>
            <dc:creator><![CDATA[AI-Advance]]></dc:creator>
            <pubDate>Fri, 29 Mar 2024 03:14:50 GMT</pubDate>
            <atom:updated>2024-03-29T05:42:17.747Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Paper: A Simple Convolutional Generative Network for Next Item Recommendation (WSDM2019)</blockquote><blockquote>TensorFlow Code: <a href="https://github.com/fajieyuan/WSDM2019-nextitnet">https://github.com/fajieyuan/WSDM2019-nextitnet</a></blockquote><blockquote>PyTorch Code：<a href="https://github.com/yuangh-x/2022-NIPS-Tenrec">https://github.com/yuangh-x/2022-NIPS-Tenrec</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*fYADF4UkZU6ebw1aEj9quA.png" /></figure><p>This paper proposes NextItNet, a generative sequence recommendation system algorithm based on autoregression. The core idea is to accumulate PixelCNN, use the existing 1-dimensional hole structure to expand the receptive field to extract item sequence features, and add a residual learning mechanism to prevent sudden changes. To solve the problem of disappearance, Next Token Prediction is used to predict the next item.</p><p>Problem<br>(1) RNN-based sequence recommendation models usually rely on the entire past hidden state and cannot fully utilize sequences for parallel computing, so the speed of the model is limited in training and evaluation.</p><p>(2) Limitations of vanilla CNN: When modeling long sequence data, max pooling cannot determine the number of occurrences of important features (there may be multiple important features, but max pooling only extracts one) and cannot determine the number of important features. Position; CNN networks such as Caser are shallow networks and are difficult to capture complex relationships and model long-term dependencies;</p><p>(3) The Transformer network is based on the Attention mechanism and has square complexity. There is a bottleneck for both ultra-long sequence training and inference speed.</p><p>Highlights</p><p>(1) NextItNet is a deep network node that can be deepened to 100 layers, see [2] for details;</p><p>(2) NextItNet can model very long sequences. For example, an item sequence with a length of more than 1000 only requires 10 CNN layers.</p><p>(3) The probability estimate explicitly models the state transition distribution of all items in the sequence, not just the last item;</p><p>(4) The convolution layer uses 1-dimensional dilated convolution instead of standard 2-dimensional convolution;</p><p>(5) No pooling layer is used</p><p>Network Architecture：</p><p>Training Loss;</p><p>AutoRegressive Next-Token Prediction</p><p>Network Masking:</p><p>In order to avoid future information leakage and prevent the network from using future item information to predict current item information, the author uses method (d) for padding, and the padding size is (f − 1) ∗ l.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*ZfeZ70bNyxi9W1KZ7BEimQ.png" /></figure><p>DataSet</p><p>Main Results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*VvaHp-QbNTrq0p0KiqA0aA.png" /></figure><p>Reference</p><p>[1] A Simple Convolutional Generative Network for Next Item Recommendation. WSDM2019</p><p>[2] StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking. SIGIR2021</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5403705140e5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An Awesome Generative Sequential Recommendation Model]]></title>
            <link>https://medium.com/@lifengyi_6964/an-awesome-generative-sequential-recommendation-model-948ef67ab28a?source=rss-398e39ee75a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/948ef67ab28a</guid>
            <dc:creator><![CDATA[AI-Advance]]></dc:creator>
            <pubDate>Fri, 29 Mar 2024 03:14:10 GMT</pubDate>
            <atom:updated>2024-03-29T05:42:37.202Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Paper: A Simple Convolutional Generative Network for Next Item Recommendation (WSDM2019)</blockquote><blockquote>TensorFlow Code: <a href="https://github.com/fajieyuan/WSDM2019-nextitnet">https://github.com/fajieyuan/WSDM2019-nextitnet</a></blockquote><blockquote>PyTorch Code：<a href="https://github.com/yuangh-x/2022-NIPS-Tenrec">https://github.com/yuangh-x/2022-NIPS-Tenrec</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*fYADF4UkZU6ebw1aEj9quA.png" /></figure><p>This paper proposes NextItNet, a generative sequence recommendation system algorithm based on autoregression. The core idea is to accumulate PixelCNN, use the existing 1-dimensional hole structure to expand the receptive field to extract item sequence features, and add a residual learning mechanism to prevent sudden changes. To solve the problem of disappearance, Next Token Prediction is used to predict the next item.</p><p>Problem<br>(1) RNN-based sequence recommendation models usually rely on the entire past hidden state and cannot fully utilize sequences for parallel computing, so the speed of the model is limited in training and evaluation.</p><p>(2) Limitations of vanilla CNN: When modeling long sequence data, max pooling cannot determine the number of occurrences of important features (there may be multiple important features, but max pooling only extracts one) and cannot determine the number of important features. Position; CNN networks such as Caser are shallow networks and are difficult to capture complex relationships and model long-term dependencies;</p><p>(3) The Transformer network is based on the Attention mechanism and has square complexity. There is a bottleneck for both ultra-long sequence training and inference speed.</p><p>Highlights</p><p>(1) NextItNet is a deep network node that can be deepened to 100 layers, see [2] for details;</p><p>(2) NextItNet can model very long sequences. For example, an item sequence with a length of more than 1000 only requires 10 CNN layers.</p><p>(3) The probability estimate explicitly models the state transition distribution of all items in the sequence, not just the last item;</p><p>(4) The convolution layer uses 1-dimensional dilated convolution instead of standard 2-dimensional convolution;</p><p>(5) No pooling layer is used</p><p>Network Architecture：</p><p>Training Loss;</p><p>AutoRegressive Next-Token Prediction</p><p>Network Masking:</p><p>In order to avoid future information leakage and prevent the network from using future item information to predict current item information, the author uses method (d) for padding, and the padding size is (f − 1) ∗ l.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*ZfeZ70bNyxi9W1KZ7BEimQ.png" /></figure><p>DataSet</p><p>Main Results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*VvaHp-QbNTrq0p0KiqA0aA.png" /></figure><p>Reference</p><p>[1] A Simple Convolutional Generative Network for Next Item Recommendation. WSDM2019</p><p>[2] StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking. SIGIR2021</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=948ef67ab28a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Title：Multimodal and Large Language Model Recommendation System (awesome Paper List)]]></title>
            <link>https://medium.com/@lifengyi_6964/title-multimodal-and-large-language-model-recommendation-system-awesome-paper-list-a51efda98e30?source=rss-398e39ee75a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/a51efda98e30</guid>
            <dc:creator><![CDATA[AI-Advance]]></dc:creator>
            <pubDate>Fri, 22 Mar 2024 05:47:38 GMT</pubDate>
            <atom:updated>2024-03-22T06:19:45.903Z</atom:updated>
            <content:encoded><![CDATA[<h3>Resources and Paper List for Multimodal and LLM-based Recommendations</h3><h3>Foundation models for Recommender System Paper List</h3><blockquote>Link： <a href="https://github.com/westlake-repl/Recommendation-Systems-without-Explicit-ID-Features-A-Literature-Review">https://github.com/westlake-repl/Recommendation-Systems-without-Explicit-ID-Features-A-Literature-Review</a></blockquote><p>Keyword: <em>Recommend System, pretraining, large language model, multimodal recommender system, transferable recommender system, foundation recommender models, universal user representation, one-model-fit-all, ID features, ID embeddings</em></p><p>These papers attempt to address the following questions:</p><p>(1) Can recommender systems have their own foundation models similar to those used in NLP and CV?</p><p>(2) Is ID embedding necessary for recommender models, can we replace or abondon it?</p><p>(3) Will recommender systems shift from a matching paradigm to a generating paradigm?</p><p>(4) How can LLM be utilized to enhance recommender systems?</p><p>(5) What does the future hold for multimodal recommender systems?</p><h3>Paper List</h3><h3>Perspective paper: ID vs. LLM &amp; ID vs. Multimodal</h3><ul><li>Where to Go Next for Recommender Systems? ID-vs. Modality-based recommender models revisited, SIGIR2023, 2022/09, <a href="https://arxiv.org/pdf/2303.13835.pdf">[paper]</a> <a href="https://github.com/westlake-repl/IDvs.MoRec">[code]</a></li><li>Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.11700.pdf">[paper]</a></li><li>Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights, WSDM2024, <a href="https://arxiv.org/pdf/2305.15036.pdf">[paper]</a> <a href="https://github.com/westlake-repl/Adapter4Rec">[code]</a></li></ul><h3>Datasets for Transferable or Multimodal RS</h3><ul><li>NineRec: A Benchmark Dataset Suite for Evaluating Transferable Recommendation, TPAMI2024, <a href="https://arxiv.org/pdf/2309.07705.pdf">[paper]</a> <a href="https://github.com/westlake-repl/NineRec">[link]</a> | Images, Text, Nine downstream datasets</li><li>TenRec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems, NeurIPS 2022 <a href="https://proceedings.neurips.cc/paper_files/paper/2022/file/4ad4fc1528374422dd7a69dea9e72948-Paper-Datasets_and_Benchmarks.pdf">[paper]</a></li><li>PixelRec: An Image Dataset for Benchmarking Recommender Systems with Raw Pixels, arxiv 2023/09 <a href="https://arxiv.org/pdf/2309.06789.pdf">[paper]</a> |<a href="https://github.com/westlake-repl/Pixelrec">[link]</a>| Images, Text, Tags, 200 million interactions</li><li>MicroLens: A Content-Driven Micro-Video Recommendation Dataset at Scale <a href="https://arxiv.org/abs/2309.15379">[paper]</a> <a href="https://github.com/westlake-repl/MicroLens">[link]</a> <a href="https://github.com/westlake-repl/MicroLens/blob/master/MicroLens_DeepMind_Talk.pdf">[DeepMind Talk]</a> | Images, Text, Video, Audio, comments, tags, etc.</li><li>MIND: A Large-scale Dataset for News Recommendation, ACL2020, <a href="https://aclanthology.org/2020.acl-main.331/">[paper]</a> | Text</li><li>Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation, SIGIR 2020 <a href="https://github.com/fajieyuan/sigir2020_peterrec">[link]</a></li><li>Where to Go Next for Recommender Systems? ID-vs. Modality-based recommender models revisited, SIGIR2023, <a href="https://arxiv.org/pdf/2303.13835.pdf">[paper]</a></li><li>Netflix: <a href="https://github.com/HKUDS/LLMRec">[link]</a> Amazon: <a href="https://cseweb.ucsd.edu/~jmcauley/datasets/amazon_v2/">[link]</a></li></ul><h3>Survey</h3><ul><li>A Survey on Large Language Models for Recommendation, arxiv 2023/05, <a href="https://arxiv.org/abs/2305.19860">[paper]</a></li><li>How Can Recommender Systems Benefit from Large Language Models: A Survey, arxiv 2023/06, <a href="https://arxiv.org/abs/2306.05817">[paper]</a></li><li>Recommender Systems in the Era of Large Language Models, arxiv, 2023/07, <a href="https://arxiv.org/pdf/2307.02046.pdf">[paper]</a></li><li>A Survey on Evaluation of Large Language Models, arxiv, 2023/07, <a href="https://arxiv.org/abs/2307.03109">[paper]</a></li><li>Self-Supervised Learning for Recommender Systems: A Survey, arxiv, 2023/06, <a href="https://arxiv.org/pdf/2203.15876.pdf">[paper]</a></li><li>Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systems, 2022/09, <a href="https://arxiv.org/pdf/2302.03735.pdf">[paper]</a></li><li>User Modeling in the Era of Large Language Models: Current Research and Future Directions,2023/12, <a href="https://www.semanticscholar.org/reader/74b0976a3a7b7013fd468a043a940dcf401e66f1">[paper]</a></li><li>USER MODELING AND USER PROFILING: A COMPREHENSIVE SURVEY,2024/02, <a href="https://arxiv.org/pdf/2402.09660.pdf">[paper]</a></li><li>Foundation Models for Recommender Systems: A Survey and New Perspectives, 2024/02, <a href="https://arxiv.org/pdf/2402.11143.pdf">[paper]</a></li></ul><h3>Large Language Models for Recommendation （LLM4Rec）</h3><h3>Scaling LLM</h3><ul><li>Emergent Abilities of Large Language Models, TMLR 2022/08, <a href="https://openreview.net/pdf?id=yzkSU5zdwD">[paper]</a></li><li>Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.11700.pdf">[paper]</a></li><li>Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.06474.pdf">[paper]</a></li><li>Scaling Law for Recommendation Models: Towards General-purpose User Representations, AAAI 2023, <a href="https://arxiv.org/abs/2111.11294">[paper]</a></li></ul><h3>Untra Wide &amp; Deep Recommender Models</h3><ul><li>StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking, SIGIR 2021, <a href="https://arxiv.org/pdf/2012.07598.pdf">[paper]</a></li><li>A User-Adaptive Layer Selection Framework for Very Deep Sequential Recommender Models, AAAI 2021, <a href="https://ojs.aaai.org/index.php/AAAI/article/view/16518">[paper]</a></li><li>A Generic Network Compression Framework for Sequential Recommender Systems, SIGIR 2020, <a href="https://arxiv.org/pdf/2004.13139.pdf">[paper]</a></li><li>Scaling Law of Large Sequential Recommendation Models, arxiv 2023/11, <a href="https://arxiv.org/pdf/2311.11351.pdf">[paper]</a></li><li>Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations, arxiv 2024/03, <a href="https://arxiv.org/pdf/2402.17152.pdf">[paper]</a></li></ul><h3>Tuning LLM</h3><ul><li>M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems,arxiv 2022/05, <a href="https://arxiv.org/pdf/2205.08084.pdf">[paper]</a></li><li>TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation, arxiv 2023/04, <a href="http://arxiv.org/abs/2305.00447v1">[paper]</a></li><li>GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretation, 2023/04, <a href="https://arxiv.org/pdf/2304.03879.pdf">[paper]</a></li><li>A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems, arxiv, 2023/08, <a href="https://arxiv.org/abs/2308.08434">[paper]</a></li><li>LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking, PGAI@CIKM 2023, <a href="https://arxiv.org/abs/2311.02089">[paper]</a> <a href="https://github.com/Yueeeeeeee/LlamaRec">[code]</a></li><li>Improving Sequential Recommendations with LLMs, arxiv 2024/02, <a href="https://arxiv.org/pdf/2402.01339.pdf">[paper]</a></li></ul><h3>Freezing LLM <a href="https://github.com/WLiK/LLM4Rec">[link]</a></h3><ul><li>CTR-BERT: Cost-effective knowledge distillation for billion-parameter teacher models，arxiv 2022/04, <a href="https://neurips2021-nlp.github.io/papers/20/CameraReady/camera_ready_final.pdf">[paper]</a></li><li>Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning, arxiv 2022/06, <a href="https://arxiv.org/abs/2206.09363">[paper]</a></li><li>Generative Recommendation: Towards Next-generation Recommender Paradigm, arxiv 2023/04, <a href="https://arxiv.org/abd/2304.03516">[paper]</a></li><li>Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.11700.pdf">[paper]</a></li><li>A First Look at LLM-Powered Generative News Recommendation, arxiv 2023/05, <a href="https://arxiv.org/abs/2305.06566">[paper]</a></li><li>Privacy-Preserving Recommender Systems with Synthetic Query Generation using Differentially Private Large Language Models, arxiv 2023/05,<a href="https://arxiv.org/abs/2305.05973">[paper]</a></li><li>RecAgent: A Novel Simulation Paradigm for Recommender Systems, arxiv 2023/06, <a href="https://arxiv.org/abs/2306.02552">[paper]</a></li><li>Zero-Shot Next-Item Recommendation using Large Pretrained Language Models, arxiv 2023/04, <a href="https://arxiv.org/abs/2304.03153">[paper]</a></li><li>Can ChatGPT Make Fair Recommendation? A Fairness Evaluation Benchmark for Recommendation with Large Language Model, RecSys 2023</li><li>Leveraging Large Language Models for Sequential Recommendation, Recsys 2023/09, <a href="https://dl.acm.org/doi/pdf/10.1145/3604915.3610639">[paper]</a></li><li>LLMRec: Large Language Models with Graph Augmentation for Recommendation, WSDM 2024 Oral, <a href="https://arxiv.org/pdf/2311.00423.pdf">[paper]</a> <a href="https://github.com/HKUDS/LLMRec">[code]</a></li><li>Are ID Embeddings Necessary? Whitening Pre-trained Text Embeddings for Effective Sequential Recommendation, arxiv 2024/02 , <a href="https://arxiv.org/pdf/2402.10602.pdf">[paper]</a></li></ul><h3>Prompt with LLM</h3><ul><li>Large Language Models are Zero-Shot Rankers for Recommender Systems, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.08845v2.pdf">[paper]</a></li><li>Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt &amp; Predict Paradigm (P5), arxiv 2022/03, <a href="https://arxiv.org/abs/2203.13366">[paper]</a></li><li>Language Models as Recommender Systems: Evaluations and Limitations, NeurIPS Workshop ICBINB 2021/10, <a href="https://openreview.net/pdf?id=hFx3fY7-m9b">[paper]</a></li><li>Prompt Learning for News Recommendation, SIGIR 2023/04, <a href="https://arxiv.org/abs/2304.05263">[paper]</a></li><li>LLM-Rec: Personalized Recommendation via Prompting Large Language Models, arxiv,2023/07 <a href="https://arxiv.org/abs/2307.15780">[paper]</a></li></ul><h3>ChatGPT <a href="https://github.com/archersama/awesome-recommend-system-pretraining-papers">[link]</a></h3><ul><li>Is ChatGPT a Good Recommender A Preliminary Study, arxiv 2023/04, <a href="https://arxiv.org/pdf/2304.10149.pdf">[paper]</a></li><li>Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent, arxiv 2023/04, <a href="https://arxiv.org/pdf/2304.09542.pdf">[paper]</a></li><li>Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System, arxiv 2023/04,<a href="https://arxiv.org/abs/2303.14524">[paper]</a></li><li>Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach, arxiv 2023/05, <a href="http://arxiv.org/abs/2305.07001v1">[paper]</a></li><li>Leveraging Large Language Models in Conversational Recommender Systems, arxiv 2023/05, <a href="http://arxiv.org/abs/2305.07961v2">[paper]</a></li><li>Uncovering ChatGPT’s Capabilities in Recommender Systems, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.02182.pdf">[paper]</a><a href="https://github.com/rainym00d/LLM4RS">[code]</a></li><li>Sparks of Artificial General Recommender (AGR): Early Experiments with ChatGPT, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.04518.pdf">[paper]</a></li><li>Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation, arxiv 2023/05,<a href="https://arxiv.org/pdf/2305.07609.pdf">[paper]</a> <a href="https://github.com/jizhi-zhang/FaiRLLM">[code]</a></li><li>Sparks of Artificial General Recommender (AGR): Early Experiments with ChatGPT, arxiv 2023/05,<a href="https://arxiv.org/abs/2305.04518">[paper]</a></li><li>PALR: Personalization Aware LLMs for Recommendation, arxiv 2023/05, <a href="http://arxiv.org/abs/2305.07622v1">[paper]</a></li><li>Privacy-Preserving Recommender Systems with Synthetic Query Generation using Differentially Private Large Language Models, arxiv 2023/05, <a href="http://arxiv.org/abs/2305.05973v1">[paper]</a></li><li>Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models, arxiv 2023/05, <a href="https://arxiv.org/abs/2305.13112">[paper]</a></li><li>CTRL: Connect Tabular and Language Model for CTR Prediction, arxiv 2023/06,<a href="https://arxiv.org/abs/2306.02841">[paper]</a>.</li></ul><h3>Multimodal Recommender System</h3><ul><li>VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback, AAAI2016, <a href="https://ojs.aaai.org/index.php/AAAI/article/view/9973">[paper]</a></li><li>Adversarial Training Towards Robust Multimedia Recommender System, TKDE2019, <a href="https://arxiv.org/pdf/1809.07062.pdf">[paper]</a></li><li>Multi-modal Knowledge Graphs for Recommender Systems, CIKM 2020, <a href="https://zheng-kai.com/paper/cikm_2020_sun.pdf">[paper]</a></li><li>Online Distillation-enhanced Multi-modal Transformer for Sequential Recommendation, ACMMM 2023, <a href="https://www.semanticscholar.org/paper/Online-Distillation-enhanced-Multi-modal-for-Ji-Liu/a1f718113c6a4ac6eeacce477eefc0a26d101e95">[paper]</a></li><li>Self-Supervised Multi-Modal Sequential Recommendation, arxiv2023/02, <a href="https://arxiv.org/pdf/2304.13277.pdf">[paper]</a></li><li>FMMRec: Fairness-aware Multimodal Recommendation, arxiv2023/10, <a href="https://www.semanticscholar.org/paper/FMMRec%3A-Fairness-aware-Multimodal-Recommendation-Chen-Chen/1c62a0d2c663837de50f729c60d90e2c5c1b69b2">[paper]</a></li><li>Self-Supervised Multi-Modal Sequential Recommendation, arxiv 2024/02, <a href="https://arxiv.org/pdf/2304.13277.pdf">[paper]</a></li><li>ID Embedding as Subtle Features of Content and Structure for Multimodal Recommendation, arxiv2023/10, <a href="https://arxiv.org/pdf/2311.05956.pdf">[paper]</a></li><li>Enhancing ID and Text Fusion via Alternative Training in Session-based Recommendation, arxiv2023/2, <a href="https://arxiv.org/pdf/2402.08921.pdf">[paper]</a></li><li>BiVRec: Bidirectional View-based Multimodal Sequential Recommendation，arxiv2023/2, <a href="https://arxiv.org/pdf/2402.17334.pdf">[paper]</a></li><li>A Large Language Model Enhanced Sequential Recommender for Joint Video and Comment Recommendation, arxiv2023/2, <a href="https://arxiv.org/pdf/2403.13574.pdf">[paper]</a></li></ul><h3>Foundation and Transferable Recommender models</h3><ul><li>TransRec: Learning Transferable Recommendation from Mixture-of-Modality Feedback, arxiv 2022/06, <a href="https://www.semanticscholar.org/paper/TransRec%3A-Learning-Transferable-Recommendation-from-Wang-Yuan/f7c9551e19fabf5d534115a5704a5f8fea097534">[paper]</a></li><li>Towards Universal Sequence Representation Learning for Recommender Systems, KDD2022,2022/06, <a href="https://arxiv.org/pdf/2206.05941.pdf">[paper]</a></li><li>Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders, WWW 2023, <a href="https://arxiv.org/abs/2210.12316">[paper]</a> <a href="https://github.com/RUCAIBox/VQ-Rec">[code]</a></li><li>UP5: Unbiased Foundation Model for Fairness-aware Recommendation, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.12090.pdf">[paper]</a></li><li>Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.15036.pdf">[paper]</a> <a href="https://github.com/westlake-repl/Adapter4Rec">[code]</a></li><li>OpenP5: Benchmarking Foundation Models for Recommendation, arxiv 2023/06, <a href="https://arxiv.org/pdf/2306.11134.pdf">[paper]</a></li><li>Thoroughly Modeling Multi-domain Pre-trained Recommendation as Language, arxiv 2023/10, <a href="https://www.semanticscholar.org/reader/05b97a88e33241bc88e69960703b97c7eb686f22">[paper]</a></li><li>MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation, arxiv 2023/10, <a href="https://www.semanticscholar.org/reader/7f1bfaa0ffd9f552750a86455117757a303648d7">[paper]</a></li><li>Collaborative Word-based Pre-trained Item Representation for Transferable Recommendation, arxiv 2023/11, <a href="https://arxiv.org/pdf/2311.10501.pdf">[paper]</a></li><li>Universal Multi-modal Multi-domain Pre-trained Recommendation, arxiv 2023/11, <a href="https://arxiv.org/pdf/2311.01831.pdf">[paper]</a></li><li>Multi-Modality is All You Need for Transferable Recommender Systems, arxiv 2023, <a href="https://www.semanticscholar.org/reader/4345216c8c99207c19d21621fcf3c6149dde97ea">[paper]</a></li><li>TransFR: Transferable Federated Recommendation with Pre-trained Language Models, arxiv 2024/02 ,<a href="https://arxiv.org/pdf/2402.01124.pdf">[paper]</a></li><li>Rethinking Cross-Domain Sequential Recommendation under Open-World Assumptions, arxiv 2024/02 ,<a href="https://arxiv.org/pdf/2311.04590.pdf">[paper]</a></li></ul><h3>Universal General-Purpose, One4all User Representation Learning</h3><ul><li>Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation, SIGIR 2020, <a href="https://arxiv.org/pdf/2001.04253.pdf">[paper]</a>, <a href="https://github.com/yuangh-x/2022-NIPS-Tenrec">[code]</a></li><li>One4all User Representation for Recommender Systems in E-commerce, arxiv 2021, <a href="https://arxiv.org/abs/2106.00573">[paper]</a></li><li>Learning Transferable User Representations with Sequential Behaviors via Contrastive Pre-training, ICDM 2021, <a href="https://fajieyuan.github.io/papers/ICDM2021.pdf">[paper]</a></li><li>User-specific Adaptive Fine-tuning for Cross-domain Recommendations, TKDE 2021, <a href="https://arxiv.org/pdf/2106.07864.pdf">[paper]</a></li><li>Scaling Law for Recommendation Models: Towards General-purpose User Representations, AAAI 2023, <a href="https://arxiv.org/abs/2111.11294">[paper]</a></li><li>U-BERT: Pre-training user representations for improved recommendation, AAAI 2021, <a href="https://ojs.aaai.org/index.php/AAAI/article/view/16557">[paper]</a></li><li>One for All, All for One: Learning and Transferring User Embeddings for Cross-Domain Recommendation, WSDM 2022, <a href="https://arxiv.org/abs/2211.11964">[paper]</a></li><li>Field-aware Variational Autoencoders for Billion-scale User Representation Learning,ICDE2022, <a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&amp;arnumber=9835571">[paper]</a></li><li>Learning Large-scale Universal User Representation with Sparse Mixture of Experts, ICML2022workshop, <a href="https://openreview.net/pdf?id=7tWNJ6NR-76">[paper]</a></li><li>Multi Datasource LTV User Representation (MDLUR), KDD2023, <a href="https://dl.acm.org/doi/pdf/10.1145/3580305.3599871">[paper]</a></li><li>Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning. arxiv2022/12, <a href="https://www.semanticscholar.org/paper/Pivotal-Role-of-Language-Modeling-in-Recommender-Shin-Kwak/7557105c9aa6a26db4f8e73fabb25e8134013fb5">[paper]</a></li><li>USER MODELING AND USER PROFILING: A COMPREHENSIVE SURVEY,2024/02, <a href="https://arxiv.org/pdf/2402.09660.pdf">[paper]</a></li><li>Generalized User Representations for Transfer Learning, arxiv 2024/03, <a href="https://arxiv.org/pdf/2403.00584.pdf">[paper]</a></li><li>Bridging Language and Items for Retrieval and Recommendation，arxiv 2024/04, <a href="https://arxiv.org/pdf/2403.03952.pdf">[paper]</a></li></ul><h3>Lifelong Universal User Representation Learning</h3><ul><li>One Person, One Model, One World: Learning Continual User Representation without Forgetting, SIGIR 2021, <a href="https://arxiv.org/pdf/2001.04253.pdf">[paper]</a>, <a href="https://github.com/yuangh-x/2022-NIPS-Tenrec">[code]</a></li><li>Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems, NeurIPS 2022 <a href="https://proceedings.neurips.cc/paper_files/paper/2022/file/4ad4fc1528374422dd7a69dea9e72948-Paper-Datasets_and_Benchmarks.pdf">[paper]</a></li><li>STAN: Stage-Adaptive Network for Multi-Task Recommendation by Learning User Lifecycle-Based Representationg, Recsys 2023, <a href="https://arxiv.org/abs/2306.12232">[paper]</a></li><li>Task Relation-aware Continual User Representation Learning, KDD2023, <a href="https://arxiv.org/pdf/2306.01792.pdf">[paper]</a></li><li>ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation, arxiv2023/08, <a href="https://www.semanticscholar.org/paper/ReLLa%3A-Retrieval-enhanced-Large-Language-Models-for-Lin-Shan/429e6c09eeadf54e2b245b8f2cddfbf157f9da4c">[paper]</a></li></ul><h3>Generative Recommender Systems <a href="https://github.com/gabriben/awesome-generative-information-retrieval#generative-recommendation">[link]</a></h3><ul><li>A Simple Convolutional Generative Network for Next Item Recommendation, WSDM 2018/08, <a href="https://arxiv.org/pdf/1808.05163.pdf">[paper]</a> <a href="https://github.com/fajieyuan/WSDM2019-nextitnet">[code]</a></li><li>Future Data Helps Training: Modeling Future Contexts for Session-based Recommendation, WWW 2020/04, <a href="https://arxiv.org/pdf/1906.04473.pdf">[paper]</a> <a href="https://github.com/fajieyuan/WWW2020-grec">[code]</a></li><li>Recommendation via Collaborative Diffusion Generative Model, KSEM 2022/08, <a href="https://dl.acm.org/doi/abs/10.1007/978-3-031-10989-8_47">[paper]</a></li><li>Blurring-Sharpening Process Models for Collaborative Filtering, arxiv 2022/09, <a href="https://arxiv.org/abs/2211.09324">[paper]</a></li><li>Generative Slate Recommendation with Reinforcement Learning, arxiv 2023/01, <a href="https://arxiv.org/abs/2301.08632">[paper]</a></li><li>Recommender Systems with Generative Retrieval, arxiv 2023/04, <a href="https://arxiv.org/pdf/2305.05065.pdf">[paper]</a></li><li>DiffuRec: A Diffusion Model for Sequential Recommendation, arxiv 2023/04, <a href="https://arxiv.org/abs/2304.00686">[paper]</a></li><li>Diffusion Recommender Model, arxiv 2023/04, <a href="https://arxiv.org/abs/2304.04971">[paper]</a></li><li>A First Look at LLM-Powered Generative News Recommendation, arxiv 2023/05, <a href="https://arxiv.org/abs/2305.06566">[paper]</a></li><li>Recommender Systems with Generative Retrieval, arxiv 2023/05, <a href="https://arxiv.org/abs/2305.05065">[paper]</a></li><li>Generative Retrieval as Dense Retrieval, arxiv 2023/06, <a href="https://arxiv.org/pdf/2306.11397.pdf">[paper]</a></li><li>RecFusion: A Binomial Diffusion Process for 1D Data for Recommendation, arxiv 2023/06, <a href="https://arxiv.org/abs/2306.08947">[paper]</a></li><li>Generative Sequential Recommendation with GPTRec, SIGIR workshop 2023, <a href="https://arxiv.org/abs/2306.11114.pdf">[paper]</a></li><li>FANS: Fast Non-Autoregressive Sequence Generation for Item List Continuation, WWW 2023, <a href="https://arxiv.org/pdf/2304.00545.pdf">[paper]</a></li><li>Generative Next-Basket Recommendation, RecSys 2023</li><li>Large Language Model Augmented Narrative Driven Recommendations, RecSys 2023, <a href="https://arxiv.org/pdf/2304.00545.pdf">[paper]</a></li><li>LightLM: A Lightweight Deep and Narrow Language Model forGenerative Recommendation, arxiv 2023/10, <a href="https://arxiv.org/pdf/2310.17488.pdf">[paper]</a></li></ul><h3>Related Resources:</h3><ul><li>Xiangyang Li Github <a href="https://github.com/archersama/awesome-recommend-system-pretraining-papers/tree/main">[Link]</a></li><li>nancheng58i Github <a href="https://github.com/nancheng58/Awesome-LLM4RS-Papers">[Link]</a></li><li>enoche Github <a href="https://github.com/enoche/MultimodalRecSys">[Link]</a></li><li>WLiK Github <a href="https://github.com/WLiK/LLM4Rec-Awesome-Papers">[Link]</a></li><li><a href="https://github.com/CHIANGEL/Awesome-LLM-for-RecSys">https://github.com/CHIANGEL/Awesome-LLM-for-RecSys</a></li><li><a href="https://medium.com/@lifengyi_6964/a-large-scale-short-video-recommender-system-dataset-160fdfe81b79">https://medium.com/@lifengyi_6964/a-large-scale-short-video-recommender-system-dataset-160fdfe81b79</a></li><li><a href="https://medium.com/@lifengyi_6964/rethinking-the-id-paradigm-in-recommender-systems-the-rise-of-modality-98f449dec016">https://medium.com/@lifengyi_6964/rethinking-the-id-paradigm-in-recommender-systems-the-rise-of-modality-98f449dec016</a></li><li><a href="https://medium.com/@lifengyi_6964/one-model-for-all-universal-recommender-system-82dab214a07d">https://medium.com/@lifengyi_6964/one-model-for-all-universal-recommender-system-82dab214a07d</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a51efda98e30" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multimodal and Large Language Model Recommendation System (awesome Paper List)]]></title>
            <link>https://medium.com/@lifengyi_6964/multimodal-and-large-language-model-recommendation-system-awesome-paper-list-a05e5fd81a79?source=rss-398e39ee75a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/a05e5fd81a79</guid>
            <dc:creator><![CDATA[AI-Advance]]></dc:creator>
            <pubDate>Fri, 22 Mar 2024 05:46:35 GMT</pubDate>
            <atom:updated>2024-03-22T06:20:07.672Z</atom:updated>
            <content:encoded><![CDATA[<h3>Foundation models for Recommender System Paper List</h3><blockquote>Link： <a href="https://github.com/westlake-repl/Recommendation-Systems-without-Explicit-ID-Features-A-Literature-Review">https://github.com/westlake-repl/Recommendation-Systems-without-Explicit-ID-Features-A-Literature-Review</a></blockquote><p>Keyword: <em>Recommend System, pretraining, large language model, multimodal recommender system, transferable recommender system, foundation recommender models, universal user representation, one-model-fit-all, ID features, ID embeddings</em></p><p>These papers attempt to address the following questions:</p><p>(1) Can recommender systems have their own foundation models similar to those used in NLP and CV?</p><p>(2) Is ID embedding necessary for recommender models, can we replace or abondon it?</p><p>(3) Will recommender systems shift from a matching paradigm to a generating paradigm?</p><p>(4) How can LLM be utilized to enhance recommender systems?</p><p>(5) What does the future hold for multimodal recommender systems?</p><h3>Paper List</h3><h3>Perspective paper: ID vs. LLM &amp; ID vs. Multimodal</h3><ul><li>Where to Go Next for Recommender Systems? ID-vs. Modality-based recommender models revisited, SIGIR2023, 2022/09, <a href="https://arxiv.org/pdf/2303.13835.pdf">[paper]</a> <a href="https://github.com/westlake-repl/IDvs.MoRec">[code]</a></li><li>Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.11700.pdf">[paper]</a></li><li>Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights, WSDM2024, <a href="https://arxiv.org/pdf/2305.15036.pdf">[paper]</a> <a href="https://github.com/westlake-repl/Adapter4Rec">[code]</a></li></ul><h3>Datasets for Transferable or Multimodal RS</h3><ul><li>NineRec: A Benchmark Dataset Suite for Evaluating Transferable Recommendation, TPAMI2024, <a href="https://arxiv.org/pdf/2309.07705.pdf">[paper]</a> <a href="https://github.com/westlake-repl/NineRec">[link]</a> | Images, Text, Nine downstream datasets</li><li>TenRec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems, NeurIPS 2022 <a href="https://proceedings.neurips.cc/paper_files/paper/2022/file/4ad4fc1528374422dd7a69dea9e72948-Paper-Datasets_and_Benchmarks.pdf">[paper]</a></li><li>PixelRec: An Image Dataset for Benchmarking Recommender Systems with Raw Pixels, arxiv 2023/09 <a href="https://arxiv.org/pdf/2309.06789.pdf">[paper]</a> |<a href="https://github.com/westlake-repl/Pixelrec">[link]</a>| Images, Text, Tags, 200 million interactions</li><li>MicroLens: A Content-Driven Micro-Video Recommendation Dataset at Scale <a href="https://arxiv.org/abs/2309.15379">[paper]</a> <a href="https://github.com/westlake-repl/MicroLens">[link]</a> <a href="https://github.com/westlake-repl/MicroLens/blob/master/MicroLens_DeepMind_Talk.pdf">[DeepMind Talk]</a> | Images, Text, Video, Audio, comments, tags, etc.</li><li>MIND: A Large-scale Dataset for News Recommendation, ACL2020, <a href="https://aclanthology.org/2020.acl-main.331/">[paper]</a> | Text</li><li>Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation, SIGIR 2020 <a href="https://github.com/fajieyuan/sigir2020_peterrec">[link]</a></li><li>Where to Go Next for Recommender Systems? ID-vs. Modality-based recommender models revisited, SIGIR2023, <a href="https://arxiv.org/pdf/2303.13835.pdf">[paper]</a></li><li>Netflix: <a href="https://github.com/HKUDS/LLMRec">[link]</a> Amazon: <a href="https://cseweb.ucsd.edu/~jmcauley/datasets/amazon_v2/">[link]</a></li></ul><h3>Survey</h3><ul><li>A Survey on Large Language Models for Recommendation, arxiv 2023/05, <a href="https://arxiv.org/abs/2305.19860">[paper]</a></li><li>How Can Recommender Systems Benefit from Large Language Models: A Survey, arxiv 2023/06, <a href="https://arxiv.org/abs/2306.05817">[paper]</a></li><li>Recommender Systems in the Era of Large Language Models, arxiv, 2023/07, <a href="https://arxiv.org/pdf/2307.02046.pdf">[paper]</a></li><li>A Survey on Evaluation of Large Language Models, arxiv, 2023/07, <a href="https://arxiv.org/abs/2307.03109">[paper]</a></li><li>Self-Supervised Learning for Recommender Systems: A Survey, arxiv, 2023/06, <a href="https://arxiv.org/pdf/2203.15876.pdf">[paper]</a></li><li>Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systems, 2022/09, <a href="https://arxiv.org/pdf/2302.03735.pdf">[paper]</a></li><li>User Modeling in the Era of Large Language Models: Current Research and Future Directions,2023/12, <a href="https://www.semanticscholar.org/reader/74b0976a3a7b7013fd468a043a940dcf401e66f1">[paper]</a></li><li>USER MODELING AND USER PROFILING: A COMPREHENSIVE SURVEY,2024/02, <a href="https://arxiv.org/pdf/2402.09660.pdf">[paper]</a></li><li>Foundation Models for Recommender Systems: A Survey and New Perspectives, 2024/02, <a href="https://arxiv.org/pdf/2402.11143.pdf">[paper]</a></li></ul><h3>Large Language Models for Recommendation （LLM4Rec）</h3><h3>Scaling LLM</h3><ul><li>Emergent Abilities of Large Language Models, TMLR 2022/08, <a href="https://openreview.net/pdf?id=yzkSU5zdwD">[paper]</a></li><li>Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.11700.pdf">[paper]</a></li><li>Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.06474.pdf">[paper]</a></li><li>Scaling Law for Recommendation Models: Towards General-purpose User Representations, AAAI 2023, <a href="https://arxiv.org/abs/2111.11294">[paper]</a></li></ul><h3>Untra Wide &amp; Deep Recommender Models</h3><ul><li>StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking, SIGIR 2021, <a href="https://arxiv.org/pdf/2012.07598.pdf">[paper]</a></li><li>A User-Adaptive Layer Selection Framework for Very Deep Sequential Recommender Models, AAAI 2021, <a href="https://ojs.aaai.org/index.php/AAAI/article/view/16518">[paper]</a></li><li>A Generic Network Compression Framework for Sequential Recommender Systems, SIGIR 2020, <a href="https://arxiv.org/pdf/2004.13139.pdf">[paper]</a></li><li>Scaling Law of Large Sequential Recommendation Models, arxiv 2023/11, <a href="https://arxiv.org/pdf/2311.11351.pdf">[paper]</a></li><li>Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations, arxiv 2024/03, <a href="https://arxiv.org/pdf/2402.17152.pdf">[paper]</a></li></ul><h3>Tuning LLM</h3><ul><li>M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems,arxiv 2022/05, <a href="https://arxiv.org/pdf/2205.08084.pdf">[paper]</a></li><li>TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation, arxiv 2023/04, <a href="http://arxiv.org/abs/2305.00447v1">[paper]</a></li><li>GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretation, 2023/04, <a href="https://arxiv.org/pdf/2304.03879.pdf">[paper]</a></li><li>A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems, arxiv, 2023/08, <a href="https://arxiv.org/abs/2308.08434">[paper]</a></li><li>LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking, PGAI@CIKM 2023, <a href="https://arxiv.org/abs/2311.02089">[paper]</a> <a href="https://github.com/Yueeeeeeee/LlamaRec">[code]</a></li><li>Improving Sequential Recommendations with LLMs, arxiv 2024/02, <a href="https://arxiv.org/pdf/2402.01339.pdf">[paper]</a></li></ul><h3>Freezing LLM <a href="https://github.com/WLiK/LLM4Rec">[link]</a></h3><ul><li>CTR-BERT: Cost-effective knowledge distillation for billion-parameter teacher models，arxiv 2022/04, <a href="https://neurips2021-nlp.github.io/papers/20/CameraReady/camera_ready_final.pdf">[paper]</a></li><li>Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning, arxiv 2022/06, <a href="https://arxiv.org/abs/2206.09363">[paper]</a></li><li>Generative Recommendation: Towards Next-generation Recommender Paradigm, arxiv 2023/04, <a href="https://arxiv.org/abd/2304.03516">[paper]</a></li><li>Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.11700.pdf">[paper]</a></li><li>A First Look at LLM-Powered Generative News Recommendation, arxiv 2023/05, <a href="https://arxiv.org/abs/2305.06566">[paper]</a></li><li>Privacy-Preserving Recommender Systems with Synthetic Query Generation using Differentially Private Large Language Models, arxiv 2023/05,<a href="https://arxiv.org/abs/2305.05973">[paper]</a></li><li>RecAgent: A Novel Simulation Paradigm for Recommender Systems, arxiv 2023/06, <a href="https://arxiv.org/abs/2306.02552">[paper]</a></li><li>Zero-Shot Next-Item Recommendation using Large Pretrained Language Models, arxiv 2023/04, <a href="https://arxiv.org/abs/2304.03153">[paper]</a></li><li>Can ChatGPT Make Fair Recommendation? A Fairness Evaluation Benchmark for Recommendation with Large Language Model, RecSys 2023</li><li>Leveraging Large Language Models for Sequential Recommendation, Recsys 2023/09, <a href="https://dl.acm.org/doi/pdf/10.1145/3604915.3610639">[paper]</a></li><li>LLMRec: Large Language Models with Graph Augmentation for Recommendation, WSDM 2024 Oral, <a href="https://arxiv.org/pdf/2311.00423.pdf">[paper]</a> <a href="https://github.com/HKUDS/LLMRec">[code]</a></li><li>Are ID Embeddings Necessary? Whitening Pre-trained Text Embeddings for Effective Sequential Recommendation, arxiv 2024/02 , <a href="https://arxiv.org/pdf/2402.10602.pdf">[paper]</a></li></ul><h3>Prompt with LLM</h3><ul><li>Large Language Models are Zero-Shot Rankers for Recommender Systems, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.08845v2.pdf">[paper]</a></li><li>Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt &amp; Predict Paradigm (P5), arxiv 2022/03, <a href="https://arxiv.org/abs/2203.13366">[paper]</a></li><li>Language Models as Recommender Systems: Evaluations and Limitations, NeurIPS Workshop ICBINB 2021/10, <a href="https://openreview.net/pdf?id=hFx3fY7-m9b">[paper]</a></li><li>Prompt Learning for News Recommendation, SIGIR 2023/04, <a href="https://arxiv.org/abs/2304.05263">[paper]</a></li><li>LLM-Rec: Personalized Recommendation via Prompting Large Language Models, arxiv,2023/07 <a href="https://arxiv.org/abs/2307.15780">[paper]</a></li></ul><h3>ChatGPT <a href="https://github.com/archersama/awesome-recommend-system-pretraining-papers">[link]</a></h3><ul><li>Is ChatGPT a Good Recommender A Preliminary Study, arxiv 2023/04, <a href="https://arxiv.org/pdf/2304.10149.pdf">[paper]</a></li><li>Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent, arxiv 2023/04, <a href="https://arxiv.org/pdf/2304.09542.pdf">[paper]</a></li><li>Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System, arxiv 2023/04,<a href="https://arxiv.org/abs/2303.14524">[paper]</a></li><li>Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach, arxiv 2023/05, <a href="http://arxiv.org/abs/2305.07001v1">[paper]</a></li><li>Leveraging Large Language Models in Conversational Recommender Systems, arxiv 2023/05, <a href="http://arxiv.org/abs/2305.07961v2">[paper]</a></li><li>Uncovering ChatGPT’s Capabilities in Recommender Systems, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.02182.pdf">[paper]</a><a href="https://github.com/rainym00d/LLM4RS">[code]</a></li><li>Sparks of Artificial General Recommender (AGR): Early Experiments with ChatGPT, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.04518.pdf">[paper]</a></li><li>Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation, arxiv 2023/05,<a href="https://arxiv.org/pdf/2305.07609.pdf">[paper]</a> <a href="https://github.com/jizhi-zhang/FaiRLLM">[code]</a></li><li>Sparks of Artificial General Recommender (AGR): Early Experiments with ChatGPT, arxiv 2023/05,<a href="https://arxiv.org/abs/2305.04518">[paper]</a></li><li>PALR: Personalization Aware LLMs for Recommendation, arxiv 2023/05, <a href="http://arxiv.org/abs/2305.07622v1">[paper]</a></li><li>Privacy-Preserving Recommender Systems with Synthetic Query Generation using Differentially Private Large Language Models, arxiv 2023/05, <a href="http://arxiv.org/abs/2305.05973v1">[paper]</a></li><li>Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models, arxiv 2023/05, <a href="https://arxiv.org/abs/2305.13112">[paper]</a></li><li>CTRL: Connect Tabular and Language Model for CTR Prediction, arxiv 2023/06,<a href="https://arxiv.org/abs/2306.02841">[paper]</a>.</li></ul><h3>Multimodal Recommender System</h3><ul><li>VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback, AAAI2016, <a href="https://ojs.aaai.org/index.php/AAAI/article/view/9973">[paper]</a></li><li>Adversarial Training Towards Robust Multimedia Recommender System, TKDE2019, <a href="https://arxiv.org/pdf/1809.07062.pdf">[paper]</a></li><li>Multi-modal Knowledge Graphs for Recommender Systems, CIKM 2020, <a href="https://zheng-kai.com/paper/cikm_2020_sun.pdf">[paper]</a></li><li>Online Distillation-enhanced Multi-modal Transformer for Sequential Recommendation, ACMMM 2023, <a href="https://www.semanticscholar.org/paper/Online-Distillation-enhanced-Multi-modal-for-Ji-Liu/a1f718113c6a4ac6eeacce477eefc0a26d101e95">[paper]</a></li><li>Self-Supervised Multi-Modal Sequential Recommendation, arxiv2023/02, <a href="https://arxiv.org/pdf/2304.13277.pdf">[paper]</a></li><li>FMMRec: Fairness-aware Multimodal Recommendation, arxiv2023/10, <a href="https://www.semanticscholar.org/paper/FMMRec%3A-Fairness-aware-Multimodal-Recommendation-Chen-Chen/1c62a0d2c663837de50f729c60d90e2c5c1b69b2">[paper]</a></li><li>Self-Supervised Multi-Modal Sequential Recommendation, arxiv 2024/02, <a href="https://arxiv.org/pdf/2304.13277.pdf">[paper]</a></li><li>ID Embedding as Subtle Features of Content and Structure for Multimodal Recommendation, arxiv2023/10, <a href="https://arxiv.org/pdf/2311.05956.pdf">[paper]</a></li><li>Enhancing ID and Text Fusion via Alternative Training in Session-based Recommendation, arxiv2023/2, <a href="https://arxiv.org/pdf/2402.08921.pdf">[paper]</a></li><li>BiVRec: Bidirectional View-based Multimodal Sequential Recommendation，arxiv2023/2, <a href="https://arxiv.org/pdf/2402.17334.pdf">[paper]</a></li><li>A Large Language Model Enhanced Sequential Recommender for Joint Video and Comment Recommendation, arxiv2023/2, <a href="https://arxiv.org/pdf/2403.13574.pdf">[paper]</a></li></ul><h3>Foundation and Transferable Recommender models</h3><ul><li>TransRec: Learning Transferable Recommendation from Mixture-of-Modality Feedback, arxiv 2022/06, <a href="https://www.semanticscholar.org/paper/TransRec%3A-Learning-Transferable-Recommendation-from-Wang-Yuan/f7c9551e19fabf5d534115a5704a5f8fea097534">[paper]</a></li><li>Towards Universal Sequence Representation Learning for Recommender Systems, KDD2022,2022/06, <a href="https://arxiv.org/pdf/2206.05941.pdf">[paper]</a></li><li>Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders, WWW 2023, <a href="https://arxiv.org/abs/2210.12316">[paper]</a> <a href="https://github.com/RUCAIBox/VQ-Rec">[code]</a></li><li>UP5: Unbiased Foundation Model for Fairness-aware Recommendation, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.12090.pdf">[paper]</a></li><li>Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights, arxiv 2023/05, <a href="https://arxiv.org/pdf/2305.15036.pdf">[paper]</a> <a href="https://github.com/westlake-repl/Adapter4Rec">[code]</a></li><li>OpenP5: Benchmarking Foundation Models for Recommendation, arxiv 2023/06, <a href="https://arxiv.org/pdf/2306.11134.pdf">[paper]</a></li><li>Thoroughly Modeling Multi-domain Pre-trained Recommendation as Language, arxiv 2023/10, <a href="https://www.semanticscholar.org/reader/05b97a88e33241bc88e69960703b97c7eb686f22">[paper]</a></li><li>MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation, arxiv 2023/10, <a href="https://www.semanticscholar.org/reader/7f1bfaa0ffd9f552750a86455117757a303648d7">[paper]</a></li><li>Collaborative Word-based Pre-trained Item Representation for Transferable Recommendation, arxiv 2023/11, <a href="https://arxiv.org/pdf/2311.10501.pdf">[paper]</a></li><li>Universal Multi-modal Multi-domain Pre-trained Recommendation, arxiv 2023/11, <a href="https://arxiv.org/pdf/2311.01831.pdf">[paper]</a></li><li>Multi-Modality is All You Need for Transferable Recommender Systems, arxiv 2023, <a href="https://www.semanticscholar.org/reader/4345216c8c99207c19d21621fcf3c6149dde97ea">[paper]</a></li><li>TransFR: Transferable Federated Recommendation with Pre-trained Language Models, arxiv 2024/02 ,<a href="https://arxiv.org/pdf/2402.01124.pdf">[paper]</a></li><li>Rethinking Cross-Domain Sequential Recommendation under Open-World Assumptions, arxiv 2024/02 ,<a href="https://arxiv.org/pdf/2311.04590.pdf">[paper]</a></li></ul><h3>Universal General-Purpose, One4all User Representation Learning</h3><ul><li>Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation, SIGIR 2020, <a href="https://arxiv.org/pdf/2001.04253.pdf">[paper]</a>, <a href="https://github.com/yuangh-x/2022-NIPS-Tenrec">[code]</a></li><li>One4all User Representation for Recommender Systems in E-commerce, arxiv 2021, <a href="https://arxiv.org/abs/2106.00573">[paper]</a></li><li>Learning Transferable User Representations with Sequential Behaviors via Contrastive Pre-training, ICDM 2021, <a href="https://fajieyuan.github.io/papers/ICDM2021.pdf">[paper]</a></li><li>User-specific Adaptive Fine-tuning for Cross-domain Recommendations, TKDE 2021, <a href="https://arxiv.org/pdf/2106.07864.pdf">[paper]</a></li><li>Scaling Law for Recommendation Models: Towards General-purpose User Representations, AAAI 2023, <a href="https://arxiv.org/abs/2111.11294">[paper]</a></li><li>U-BERT: Pre-training user representations for improved recommendation, AAAI 2021, <a href="https://ojs.aaai.org/index.php/AAAI/article/view/16557">[paper]</a></li><li>One for All, All for One: Learning and Transferring User Embeddings for Cross-Domain Recommendation, WSDM 2022, <a href="https://arxiv.org/abs/2211.11964">[paper]</a></li><li>Field-aware Variational Autoencoders for Billion-scale User Representation Learning,ICDE2022, <a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&amp;arnumber=9835571">[paper]</a></li><li>Learning Large-scale Universal User Representation with Sparse Mixture of Experts, ICML2022workshop, <a href="https://openreview.net/pdf?id=7tWNJ6NR-76">[paper]</a></li><li>Multi Datasource LTV User Representation (MDLUR), KDD2023, <a href="https://dl.acm.org/doi/pdf/10.1145/3580305.3599871">[paper]</a></li><li>Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning. arxiv2022/12, <a href="https://www.semanticscholar.org/paper/Pivotal-Role-of-Language-Modeling-in-Recommender-Shin-Kwak/7557105c9aa6a26db4f8e73fabb25e8134013fb5">[paper]</a></li><li>USER MODELING AND USER PROFILING: A COMPREHENSIVE SURVEY,2024/02, <a href="https://arxiv.org/pdf/2402.09660.pdf">[paper]</a></li><li>Generalized User Representations for Transfer Learning, arxiv 2024/03, <a href="https://arxiv.org/pdf/2403.00584.pdf">[paper]</a></li><li>Bridging Language and Items for Retrieval and Recommendation，arxiv 2024/04, <a href="https://arxiv.org/pdf/2403.03952.pdf">[paper]</a></li></ul><h3>Lifelong Universal User Representation Learning</h3><ul><li>One Person, One Model, One World: Learning Continual User Representation without Forgetting, SIGIR 2021, <a href="https://arxiv.org/pdf/2001.04253.pdf">[paper]</a>, <a href="https://github.com/yuangh-x/2022-NIPS-Tenrec">[code]</a></li><li>Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems, NeurIPS 2022 <a href="https://proceedings.neurips.cc/paper_files/paper/2022/file/4ad4fc1528374422dd7a69dea9e72948-Paper-Datasets_and_Benchmarks.pdf">[paper]</a></li><li>STAN: Stage-Adaptive Network for Multi-Task Recommendation by Learning User Lifecycle-Based Representationg, Recsys 2023, <a href="https://arxiv.org/abs/2306.12232">[paper]</a></li><li>Task Relation-aware Continual User Representation Learning, KDD2023, <a href="https://arxiv.org/pdf/2306.01792.pdf">[paper]</a></li><li>ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation, arxiv2023/08, <a href="https://www.semanticscholar.org/paper/ReLLa%3A-Retrieval-enhanced-Large-Language-Models-for-Lin-Shan/429e6c09eeadf54e2b245b8f2cddfbf157f9da4c">[paper]</a></li></ul><h3>Generative Recommender Systems <a href="https://github.com/gabriben/awesome-generative-information-retrieval#generative-recommendation">[link]</a></h3><ul><li>A Simple Convolutional Generative Network for Next Item Recommendation, WSDM 2018/08, <a href="https://arxiv.org/pdf/1808.05163.pdf">[paper]</a> <a href="https://github.com/fajieyuan/WSDM2019-nextitnet">[code]</a></li><li>Future Data Helps Training: Modeling Future Contexts for Session-based Recommendation, WWW 2020/04, <a href="https://arxiv.org/pdf/1906.04473.pdf">[paper]</a> <a href="https://github.com/fajieyuan/WWW2020-grec">[code]</a></li><li>Recommendation via Collaborative Diffusion Generative Model, KSEM 2022/08, <a href="https://dl.acm.org/doi/abs/10.1007/978-3-031-10989-8_47">[paper]</a></li><li>Blurring-Sharpening Process Models for Collaborative Filtering, arxiv 2022/09, <a href="https://arxiv.org/abs/2211.09324">[paper]</a></li><li>Generative Slate Recommendation with Reinforcement Learning, arxiv 2023/01, <a href="https://arxiv.org/abs/2301.08632">[paper]</a></li><li>Recommender Systems with Generative Retrieval, arxiv 2023/04, <a href="https://arxiv.org/pdf/2305.05065.pdf">[paper]</a></li><li>DiffuRec: A Diffusion Model for Sequential Recommendation, arxiv 2023/04, <a href="https://arxiv.org/abs/2304.00686">[paper]</a></li><li>Diffusion Recommender Model, arxiv 2023/04, <a href="https://arxiv.org/abs/2304.04971">[paper]</a></li><li>A First Look at LLM-Powered Generative News Recommendation, arxiv 2023/05, <a href="https://arxiv.org/abs/2305.06566">[paper]</a></li><li>Recommender Systems with Generative Retrieval, arxiv 2023/05, <a href="https://arxiv.org/abs/2305.05065">[paper]</a></li><li>Generative Retrieval as Dense Retrieval, arxiv 2023/06, <a href="https://arxiv.org/pdf/2306.11397.pdf">[paper]</a></li><li>RecFusion: A Binomial Diffusion Process for 1D Data for Recommendation, arxiv 2023/06, <a href="https://arxiv.org/abs/2306.08947">[paper]</a></li><li>Generative Sequential Recommendation with GPTRec, SIGIR workshop 2023, <a href="https://arxiv.org/abs/2306.11114.pdf">[paper]</a></li><li>FANS: Fast Non-Autoregressive Sequence Generation for Item List Continuation, WWW 2023, <a href="https://arxiv.org/pdf/2304.00545.pdf">[paper]</a></li><li>Generative Next-Basket Recommendation, RecSys 2023</li><li>Large Language Model Augmented Narrative Driven Recommendations, RecSys 2023, <a href="https://arxiv.org/pdf/2304.00545.pdf">[paper]</a></li><li>LightLM: A Lightweight Deep and Narrow Language Model forGenerative Recommendation, arxiv 2023/10, <a href="https://arxiv.org/pdf/2310.17488.pdf">[paper]</a></li></ul><h3>Related Resources:</h3><ul><li>Xiangyang Li Github <a href="https://github.com/archersama/awesome-recommend-system-pretraining-papers/tree/main">[Link]</a></li><li>nancheng58i Github <a href="https://github.com/nancheng58/Awesome-LLM4RS-Papers">[Link]</a></li><li>enoche Github <a href="https://github.com/enoche/MultimodalRecSys">[Link]</a></li><li>WLiK Github <a href="https://github.com/WLiK/LLM4Rec-Awesome-Papers">[Link]</a></li><li><a href="https://github.com/CHIANGEL/Awesome-LLM-for-RecSys">https://github.com/CHIANGEL/Awesome-LLM-for-RecSys</a></li><li><a href="https://medium.com/@lifengyi_6964/a-large-scale-short-video-recommender-system-dataset-160fdfe81b79">https://medium.com/@lifengyi_6964/a-large-scale-short-video-recommender-system-dataset-160fdfe81b79</a></li><li><a href="https://medium.com/@lifengyi_6964/rethinking-the-id-paradigm-in-recommender-systems-the-rise-of-modality-98f449dec016">https://medium.com/@lifengyi_6964/rethinking-the-id-paradigm-in-recommender-systems-the-rise-of-modality-98f449dec016</a></li><li><a href="https://medium.com/@lifengyi_6964/one-model-for-all-universal-recommender-system-82dab214a07d">https://medium.com/@lifengyi_6964/one-model-for-all-universal-recommender-system-82dab214a07d</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a05e5fd81a79" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cross-domain & Cross-Platform Recommendation Systems: Dataset and Benchmark]]></title>
            <link>https://medium.com/@lifengyi_6964/cross-domain-cross-platform-recommendation-systems-dataset-and-benchmark-3836fe22e69c?source=rss-398e39ee75a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/3836fe22e69c</guid>
            <dc:creator><![CDATA[AI-Advance]]></dc:creator>
            <pubDate>Wed, 20 Mar 2024 02:52:32 GMT</pubDate>
            <atom:updated>2024-03-20T02:52:32.101Z</atom:updated>
            <content:encoded><![CDATA[<p>Title:Cross-domain &amp; Cross-Platform Recommendation Systems: Dataset and Benchmark</p><p>In response to problems such as the small size and limited form of user feedback existing in current data sets, Tencent and West Lake University released a large-scale multi-purpose recommendation system data set Tenrec. Based on this dataset, the baseline performance on 10 different recommendation tasks was also verified.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/897/1*ctvWWRfwZArchT6sodsiWQ.png" /></figure><p>Paper: <a href="https://arxiv.org/abs/2210.10629">https://arxiv.org/abs/2210.10629</a></p><p>The code, data sets and lists involved in the paper can be obtained at the following link:<br><a href="https://github.com/yuangh-x/2022-NIPS-Tenrec">https://github.com/yuangh-x/2022-NIPS-Tenrec</a></p><p>This data set comes from Tencent’s two information flow (article and video) recommendation platforms, namely the OK platform and the QB platform. Each platform has two information sources, namely articles and videos, and the two platforms have varying degrees of overlap.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*Xggx5So9dcE_eCaWIsgnbg.png" /></figure><p>The statistical information of this data set is as follows, counting the number of users, number of items, number of clicks, number of likes and other commonly used indicators for each platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*jfKrt9dNHuklTMo19vdKzg.png" /></figure><p>For this data set, the user behavior logs of QK and QB from September 17 to December 7, 2021 were collected. The specific collection process is as follows: First, approximately 5.02 million users are randomly selected from the QK-video database, and each user is required to have at least 5 video click behaviors; then, their feedback (approximately 493 million) is extracted, including positive feedback (i.e., video clicks , shares, likes and follows) and negative feedback (exposure but no user action); in the end, it received about 142 million clicks, 10 million likes, 1 million shares and 860,000 follows, as well as 3.75 million videos . In addition, there are age and gender characteristics of users, and video type characteristics of items. In addition, similar data extraction strategies are implemented for QK-article, QB-video and QB-article.</p><p>It is worth mentioning that in this data set (1) all users, items and related features have been desensitized; (2) the sota results can notify the author and the leaderboard can be updated in a timely manner; (3) new tasks based on Tenrec can create a new leaderboard . The relevant website is currently under construction.</p><p>Specifically, this data set contains the following characteristics:</p><ol><li>Large scale, containing approximately 5 million users and 140 million interaction data;</li><li>Feedback comes in various forms, including not only positive feedback, but also real negative samples;</li><li>There is overlap in users/items, and there are overlapping users and items in 4 scenarios;</li><li>Positive feedback comes in various forms, including clicks, likes, shares, etc.;</li><li>Rich side information, in addition to user and item IDs, there is also other auxiliary information.</li></ol><p>Figures a and b below show the item popularity distribution of click behavior in the QK-video data set, which clearly exhibits the characteristics of a long-tail distribution. Figure c shows the length distribution of sessions. It can be seen that most of the lengths are distributed in the [0–20] interval.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*oNu-DGVwPvSO4FN7RgwMCA.png" /></figure><p>Based on the above advantages, Tenrec can be used to evaluate a wide range of recommendation tasks. In this paper, its characteristics are studied through ten recommendation tasks, including (1) click-through rate estimation, (2) session-based recommendation, (3) multi-task recommendation, (4) cross-domain recommendation, (5) User attribute prediction, (6) cold start recommendation, (7) lifelong user representation learning, (8) model compression, (9) model training acceleration, and (10) model inference acceleration. In addition to these tasks, some of the above features can be easily integrated to propose more or new tasks.</p><p>Next, the experimental results of each recommendation task are introduced. For the specific experimental settings and data sets used for each task, you can read the original text for detailed reading. Only part of the experimental data is listed here.</p><h3>Click rate prediction experiment results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*oKwPs0F9S1XKyDZzek1Rdg.png" /></figure><h3>Session recommendation experiment results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*3d8FgagbwKhMMc-HwIahbw.png" /></figure><h3>Multi-task recommendation experimental results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*mZxZr30FtdiXF-xV8jFaTw.png" /></figure><h3>Cross-domain recommendation experiment results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*9yQ16rN_vLLEJMkoJ74uXQ.png" /></figure><h3>User attribute prediction experiment results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/1*JXZaogTkXsO1HxCDgAXfOA.png" /></figure><h3>Cold start recommendation experiment results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*T8cGQm9D_cWj_AjRTuQ3JA.png" /></figure><h3>Lifelong representation learning experimental results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*JTLTptbgW4IEcxOnxDwGXQ.png" /></figure><h3>Recommended model compression experimental results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*LVTUn9loS7r0FBiLAUojXw.png" /></figure><h3>Model training acceleration experimental results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*N1XpgBX79fSXjdqWg5_WBQ.png" /></figure><h3>Model inference accelerates experimental results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*KY1GdhOp9IEPGR0aLGL_Zw.png" /></figure><p>This paper proposes Tenrec, one of the largest and most general recommendation datasets covering real-world scenarios with various user feedback types. To show its broad utility, experimental validation is performed on ten different recommendation tasks and classic deep models are tested. Promote research in the recommendation community by publishing code, datasets, and rankings for each task. Cooperating units: SIAT, USTC, SYSU.</p><p>Interested readers can go to the original text for detailed reading.</p><p>Original source: zhihu <a href="https://zhuanlan.zhihu.com/p/579806424">https://zhuanlan.zhihu.com/p/579806424</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3836fe22e69c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dataset and Benchmark for Recommender Systems: CTR，Top-N，Sequential, and Multi-domain…]]></title>
            <link>https://medium.com/@lifengyi_6964/dataset-and-benchmark-for-recommender-systems-ctr-top-n-multi-domain-recommendation-0f80a505bd32?source=rss-398e39ee75a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/0f80a505bd32</guid>
            <dc:creator><![CDATA[AI-Advance]]></dc:creator>
            <pubDate>Wed, 20 Mar 2024 02:52:05 GMT</pubDate>
            <atom:updated>2024-03-21T02:32:29.056Z</atom:updated>
            <content:encoded><![CDATA[<h3>Dataset and Benchmark for Recommender Systems: CTR，Top-N，Sequential, and Multi-domain recommendation</h3><p>Title:Dataset and Benchmark for Recommender Systems: CTR，Top-N，Sequential, and Multi-domain recommendation</p><p>In response to problems such as the small size and limited form of user feedback existing in current data sets, Tencent and West Lake University released a large-scale multi-purpose recommendation system data set Tenrec. Based on this dataset, the baseline performance on 10 different recommendation tasks was also verified.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/897/1*ctvWWRfwZArchT6sodsiWQ.png" /></figure><p>Paper: <a href="https://arxiv.org/abs/2210.10629">https://arxiv.org/abs/2210.10629</a></p><p>The code, data sets and lists involved in the paper can be obtained at the following link:<br><a href="https://github.com/yuangh-x/2022-NIPS-Tenrec">https://github.com/yuangh-x/2022-NIPS-Tenrec</a></p><p>This data set comes from Tencent’s two information flow (article and video) recommendation platforms, namely the OK platform and the QB platform. Each platform has two information sources, namely articles and videos, and the two platforms have varying degrees of overlap.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*Xggx5So9dcE_eCaWIsgnbg.png" /></figure><p>The statistical information of this data set is as follows, counting the number of users, number of items, number of clicks, number of likes and other commonly used indicators for each platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*jfKrt9dNHuklTMo19vdKzg.png" /></figure><p>For this data set, the user behavior logs of QK and QB from September 17 to December 7, 2021 were collected. The specific collection process is as follows: First, approximately 5.02 million users are randomly selected from the QK-video database, and each user is required to have at least 5 video click behaviors; then, their feedback (approximately 493 million) is extracted, including positive feedback (i.e., video clicks , shares, likes and follows) and negative feedback (exposure but no user action); in the end, it received about 142 million clicks, 10 million likes, 1 million shares and 860,000 follows, as well as 3.75 million videos . In addition, there are age and gender characteristics of users, and video type characteristics of items. In addition, similar data extraction strategies are implemented for QK-article, QB-video and QB-article.</p><p>It is worth mentioning that in this data set (1) all users, items and related features have been desensitized; (2) the sota results can notify the author and the leaderboard can be updated in a timely manner; (3) new tasks based on Tenrec can create a new leaderboard . The relevant website is currently under construction.</p><p>Specifically, this data set contains the following characteristics:</p><ol><li>Large scale, containing approximately 5 million users and 140 million interaction data;</li><li>Feedback comes in various forms, including not only positive feedback, but also real negative samples;</li><li>There is overlap in users/items, and there are overlapping users and items in 4 scenarios;</li><li>Positive feedback comes in various forms, including clicks, likes, shares, etc.;</li><li>Rich side information, in addition to user and item IDs, there is also other auxiliary information.</li></ol><p>Figures a and b below show the item popularity distribution of click behavior in the QK-video data set, which clearly exhibits the characteristics of a long-tail distribution. Figure c shows the length distribution of sessions. It can be seen that most of the lengths are distributed in the [0–20] interval.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*oNu-DGVwPvSO4FN7RgwMCA.png" /></figure><p>Based on the above advantages, Tenrec can be used to evaluate a wide range of recommendation tasks. In this paper, its characteristics are studied through ten recommendation tasks, including (1) click-through rate estimation, (2) session-based recommendation, (3) multi-task recommendation, (4) cross-domain recommendation, (5) User attribute prediction, (6) cold start recommendation, (7) lifelong user representation learning, (8) model compression, (9) model training acceleration, and (10) model inference acceleration. In addition to these tasks, some of the above features can be easily integrated to propose more or new tasks.</p><p>Next, the experimental results of each recommendation task are introduced. For the specific experimental settings and data sets used for each task, you can read the original text for detailed reading. Only part of the experimental data is listed here.</p><h3>Click rate prediction experiment results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*oKwPs0F9S1XKyDZzek1Rdg.png" /></figure><h3>Session recommendation experiment results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*3d8FgagbwKhMMc-HwIahbw.png" /></figure><h3>Multi-task recommendation experimental results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*mZxZr30FtdiXF-xV8jFaTw.png" /></figure><h3>Cross-domain recommendation experiment results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*9yQ16rN_vLLEJMkoJ74uXQ.png" /></figure><h3>User attribute prediction experiment results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/1*JXZaogTkXsO1HxCDgAXfOA.png" /></figure><h3>Cold start recommendation experiment results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*T8cGQm9D_cWj_AjRTuQ3JA.png" /></figure><h3>Lifelong representation learning experimental results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*JTLTptbgW4IEcxOnxDwGXQ.png" /></figure><h3>Recommended model compression experimental results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*LVTUn9loS7r0FBiLAUojXw.png" /></figure><h3>Model training acceleration experimental results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*N1XpgBX79fSXjdqWg5_WBQ.png" /></figure><h3>Model inference accelerates experimental results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*KY1GdhOp9IEPGR0aLGL_Zw.png" /></figure><p>This paper proposes Tenrec, one of the largest and most general recommendation datasets covering real-world scenarios with various user feedback types. To show its broad utility, experimental validation is performed on ten different recommendation tasks and classic deep models are tested. Promote research in the recommendation community by publishing code, datasets, and rankings for each task. Cooperating units: SIAT, USTC, SYSU.</p><p>Interested readers can go to the original text for detailed reading.</p><p>Original source: zhihu <a href="https://zhuanlan.zhihu.com/p/579806424">https://zhuanlan.zhihu.com/p/579806424</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0f80a505bd32" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>