Researchers introduced the QueryFormer architecture for conversion prediction

The team of developers introduced QueryFormer — an architecture for predicting the probability of conversion after a click (pCVR), which took first place in the industrial track of the KDD Cup 2026 Tencent UniRec Challenge.

The solution is based on a unified block that combines non-sequential features with user behavioral sequences. Unlike traditional approaches that use projection-based multilayer perceptrons, the model applies a cross-attention mechanism to refine queries. This allowed achieving an official score of AUC 0.83254 on the test set, and after additional scaling, the result grew to 0.832713.

The authors conducted a study on the scalability of the model in terms of width and depth of layers. Using packed cross-attention with shared parameters provides high efficiency: when increasing the width H from 1 to 8, the inference delay increases only 1.89 times. In the course of ablation analysis, it was confirmed that generating queries through the attention mechanism makes the largest contribution to the overall accuracy of the system.

The architecture outperforms the HyFormer model with comparable computational budgets. The study is published under the Creative Commons Attribution 4.0 license, which allows the use of the proposed methods in industrial recommendation systems for processing complex user data.