Qing Da (Q. Da), www.daqings.net
daqingacm@sina.com / linkedin / Google scholar / DBLP
现任阿里巴巴资深算法专家（花名达卿），中国计算机学会会员，本硕均毕业于南京大学计算机系，师从周志华教授，从事机器学习相关的研究工作。2015年初，于博士二年级退学，加入当时的阿里巴巴搜索推荐事业部，从事搜索推荐算法相关的工作至今。
本职工作之外，对内承担了一些算法新人培训的工作，2019年集团新人课程《搜索推荐广告-算法体系概论》作者之一，杭州场主讲人，亦承担了次年（2020年）搜索推荐事业部算法新人班的班主任；对外则担任了TNNLS、AAAI、IJCAI、ICML等领域期刊会议的审稿人，2016 中国计算机学会第一届强化学习论坛主讲嘉宾，天池深度学习课程强化学习章节讲师等。
教育经历
本科：南京大学，计算机科学与技术系，2006-09 至 2010-06
- 全国大学生数学建模大赛全国一等奖，第一完成人，2008，论文.
- 教育部-Sun大学生创新实验计划杰出项目(Outstanding project)，第一完成人，2009，项目地址.
- 南京大学本科优秀毕业论文（设计）一等奖，2010.
- 保送至本系继续攻读研究生学位（Rank 1st）.
硕士：南京大学，计算机科学与技术系，机器学习与数据挖掘研究所（LAMDA），2010-09 至 2013-06
- PAKDD数据挖掘竞赛公开组冠军，第五完成人，2012.
- 首届“中国云·移动互联网创新大奖赛”，一等奖1项，二等奖1项，三等奖2项，第一完成人，2013. [央视新闻]
- 硕士毕业论文，《基于直接策略搜索的强化学习方法研究》.
博士：南京大学，计算机科学与技术系，机器学习与数据挖掘研究所（LAMDA），2013-09 至 2015-01，肄业
- 博一期间以一作身份分别发表CCF A类一篇[21]，CCF B类一篇[22]，博士生中期考核中优秀（Rank 1st）.
- 博二上学期因个人原因退学进入工业界.
职业经历
2015.01 – 2016.06: 资深算法工程师 搜索事业部 阿里巴巴
- 从事淘宝搜索算法相关工作
- 通过和工程团队的配合，上线第一版基于parameter server的大规模在线学习模型，推动了在线学习在手淘搜索的遍地开花，相关工作《基于在线矩阵分解的淘宝搜索实时个性化》获得集团2015年十大算法奖
- 首次将最优分配+PID在线调节的技术引入双十一红包的发放，该技术已成为目前集团内权益发放+流量调控的基础通用方案，并沉淀了专利《一种通过关键词发放红包出售流量的方式》
- 2015年搜索事业部最佳新人奖，双11疯狂搜索人奖，技术卓越奖团队成员
- 集团内部算法竞赛奖项若干：新浪微博互动预测大赛亚军；简历智能评分大赛亚军，菜鸟-需求预测&分仓规划优胜奖（rank 4）
2016.07 – 2017.12: 算法专家 搜索事业部 阿里巴巴
- 首次将强化学习引入电商搜索排序中，并且在2016年双十一进行上线应用，相关工作被多家媒体报道。[机器之心报道]
- 设计实现了搜索内部的强化学习框架AI4B-RL，在多个业务线尝试进行强化学习应用，包括搜索排序[17]、锦囊展示学习[16]、引擎性能优化[11]、卖家分层调控、流量调控[15]、虚拟淘宝[13]等工作，相关工作已经整理发表在KDD，AAAI，ECML上。
- 组织集团内所有相关算法团队，发布电子书《强化学习在阿里的技术演进与业务创新》，其内容包含搜索、推荐、广告、物流、智能客服等广泛领域，次年其实体书由电子工业出版社出版发行。
2018.01 – 2020.07: 高级算法专家 AI国际事业部 阿里巴巴
- 带队参加OpenAI举办的强化学习算法竞赛，获得全球总冠军，第一完成人。[OpenAI官网新闻][阿里技术报道]
- 2018年8月开始担任AliExpress搜索算法负责人角色，开始组建团队和技术升级，同时针对国际化业务特点进行技术创新。目前从2-3人的种子团队成长为20+的算法团队，覆盖搜索相关性、搜索效率、搜索导购产品、流量调控等4个子方向。在先后2个财年内，搜索转化率在原先优化了8年的baseline基础上，分别提升了30%和20%，同时在搜索相关性、搜索导购产品等相关业务上也有较大幅度的提升和改进。
- 针对国际化业务的多语言问题，在facebook的XLM提出来同期就开始研究跨语言的向量模型，目前覆盖英、俄、西、法、葡，并最终全量上线，用于搜索的语义向量召回，相关工作已经投稿至EMNLP[6]。
- 针对单一模型很难捕捉到全球国家用户的行为差异，将经典的MOE结构升级为层次MOE结构，并通过场景子网络和场景梯度隔断来保障最后的融合效果，相关工作已经全量上线，并被CIKM’20录取[12]。
- 针对重排序场景，为了捕捉用户对商品整体排列的行为偏好，提出了基于评估器-生成器架构的learning to rank框架，该框架可以直接生成一个商品排列，以最优化全页面的用户行为，相关工作已经全量上线，并且被TKDE收录[9]，以及相关领域媒体报道。
2020.08 – 现在 : 资深算法专家 AE技术部 阿里巴巴
- 设计并产品化了AliExpress的流量调控系统，用于商业策略和效率策略的高效整合，其中涉及的相关工作：online matching[10]已经在AAAI’10发表，其余部分还在整理投稿中[2]。
- 目前除了担任AliExpress搜索算法负责人外，同时还负责AliExpress整体搜索、推荐、广告、用增等多条业务算法线的横向能力建设。
书籍出版
- 笪庆，曾安祥主编，《强化学习实战：强化学习在阿里的技术演进和业务创新》，电子工业出版社，2018，书籍链接
推荐系统开源环境
1. VirtualTaobao[AAAI’19], Github.
This project provides VirtualTaobao simulators trained from the real-data of Taobao, one of the largest online retail platforms. In Taobao, when a customer entered some query, the recommondation system returns a list of items according to the query and the customer profile. The system is expected to return a good list such that customers will have high chances of clicking the items.
Using VirtualTaobao simulator, one can access a “live” environment just like the real Taobao environment. Virtual customers will be generated once at a time, the virtual customer starts a query, and the recommendation system needs to return a list of items. The virtual customer will decide if it would like to click the items in the list, similar to a real customer.
How VirtualTaobao was trained is described in：
- Jing-Cheng Shi, Yang Yu, Qing Da, Shi-Yong Chen, An-Xiang Zeng. Virtual-Taobao: Virtualizing real-world online retail environment for reinforcement learning. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19), Honolulu, HI, 2019. PDF
2. SESim, Github.
SESim is an E-Commerce search engine simulation platform for model examinations, which was a missing piece to connect evaluations of LTR researches and business objectives of real-world applications. SESim can examine models in the simulation E-commerce environment with dynamic responses, and its framework can be easily extended to other scenarios that items and users have differ- ent features. We hope to see the development of a dynamic dataset that facilitates industrial LTR researches in the future.
A typical process of industrial search engines contains three stages to produce a display list from a user query. A search engine first retrieves related items with intend of the user (i.e. the user query), then the ranker ranks these items by a fine-tuned deep LTR model, finally, the re-ranker rearranges the order of items to achieve some businesses goals such as diversity and advertising. Our proposed simulation platform SESim contains these three stages. We replace queries with category indices in our work, therefore SESim can retrieve items from a desensitized items database by the category index. After that, a customizable ranker and a customizable re-ranker produce the final item list. SESim allows us to study joint learning of multiple models, we left it as future work and focus on the correct evaluation for a single model.
Besides the set of real items, two important modules make SESim vividly reflect the behaviors of real users. Virtual user module aims at generating embeddings of virtual users and their query, and it follows the paradigm of Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP). Feedback module inputs the display list and the information of the user, then outputs the feedback of users on the display list. To model the decision process of users, we train the feedback module by Generative Adversarial Imitation Learning (GAIL). For diversifying behaviors, we consider clicking and purchasing, which are two of the most important feedback of users in E-commerce.
How SESim was trained is described in：
- Yongqing Gao, Guangda Huzhang, Weijie Shen, Yawen Liu, Wen-Ji Zhou, Qing Da, Yang Yu. Imitate The World: A Search Engine Simulation Platform. Submitted to CIKM’21, CORR abs/2107.07693.
论文发表
- Wen-Ji Zhou, Yunan Ye, Qing Da, Yinfu Feng, Anxiang Zeng, Han Yu and Chunyan Miao. Policy Gradient Matching for Recommendation Systems. Submitted to CIKM’21.
- Chenlin Shen, Guangda Huzhang, Yu-Hang Zhou, Shen Liang, Qing Da. A General Traffic Shaping Protocol in E-Commerce. Submitted to CIKM’21.
- Xuesi Wang, Guangda Huzhang, Qianying Lin, Qing Da. Learning-To-Ensemble by Contextual Rank Aggregation in E-Commerce. Submitted to CIKM’21.
- Yongqing Gao, Guangda Huzhang, Weijie Shen, Yawen Liu, Wen-Ji Zhou, Qing Da, Yang Yu. Imitate The World: A Search Engine Simulation Platform. Submitted to CIKM’21, CORR abs/2107.07693.
- Wenya Zhu, Yinghua Zhang, Yu Zhang, Yu-Hang Zhou, Yinfu Feng, Qing Da, Yuxiang Wu and Xiaoyu Lv. DHA: Product Title Generation with Discriminative Hierarchical Attention for E-commerce. Submitted to CIKM’21.
- Wenya Zhu, Xiaoyu Lv, Baosong Yang, Yinghua Zhang, xu yong, Dayiheng Liu, Linlong Xu, yinfu feng, Haibo Zhang,Qing Da and Weihua Luo. CLPR-9M: An E-Commerce Dataset for Cross-Lingual Product Retrieval. Submitted to EMNLP’21.
- Junmei Hao, Jingcheng Shi, Qing Da, Anxiang Zeng, Yujie Dun, Xueming Qian, Qianying Lin. Diversity Regularized Interests Modeling for Recommender Systems. Submitted to TMM. CORR abs/2103.12404
- Yanshi Wang, Jie Zhang, Qing Da, Anxiang Zeng. Delayed Feedback Modeling for the Entire Space Conversion Rate Prediction, 2020, CORR abs/2011.11826
- Guangda Huzhang, Zhen-Jia Pang, Yongqing Gao, Yawen Liu, Weijie Shen, Wen-Ji Zhou, Qianying Lin, Qing Da, An-Xiang Zeng, Han Yu, Yang Yu, Zhi-Hua Zhou. AliExpress Learning-To-Rank: Maximizing Online Model Performance without Going Online. IEEE Transactions on Knowledge and Data Engineering (TKDE). CORR abs/2003.11941
- Yu-Hang Zhou, Peng Hu, Chen Liang, Huan Xu, Guangda Huzhang, Yinfu Feng, Qing Da, XInshang Wang, An-Xiang Zeng. A Primal-Dual Online Algorithm for Online Matching Problem in Dynamic Environments. In: Proceedings of the 34rd AAAI Conference on Artificial Intelligence (AAAI-21)
- Anxiang Zeng, Han Yu, Qing Da, Yusen Zhan, Chun-yanMiao, Accelerating E-Commerce Search Engine Ranking by Contextual Factor Selection. In: Proceedings of the 34rd AAAI Conference on Artificial Intelligence (AAAI-20 / IAAI-20), New York, USA, 2020. PDF
- Pengcheng Li, Runze Li, Qing Da, An-Xiang Zeng, Lijun Zhang. Improving Multi-Scenario Learning to Rank in E-commerce by Exploiting Task Relationships in the Label Space. In: Proceedings of the 29th International Conference on Information and Knowledge Management (CIKM’20), Virtual Event, Ireland, 2020. PDF
- Jing-Cheng Shi, Yang Yu, Qing Da, Shi-Yong Chen, An-Xiang Zeng. Virtual-Taobao: Virtualizing real-world online retail environment for reinforcement learning. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19), Honolulu, HI, 2019. PDF
- Feiyang Pan, Qingpeng Cai , An-Xiang Zeng , Chun-Xiang Pan, Qing Da, Hualin He, Qing He, Pingzhong Tang. Policy Optimization with Model-based Explorations. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19), Honolulu, HI, 2019. PDF
- Hua-Lin Hei, Chun-Xiang Pan, Qing Da, An-Xiang Zeng. Speeding up the Metabolism in E-commerce by Reinforcement Mechanism Design. In: “Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD’18)“, Dublin, Ireland, 2018. PDF
- Shi-Yong Chen, Yang Yu, Qing Da, Jun Tan, Hai-Kuan Huang and Hai-Hong Tang. Stablizing reinforcement learning in dynamic environment with application to online recommendation. In: Proceedings of the 24th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’18) (Research Track), London, UK, 2018. PDF
- Yujing Hu, Qing Da, Anxiang Zeng, Yang Yu, Yinghui Xu, Reinforcement Learning to Rank in E-Commerce Search Engine: Formalization, Analysis, and Application. In: Proceedings of the 24th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’18) (Applied Track), London, UK, 2018. PDF
- Yang Yu, Shi-Yong Chen, Qing Da, Zhi-Hua Zhou. Reusable reinforcement learning via shallow trails. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(6): 2204-2215. PDF
- Yang Yu, Peng-Fei Hou, Qing Da, and Yu Qian. Boosting nonparametric policies. In: Proceedings of the 2016 International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS’16), Singapore, 2016, pp.477-484. PDF
- Yang Yu and Qing Da, PolicyBoost: Functional policy gradient with ranking-based reward objective. In: Proceedings of AAAI Workshop on AI and Robotics (AIRob’14), Quebec City, Canada, 2014. PDF
- Qing Da, Yang Yu, and Zhi-Hua Zhou. Learning with Augmented Class by Exploiting Unlabeled Data. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI’14), Québec city, Canada, 2014. PDF
- Qing Da, Yang Yu, and Zhi-Hua Zhou. Napping for Functional Representation of Policy. In: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS’14), Paris, France, 2014. PDF
- Qing Da, Yang Yu, and Zhi-Hua Zhou. Self-Practice Imitation Learning from Weak Policy. In: Proceedings of the 2nd IAPR International Workshop on Partially Supervised Learning (PSL’13), Nanjing, China, 2013, pp.9-20. PDF