Optimal Rates of Distributed Regression with Imperfect Kernels

讲座时间 Datetime: 2022年12月9日，星期五，15：00-16：00

地点 Venue: 腾讯会议505-746-358

主持人 Host：张海樟 教授

报告人 Speaker: 孙红卫 教授

单位 Affiliation: 济南大学数学科学学院

报告摘要 Abstract:

In this paper, we study the distributed kernel regression via the divide and conquer approach. This learning approach has been proved asymptotically minimax optimal if the kernel is perfectly selected so that the true regression function lies in the associated reproducing kernel Hilbert space. However, this is usually, if not always, impractical because kernels that can only be selected via prior knowledge or a tuning process are hardly perfect. Instead it is more common that the kernel is good enough but imperfect in the sense that the true regression can be well approximated by but does not lie exactly in the kernel space. We show distributed kernel regression can still achieves capacity independent optimal rate in this case. To this end, we first establish a general framework that allows to analyze distributed regression with response weighted base algorithms by bounding the error of such algorithms on a single data set, provided that the error bounds has factored the impact of the unexplained variance of the response variable. Then we perform a leave one out analysis of the kernel ridge regression and bias corrected kernel ridge regression, which in combination with the aforementioned framework allows us to derive sharp error bounds and capacity independent optimal rates for the associated distributed kernel regression algorithms. As a byproduct of the thorough analysis, we also prove the kernel ridge regression can achieve rates faster than O(1/N) (where N is the sample size) in the noise free setting which, to our best knowledge, are first observed and novel in regression learning.

报告人简介：

孙红卫，济南大学数学科学学院教授，博士，硕士生导师。主持完成国家自然科学基金面上项目两项，省自然科学基金面上项目两项。主要研究领域为核机器学习理论。在《IEEE Transactions on Neural Networks and Learning Systems》、 《Applied and Computational Harmonic Analysis》、《Advances in Computational Mathematics》、 《Journal of Machine Learning Research》等国内外学术期刊发表研究论文40余篇，SCI收录论文30余篇，其中1篇入选ESI高被引论文。获山东省高校优秀科研成果一等奖1项。