基于Dantzig Selector的迁移学习——应用于广义线性模型
Transfer Learning Based on Dantzig Selector—Applied to Generalized Linear Models
摘要: 小样本高维度的n-p问题一直是统计学家的研究热点，不同于传统的变量选择的参数估计方法，在本篇论文中，我们应用迁移学习的相关知识，借用与需要预测数据相关但分布不同的数据，有效的帮助我们解决目标域数据参数的预测问题。我们提出了一种新颖的基于Dantzig selector的两步迁移学习算法，在数值模拟中，我们验证了提出的迁移学习算法在广义线性模型以及不同的协方差数据设计中的稳健性和有效性，这表明提出的算法具有一定的实际应用价值。
Abstract: The problem of small samples and high dimensionality has always been a research hotspot for stat-isticians. Different from the traditional parameter estimation method of variable selection, in this paper, we apply the relevant knowledge of transfer learning, and borrow data with different distri-butions from those that need to be predicted. It effectively helps us solve the prediction problem of target domain data. We propose a novel two-step transfer learning algorithm based on Dantzig se-lector. In numerical simulations, we verify the robustness and effectiveness of the proposed transfer learning algorithm in generalized linear models as well as in different covariance data designs, which shows that the proposed algorithm has certain practical application value.
文章引用：孙飞, 梁淑娜. 基于Dantzig Selector的迁移学习——应用于广义线性模型[J]. 应用数学进展, 2022, 11(9): 6779-6786. https://doi.org/10.12677/AAM.2022.119718

参考文献

[1] 王晋东. 迁移学习简明手册[M/OL].
https://www.doc88.com/p-79899021072337.html, 2020-08-13.
[2] Zhao, L., Pan, S.J., Xiang, E.W., Zhong, E., Lu, Z. and Yang, Q. (2013) Active Transfer Learning for Cross-System Recom-mendation. Proceedings of the AAAI Conference on Artificial Intelligence, 27, 1205-1211. [Google Scholar] [CrossRef] 
[3] Erbe, R., Kessler, M.D., Favorov, A.V., Easwaran, H., Gaykalova, D.A. and Fertig, E.J. (2020) Matrix Factorization and Transfer Learning Uncover Regulatory Biology across Multiple Single-Cell ATAC-Seq Data Sets. Nucleic Acids Research, 48, e68. [Google Scholar] [CrossRef] [PubMed]
[4] Mesnil, G., Dauphin, Y., Glorot, X., et al. (2012) Unsupervised and Transfer Learning Challenge: A Deep Learning Approach. Proceedings of the 2011 International Conference on Unsu-pervised and Transfer Learning Workshop, Vol. 27, Washington DC, 2 July 2011, 97-111.
[5] Hu, W., Qian, Y., Soong, F.K., et al. (2015) Improved Mispronunciation Detection with Deep Neural Network Trained Acoustic Models and Transfer Learning Based Logistic Regression Classifiers. Speech Communication, 67, 154-166. [Google Scholar] [CrossRef] 
[6] Qi, G.-J., Liu, W., Aggarwal, C. and Huang, T. (2016) Joint Intermodal and Intramodal Label Transfers for Extremely Rare or Unseen Classes. IEEE Transactions on Pattern Analy-sis and Machine Intelligence, 39, 1360-1373. [Google Scholar] [CrossRef] 
[7] Tibshirani, R. (1996) Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58, 267-288. [Google Scholar] [CrossRef] 
[8] Candes, E. and Tao, T. (2007) The Dantzig Selector: Sta-tistical Estimation When p Is Much Larger than n. The Annals of Statistics, 35, 2313-2351. [Google Scholar] [CrossRef] 
[9] Zou, H. (2006) The Adaptive Lasso and Its Oracle Properties. Journal of the American Statistical Association, 101, 1418-1429. [Google Scholar] [CrossRef] 
[10] Yuan, M. and Lin, Y. (2006) Model Selection and Estimation in Regression with Grouped Variables, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68, 49-67. [Google Scholar] [CrossRef] 
[11] Zhang, C.-H. (2010) Nearly Unbiased Variable Selection under Minimax Concave Penalty. The Annals of statistics, 38, 894-942. [Google Scholar] [CrossRef] 
[12] Tucker, D.C., Wu, Y. and Muller, H.-G. (2021) Variable Selection for Global Frëchet Regression. Journal of the American Statistical Association, 1-15. [Google Scholar] [CrossRef] 
[13] Ben-David, S., Blitzer, J., Crammer, K. and Pereira, F. (2006) Analysis of Representations for Domain Adaptation. Advances in Neural Information Processing Systems, 19, 137-144.
[14] Blitzer, J., McDonald, R. and Pereira, F. (2006) Domain Adaptation with Structural Correspondence Learning. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, 22-23 July 2006, 120-128. [Google Scholar] [CrossRef] 
[15] Gong, B., Shi, Y., Sha, F. and Grauman, K. (2012) Geodesic Flow Kernel for Unsupervised Domain Adaptation. 2012 IEEE Conference on Computer Vision and Pattern Recognition, 16-21 June 2012, Providence, 2066-2073. [Google Scholar] [CrossRef] 
[16] Li, S., Cai, T.T. and Li, H. (2022) Transfer Learning for High-Dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality. Journal of the Royal Statistical Society: Series B (Methodological), 84, 149-173. [Google Scholar] [CrossRef] [PubMed]
[17] Bastani, H. (2021) Predicting with Proxies: Transfer Learning in High Di-mension. Management Science, 67, 2964- 2984. [Google Scholar] [CrossRef] 
[18] Cai, T.T. and Wei, H. (2021) Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier, The Annals of Statis-tics, 49, 100-128. [Google Scholar] [CrossRef] 
[19] Tian, Y. and Feng, Y. (2022) Transfer Learning under High-Dimensional Generalized Linear Models. Journal of the American Statistical Association, 1-30. [Google Scholar] [CrossRef] [PubMed]
[20] Nelder, J.A. and Wedderburn, R.W.M. (1972) Generalized Linear Models. Journal of the Royal Statistical Society: Series A (General), 135, 370-384. [Google Scholar] [CrossRef] 
[21] Duchi, J. and Namkoong, H. (2018) Learning Models with Uniform Perfor-mance via Distributionally Robust Optimization. arXiv preprint arXiv:1810.08750.

为你推荐