迁移学习

迁移学习（英語：Transfer learning）是属于机器学习的一种研究领域。它专注于存储已有问题的解决模型，并将其利用在其他不同但相关问题上。^[1] 比如说，用来辨识汽车的知识（或者是模型）也可以被用来提升识别卡车的能力。计算机领域的迁移学习和心理学常常提到的学习迁移在概念上有一定关系，但是两个领域在学术上的关系非常有限。

最早被引用的关于迁移学习的工作被认为属于洛麗安·普拉特（英语：Lorien Pratt）。他在1993年制定了基于可辨识性的转移（DBT）算法。^[2]

1997年，机器学习期刊发表了一期专门讨论迁移学习的期刊，^[3] 而到了1998年，该领域已经发展到包括多任务学习，^[4] 以及对其理论基础的更深入完善的分析。^[5] 1998年，由普拉特和塞巴斯蒂安·特龍编辑的《Learning to Learn》^[6]便是对该主题的回顾。

迁移学习也被应用于认知科学，比如《Connection Science》杂志就于1996年出版了一版特殊期刊，描述了如何通过使用迁移学习重新利用已有神经网络。^[7]

迁移学习是由域和任务定义的。域 ${\mathcal {D}}$ 由特征空间 ${\mathcal {X}}$ 和边缘概率分布 $P(X)$ 构成，其中 $X=\{x_{1},...,x_{n}\}\in {\mathcal {X}}$ 。给定域 ${\mathcal {D}}=\{{\mathcal {X}},P(X)\}$ ，任务由标签空间 ${\mathcal {Y}}$ 和目标预测函数 $f:{\mathcal {X}}\rightarrow {\mathcal {Y}}$ 两部分组成，函数 $f$ 预测 $x$ 对应的标签 $f(x)$ 。任务 ${\mathcal {T}}=\{{\mathcal {Y}},f(x)\}$ 是从含有样本对 $\{x_{i},y_{i}\}$ 的训练数据中学习得到的，其中 $x_{i}\in X,y_{i}\in {\mathcal {Y}}$ 。^[8]

给定原域 ${\mathcal {D}}_{S}$ 及其任务 ${\mathcal {T}}_{S}$ ，目标域 ${\mathcal {D}}_{T}$ 及其任务 ${\mathcal {T}}_{T}$ （满足 ${\mathcal {D}}_{S}\neq {\mathcal {D}}_{T}$ 或 ${\mathcal {T}}_{S}\neq {\mathcal {T}}_{T}$ ），迁移学习旨在通过利用 ${\mathcal {D}}_{S}$ 和 ${\mathcal {T}}_{S}$ 的知识，帮助学习 ${\mathcal {D}}_{T}$ 域的目标预测函数 $f_{T}(\cdot )$ 。^[8]

迁移学习的算法基础可以源自马尔可夫逻辑网络^[9]和贝叶斯网络。^[10]迁移网络还被利用与发现癌症种类 ^[11]、建筑物人员限额^[12]、普适智能游戏玩家^[13]、语句分类^[14]^[15]以及筛选垃圾邮件（短信）。^[16]

Thrun, Sebastian; Pratt, Lorien. Learning to Learn. Springer Science & Business Media. 6 December 2012 [2022-12-23]. ISBN 978-1-4615-5529-2. （原始内容存档于2022-07-02）.

[1]
West, Jeremy; Ventura, Dan; Warnick, Sean. Spring Research Presentation: A Theoretical Foundation for Inductive Transfer. Brigham Young University, College of Physical and Mathematical Sciences. 2007 [2007-08-05]. （原始内容存档于2007-08-01）.
[2]
Pratt, L. Y. Discriminability-based transfer between neural networks (PDF). NIPS Conference: Advances in Neural Information Processing Systems 5. Morgan Kaufmann Publishers. 1993: 204–211 [2022-12-23]. （原始内容存档于2022-07-02）.
[3]
Pratt, L. Y.; Thrun, Sebastian. Machine Learning - Special Issue on Inductive Transfer. link.springer.com. Springer. July 1997 [2017-08-10]. （原始内容存档于2019-03-27）（英语）. |volume=被忽略 (帮助); |issue=被忽略 (帮助)
[4]
Caruana, R., "Multitask Learning", pp. 95-134 in Pratt & Thrun 1998
[5]
Baxter, J., "Theoretical Models of Learning to Learn", pp. 71-95 Pratt & Thrun 1998
[6]
Thrun & Pratt 2012.
[7]
Pratt, L. Special Issue: Reuse of Neural Networks through Transfer. Connection Science. 1996 [2017-08-10]. （原始内容存档于2019-03-27）（英语）. 参数|journal=与模板{{cite web}}不匹配（建议改用{{cite journal}}或|website=） (帮助); |volume=被忽略 (帮助); |issue=被忽略 (帮助)
[8]
Lin, Yuan-Pin; Jung, Tzyy-Ping. Improving EEG-Based Emotion Classification Using Conditional Transfer Learning. Frontiers in Human Neuroscience. 27 June 2017, 11: 334. PMC 5486154 . PMID 28701938. doi:10.3389/fnhum.2017.00334 . Material was copied from this source, which is available under a Creative Commons Attribution 4.0 International License.
[9]
Mihalkova, Lilyana; Huynh, Tuyen; Mooney, Raymond J., Mapping and Revising Markov Logic Networks for Transfer (PDF), Learning Proceedings of the 22nd AAAI Conference on Artificial Intelligence (AAAI-2007), Vancouver, BC: 608–614, July 2007 [2007-08-05], （原始内容 (PDF)存档于2019-11-13）
[10]
Niculescu-Mizil, Alexandru; Caruana, Rich, Inductive Transfer for Bayesian Network Structure Learning (PDF), Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS 2007), March 21–24, 2007 [2007-08-05], （原始内容存档 (PDF)于2010-06-20）
[11]
Hajiramezanali, E. & Dadaneh, S. Z. & Karbalayghareh, A. & Zhou, Z. & Qian, X. Bayesian multi-domain learning for cancer subtype discovery from next-generation sequencing count data. 32nd Conference on Neural Information Processing Systems (NIPS 2018), Montréal, Canada. https://arxiv.org/pdf/1810.09433.pdf （页面存档备份，存于互联网档案馆）
[12]
Arief-Ang, I.B.; Salim, F.D.; Hamilton, M. DA-HOC: semi-supervised domain adaptation for room occupancy prediction using CO2 sensor data. 4th ACM International Conference on Systems for Energy-Efficient Built Environments (BuildSys). Delft, Netherlands: 1–10. 2017-11-08 [2018-11-29]. ISBN 978-1-4503-5544-5. doi:10.1145/3137133.3137146. （原始内容存档于2023-01-29）.
[13]
Banerjee, Bikramjit, and Peter Stone. "General Game Learning Using Knowledge Transfer （页面存档备份，存于互联网档案馆）." IJCAI. 2007.
[14]
Do, Chuong B.; Ng, Andrew Y. Neural Information Processing Systems Foundation, NIPS*2005 (PDF). 2005 [2007-08-05]. （原始内容 (PDF)存档于2020-10-12）. |contribution=被忽略 (帮助)
[15]
Rajat, Raina; Ng, Andrew Y.; Koller, Daphne. Twenty-third International Conference on Machine Learning (PDF). 2006 [2007-08-05]. （原始内容存档 (PDF)于2007-07-08）. |contribution=被忽略 (帮助)
[16]
Bickel, Steffen. ECML-PKDD Discovery Challenge Workshop (PDF). 2006 [2007-08-05]. （原始内容 (PDF)存档于2017-08-10）. |contribution=被忽略 (帮助)

[1] [1]
West, Jeremy; Ventura, Dan; Warnick, Sean. Spring Research Presentation: A Theoretical Foundation for Inductive Transfer. Brigham Young University, College of Physical and Mathematical Sciences. 2007 [2007-08-05]. （原始内容存档于2007-08-01）.

[2] [2]
Pratt, L. Y. Discriminability-based transfer between neural networks (PDF). NIPS Conference: Advances in Neural Information Processing Systems 5. Morgan Kaufmann Publishers. 1993: 204–211 [2022-12-23]. （原始内容存档于2022-07-02）.

[3] [3]
Pratt, L. Y.; Thrun, Sebastian. Machine Learning - Special Issue on Inductive Transfer. link.springer.com. Springer. July 1997 [2017-08-10]. （原始内容存档于2019-03-27）（英语）. |volume=被忽略 (帮助); |issue=被忽略 (帮助)

[4] [4]
Caruana, R., "Multitask Learning", pp. 95-134 in Pratt & Thrun 1998

[5] [5]
Baxter, J., "Theoretical Models of Learning to Learn", pp. 71-95 Pratt & Thrun 1998

[FOOTNOTEThrunPratt2012-6] [6]
Thrun & Pratt 2012.

[7] [7]
Pratt, L. Special Issue: Reuse of Neural Networks through Transfer. Connection Science. 1996 [2017-08-10]. （原始内容存档于2019-03-27）（英语）. 参数|journal=与模板{{cite web}}不匹配（建议改用{{cite journal}}或|website=） (帮助); |volume=被忽略 (帮助); |issue=被忽略 (帮助)

[Lin,_Jung_2017-8] [8]
Lin, Yuan-Pin; Jung, Tzyy-Ping. Improving EEG-Based Emotion Classification Using Conditional Transfer Learning. Frontiers in Human Neuroscience. 27 June 2017, 11: 334. PMC 5486154 . PMID 28701938. doi:10.3389/fnhum.2017.00334 . Material was copied from this source, which is available under a Creative Commons Attribution 4.0 International License.

[9] [9]
Mihalkova, Lilyana; Huynh, Tuyen; Mooney, Raymond J., Mapping and Revising Markov Logic Networks for Transfer (PDF), Learning Proceedings of the 22nd AAAI Conference on Artificial Intelligence (AAAI-2007), Vancouver, BC: 608–614, July 2007 [2007-08-05], （原始内容 (PDF)存档于2019-11-13）

[10] [10]
Niculescu-Mizil, Alexandru; Caruana, Rich, Inductive Transfer for Bayesian Network Structure Learning (PDF), Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS 2007), March 21–24, 2007 [2007-08-05], （原始内容存档 (PDF)于2010-06-20）

[:bmdl-11] [11]
Hajiramezanali, E. & Dadaneh, S. Z. & Karbalayghareh, A. & Zhou, Z. & Qian, X. Bayesian multi-domain learning for cancer subtype discovery from next-generation sequencing count data. 32nd Conference on Neural Information Processing Systems (NIPS 2018), Montréal, Canada. https://arxiv.org/pdf/1810.09433.pdf （页面存档备份，存于互联网档案馆）

[12] [12]
Arief-Ang, I.B.; Salim, F.D.; Hamilton, M. DA-HOC: semi-supervised domain adaptation for room occupancy prediction using CO2 sensor data. 4th ACM International Conference on Systems for Energy-Efficient Built Environments (BuildSys). Delft, Netherlands: 1–10. 2017-11-08 [2018-11-29]. ISBN 978-1-4503-5544-5. doi:10.1145/3137133.3137146. （原始内容存档于2023-01-29）.

[13] [13]
Banerjee, Bikramjit, and Peter Stone. "General Game Learning Using Knowledge Transfer （页面存档备份，存于互联网档案馆）." IJCAI. 2007.

[14] [14]
Do, Chuong B.; Ng, Andrew Y. Neural Information Processing Systems Foundation, NIPS*2005 (PDF). 2005 [2007-08-05]. （原始内容 (PDF)存档于2020-10-12）. |contribution=被忽略 (帮助)

[15] [15]
Rajat, Raina; Ng, Andrew Y.; Koller, Daphne. Twenty-third International Conference on Machine Learning (PDF). 2006 [2007-08-05]. （原始内容存档 (PDF)于2007-07-08）. |contribution=被忽略 (帮助)

[16] [16]
Bickel, Steffen. ECML-PKDD Discovery Challenge Workshop (PDF). 2006 [2007-08-05]. （原始内容 (PDF)存档于2017-08-10）. |contribution=被忽略 (帮助)

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

Wikiwand in your browser!

迁移学习

Wikiwand in your browser!

历史

定义

应用

来源

参见

引用