可微分编程是一种编程范型,在其中数值计算程序始终可通过自动微分来求导数[1][2][3][4]。这允许了对程序中的参数的基于梯度优化,通常通过梯度下降。可微分编程广泛用于各种领域,特别是科学计算和人工智能[4]。
多数可微分编程框架是通过构造包含程序中的控制流和数据结构的图来进行工作的[5]。各种尝试一般可归入两组之中:
早期方式的局限在于,它们都是以适合于这些框架的风格书写求微分的代码,这限制了同其他程序的互操作性。新近的方式,通过从语言的语法或中间表示构造图来解决了这种问题,允许任意代码都是可求微分的[5][6]。
Wang, Fei; Decker, James; Wu, Xilun; Essertel, Gregory; Rompf, Tiark, Bengio, S.; Wallach, H.; Larochelle, H.; Grauman, K. , 编, Backpropagation with Callbacks: Foundations for Efficient and Expressive Differentiable Programming (PDF), Advances in Neural Information Processing Systems 31 (Curran Associates, Inc.), 2018: 10201–10212 [2019-02-13], (原始内容存档 (PDF)于2021-02-15)
Innes, Mike; Edelman, Alan; Fischer, Keno; Rackauckas, Chris; Saba, Elliot; Viral B Shah; Tebbutt, Will, ∂P: A Differentiable Programming System to Bridge Machine Learning and Scientific Computing, 2019, arXiv:1907.07587
Zygote. [2021-01-14]. (原始内容存档于2021-02-14).
Innes, Michael. Don't Unroll Adjoint: Differentiating SSA-Form Programs. 2018-10-18. arXiv:1810.07951 [cs.PL].
Degrave, Jonas; Hermans, Michiel; Dambre, Joni; wyffels, Francis. A Differentiable Physics Engine for Deep Learning in Robotics. 2016-11-05. arXiv:1611.01652 [cs.NE].
Li, Li; Hoyer, Stephan; Pederson, Ryan; Sun, Ruoxi; Cubuk, Ekin D.; Riley, Patrick; Burke, Kieron. Kohn-Sham Equations as Regularizer: Building Prior Knowledge into Machine-Learned Physics. Physical Review Letters. 2021, 126 (3): 036401. doi:10.1103/PhysRevLett.126.036401.