Clad: Automatic differentiation plugin for C++

Release v2.6.0~dev.

Overview

Clad enables automatic differentiation (AD) for C++. It is based on LLVM compiler infrastructure and is a plugin for Clang compiler. Clad is based on source code transformation. Given C++ source code of a mathematical function, it can automatically generate C++ code for computing derivatives of the function.

Clad supports a large set of C++ features including control flow statements and function calls. Supported C++ features covers what it handles and what it does not, and Using Clad works through how. Clad supports reverse-mode AD (a.k.a backpropagation) as well as forward-mode AD, and it also facilitates computation of hessian matrix and jacobian matrix of any arbitrary function.

Automatic differentiation avoids the precision loss of numerical differentiation, and the closed-form restrictions and expression swell of symbolic differentiation.

If you are just getting started, Using Clad and Tutorials are the places to begin. Clad is a Clang plugin, so it is loaded with -fplugin= at compile time; Installation and usage has the details.

---

Clad example use:

#include "clad/Differentiator/Differentiator.h"
#include <iostream>

double f(double x, double y) { return x * y; }

int main() {
  auto f_dx = clad::differentiate(f, "x");
  // Computes the derivative of 'f' at (x, y) = (3, 4) and prints it.
  std::cout << f_dx.execute(3, 4) << std::endl; // prints: 4
  f_dx.dump();
  // prints: double f_darg0(double x, double y) {
  // prints:     double _d_x = 1;
  // prints:     double _d_y = 0;
  // prints:     return _d_x * y + x * _d_y;
  // prints: }
}

Features

• Requires little to no code modification for computing derivatives of existing codebase.

• Features both reverse mode AD (backpropagation) and forward mode AD.

• Computes derivatives of functions, member functions, functors and lambda expressions.

• Supports large subset of C++ including if statements, for, while loops and so much more; it is actively being developed with the goal of supporting all of C++ syntax.

• Provides direct functions for computation of Hessian matrix and Jacobian matrix.

• Supports array differentiation, that is, it can differentiate either with respect to whole arrays or particular indices of the array.

• Features numerical differentiation support, to be used as a fallback where automatic differentiation is not feasible.

The User Guide

Getting started

• Installation and usage
  • Conda Installation
  • Building from source
  • How to use Clad
• Tutorials
• FAQ
  • Clang says “Clad doesn’t appear to be loaded”
  • Clang cannot load clad.so, or reports an undefined symbol
  • Should I use forward mode or reverse mode?
  • How do I differentiate with respect to an array?
  • Clad warns that a function has no definition
  • How do I see the code Clad generated?
  • Can Clad differentiate templates and overloaded functions?
  • I think a derivative is wrong. What should I report?

Guides

• Using Clad
  • Automatic Differentiation
  • Numerical Differentiation Fallback
  • Error Estimation
  • Debug functionalities
  • Other ways to differentiate
• Custom derivatives
  • What are custom derivatives?
  • Where to define custom derivatives?
  • Adjoint construction and initialization
  • Pushforward custom derivatives
  • Pullback custom derivatives
  • Reverse-forward custom derivatives
  • Member functions custom derivatives
  • Constructor custom derivatives
  • Porting hints: discovering which custom derivatives to write
• Floating-point error estimation
  • Introduction

Reference

• API reference
• Supported C++ features
  • If Clad cannot differentiate something
  • Templates
• Demos
  • Building one
  • The basics
  • Inside a real program
  • Without installing anything
• Core concepts
  • How Clad Works
  • Forward and Reverse Mode Automatic Differentiation
  • Vectorized Forward Mode Automatic Differentiation
  • Derived Function Types and Derivative Types
  • Custom Derivatives
  • Pushforward and Pullback functions
  • Differentiable Class Types
  • Numerical Differentiation
  • Error Estimation Core Concepts
  • Further Reading
• What the analyses look for
  • Seeing what clad declined
  • CLAD1001: a counted loop
  • CLAD1002: a broadcast read
  • CLAD1003: a bounded write
  • CLAD1004: a call clad can look inside
  • CLAD1005: a loop with a literal count
• What clad says
• Options

Contributing

• Developer guide
  • Building from source
  • Clad Internal Documentation
  • Clad’s Tests
  • Debugging Clang
  • Debugging Github runners
  • Debugging x86 builds locally
• Clang for contributors
  • The big picture
  • The AST
  • Flow graph of Clad’s compilation
  • Identifying functions to derive
  • Code generation and insertion
  • Statement differentiation
  • Less relevant notes
  • Further Reading
• Internal documentation

Citing Clad

If Clad contributed to your work, please cite the paper describing it:

% 16th International workshop on Advanced Computing and Analysis Techniques
% in physics research (ACAT), 1-5 September, 2014, Prague, The Czech Republic
@inproceedings{Vassilev_Clad,
  author = {Vassilev,V. and Vassilev,M. and Penev,A. and Moneta,L. and Ilieva,V.},
  title = {{Clad -- Automatic Differentiation Using Clang and LLVM}},
  journal = {Journal of Physics: Conference Series},
  year = 2015,
  month = {may},
  volume = {608},
  number = {1},
  pages = {012055},
  doi = {10.1088/1742-6596/608/1/012055},
  url = {https://iopscience.iop.org/article/10.1088/1742-6596/608/1/012055/pdf},
  publisher = {{IOP} Publishing}
}

Founders

Clad was founded by Vassil Vassilev, as part of his research interests and vision. He holds the exclusive copyright and other related rights, described in Copyright.txt.

License

Clad is an open source project, licensed under the GNU Lesser General Public License. A module under a different license says so in the License.txt of its own source folder. See License.txt for the full text.