\(\newcommand{\W}[1]{ \; #1 \; }\) \(\newcommand{\R}[1]{ {\rm #1} }\) \(\newcommand{\B}[1]{ {\bf #1} }\) \(\newcommand{\D}[2]{ \frac{\partial #1}{\partial #2} }\) \(\newcommand{\DD}[3]{ \frac{\partial^2 #1}{\partial #2 \partial #3} }\) \(\newcommand{\Dpow}[2]{ \frac{\partial^{#1}}{\partial {#2}^{#1}} }\) \(\newcommand{\dpow}[2]{ \frac{ {\rm d}^{#1}}{{\rm d}\, {#2}^{#1}} }\)
exp_reverse¶
Exponential Function Reverse Mode Theory¶
We use the reverse theory standard math function definition for the functions \(H\) and \(G\). The zero order forward mode formula for the exponential is
\[z^{(0)} = F ( x^{(0)} )\]
and for \(j > 0\),
\[z^{(j)} = x^{(j)} d^{(0)}
+ \frac{1}{j} \sum_{k=1}^{j} k x^{(k)} z^{(j-k)}\]
where
\[\begin{split}d^{(0)} = \left\{ \begin{array}{ll}
0 & \R{if} \; F(x) = \R{exp}(x)
\\
1 & \R{if} \; F(x) = \R{expm1}(x)
\end{array} \right.\end{split}\]
For order \(j = 0, 1, \ldots\) we note that
\begin{eqnarray}
\D{H}{ x^{(j)} }
& = & \D{G}{ x^{(j)} } + \D{G}{ z^{(j)} } \D{ z^{(j)} }{ x^{(j)} }
\\
& = & \D{G}{ x^{(j)} } + \D{G}{ z^{(j)} } ( d^{(0)} + z^{(0)} )
\end{eqnarray}
If \(j > 0\), then for \(k = 1 , \ldots , j\)
\begin{eqnarray}
\D{H}{ x^{(k)} } & = &
\D{G}{ x^{(k)} } + \D{G}{ z^{(j)} } \frac{1}{j} k z^{(j-k)}
\\
\D{H}{ z^{(j-k)} } & = &
\D{G}{ z^{(j-k)} } + \D{G}{ z^{(j)} } \frac{1}{j} k x^{(k)}
\end{eqnarray}