"scmutils" refers to
MIT Scheme
running the scmutils
implementation of forward AD.
"MIT Scheme" refers to
MIT Scheme running the same
implementation of AD as the other
Scheme implementations.
Many Scheme implementations do
not allow redefinition of builtin arithmetic procedures.
Thus the Scheme variants of our
examples (except for scmutils)
are written using alternate arithmetic procedures d+, d-,
etc.
scmutils incorrectly
implements, or fails to implement, the overloaded versions of
=, <=, zero?, and positive?.
Thus the scmutils variants of
our examples are written using alternate procedures d=,
d<=, dzero?, and dpositive?.
It is conjectured to be impossible to implement AD in
ml in a fashion that applies to
unmodified source code.
Thus the ml variants of our
examples require that (a) all code which implements the function whose
derivative is taken, including all code called by such code, be syntactically
nested inside the redefinition of the primitives and (b) all real constants in
that code be syntactically wrapped with the BASE constructor.
It is conjectured to be impossible to implement nestable forward AD in
Haskell in a fashion that
applies to unmodified source code.
Thus the Haskell variants of
our examples require that the code that implements the functions whose
derivatives are taken be manually annotated with appropriate calls to
lift to properly handle nesting.
Code must be written with templates in
fadbad++ to support taking
derivatives of different orders.
Code that is transformed by
Tapenade multiple times must be
post-edited.
The flow analysis used by
adifor yields incorrect
derivative code that produces the wrong answer, without warning, when using the
same file organization as all of the other variants, where each example is
contained in a single file.
Thus the
adifor variants of our examples
circumvent this flaw by manually partitioning each example into three files
that contain the code that is transformed zero, one, and two times.
Tapenade cannot transform code
that relies on indirect (i.e., external) subroutine calls.
Thus the Tapenade variants of
our examples require manual specialization of the
multivariate_argmin subroutine.
We could not construct
adic variants of
particle-FF and saddle-FF because we were unsuccessful in
getting adic to transform its
own generated code.
http://docs.lib.purdue.edu/ecetr/368
discusses the limitations of
adifor,
Tapenade,
adic, and
fadbad++ in greater detail.
These AD-specific limitations of existing languages and systems are in
addition to the standard limitations of languages like
Fortran,
c, and
c++ relative to
higher-order functional-programming languages.
For example, since our particle and saddle examples make use
of nested lambda expressions with lexically-scoped free variables to implement
the potential function in particle and the nested minimax
optimization in saddle, the
adifor,
Tapenade, and
fadbad++ variants of these
examples manually implement the requisite closures by way of common blocks in
Fortran and global variables in
c++.