Inlining the Jet initialzation is mandatory for good performance
in autodiff, because all the constants in the dual part can be
propagated into the cost functor.
This patch unrolls the initialization loop with templates and adds
EIGEN_ALWAYS_INLINE to the constructors.
Change-Id: Ic89d645984f3e1df6c63948236da823ba60d9620
The class ArraySelector is now used in autodiff to store the
parameters and residuals. This reduces overhead of FixedArray
for fixed-sized residuals and allows more optimizations due
to inlining and unrolling.
Change-Id: Ibadc5644e64d672f7a555e250fb1f8da262f9d4f
The compile-time constant kNumResiduals is now passed to the
autodiff functions as a template parameter. This will be used
by future patches to optimize autodiff performance.
Change-Id: Ia2b2cc99b88752e8f12f4ce2542b1963bda552f5
Since the output of LossFunction::Evaluate is multiplied by 0.5, the
current implementation of the Tukey loss function must be multiplied
by 2.
Change-Id: Ia94753eef1a375fe48cc2e0d2cc61350904c8248
The default constructor of ExpressionRef now creates a compile
time constant zero assignment. This patch is required, because
a reason change in Jet assumes default 0 initialization.
Change-Id: I8185cef587c17ab828896bce1e768170f8229d4e
In Jet types, there is an attempt to set the derivative to 0.
However, if the derivative is not directly a scalar but a Jet<Jet>,
this cannot be constructed directly from a 0 literal. This is solved
by using the default constructor for the scalar type instead of 0.
The typical use case for this is adding constraints to the second
derivative of a curve in an Autodiff cost function.
Change-Id: Id480096632a731f312be12e294b3d6e244211529
This patch adds the classes Optimizer and OptimizationPass, which
define the core API of the codegen optimization.
A single (trivial) optimization is added that removes NOP
expressions from a graph.
Change-Id: I0430d6ffc61a474f997475ff787a81a9c69cfe23
This patch integrates the code generation module into the build
system. All depenendcies are tracked through CMake targets.
Modifying the cost functor will automatically trigger code
re-generation.
All this functionality is defined in the CMake function
ceres_generate_cost_function_implementation_for_functor
in CeresCodeGeneration.cmake. A hello world usage example
is included in examples/CMakeLists.txt.
Change-Id: I23b8b6698d1ea51cf3d788a47afcf39f8c5ce327
Add the function ExpressionGraph::Erase and a test-case for it.
Erase removes the given expression from the graph by shifting
all later expressions to the front. Indices and references
are updated accordingly.
Change-Id: Ic0449ccf28b369600fd2959a7e2a919d47f4cbe3
The generated function names now include the containing namespace.
For example:
std::abs(...)
std::sin(...)
ceres::Ternary(...)
This patch also fixes the generation of inf/nan compile time constants,
using std::numeric_limits.
Change-Id: I4a36b09c68dd2adabed49fd4f7f37c8229ab7377
- As Problem contains a unique_ptr<ProblemImpl> and as defaulted
methods are declared inline where ProblemImpl's implementation is not
available use of '= default' will generate compile errors on strict
compilers.
- This CL moves the definition of defaulted methods to the
implementation.
Change-Id: I7d79757cf684378fadef8e6f4000c84387018cdb
The move constructor and move =operator are not required. They make
the code more complex and prone to bugs. The few saved assignments
are all trivial and are optimized away by the compiler or our
optimizer.
In fact, there is a bug in the current move-constructor implementation
that occurs, for example, when moving Eigen matrices around.
Change-Id: I013796495bb39f3f27677111bd0aaf49e2454e20
This CL improves the build experience with MSVC:
- It adds the build flag '/bigobj' otherwise the build of
the unit test fails.
- It adds the flag '/wd4267' to suppress signed / unsigned int
conversions (size_t to int).
- It removes the use of std::aligned_storage from FixedArray.
This has been done from the Abseil Team and is in the
absl::FixedArray. Those changes has been ported to ceres.
This fixes the alignemnt for older MSVC versions due to a
bug in the implementation of std::aligned_storage, and
prevents the use of the macro '_ENABLE_EXTENDED_ALIGNED_STORAGE'
for newer MSVC versions, which is problematic as it could affect
user code.
- Fix of the fixed array unit test. Due to the use of std::tuple
instead of absl::tuple in ceres::internal::FixedArray, the
unit test needs to reflect that change as well.
- Replaces 'add_definitions' with 'add_compile_options' for
compiler flags as suggested by the cmake documentation.
Change-Id: I63f08cd6c0a8db8c9931289b909b4deafd75b039
ExpressionId FindMatchingEndif(ExpressionId id)
- Finds the closing ENDIF expression for a given IF expression.
ExpressionId FindMatchingElse(ExpressionId id)
- Finds the ELSE expression for a given IF expression.
- Returns -1, if this IF doesn't have an else
This patch is a preparation to the code analyzing required
for various optimization passes.
Change-Id: I893102a757c6a0bcacbcc1190c2b8ef08314eb97
This CL removes a not used using declaration
which will cause a warning when compiled with
-Wunused-local-typedef under gcc.
Change-Id: Ie4cfd5d1fed9bee860e387545e60d92d257da4f4
The comment expression allows the user to insert single-line
comments into the generated code. These will not be moved or
optimized away.
Comment expressions should make it easier to understand the
generated autodiff code.
Change-Id: Ie87d0698aa6af2aac8a437a13b25e0fef33dbfdc
The autodiff code generator is extended in the following ways:
- Evaluate the complete jacobian into a temporary array
- Copy the temporary jacobian to the result if != NULL
Change-Id: If5ed0bff3b31e93a2d7d5dc718862cb648dba474
Before this patch the return type was implicitly defined by the
ExpressionType. This patch separates this connection and allows
each Expression to have one of the predefined types (scalar,
boolean, void).
This patch is required to add support for the functions isfinite,
isinf, isnan, and isnormal. These are function taking a double and
returning a bool.
This also moves some complexity of the code generator to the
Expression, because the generator can direclty get the c++ type.
Change-Id: I8b32bab1bfab2f668875e506d6f3b789a5d1f3fd
- Move codegen tests to a sub directory
- Add tests for all functions of Expression, ExpressionRef, and ExpressionGraph
- Respect dependencies during tests: The ExpressionGraph test doesn't
use ExpressionRef anymore.
The new tests revealed a few bugs so the following changes were made:
- Expression::MakeNop now resets the current expression with the default
constructed NOP expression
- ExpressionGraph::Insert now updates the lhs_id the same way as
InsertBack()
Change-Id: I6a18925c1e4d972c29ec1219f2073b4eaf2df737
Objects of 'Expression' can now be freely created and copied around.
This creation does NOT add them to the active ExpressionGraph any
more. The insertion into the graph is now done by ExpressionRef,
which explicitly call graph.add(...) in each operation.
This change brings the following advantages:
1. 'Expression' is now stand-alone and side-effect free
- Remove the dependency Expression->ExpressionGraph
- Expressions can be created by the optimizer
2. Explicit graph insertion
- Previously CreateCompileTimeConstant not only created an
expression, but also inserted it into the active graph. Now
this insertion is done explicitly by ExpressionRef
3. It is now easier to insert new types and members
- Should be straight forward now, because we got rid of the
3-way dependency
- This is a preparation patch for the new return-type member in
https://ceres-solver-review.googlesource.com/c/ceres-solver/+/16224
Change-Id: Icef2fe529a4db001a10d1fb6816c9dc681b14ff2
The public API for the Autodiff Codegen system consist of a single
function, GenerateCodeForFunctor. This function takes as template
argument a cost functor type and the residual/parameter structure.
The output is the C++ for the residual and derivative. This class
mainly serves as a wrapper for the different codegen modules.
Change-Id: I8ee974199219805d54eed5e07c0e8d1394940779
User defined cost functors have to include this new header
instead of internal/expression_ref.h. This hides some
complexity of ExpressionRef and reduces compile time outside
of code generation mode.
This patch also removes the dependency ExpressionRef->Jet.
Change-Id: Ie3f93648775e14881dc5cfab213bbc983c6cfeee
The move constructor of ExpressionRef now implements the copy
elision, which was incorrectly done by the copy constructor.
This patch also updates the CMakeList to only add the
Expression tests for gcc. This is currently required, because
the AutoDiffCodeGen system relies on implementation defined
behaviour. (Evalution order + copy elision)
Change-Id: Ib17aeb22bd4d81de6d838b55c1b6497fb3740d0e
Currently there is a bug in the copy constructor of ExpressionRef.
Fixing this bug, will create new expressions each time a copy is
executed. To reduce the amount of copy expression we pass
ExpressionRef by const reference now.
Change-Id: I0f81fb3d08aa4c66b9191f3138650f17023af67e
This functions allows the insertion of new expression anywhere into
the graph. All references are updated. This will be used by semantic
expression testing and by the optimizer.
Change-Id: Ieafcfec7672f6328106dc0511c85d6fb5bd64d97
- Add operator== for comparing Expressions and ExpressionGraphs.
- Remove the broken Jet-test for now. (Will be added again later)
Change-Id: I3ee1599e8a0b50fa61c16f30812c78db7333f273
Due to the removed definition, calling this method will result
in a compile-time error instead of a run-time error.
Change-Id: Iac2514c05d79a66fcbad124587d08eb9309aa6a5
- Remove TERNARY (replaced by function call)
- Combine PARAMETER and RUNTIME_CONSTANT into INPUT_ASSIGNMENT
Change-Id: Id35fb45da08a9f80c1ac0dab7008deb0b3922835
- Add missing copy constructor
- Add make functions for input/output expressions
- Add missing ExpressionRef overload for MakeRuntimeConstant
- Add ExpressionRef overloads for common functions (cos,exp,...)
- Make ExpressionRef constructor from double implicit
- Add simple test for using ExpressionRef in Jets.
- Rename expression test macro
Change-Id: I071dc6717e4034a281662021998a91e80636da27
- Expression types for if/else blocks
- CERES_IF/ELSE macros
- Multiple assignment to the same variable
Change-Id: If529516f243f31823d1ef7b8827bb6f2390e418d
This patch adds the 'Expression' class, which is a fundamental
building block of automatic code generation. The expressions can
be used as scalar types for cost functors as well as Jets.
Dynamic branching is not yet supported.
Change-Id: I8c61bee5c307e0eec20fd39382683ea90f720dff
Now that we are using c++11, it is safer to use override in subclasses
instead of virtual.
This CL does it for the interface, a follow up CL will do it for
other parts of the code base.
Change-Id: Ice8d0f4355cb700019d7a9c1566fbff0099e97d6
https://github.com/ceres-solver/ceres-solver/issues/270
Detailed list of changes:
1. Add SUBSET to the PreconditionerType enum.
2. Add Solver::Options::residual_blocks_for_subset_preconditioner
3. Integrate SubsetPreconditioner into the CGNR solver.
4. Add the reordering logic needed for this to TrustRegionPreprocessor.
5. Expect CreateJacobianBlockTranspose to take the starting row block
so that we can work with subparts of the Jacobian matrix.
6. Extend the denoising example to use this preconditioner.
As an illustration of its performance, we consider the performance of
denoising -input ../data/ceres_noisy.pgm --foe_file ../data/5x5.foe
tl;dr
For the same cost,
SPARSE_NORMAL_CHOLESKY - 81s
CGNR + JACOBI - 718s
CGNR + SUBSET - 57s
SPARSE_NORMAL_CHOLESKY
======================
Cost:
Initial 2.317806e+05
Final 2.232323e+04
Change 2.094574e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 2.999746
Residual only evaluation 2.306811 (10)
Jacobian & residual evaluation 7.421727 (10)
Linear solver 65.517273 (10)
Minimizer 78.731011
Postprocessor 0.026079
Total 81.756836
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.573046e-04 <= 1.000000e-03)
CGNR + JACOBI
=============
Cost:
Initial 2.317806e+05
Final 2.232344e+04
Change 2.094572e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 0.648814
Residual only evaluation 2.297607 (10)
Jacobian & residual evaluation 7.327886 (10)
Linear solver 699.601248 (10)
Minimizer 712.419493
Postprocessor 0.024014
Total 713.092321
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.528538e-04 <= 1.000000e-03)
CGNR + SUBSET (random 20% residuals used for the preconditioner)
===============================================================
Cost:
Initial 2.317806e+05
Final 2.232327e+04
Change 2.094574e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 1.472743
Residual only evaluation 2.428315 (10)
Jacobian & residual evaluation 7.367796 (10)
Linear solver 42.585999 (10)
Minimizer 55.664459
Postprocessor 0.024098
Total 57.161301
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.538277e-04 <= 1.000000e-03)
Change-Id: Ifb011408bd53edbb9439b0b7345649a38f999e18
- Accelerate currently does not support dynamic sparsity, whereas the
other sparse linear algebra libraries do (outstanding issue to update)
- Previously we preferred Accelerate to all but SuiteSparse if it was
available, which breaks the dynamic_sparsity_test if SuiteSparse is
*not* available (even if others are) as Accelerate does not support
dynamic sparsity.
Change-Id: Ibc2dd2f14f83cffbecca38097d02bb2188aaaa05
Some methods in Problem do not modify the parameter block
and those methods now allow the user to call them with const double*.
The methods are
RemoveParameterBlock
SetParameterBlockConstant
IsParameterBlockConstant
GetParameterization
GetParameterLowerBound
GetParameterUpperBound
https://github.com/ceres-solver/ceres-solver/issues/479
Change-Id: I59dcb77134f59576dd498bd732e29aae9abd28b1
The implicitly-declared copy constructor and copy assignment operator
of ProductParameterization are not deleted since the corresponding
functions of the member vector are defined, though ill-formed if they
are instantiated.
With dllexport, the compiler must generate all of the member functions
of the class, including those are implicitly-declared, so we need
explicitly delete the ill-formed ones.
Change-Id: Iaa763dfaefdc74a9840f244ba05c6bfd0bd99492
Darius Rueckert