Up till now ITERATIVE_SCHUR evaluates matrix-vector products
between the Schur complement and a vector implicitly by exploiting
the algebraic expression for the Schur complement.
This cost of this evaluation scales with the number of non-zeros
in the Jacobian.
For small to medium sized problems there is a sweet spot where
computing the Schur complement is cheap enough that it is much
more efficient to explicitly compute it and use it for evaluating
the matrix-vector products.
This changes implements support for an explicit Schur complement
in ITERATIVE_SCHUR in combination with the SCHUR_JACOBI preconditioner.
API wise a new bool Solver::Options::use_explicit_schur_complement
has been added.
The implementation extends the SparseSchurComplementSolver to use
Conjugate Gradients.
Example speedup:
use_explicit_schur_complement = false
Time (in seconds):
Preprocessor 0.585
Residual evaluation 0.319
Jacobian evaluation 1.590
Linear solver 25.685
Minimizer 27.990
Postprocessor 0.010
Total 28.585
use_explicit_schur_complement = true
Time (in seconds):
Preprocessor 0.638
Residual evaluation 0.318
Jacobian evaluation 1.507
Linear solver 5.930
Minimizer 8.144
Postprocessor 0.010
Total 8.791
Which indicates an end-to-end speedup of more than 3x, with the linear
solver being sped up by > 4x.
The idea to explore this optimization was inspired by the recent paper:
Mining structure fragments for smart bundle adjustment
L. Carlone, P. Alcantarilla, H. Chiu, K. Zsolt, F. Dellaert
British Machine Vision Conference, 2014
which uses a more complicated algorithm to compute parts of the
Schur complement to speed up the matrix-vector product.
Change-Id: I95324af0ab351faa1600f5204039a1d2a64ae61d
1. Extend the implementation of BlockRandomAccessDiagonalMatrix
by adding Invert and RightMultiply methods.
2. Simplify the implementation of the Schur Jacobi preconditioner
using these new methods.
3. Replace the custom storage used inside Block Jacobi preconditioner
with BlockRandomAccessDiagonalMatrix and simplify its implementation
too.
Change-Id: I9d4888b35f0f228c08244abbdda5298b3ce9c466
- CreateBlockJacobian() is only ever used when Eigen >= 3.2.2 is
detected, but was previously defined whenever CERES_USE_EIGEN_SPARSE
was defined with no check on the Eigen version.
- This resulted in an unused-function compile warning that became an
error due to -Werror, preventing compilation when using Eigen < 3.2.2.
Change-Id: I24628ff329f14b087ece66bf2626bdc0de4ba224
The line search minimizer in Ceres does not require that the
problems that is solving is a sum of squares. Over the past
year there have been multiple requests to expose this algorithm
on its own so that it can be used to solve unconstrained
non-linear minimization problems on its own.
With this change, a new optimization problem called
GradientProblem is introduced which is basically a thin
wrapper around a user defined functor that evaluates cost
and gradients (FirstOrderFunction) and an optional LocalParameterization.
Corresponding to it, a GradientProblemSolver and its associated
options and summary structs are introduced too.
An example that uses the new API to find the minimum of Rosenbrock's
function is also added.
Change-Id: I42bf687540da25de991e9bdb00e321239244e8b4
Problem::GetCostFunctionForResidualBlock
Problem::GetLossFunctionForResidualBlock
are added, so that users do not have to maintain this mapping
outside the Problem.
Change-Id: I38356dfa094b2c7eec90651dafeaf3a33c5f5f56
The row/column blocks can be huge when using dynamic sparsity. This
can result in very large memory usage when augmenting the jacobian
with the LM diagonal.
Thanks to Mingsong Dou for reporting this.
Change-Id: I6aa140ceefa98389ae17958f89ca76e0c76f95b8
This moves a couple of routines from solver.cc into solver_utils.h/cc
so that they can also be used by the upcoming GradientProblemSolver.
Change-Id: I627b32ad3dc639422aacde78a8e391459d947e99
Change the logic for how IterationSummary objects are added to
Summary::iterations to match the one in TrustRegionMinimizer.
Change-Id: I57851ad8294e58f83b9115cca9c24695d86ee92a
Its API was broken, and its implementation was an unnecessary
layer of abstraction over CostFunctionToFunctor.
Change-Id: I18fc261fc6a3620b51a9eeb4dde0af03d753af69
The code itself was perfectly capable of handling residuals, but there
was an overly strict runtime check that had to be removed.
Thanks to Domink Reitzle for reporting this.
Change-Id: I6a6d000a7c5203dd5945a61b4caeda1b8aeb09c9
Ordering routines for the Schur complement when using EIGEN_SPARSE.
Also integration into SchurComplementSolver.
Part of this CL is also a refactoring of the block jacobian matrix
construction.
Change-Id: I11d665cc7d4867c64190e6fed1118f4d2e13d59b
This is just the reordering routine. The integration with
SparseNormalCholesky shall happen in a subsequent CL.
Change-Id: I39ddc32aa66b11c368faf75404850fa0ae0d2b3a
Program::SetParameterOffsetsAndIndex() was being called willy nilly.
Now the invariant is that any function that actually reorders the
program, updates the offsets and indices.
Also the logic around handling EIGEN_SPARSE has been simplified in
anticipation of the block AMD code that is forthcoming.
Last but not the least, num_eliminate_blocks, which is a rather
cryptic name to begin with has been replaced by the more meaningful
size_of_first_elimination_group.
Change-Id: I77e684f699a93b53e76aa406d64f40f8704df813
The header of Summary::FullReport now looks like
Solver Summary (v 1.10.0-suitesparse-cxsparse-lapack-no_openmp)
Original Reduced
Parameter blocks 22122 22122
Parameters 66462 66462
Residual blocks 83718 83718
Residual 167436 167436
Change-Id: Id1b81bbf90ba412d19e2dd3687eeb9d372b72c1b
The LinearSolver factory was creating a NULL linear solver
if only Eigen's sparse linear algebra backend was available.
Thanks to Michael Samples and Domink Reitzle for reporting this.
Change-Id: I35e3a6c0fd0da2a31934adb5dfe4cad29577cc73
If the parameter block size is 1, asking Eigen to create
a row-major matrix triggers a compile time error. Previously
we were handling the case where the number of rows in the
jacobian block was known statically, but the problem is present
when the nummber of rows is dynamic.
This CL fixes this problem.
Thanks to Dominik Reitzle for reporting this.
Change-Id: I99c3eec3558e66ebf4efa51c4dee8ce292ffe0c1
- Previously if options.num_threads > 1 was given to Covariance compiled
without OpenMP, a CHECK() would be triggered in program_evalutor.
Change-Id: Iaade4f5ed5326b0c59a7014c750c41ee026e1124
1. Allow the minimum number of linear solver iterations to be zero.
2. Fix conjugate gradients solver's iteration loop to be sane again.
Change-Id: I8594815fec940c2b30e28eb58ec5d8baacf13dae
1. Fix a build breakage in graph_test.
2. Respect Solver::Options::min_num_linear_solver_iterations in
conjugate_gradients_solver.cc
Thanks to Johannes Schönberger for reporting these.
Change-Id: Ib32e3929bf5d92dd576ae5b53d4d88797095136e
Rename Graph -> WeightedGraph.
Add a new Graph class, which is cheaper to construct and
work with if the weights are not needed.
This cuts down the cost of building the Hessian graph
significantly.
Change-Id: Id0cfc81dd2c0bb5ff8f63a1b55aa133c53c0c869
I got the following error with MSVC:
Syntax error at index 9 in simple regular expression "NumGroups()": '(' is unsupported.
Change-Id: Id1952831d81d3eb5d73bbed8c311914c4c8ab51f
Alternatively, if quiet_NaN is not available on all platforms a workaround would be:
volatile double zero = 0.0;
double x = 1.0/zero;
The 'volatile' is needed to shut up "warning C4723: potential divide by 0".
Change-Id: If2bbdab8540595aa2e0079e1eb6b6fed6d4a6ef7
TrustRegionPreprocessor was not setting Minimizer::Options::is_constrained.
This meant that the line search for bounds constraints was not being
invoked for bounds constrained problems.
And some minor lint cleanup.
Change-Id: I18852cfaf1b33fd90b7d8c196f2063c128126658
Replace SolverImpl with
a. A minimizer specific preprocessor class.
b. A generic Solve function inside solver.cc
c. Presummarize and Postsummarize functions to handle
updates to the summary object.
The existing SolverImpl class was a mixture of the above three
things and was increasingly complicated code to follow. This change,
breaks it into its three separate constituents, with the aims of
better separation of concerns and thus better testability and
reliability.
The call to Solver::Solve() now consists of
1. Presummarize - summarize the given state of the problem and solver
options.
2. Preprocess - Setup everything that is needed to call the minimizer.
This includes, removing redundant parameter and residual blocks,
setting up the reordering for the linear solver, creating the
linear solver, evaluator, inner iteration minimizer etc.
3. Minimize.
4. Post summarize - summarize the result of the preprocessing and the
solve.
Change-Id: I80f35cfc9f2cbf78f1df4aceace27075779d8a3a
1. Base class for preprocessors.
2. A preprocessor for problems that will be solved using
the trust region minimizer.
3. Added sanity tests to the program reordering options
for Schur type linear solvers.
4. Tests for the TrustRegionPreprocessor.
Change-Id: I88cd926f0053bbbf2bd6b11e03ec55b8bf473cf1
This is a intermediate change to clean things up
in preparation for a broader refactoring of the SolverImpl.
Essentially we are replacing raw pointers in Minimizer::Options
with shared_ptr objects. For now this only makes things a bit
more complicated looking inside solver_impl.cc, but going
forward this will lead to considerable simplifications in
tracking ownership of various pointers.
Change-Id: I21db8fc6763c29b0d15e834d7c968a0f514042a0
Ensure that when a new problem object is constructed for validing
gradients, the parameter blocks have their data pointers point to
the user's parameter blocks.
We used to do this inside solver_impl.cc, but doing this at
construction is the right thing to do.
Change-Id: I3bfdc89bb0027c8d67cde937e8f2fa385d89c30c
Sameer Agarwal