Up till now ITERATIVE_SCHUR evaluates matrix-vector products
between the Schur complement and a vector implicitly by exploiting
the algebraic expression for the Schur complement.
This cost of this evaluation scales with the number of non-zeros
in the Jacobian.
For small to medium sized problems there is a sweet spot where
computing the Schur complement is cheap enough that it is much
more efficient to explicitly compute it and use it for evaluating
the matrix-vector products.
This changes implements support for an explicit Schur complement
in ITERATIVE_SCHUR in combination with the SCHUR_JACOBI preconditioner.
API wise a new bool Solver::Options::use_explicit_schur_complement
has been added.
The implementation extends the SparseSchurComplementSolver to use
Conjugate Gradients.
Example speedup:
use_explicit_schur_complement = false
Time (in seconds):
Preprocessor 0.585
Residual evaluation 0.319
Jacobian evaluation 1.590
Linear solver 25.685
Minimizer 27.990
Postprocessor 0.010
Total 28.585
use_explicit_schur_complement = true
Time (in seconds):
Preprocessor 0.638
Residual evaluation 0.318
Jacobian evaluation 1.507
Linear solver 5.930
Minimizer 8.144
Postprocessor 0.010
Total 8.791
Which indicates an end-to-end speedup of more than 3x, with the linear
solver being sped up by > 4x.
The idea to explore this optimization was inspired by the recent paper:
Mining structure fragments for smart bundle adjustment
L. Carlone, P. Alcantarilla, H. Chiu, K. Zsolt, F. Dellaert
British Machine Vision Conference, 2014
which uses a more complicated algorithm to compute parts of the
Schur complement to speed up the matrix-vector product.
Change-Id: I95324af0ab351faa1600f5204039a1d2a64ae61d
This moves a couple of routines from solver.cc into solver_utils.h/cc
so that they can also be used by the upcoming GradientProblemSolver.
Change-Id: I627b32ad3dc639422aacde78a8e391459d947e99
The header of Summary::FullReport now looks like
Solver Summary (v 1.10.0-suitesparse-cxsparse-lapack-no_openmp)
Original Reduced
Parameter blocks 22122 22122
Parameters 66462 66462
Residual blocks 83718 83718
Residual 167436 167436
Change-Id: Id1b81bbf90ba412d19e2dd3687eeb9d372b72c1b
1. Allow the minimum number of linear solver iterations to be zero.
2. Fix conjugate gradients solver's iteration loop to be sane again.
Change-Id: I8594815fec940c2b30e28eb58ec5d8baacf13dae
TrustRegionPreprocessor was not setting Minimizer::Options::is_constrained.
This meant that the line search for bounds constraints was not being
invoked for bounds constrained problems.
And some minor lint cleanup.
Change-Id: I18852cfaf1b33fd90b7d8c196f2063c128126658
Replace SolverImpl with
a. A minimizer specific preprocessor class.
b. A generic Solve function inside solver.cc
c. Presummarize and Postsummarize functions to handle
updates to the summary object.
The existing SolverImpl class was a mixture of the above three
things and was increasingly complicated code to follow. This change,
breaks it into its three separate constituents, with the aims of
better separation of concerns and thus better testability and
reliability.
The call to Solver::Solve() now consists of
1. Presummarize - summarize the given state of the problem and solver
options.
2. Preprocess - Setup everything that is needed to call the minimizer.
This includes, removing redundant parameter and residual blocks,
setting up the reordering for the linear solver, creating the
linear solver, evaluator, inner iteration minimizer etc.
3. Minimize.
4. Post summarize - summarize the result of the preprocessing and the
solve.
Change-Id: I80f35cfc9f2cbf78f1df4aceace27075779d8a3a
A number of typos in solver.cc which were hidden behind ifdefs
and were only revealed when building under android.
Change-Id: I89cbe8056da071479d2d1a251a8919b0be1f1cd4
This replaces the broken CERES_VERSION and CERES_ABI_VERSION
defines with a different set, including integer versions for
MAJOR/MINOR/etc.
This also adds the Ceres version to Solver::FullReport().
Example report from powell:
Ceres Solver v1.10.0 Solve Report
----------------------------------
Original Reduced
Parameter blocks 4 4
Parameters 4 4
Residual blocks 4 4
Residual 4 4
Minimizer TRUST_REGION
Dense linear algebra library EIGEN
Trust region strategy LEVENBERG_MARQUARDT
Given Used
Linear solver DENSE_QR DENSE_QR
Threads 1 1
Linear solver threads 1 1
Cost:
Initial 1.075000e+02
Final 1.791438e-14
Change 1.075000e+02
Minimizer iterations 14
Successful steps 14
Unsuccessful steps 0
Time (in seconds):
Preprocessor 0.001
Residual evaluation 0.000
Jacobian evaluation 0.000
Linear solver 0.000
Minimizer 0.001
Postprocessor 0.000
Total 0.003
Change-Id: I5bf0e8023693e9195276b1f1e881b13121ba1196
Termination: CONVERGENCE (Gradient tolerance reached. Gradient max norm: 3.642190e-11 <= 1.000000e-10)
Solver::Options::linear_solver_ordering and
Solver::Options::inner_iteration_ordering
were bare pointers even though Solver::Options took ownership of these
objects.
This lead to buggy user code and the inability to copy Solver::Options
objects around.
With this change, these naked pointers have been replaced by a
shared_ptr object which will managed the lifetime of these objects. This
also leads to simplification of the lifetime handling of these objects
inside the solver.
The Android.mk and Application.mk files have also been updated
to use a newer NDK revision which ships with LLVM's libc++.
Change-Id: I25161fb3ddf737be0b3e5dfd8e7a0039b22548cd
The original implementation for computing the norm of the gradient was
gradient_norm = norm(gradient)
when the gradient vector lies in the same space as the parameter
vector, this value is meaningful. When there is a local parameterization
involved, interpreting this value and diagnosing convergence using it
is hard.
Further, this expression does not respect the bounds constraints
on the parmeters. Measuring the norm of the gradient only makes
sense when the optimization being performed is unconstrained.
A better solution, used by LANCELOT is the expression
gradient_norm = norm(x - P(x - gradient))
Here, P is the projection operator onto the bounds constraints.
x - gradient is computed by computing Plus(x, -gradient), thus the
actual expression becomes
gradient_norm = norm(x - P(Plus(x, -gradient)));
Which in the case where there are no bounds constraints, and there
are no local parameterizations, reduces to the usual Euclidean
expression from above, since
Plus(x, -gradient) = x - gradient
and P(x - gradient) = x - gradient.
This change implements this change. Further, the convergence
test using the gradient tolerance now uses an absolute measure
rather than a relative measure. This is a forward looking change
as we start implementing the Augmented Lagrangian solver.
Last but not the least, various "Terminating: Foo" messages
have been changed so that "Terminating: " is logged but is
not part of the Solver::Summary::message string as it is
pointless.
Change-Id: I943146f71a1da47c8c7592986039b4112781b99b
1. Rename SolverTerminationType to TerminationType.
2. Consolidate the enum as
a. CONVERGENCE - subsumes FUNCTION_TOLERANCE, PARAMETER_TOLERANCE and GRADIENT_TOLERANCE
b. NO_CONVERGENCE
c. FAILURE - captures all kinds of failures including DID_NOT_RUN.
d. USER_SUCCESS
e. USER_FAILURE
3. Solver::Summary::error is renamed to be Solver::Summary::message, to both
reduce confusion as well as capture its true meaning.
Change-Id: I27a382e66e67f5a4750d0ee914d941f6b53c326d
- Increase precision of numeric values output in error messages to
allow for easier debugging.
- Ensure termination after Wolfe search bracketing phase if bracket
width has been shrunk to below tolerance.
- Cleaned up return value for BracketingPhase(), now false iff
optimisation should stop, true otherwise.
- Fix bug whereby we would mark a step size as satisfying the Wolfe
conditions when it did not due to numerical issues in the cost
function.
- Adding explanation of a subtlety in which a zoom could still be
acceptably invoked with bracket_low.f > bracket_high.f.
- Replacing hard check of a pre-condition of ZoomPhase() with a
conditional return if not satisfied to address issue whereby a
bracket could be incorrectly identified due to inconsistent values
& gradients returned from the cost function.
- Adding missing check for step size validity in line search minimizer.
- Adding ToDebugString() for FunctionSample.
Change-Id: Iad98e635749877f80c079ebad126bf022d82232d
The original visibility based preconditioning paper and
implementation only used the canonical views algorithm.
This algorithm for large dense graphs can be particularly
expensive. As its worst case complexity is cubic in size
of the graph.
Further, for many uses the SCHUR_JACOBI preconditioner
was both effective enough while being cheap. It however
suffers from a fatal flaw. If the camera parameter blocks
are split between two or more parameter blocks, e.g,
extrinsics and intrinsics. The preconditioner because
it is block diagonal will not capture the interactions
between them.
Using CLUSTER_JACOBI or CLUSTER_TRIDIAGONAL will fix
this problem but as mentioned above this can be quite
expensive depending on the problem.
This change extends the visibility based preconditioner
to allow for multiple clustering algorithms. And adds
a simple thresholded single linkage clustering algorithm
which allows you to construct versions of CLUSTER_JACOBI
and CLUSTER_TRIDIAGONAL preconditioners that are cheap
to construct and are more effective than SCHUR_JACOBI.
Currently the constants controlling the threshold above
which edges are considered in the single linkage algorithm
are not exposed. This would be done in a future change.
Change-Id: I7ddc36790943f24b19c7f08b10694ae9a822f5c9
1. When a LAPACK implementation is present, then
DENSE_QR, DENSE_NORMAL_CHOLESKY and DENSE_SCHUR
can use it for doing dense linear algebra operations.
2. The user can switch dense linear algebra libraries
by setting Solver::Options::dense_linear_algebra_library_type.
3. Solver::Options::sparse_linear_algebra_library is now
Solver::Options::sparse_linear_algebra_library_type to be consistent
with all the other enums in Solver::Options.
4. Updated documentation as well as Solver::Summary::FullReport
to reflect these changes.
Change-Id: I5ab930bc15e90906b648bc399b551e6bd5d6498f
Also
1. Remove an inadvertent LOG(INFO) from trust_region_minimizer.cc
2. Refactor some of the code in FullReport to reduce duplication
across line search and trust region minimizers.
3. Consistent capitalization.
Change-Id: I9078b1704efab23d2858530636f524e60c7d9016
Here is an example report, obtained by running:
bin/Debug/bundle_adjuster \
--input=../ceres-solver/data/problem-16-22106-pre.txt \
--linear_solver=iterative_schur \
--num_iterations=1 \
--alsologtostderr \
--use_local_parameterization \
--use_quaternions
Note that effective parameters is less than parameters by 16, which is the
number of cameras. In this case the local parameterization has a 3 dimensional
tangent space for the 4-dimensional quaternions.
Ceres Solver Report
-------------------
Original Reduced
Parameter blocks 22138 22138
Parameters 66478 66478
Effective parameters 66462 66462
Residual blocks 83718 83718
Residual 167436 167436
Minimizer TRUST_REGION
Trust Region Strategy LEVENBERG_MARQUARDT
Given Used
Linear solver ITERATIVE_SCHUR ITERATIVE_SCHUR
Preconditioner JACOBI JACOBI
Threads: 1 1
Linear solver threads 1 1
Linear solver ordering AUTOMATIC 22106, 32
Cost:
Initial 4.185660e+06
Final 7.221647e+04
Change 4.113443e+06
Number of iterations:
Successful 1
Unsuccessful 0
Total 1
Time (in seconds):
Preprocessor 0.697
Residual Evaluations 0.063
Jacobian Evaluations 27.608
Linear Solver 13.360
Minimizer 43.973
Postprocessor 0.004
Total 44.756
Termination: NO_CONVERGENCE
Change-Id: I6b6b8ac24f71bd187e67d95651290917642be74f
1. Added a Preconditioner interface.
2. SCHUR_JACOBI is now its own class and is independent of
SuiteSparse.
Change-Id: Id912ab19cf3736e61d1b90ddaf5bfba33e877ec4
1. Add an ExecutionSummary object to record execution
information about Ceres objects.
2. Add an EventLogger object to log events in a function call.
3. Add a ScopedExecutionTimer object to log times in ExecutionSummary.
4. Instrument ProgramEvaluator and all the linear solvers
to report their timing statistics.
5. Connect the timing statistics to Summary::FullReport.
6. Add high precision timer on unix systems using
gettimeofday() call.
7. Various minor clean ups all around.
Change-Id: I5e09804b730b09535484124be7dbc1c58eccd1d4
Add automatic recursive independent set decomposition.
Clean up the naming and the API for inner iterations.
Change-Id: I3d7d6babb9756842d7367e14b7279d2df98fb724
Remove the old ordering API, and modify solver_impl.cc
to use the new API everywhere.
In the process also clean up the linear solver instantion
logic in solver_impl.cc a bit too.
Change-Id: Ia66898abc7f622070b184b21fce8cc6140c4cebf
This fixes the bug introduced in a previous commit,
and adds a test to check that constant parameter
blocks work as expected.
This also refactors the Solver/SolverImpl split so
that SolverImpl is no longer a friend of Problem;
instead, Solver is. This makes it possible to
verify the invariant on parameter block states in
the unit test, and is a more symmetric design
anyway.
Bug: 51
Change-Id: Id503f5b526cfb8bc24aae3aaad2e414b14063d78
1. Document the use of dogleg and a general discussion of
trust region methods.
2. Added a TBD section on compiler/linker flags.
3. Summary::FullReport now prints out sparse_linear_algebra_library
and trust_region_strategy_type.
Change-Id: I01f680070d510715900f345364855689005d54bb
Sameer Agarwal