With this change, the user can now choose between Approximate Minimum
Degree and Nested Dissection as a fill reducing algorithm when using
a sparse direct factorization based linear solver like SPARSE_NORMAL_CHOLESKY
or SPARSE_SCHUR.
Currenly only SUITE_SPARSE is supported. It requires that
SuiteSparse be compiled with Metis support enabled.
On most problems AMD is still the better choice, but in some cases
like the grid3D dataset from https://lucacarlone.mit.edu/datasets/
the solution time with AMD is 57s and with NESDIS 38 on my M1 Mac.
On some other problems at Google we have observed speedups of 10x,
there is also a corresponding decrease in the total amount of memory
used.
This patch is based on the original work done by NeroBurner in
https://ceres-solver-review.googlesource.com/c/ceres-solver/+/20580
1. Add a new enum to the public api LinearSolverOrderingType and
a setting Solver::Options::linear_solver_ordering_type.
2. TrustRegionPreprocessor had some complicated logic which determined
when linear solvers should reorder their matrices on their own and not
this has been refactored into a more readable function that lives
inside reorder_program.h/cc.
3. Plumbing in reorder_program.cc and trust_region_processor.cc to use
nested dissection.
4. Update bundle_adjuster.cc to use nested dissection.
Change-Id: I388b027934f86c58b4da2b65a4fa5204ea73bf40
Currently, the logic for exporting symbols is rather complicated: when
tests are enabled internal symbols are exported in addition to the
public symbols. Such logic causes several problems. (1) Test binaries
link against a Ceres build that is different from the final release
since fewer optimizations are applied if more symbols are exported. (2)
Also, some toolchains hide symbols by default breaking the existing
logic eventually causing linker errors.
Since internal symbols are not intended to be used outside of the
project, we can compile them into object files and use exactly the same
binary code both for the final build and the tests without relying on
conditionals.
By default, all symbols are now hidden unless annotated as public.
Internal symbols are explicitly marked as not being exported in case
users chose not to hide symbols by default.
Change-Id: I589dd10be2f6f438508783cf99d141af0120057b
1. Add CUDADenseCholesky64Bit, CUDADenseCholesky32Bit, & tests.
CUDADenseCholesky32Bit uses the legacy versions of potrf/potrs
in cuSolverDN, while CUDADenseCholesky64Bit uses the new 64-bit
versions available since Cuda 11.1. The legacy versions are
provided since some platforms such as the Nvidia Jetsons only
support Cuda 10.2.
2. Expose CUDA as a new option under DenseLinearAlgebraLibraryType.
The relevant option to string and string to option helper functions
are modified accordingly.
3. Add cuda as a dense_linear_algebra_library option in bundle_adjuster
to demonstrate the use of the new CUDA option.
Change-Id: I23615e1d301df5185ed646b3e33ee802508dae86
- ensure all public headers files adhere to clang-format
- preserve one-per-line for enums by adding trailing comma
- preserve include order for en/disable_warning.h
Change-Id: I78dbd0527a294ab2ec5f074fb426e48b20c393e6
https://github.com/ceres-solver/ceres-solver/issues/270
Detailed list of changes:
1. Add SUBSET to the PreconditionerType enum.
2. Add Solver::Options::residual_blocks_for_subset_preconditioner
3. Integrate SubsetPreconditioner into the CGNR solver.
4. Add the reordering logic needed for this to TrustRegionPreprocessor.
5. Expect CreateJacobianBlockTranspose to take the starting row block
so that we can work with subparts of the Jacobian matrix.
6. Extend the denoising example to use this preconditioner.
As an illustration of its performance, we consider the performance of
denoising -input ../data/ceres_noisy.pgm --foe_file ../data/5x5.foe
tl;dr
For the same cost,
SPARSE_NORMAL_CHOLESKY - 81s
CGNR + JACOBI - 718s
CGNR + SUBSET - 57s
SPARSE_NORMAL_CHOLESKY
======================
Cost:
Initial 2.317806e+05
Final 2.232323e+04
Change 2.094574e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 2.999746
Residual only evaluation 2.306811 (10)
Jacobian & residual evaluation 7.421727 (10)
Linear solver 65.517273 (10)
Minimizer 78.731011
Postprocessor 0.026079
Total 81.756836
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.573046e-04 <= 1.000000e-03)
CGNR + JACOBI
=============
Cost:
Initial 2.317806e+05
Final 2.232344e+04
Change 2.094572e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 0.648814
Residual only evaluation 2.297607 (10)
Jacobian & residual evaluation 7.327886 (10)
Linear solver 699.601248 (10)
Minimizer 712.419493
Postprocessor 0.024014
Total 713.092321
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.528538e-04 <= 1.000000e-03)
CGNR + SUBSET (random 20% residuals used for the preconditioner)
===============================================================
Cost:
Initial 2.317806e+05
Final 2.232327e+04
Change 2.094574e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 1.472743
Residual only evaluation 2.428315 (10)
Jacobian & residual evaluation 7.367796 (10)
Linear solver 42.585999 (10)
Minimizer 55.664459
Postprocessor 0.024098
Total 57.161301
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.538277e-04 <= 1.000000e-03)
Change-Id: Ifb011408bd53edbb9439b0b7345649a38f999e18
This commit includes the following:
- Changes to CMake to make it safer to see which compiler flags are supported,
so this way we do not need to worry about version checks in CMake.
- Unix platforms (which includes both Linux and Apple as far as i can tell)
will now enable -Wmissing-declarations warning for the whole Ceres.
- Changes in all sources to solve missing declaration warning. In most cases
it was either matter of using static qualifier or moving functions to an
anonymous namespace.
In one case the function got removed, since it seems to be unused.
Additionally, in slam examples there was a non-inlined function implementation
in a header, which is a direct way to cause linking errors if other .cc file
will include that helper header.
- All third party sources (which is currently only gmock) has this extra
paranoid warning disabled.
This warning is important in the following cases:
- Detect helper functions which are not needed anymore.
- Avoid unnoticed pollution of namespace.
- Avoid bad level calls.
- Avoid missing updates in header files after changes in implementation file.
- Helps integrating Ceres into software where paranoid warnings are important.
Change-Id: I9b1044aced3910d8c6b2356cfe2bf57f3c8c58db
This header defined integral types in the pre-C++11 days, and can
be replaced with <cstdint> and the types defined therein.
Also remove a shallow (and incorrect) typedef in include/ceres/types.h
https://github.com/ceres-solver/ceres-solver/issues/409
Change-Id: I398c652f74d24bbeea459672508bf28f591b100f
THIS IS AN API BREAKING CHANGE.
Decouple the algorithm from the sparse linear algebra
library being used to perform the computation.
Before this change
Covariance::AlgorithmType had values
DENSE_SVD
EIGEN_SPARSE_QR
SUITE_SPARSE_QR
This has been replaced by two enums now.
Covariance::Options::sparse_linear_algebra_library_type
which can take values EIGEN_SPARSE, SUITE_SPARSE or CX_SPARSE.
The last one is currently not supported.
And Covariance::Options::algorithm_type takes values
DENSE_SVD
SPARSE_QR
This sets the stage for future extensions of the covariance
computation algorithm.
Also as part of this change, the covariance computation chapter
has been made a top level chapter on its own instead of being
buried deep inside the Solving Non-linear Least Squares problem.
Change-Id: Ibfbf60902d8d17694d9ff585047a5a57d329ab22
ResidualBlock evaluation has logic to ensure that CostFunction
should always fill out the residual and jacobian arrays completely
by using a special value to pre-populate these arrays.
This works for CostFunctions with analytical Jacobians but not for
AutoDiffCostFunction and NumericDiffCostFunction Jacobians.
There is no way to fix this for NumericDiffCostFunctions without
introducing significant performance penalties but the residual
evaluation fails, which should be enough to catch such errors.
For AutoDiffCostFunction the way the Jets are default initialized
was sidestepping this check. So now, the Jet that is used to
capture the output residuals is now initialized with
kImpossibleValue, which will ensure that if the user forgets
to fill all output fields, it triggers an evaluation error.
This change required that ceres::internal::kImpossibleValue be moved
out of array_utils.h/cc to types.h.
Change-Id: I35bb0946cf0785a5d43c7b5459a2272848fb2a9b
This method numerically computes function derivatives in different
scales, extrapolating between intermediate results to conserve function
evaluations. Adaptive differentiation is essential to produce accurate
results for functions with noisy derivatives.
Full changelist:
-Created a new type of NumericDiffMethod (RIDDERS).
-Implemented EvaluateRiddersJacobianColumn in NumericDiff.
-Created unit tests with f(x) = x^2 + [random noise] and
f(x) = exp(x).
Change-Id: I2d6e924d7ff686650272f29a8c981351e6f72091
Since Ceres is moving to using GitHub for issues, and the Google
Code URL in the current copyright header will soon become invalid,
update all the headers.
Change-Id: I1fce70375d1bcf098591f07b4d8f01a5c1e0789c
- Previously we had no defined default value for
sparse_linear_algebra_library_type in Solver::Options if Ceres
was compiled with no sparse library available. Thus in that case,
the default value (dependent upon the compiler) would indicate that
one was available.
- Now we have an explicit option that means no sparse library is
available, which is now the default value in Solver::Options in this
case.
- Add a warning in CMake when the user disables all sparse libraries.
- Fix typos in trust_region_preprocessor_test:
(SUITE/CX)_SPARSE -> (SUITE/CX)SPARSE that induced failures when
no sparse libraries were available.
Change-Id: I869c399a12d42bfc44220cbb25ce6d6dd80236bd
SPARSE_NORMAL_CHOLESKY and SPARSE_SCHUR can now be used
with EIGEN_SPARSE as the backend.
The performance is not as good as CXSparse. This needs to be
investigated. Is it because the quality of AMD ordering that
we are computing is not as good as the one for CXSparse? This
could be because we are working with the scalar matrix instead
of the block matrix.
Also, the upper/lower triangular story is not completely clear.
Both of these issues will be benchmarked and tackled in the
near future.
Also included in this change is a bunch of cleanup to the
SparseNormalCholeskySolver and SparseSchurComplementSolver
classes around the use of the of defines used to conditionally
compile out parts of the code.
The system_test has been updated to test EIGEN_SPARSE also.
Change-Id: I46a57e9c4c97782696879e0b15cfc7a93fe5496a
Sparse Cholesky factorization is not rank revealing. Therefore
this algorithm cannot reliably tell when the Jacobian matrix is
rank deficient or so poorly conditioned that the covariance matrix
cannot be estimated.
Making things worse, this algorithm works on the normal equations,
which makes the conditioning problem much worse.
This change, deletes the SPARSE_CHOLESKY algorithm in the covariance
estimation code. Also to make the naming consistent, it renames
SPARSE_QR -> SUITE_SPARSE_QR
so that it parallels EIGEN_SPARSE_QR.
Also, since we now have EIGEN_SPARSE_QR, we can default to using
it when SuiteSparse is not available instead of DENSE_SVD, which
generally speaking should only be used by folks who are dealing
with small rank deficient jacobians.
Change-Id: I8b134c7e8a2e86ca374371f185b19f1c3e74349c
For smaller problems Eigen is faster than SuiteSparseQR. This has been
tested with Eigen 3.2.1. Below are detailed timings. Problem 1 is the
smallest and problem 3 is the largest. The timings below are:
mean +- standard deviation.
Problem 1:
Eigen 0.0009218 +- 0.0002755
SuiteSparse 0.001406 +- 0.001610
Problem 2:
Eigen 0.002338 +- 0.001005
SuiteSparse 0.001910 +- 0.0004513
Problem 3:
Eigen 0.005455 +- 0.001759
SuiteSparse 0.002411 +- 0.0004974
Detailed problem descriptions:
Problem 1 size:
Original Reduced
Parameter blocks 533 54
Parameters 368 104
Effective parameters 1201 94
Residual blocks 233 77
Residual 1194 258
Problem 2 size:
Original Reduced
Parameter blocks 573 84
Parameters 1458 184
Effective parameters 1281 164
Residual blocks 263 107
Residual 1314 378
Problem 3 size:
Original Reduced
Parameter blocks 613 114
Parameters 1548 264
Effective parameters 1361 234
Residual blocks 293 137
Residual 1434 498
Change-Id: I884a67e2f728fe2992812148d82ccf5f27864fd7
1. Update version history.
2. Minor changes to the tutorial to reflect the bounds constrained
problem.
3. Added static factory methods to the SnavelyReprojectionError.
4. Removed relative gradient tolerance from types.h as it is
not true anymore.
Change-Id: I8de386e5278a008c84ef2d3290d2c4351417a9f1
1. Rename SolverTerminationType to TerminationType.
2. Consolidate the enum as
a. CONVERGENCE - subsumes FUNCTION_TOLERANCE, PARAMETER_TOLERANCE and GRADIENT_TOLERANCE
b. NO_CONVERGENCE
c. FAILURE - captures all kinds of failures including DID_NOT_RUN.
d. USER_SUCCESS
e. USER_FAILURE
3. Solver::Summary::error is renamed to be Solver::Summary::message, to both
reduce confusion as well as capture its true meaning.
Change-Id: I27a382e66e67f5a4750d0ee914d941f6b53c326d
1. Move LinearSolverTerminationType to ceres::internal.
2. Add FATAL_ERROR as a new enum to LinearSolverTerminationType.
3. Pipe SuiteSparse errors via a LinearSolverTerminationType so
to distinguish between fatal and non-fatal errors.
4. Update levenberg marquardt and dogleg strategies to deal
with FATAL_ERROR.
5. Update trust_region_minimizer to terminate when FATAL_ERROR
is encountered.
6. Remove SuiteSparse::SolveCholesky as it screws up the error
handling.
7. Fix all clients calling SuiteSparse to handle the result of
SuiteSparse::Cholesky correctly.
8. Remove fatal failures in SuiteSparse when symbolic factorization
fails.
9. Fix all clients of SuiteSparse to deal with null symbolic factors.
This is a temporary fix to deal with some production problems. A more
extensive cleanup and testing regime will be put in place in a
subsequent CL.
Change-Id: I1f60d539799dd95db7ecc340911e261fa4824f92
The original visibility based preconditioning paper and
implementation only used the canonical views algorithm.
This algorithm for large dense graphs can be particularly
expensive. As its worst case complexity is cubic in size
of the graph.
Further, for many uses the SCHUR_JACOBI preconditioner
was both effective enough while being cheap. It however
suffers from a fatal flaw. If the camera parameter blocks
are split between two or more parameter blocks, e.g,
extrinsics and intrinsics. The preconditioner because
it is block diagonal will not capture the interactions
between them.
Using CLUSTER_JACOBI or CLUSTER_TRIDIAGONAL will fix
this problem but as mentioned above this can be quite
expensive depending on the problem.
This change extends the visibility based preconditioner
to allow for multiple clustering algorithms. And adds
a simple thresholded single linkage clustering algorithm
which allows you to construct versions of CLUSTER_JACOBI
and CLUSTER_TRIDIAGONAL preconditioners that are cheap
to construct and are more effective than SCHUR_JACOBI.
Currently the constants controlling the threshold above
which edges are considered in the single linkage algorithm
are not exposed. This would be done in a future change.
Change-Id: I7ddc36790943f24b19c7f08b10694ae9a822f5c9
1. When a LAPACK implementation is present, then
DENSE_QR, DENSE_NORMAL_CHOLESKY and DENSE_SCHUR
can use it for doing dense linear algebra operations.
2. The user can switch dense linear algebra libraries
by setting Solver::Options::dense_linear_algebra_library_type.
3. Solver::Options::sparse_linear_algebra_library is now
Solver::Options::sparse_linear_algebra_library_type to be consistent
with all the other enums in Solver::Options.
4. Updated documentation as well as Solver::Summary::FullReport
to reflect these changes.
Change-Id: I5ab930bc15e90906b648bc399b551e6bd5d6498f
1. Added a Preconditioner interface.
2. SCHUR_JACOBI is now its own class and is independent of
SuiteSparse.
Change-Id: Id912ab19cf3736e61d1b90ddaf5bfba33e877ec4
The interface for NumericDiffCostFunction and AutoDiffCostFunction
are not comparable. They both accept variadic functors.
The change is backward compatible, as it still supports numeric
differentiation of CostFunction objects.
Some refactoring of documentation and code in auto_diff_cost_function
and its relatives was also done to make things consistent.
Change-Id: Ib5f230a1d4a85738eb187803b9c1cd7166bb3b92
1. Add a line search based minimization loop.
2. Currently this loop supports steepest descent and three
kinds of non-linear conjugate gradient algorithms.
3. Update SolverImpl to talk to LineSearchMinimizer.
4. Update IterationCallback to carry information about
line search.
5. Update LineSearch to take the initial point as input,
saving on one function evaluation.
6. Updates to the external API.
Change-Id: I901a0e89fc948451ab34c743e70f3dec57c9405e
Remove the old ordering API, and modify solver_impl.cc
to use the new API everywhere.
In the process also clean up the linear solver instantion
logic in solver_impl.cc a bit too.
Change-Id: Ia66898abc7f622070b184b21fce8cc6140c4cebf
More flexible testing.
Read and parse the certified cost value from the data file.
Remove the ugly hack for computing the certified cost.
Refactored the flags parsing logic
Change-Id: I8f2e6be183b758b2453302fcdc6696bfa0db5eb8
For problems with a small number of variables, but a large
number of residuals, it is sometimes beneficial to use the
Cholesky factorization on the normal equations, instead of
the dense QR factorization of the Jacobian, even though it
is numerically the better thing to do.
Change-Id: I3506b006195754018deec964e6e190b7e8c9ac8f
1. A new dogleg trust region strategy.
2. Consistent naming of all variables taking and reporting
time. Also all are doubles now.
3. Enum to stringification routines.
4. bundle_adjuster.cc accepts max solver time and trust_region_strategy.
5. Time accounting is pushed into solver_impl.cc and there is now
postprocessing time accounted for explicitly.
6. IterationCallback now has cumulative time.
7. LoggingCallback logs per iteration and cumulative time.
8. TrustRegionStrategy now allows for Invalid steps to be indicated
explicitly.
9. Trust region minimizer actually terminates on max_solver_time.
Change-Id: I7e3b82c8beebc17b6b355ea46ddd280754a2d8b2
1. New TrustRegionMinimizer and basic tests for it.
2. New TrustRegionStrategy interface.
3. New LevenbergMarquardtStrategy and tests for it.
4. Updates to SolverImpl to reflect this.
5. Changes to Solver::Options and IterationSummary related to this.
6. Deleted levenberg_marquardt.cc/h/_test.cc
Change-Id: I6c1d1a7c774f014856f9f26263a830aa886e1400
Sameer Agarwal