THIS IS AN API BREAKING CHANGE.
Decouple the algorithm from the sparse linear algebra
library being used to perform the computation.
Before this change
Covariance::AlgorithmType had values
DENSE_SVD
EIGEN_SPARSE_QR
SUITE_SPARSE_QR
This has been replaced by two enums now.
Covariance::Options::sparse_linear_algebra_library_type
which can take values EIGEN_SPARSE, SUITE_SPARSE or CX_SPARSE.
The last one is currently not supported.
And Covariance::Options::algorithm_type takes values
DENSE_SVD
SPARSE_QR
This sets the stage for future extensions of the covariance
computation algorithm.
Also as part of this change, the covariance computation chapter
has been made a top level chapter on its own instead of being
buried deep inside the Solving Non-linear Least Squares problem.
Change-Id: Ibfbf60902d8d17694d9ff585047a5a57d329ab22
ResidualBlock evaluation has logic to ensure that CostFunction
should always fill out the residual and jacobian arrays completely
by using a special value to pre-populate these arrays.
This works for CostFunctions with analytical Jacobians but not for
AutoDiffCostFunction and NumericDiffCostFunction Jacobians.
There is no way to fix this for NumericDiffCostFunctions without
introducing significant performance penalties but the residual
evaluation fails, which should be enough to catch such errors.
For AutoDiffCostFunction the way the Jets are default initialized
was sidestepping this check. So now, the Jet that is used to
capture the output residuals is now initialized with
kImpossibleValue, which will ensure that if the user forgets
to fill all output fields, it triggers an evaluation error.
This change required that ceres::internal::kImpossibleValue be moved
out of array_utils.h/cc to types.h.
Change-Id: I35bb0946cf0785a5d43c7b5459a2272848fb2a9b
This method numerically computes function derivatives in different
scales, extrapolating between intermediate results to conserve function
evaluations. Adaptive differentiation is essential to produce accurate
results for functions with noisy derivatives.
Full changelist:
-Created a new type of NumericDiffMethod (RIDDERS).
-Implemented EvaluateRiddersJacobianColumn in NumericDiff.
-Created unit tests with f(x) = x^2 + [random noise] and
f(x) = exp(x).
Change-Id: I2d6e924d7ff686650272f29a8c981351e6f72091
Since Ceres is moving to using GitHub for issues, and the Google
Code URL in the current copyright header will soon become invalid,
update all the headers.
Change-Id: I1fce70375d1bcf098591f07b4d8f01a5c1e0789c
- Previously we had no defined default value for
sparse_linear_algebra_library_type in Solver::Options if Ceres
was compiled with no sparse library available. Thus in that case,
the default value (dependent upon the compiler) would indicate that
one was available.
- Now we have an explicit option that means no sparse library is
available, which is now the default value in Solver::Options in this
case.
- Add a warning in CMake when the user disables all sparse libraries.
- Fix typos in trust_region_preprocessor_test:
(SUITE/CX)_SPARSE -> (SUITE/CX)SPARSE that induced failures when
no sparse libraries were available.
Change-Id: I869c399a12d42bfc44220cbb25ce6d6dd80236bd
SPARSE_NORMAL_CHOLESKY and SPARSE_SCHUR can now be used
with EIGEN_SPARSE as the backend.
The performance is not as good as CXSparse. This needs to be
investigated. Is it because the quality of AMD ordering that
we are computing is not as good as the one for CXSparse? This
could be because we are working with the scalar matrix instead
of the block matrix.
Also, the upper/lower triangular story is not completely clear.
Both of these issues will be benchmarked and tackled in the
near future.
Also included in this change is a bunch of cleanup to the
SparseNormalCholeskySolver and SparseSchurComplementSolver
classes around the use of the of defines used to conditionally
compile out parts of the code.
The system_test has been updated to test EIGEN_SPARSE also.
Change-Id: I46a57e9c4c97782696879e0b15cfc7a93fe5496a
Sparse Cholesky factorization is not rank revealing. Therefore
this algorithm cannot reliably tell when the Jacobian matrix is
rank deficient or so poorly conditioned that the covariance matrix
cannot be estimated.
Making things worse, this algorithm works on the normal equations,
which makes the conditioning problem much worse.
This change, deletes the SPARSE_CHOLESKY algorithm in the covariance
estimation code. Also to make the naming consistent, it renames
SPARSE_QR -> SUITE_SPARSE_QR
so that it parallels EIGEN_SPARSE_QR.
Also, since we now have EIGEN_SPARSE_QR, we can default to using
it when SuiteSparse is not available instead of DENSE_SVD, which
generally speaking should only be used by folks who are dealing
with small rank deficient jacobians.
Change-Id: I8b134c7e8a2e86ca374371f185b19f1c3e74349c
For smaller problems Eigen is faster than SuiteSparseQR. This has been
tested with Eigen 3.2.1. Below are detailed timings. Problem 1 is the
smallest and problem 3 is the largest. The timings below are:
mean +- standard deviation.
Problem 1:
Eigen 0.0009218 +- 0.0002755
SuiteSparse 0.001406 +- 0.001610
Problem 2:
Eigen 0.002338 +- 0.001005
SuiteSparse 0.001910 +- 0.0004513
Problem 3:
Eigen 0.005455 +- 0.001759
SuiteSparse 0.002411 +- 0.0004974
Detailed problem descriptions:
Problem 1 size:
Original Reduced
Parameter blocks 533 54
Parameters 368 104
Effective parameters 1201 94
Residual blocks 233 77
Residual 1194 258
Problem 2 size:
Original Reduced
Parameter blocks 573 84
Parameters 1458 184
Effective parameters 1281 164
Residual blocks 263 107
Residual 1314 378
Problem 3 size:
Original Reduced
Parameter blocks 613 114
Parameters 1548 264
Effective parameters 1361 234
Residual blocks 293 137
Residual 1434 498
Change-Id: I884a67e2f728fe2992812148d82ccf5f27864fd7
1. Update version history.
2. Minor changes to the tutorial to reflect the bounds constrained
problem.
3. Added static factory methods to the SnavelyReprojectionError.
4. Removed relative gradient tolerance from types.h as it is
not true anymore.
Change-Id: I8de386e5278a008c84ef2d3290d2c4351417a9f1
1. Rename SolverTerminationType to TerminationType.
2. Consolidate the enum as
a. CONVERGENCE - subsumes FUNCTION_TOLERANCE, PARAMETER_TOLERANCE and GRADIENT_TOLERANCE
b. NO_CONVERGENCE
c. FAILURE - captures all kinds of failures including DID_NOT_RUN.
d. USER_SUCCESS
e. USER_FAILURE
3. Solver::Summary::error is renamed to be Solver::Summary::message, to both
reduce confusion as well as capture its true meaning.
Change-Id: I27a382e66e67f5a4750d0ee914d941f6b53c326d
1. Move LinearSolverTerminationType to ceres::internal.
2. Add FATAL_ERROR as a new enum to LinearSolverTerminationType.
3. Pipe SuiteSparse errors via a LinearSolverTerminationType so
to distinguish between fatal and non-fatal errors.
4. Update levenberg marquardt and dogleg strategies to deal
with FATAL_ERROR.
5. Update trust_region_minimizer to terminate when FATAL_ERROR
is encountered.
6. Remove SuiteSparse::SolveCholesky as it screws up the error
handling.
7. Fix all clients calling SuiteSparse to handle the result of
SuiteSparse::Cholesky correctly.
8. Remove fatal failures in SuiteSparse when symbolic factorization
fails.
9. Fix all clients of SuiteSparse to deal with null symbolic factors.
This is a temporary fix to deal with some production problems. A more
extensive cleanup and testing regime will be put in place in a
subsequent CL.
Change-Id: I1f60d539799dd95db7ecc340911e261fa4824f92
The original visibility based preconditioning paper and
implementation only used the canonical views algorithm.
This algorithm for large dense graphs can be particularly
expensive. As its worst case complexity is cubic in size
of the graph.
Further, for many uses the SCHUR_JACOBI preconditioner
was both effective enough while being cheap. It however
suffers from a fatal flaw. If the camera parameter blocks
are split between two or more parameter blocks, e.g,
extrinsics and intrinsics. The preconditioner because
it is block diagonal will not capture the interactions
between them.
Using CLUSTER_JACOBI or CLUSTER_TRIDIAGONAL will fix
this problem but as mentioned above this can be quite
expensive depending on the problem.
This change extends the visibility based preconditioner
to allow for multiple clustering algorithms. And adds
a simple thresholded single linkage clustering algorithm
which allows you to construct versions of CLUSTER_JACOBI
and CLUSTER_TRIDIAGONAL preconditioners that are cheap
to construct and are more effective than SCHUR_JACOBI.
Currently the constants controlling the threshold above
which edges are considered in the single linkage algorithm
are not exposed. This would be done in a future change.
Change-Id: I7ddc36790943f24b19c7f08b10694ae9a822f5c9
1. When a LAPACK implementation is present, then
DENSE_QR, DENSE_NORMAL_CHOLESKY and DENSE_SCHUR
can use it for doing dense linear algebra operations.
2. The user can switch dense linear algebra libraries
by setting Solver::Options::dense_linear_algebra_library_type.
3. Solver::Options::sparse_linear_algebra_library is now
Solver::Options::sparse_linear_algebra_library_type to be consistent
with all the other enums in Solver::Options.
4. Updated documentation as well as Solver::Summary::FullReport
to reflect these changes.
Change-Id: I5ab930bc15e90906b648bc399b551e6bd5d6498f
1. Added a Preconditioner interface.
2. SCHUR_JACOBI is now its own class and is independent of
SuiteSparse.
Change-Id: Id912ab19cf3736e61d1b90ddaf5bfba33e877ec4
The interface for NumericDiffCostFunction and AutoDiffCostFunction
are not comparable. They both accept variadic functors.
The change is backward compatible, as it still supports numeric
differentiation of CostFunction objects.
Some refactoring of documentation and code in auto_diff_cost_function
and its relatives was also done to make things consistent.
Change-Id: Ib5f230a1d4a85738eb187803b9c1cd7166bb3b92
1. Add a line search based minimization loop.
2. Currently this loop supports steepest descent and three
kinds of non-linear conjugate gradient algorithms.
3. Update SolverImpl to talk to LineSearchMinimizer.
4. Update IterationCallback to carry information about
line search.
5. Update LineSearch to take the initial point as input,
saving on one function evaluation.
6. Updates to the external API.
Change-Id: I901a0e89fc948451ab34c743e70f3dec57c9405e
Remove the old ordering API, and modify solver_impl.cc
to use the new API everywhere.
In the process also clean up the linear solver instantion
logic in solver_impl.cc a bit too.
Change-Id: Ia66898abc7f622070b184b21fce8cc6140c4cebf
More flexible testing.
Read and parse the certified cost value from the data file.
Remove the ugly hack for computing the certified cost.
Refactored the flags parsing logic
Change-Id: I8f2e6be183b758b2453302fcdc6696bfa0db5eb8
For problems with a small number of variables, but a large
number of residuals, it is sometimes beneficial to use the
Cholesky factorization on the normal equations, instead of
the dense QR factorization of the Jacobian, even though it
is numerically the better thing to do.
Change-Id: I3506b006195754018deec964e6e190b7e8c9ac8f
1. A new dogleg trust region strategy.
2. Consistent naming of all variables taking and reporting
time. Also all are doubles now.
3. Enum to stringification routines.
4. bundle_adjuster.cc accepts max solver time and trust_region_strategy.
5. Time accounting is pushed into solver_impl.cc and there is now
postprocessing time accounted for explicitly.
6. IterationCallback now has cumulative time.
7. LoggingCallback logs per iteration and cumulative time.
8. TrustRegionStrategy now allows for Invalid steps to be indicated
explicitly.
9. Trust region minimizer actually terminates on max_solver_time.
Change-Id: I7e3b82c8beebc17b6b355ea46ddd280754a2d8b2
1. New TrustRegionMinimizer and basic tests for it.
2. New TrustRegionStrategy interface.
3. New LevenbergMarquardtStrategy and tests for it.
4. Updates to SolverImpl to reflect this.
5. Changes to Solver::Options and IterationSummary related to this.
6. Deleted levenberg_marquardt.cc/h/_test.cc
Change-Id: I6c1d1a7c774f014856f9f26263a830aa886e1400
1. Added support for CXSparse - SparseNormalCholesky and
SchurComplementSolver support SuiteSparse and CXSparse now.
I am not sure I will add suport for visibility based
preconditioning using CXSparse. Its not a high priority.
2. New enum SparseLinearAlgebraLibraryType which allows the user
to indicate which sparse linear algebra library should be used.
3. Updated tests for SolverImpl and system_test.
4. Build system changes to automatically detect CXSparse and
link to it by default -- just like SuiteSparse.
5. Minor bug fixes dealing in the cmake files and VBP.
6. Changed the order of the system test.
7. Deduped the unsymmetric linear solver test.
Change-Id: I33252a103c87b722ecb7ed7b5f0ae7fd91249244
This commit modifies the only function in autodiff that takes a
templated number of outputs (i.e. residuals) and makes that
template parameter a normal parameter. With that change, it
is a trivial matter to support a dynamic number of residuals.
The API for dynamic residuals is to pass a fake number of
residuals as the second template argument to
AutoDiffCostFunction, and to pass the real number of
parameters as a second constructor argument.
1. Make the mechanism for writing problems to disk, generic and
controllable using an enum DumpType visible in the API.
2. Instead of single file containing protocol buffers, now matrices can
be written in a matlab/octave friendly format. This is now the default.
3. The support for writing problems to disk is moved into
linear_least_squares_problem.cc/h
4. SparseMatrix now has a ToTextFile virtual method which is
implemented by each of its subclasses to write a (i,j,s) triplets.
5. Minor changes to simple_bundle_adjuster to enable logging at startup.
- Rename BlockDiagonalPreconditioner to BlockJacobiPreconditioner
- Include the diagonal in the block jacobi preconditioner.
- Better flag help for eta.
- Enable test for CGNR
- Rename CONJUGATE_GRADIENTS to CGNR.
- etc.
Sameer Agarwal