1. Add a version history
2. Update copyright years across the code base
3. Run format_all.sh
4. Update version strings from 2.1.0 to 2.2.0 in the docs and
elsewhere.
Change-Id: I46d8d479d54bd6002d532785e67342106e73c9ac
1. Use hardware_concurrency to configure number of threads.
2. Use user ordering instead of automatic ordering.
Fixes https://github.com/ceres-solver/ceres-solver/issues/874
Change-Id: I4d7d69612e1ee40358943019a0c3a909e419c4e4
* Added CudaCgnrSolver, a new CUDA-accelerated CGNR.
* To use CudaCgnrSolver, the user must select CGNR as the linear_solver
and CUDA_SPARSE as the sparse_linear_algebra_library.
* Updated ConjugateGradientSolver to work with an array of pointers to
scratch to support CudaVectors as scratch.
* Moved CUDA initialization to run in Solver::Solve as needed.
Some performance comparisons on an Ubuntu 20.04 desktop with an
Intel i9-9940X CPU @ 3.30GHz, and an nVidia Quadro RTX 6000,
all configurations run with 24 threads, and 10 iterations.
=================================================
CGNR + CUDA_SPARSE + IDENTITY Preconditioner
problem-1778-993923-pre.txt
=================================================
Cost:
Initial 2.563973e+08
Final 1.724755e+06
Change 2.546725e+08
Minimizer iterations 11
Successful steps 7
Unsuccessful steps 4
Time (in seconds):
Preprocessor 4.020158
Residual only evaluation 1.567092 (10)
Jacobian & residual evaluation 7.847130 (7)
Linear solver 31.688898 (10)
Minimizer 46.834987
Postprocessor 0.353974
Total 51.209120
=================================================
SPARSE_SCHUR (CPU) + SUITE_SPARSE + AMD
problem-1778-993923-pre.txt
=================================================
Cost:
Initial 2.563973e+08
Final 1.651617e+06
Change 2.547457e+08
Minimizer iterations 11
Successful steps 11
Unsuccessful steps 0
Time (in seconds):
Preprocessor 35.812003
Residual only evaluation 1.658980 (10)
Jacobian & residual evaluation 12.218799 (11)
Linear solver 76.409992 (10)
Minimizer 98.809773
Postprocessor 0.372712
Total 134.994489
=================================================
ITERATIVE_SCHUR (CPU) + JACOBI Preconditioner
problem-1778-993923-pre.txt
=================================================
Cost:
Initial 2.563973e+08
Final 1.684447e+06
Change 2.547128e+08
Minimizer iterations 11
Successful steps 8
Unsuccessful steps 3
Time (in seconds):
Preprocessor 15.331614
Residual only evaluation 1.606114 (10)
Jacobian & residual evaluation 8.502166 (8)
Linear solver 351.910080 (10)
Minimizer 368.797327
Postprocessor 0.363536
Total 384.492478
=================================================
CGNR + CUDA_SPARSE + IDENTITY Preconditioner
problem-13682-4456117-pre.txt
=================================================
Cost:
Initial 1.126372e+09
Final 2.269329e+07
Change 1.103678e+09
Minimizer iterations 11
Successful steps 7
Unsuccessful steps 4
Time (in seconds):
Preprocessor 19.140087
Residual only evaluation 8.721920 (10)
Jacobian & residual evaluation 41.955923 (7)
Linear solver 214.121861 (10)
Minimizer 296.636890
Postprocessor 1.971827
Total 317.748804
Change-Id: I3a09f31aa6903f661e91f595afd39d427583e856
With this change, the user can now choose between Approximate Minimum
Degree and Nested Dissection as a fill reducing algorithm when using
a sparse direct factorization based linear solver like SPARSE_NORMAL_CHOLESKY
or SPARSE_SCHUR.
Currenly only SUITE_SPARSE is supported. It requires that
SuiteSparse be compiled with Metis support enabled.
On most problems AMD is still the better choice, but in some cases
like the grid3D dataset from https://lucacarlone.mit.edu/datasets/
the solution time with AMD is 57s and with NESDIS 38 on my M1 Mac.
On some other problems at Google we have observed speedups of 10x,
there is also a corresponding decrease in the total amount of memory
used.
This patch is based on the original work done by NeroBurner in
https://ceres-solver-review.googlesource.com/c/ceres-solver/+/20580
1. Add a new enum to the public api LinearSolverOrderingType and
a setting Solver::Options::linear_solver_ordering_type.
2. TrustRegionPreprocessor had some complicated logic which determined
when linear solvers should reorder their matrices on their own and not
this has been refactored into a more readable function that lives
inside reorder_program.h/cc.
3. Plumbing in reorder_program.cc and trust_region_processor.cc to use
nested dissection.
4. Update bundle_adjuster.cc to use nested dissection.
Change-Id: I388b027934f86c58b4da2b65a4fa5204ea73bf40
In many cases, manifolds stored in ProductManifold have a default
constructor which can simplify ProductManifold initialization even
further. Allow default construction of ProductManifold in this case.
Change-Id: I29b2612870c02232556688019a77049709684a55
Since the number of manifolds used to initialize ProductManifold and
their types are known at compile-time, it is possible to avoid storing
pointers to the base class as required by a homogeneous, currently
dynamically sized container. Instead, we can use std::tuple<> as a
heterogenous container with the number of elements fixed at compile-time
that allows us to store the concrete manifold realizations.
The advantage of this approach is that we can bypass the vtable when
iterating over each manifold within ProductManifold. The indirection is
invoked only once while accessing the ProductManifoldImpl members.
Additionally, potential dynamic memory allocations by a std::vector can
be completely avoided. This makes the ProductManifold implementation
more efficient both in memory and runtime.
Change-Id: Ic71b0c175ab726f8992e9703f7666bca477baf19
This brings it in line with other manifolds like SphereManifold
and LineManifold, where the user has the choice to specify the size
of the manifold at compile time or runtime.
Most of the time the size is known at compile time so this will
speed up the common case.
Change-Id: I0c7ff8b7a9a64a81203eb11afc074874e208815a
1. Add CUDADenseCholesky64Bit, CUDADenseCholesky32Bit, & tests.
CUDADenseCholesky32Bit uses the legacy versions of potrf/potrs
in cuSolverDN, while CUDADenseCholesky64Bit uses the new 64-bit
versions available since Cuda 11.1. The legacy versions are
provided since some platforms such as the Nvidia Jetsons only
support Cuda 10.2.
2. Expose CUDA as a new option under DenseLinearAlgebraLibraryType.
The relevant option to string and string to option helper functions
are modified accordingly.
3. Add cuda as a dense_linear_algebra_library option in bundle_adjuster
to demonstrate the use of the new CUDA option.
Change-Id: I23615e1d301df5185ed646b3e33ee802508dae86
This is needed to add a layer of indirection which will allow
us to use gflags in the public version and absl inside Google.
Change-Id: I32f3da23200a01c9b658bbf8aaa66cb8fddc2cc3
This is mostly just applying the existing clang format config, except:
- Use NOLINT on overlong comment lines.
- Wrap some sections in 'clang-format off' / 'clang format on'.
- Manually split or join some multi-line strings.
Change-Id: Ia1a40eeb92112e12c3a169309afe087af55b2f4f
This commit includes the following:
- Changes to CMake to make it safer to see which compiler flags are supported,
so this way we do not need to worry about version checks in CMake.
- Unix platforms (which includes both Linux and Apple as far as i can tell)
will now enable -Wmissing-declarations warning for the whole Ceres.
- Changes in all sources to solve missing declaration warning. In most cases
it was either matter of using static qualifier or moving functions to an
anonymous namespace.
In one case the function got removed, since it seems to be unused.
Additionally, in slam examples there was a non-inlined function implementation
in a header, which is a direct way to cause linking errors if other .cc file
will include that helper header.
- All third party sources (which is currently only gmock) has this extra
paranoid warning disabled.
This warning is important in the following cases:
- Detect helper functions which are not needed anymore.
- Avoid unnoticed pollution of namespace.
- Avoid bad level calls.
- Avoid missing updates in header files after changes in implementation file.
- Helps integrating Ceres into software where paranoid warnings are important.
Change-Id: I9b1044aced3910d8c6b2356cfe2bf57f3c8c58db
1. Add Solver::Options::use_mixed_precision_solves,
and Solver::Options::max_num_refinement_iterations.
2. Make SparseCholesky::Create return a unique_ptr.
3. SparseCholesky::Create now takes LinearSolver::Options
as an argument.
4. IterativeRefiner's constructor does not require num_cols
as an argument.
5. SparseNormalCholeskySolver now uses a separate rhs vector.
This basic implementation results in a 10% reduction in solver time
and 30% reduction in linear solver memory usage.
Change-Id: I6830f32cae2febf082d2733262eb2c9f0482b0ea
1. Solver::Options::num_threads now controls parallelism in Ceres
Solver. The user specified value of
Solver::Options::num_linear_solver_threads is ignored.
2. If the user specifies Solver::Options::num_linear_solver_threads
and it is different from Solver::Options::num_threads,
a warning is printed.
3. Solver::Summary:num_linear_solver_threads_given and
Solver::Summary::num_linear_solver_threads_used are also
deprecated and are always set to Solver::Summary::num_threads_given
and Solver::Summary::num_threads_used.
Change-Id: I20b9336d9336e400e6f0a15b63857c0c43eb271c
Previously, when using a quaternion to parameterize the camera
orientation, the camera parameter block was split into two
parameter blocks. One for the rotation and another for the
translation and intrinsics. This was to enable the use of the
Quaternion parameterization.
Now that we have a ProductParameterization which allows us
to compose multiple parameterizations, this is no longer needed
and we use a size 10 parameter block instead.
This leads to a more than 2x improvements in the linear solver time.
Change-Id: I78b8f06696f81fee54cfe1a4ae193ee8a5f8e920
Since Ceres is moving to using GitHub for issues, and the Google
Code URL in the current copyright header will soon become invalid,
update all the headers.
Change-Id: I1fce70375d1bcf098591f07b4d8f01a5c1e0789c
- At version 2.1, gflags changed from using the google namespace, to
using gflags by default. However, it can be configured at build time
to be something else (which would be google for legacy compatibility
unless you were evil).
- Ceres previously assumed that gflags was in the google namespace.
- Now, FindGFlags.cmake extracts the namespace when gflags.h is found
and saves it in GFLAGS_NAMESPACE.
- When building the tests and examples that require gflags,
CERES_GFLAGS_NAMESPACE is defined to be the detected namespace, and
all tests/examples now use CERES_GFLAGS_NAMESPACE:: instead of
google:: when calling gflags functions.
Change-Id: Ia333df7a7e2f08ba9f26bbd339c3a785b88f04c4
Up till now ITERATIVE_SCHUR evaluates matrix-vector products
between the Schur complement and a vector implicitly by exploiting
the algebraic expression for the Schur complement.
This cost of this evaluation scales with the number of non-zeros
in the Jacobian.
For small to medium sized problems there is a sweet spot where
computing the Schur complement is cheap enough that it is much
more efficient to explicitly compute it and use it for evaluating
the matrix-vector products.
This changes implements support for an explicit Schur complement
in ITERATIVE_SCHUR in combination with the SCHUR_JACOBI preconditioner.
API wise a new bool Solver::Options::use_explicit_schur_complement
has been added.
The implementation extends the SparseSchurComplementSolver to use
Conjugate Gradients.
Example speedup:
use_explicit_schur_complement = false
Time (in seconds):
Preprocessor 0.585
Residual evaluation 0.319
Jacobian evaluation 1.590
Linear solver 25.685
Minimizer 27.990
Postprocessor 0.010
Total 28.585
use_explicit_schur_complement = true
Time (in seconds):
Preprocessor 0.638
Residual evaluation 0.318
Jacobian evaluation 1.507
Linear solver 5.930
Minimizer 8.144
Postprocessor 0.010
Total 8.791
Which indicates an end-to-end speedup of more than 3x, with the linear
solver being sped up by > 4x.
The idea to explore this optimization was inspired by the recent paper:
Mining structure fragments for smart bundle adjustment
L. Carlone, P. Alcantarilla, H. Chiu, K. Zsolt, F. Dellaert
British Machine Vision Conference, 2014
which uses a more complicated algorithm to compute parts of the
Schur complement to speed up the matrix-vector product.
Change-Id: I95324af0ab351faa1600f5204039a1d2a64ae61d
1. Update version history.
2. Minor changes to the tutorial to reflect the bounds constrained
problem.
3. Added static factory methods to the SnavelyReprojectionError.
4. Removed relative gradient tolerance from types.h as it is
not true anymore.
Change-Id: I8de386e5278a008c84ef2d3290d2c4351417a9f1
Solver::Options::linear_solver_ordering and
Solver::Options::inner_iteration_ordering
were bare pointers even though Solver::Options took ownership of these
objects.
This lead to buggy user code and the inability to copy Solver::Options
objects around.
With this change, these naked pointers have been replaced by a
shared_ptr object which will managed the lifetime of these objects. This
also leads to simplification of the lifetime handling of these objects
inside the solver.
The Android.mk and Application.mk files have also been updated
to use a newer NDK revision which ships with LLVM's libc++.
Change-Id: I25161fb3ddf737be0b3e5dfd8e7a0039b22548cd
The original visibility based preconditioning paper and
implementation only used the canonical views algorithm.
This algorithm for large dense graphs can be particularly
expensive. As its worst case complexity is cubic in size
of the graph.
Further, for many uses the SCHUR_JACOBI preconditioner
was both effective enough while being cheap. It however
suffers from a fatal flaw. If the camera parameter blocks
are split between two or more parameter blocks, e.g,
extrinsics and intrinsics. The preconditioner because
it is block diagonal will not capture the interactions
between them.
Using CLUSTER_JACOBI or CLUSTER_TRIDIAGONAL will fix
this problem but as mentioned above this can be quite
expensive depending on the problem.
This change extends the visibility based preconditioner
to allow for multiple clustering algorithms. And adds
a simple thresholded single linkage clustering algorithm
which allows you to construct versions of CLUSTER_JACOBI
and CLUSTER_TRIDIAGONAL preconditioners that are cheap
to construct and are more effective than SCHUR_JACOBI.
Currently the constants controlling the threshold above
which edges are considered in the single linkage algorithm
are not exposed. This would be done in a future change.
Change-Id: I7ddc36790943f24b19c7f08b10694ae9a822f5c9
1. When a LAPACK implementation is present, then
DENSE_QR, DENSE_NORMAL_CHOLESKY and DENSE_SCHUR
can use it for doing dense linear algebra operations.
2. The user can switch dense linear algebra libraries
by setting Solver::Options::dense_linear_algebra_library_type.
3. Solver::Options::sparse_linear_algebra_library is now
Solver::Options::sparse_linear_algebra_library_type to be consistent
with all the other enums in Solver::Options.
4. Updated documentation as well as Solver::Summary::FullReport
to reflect these changes.
Change-Id: I5ab930bc15e90906b648bc399b551e6bd5d6498f
Sparse Cholesky factorization algorithms use a fill-reducing
ordering to permute the columns of the Jacobian matrix. There
are two ways of doing this.
1. Compute the Jacobian matrix in some order and then have the
factorization algorithm permute the columns of the Jacobian.
2. Compute the Jacobian with its columns already permuted.
The first option incurs a significant memory penalty. The
factorization algorithm has to make a copy of the permuted
Jacobian matrix.
Starting with this change Ceres pre-permutes the columns of the
Jacobian matrix and generally speaking, there is no performance
penalty for doing so.
In some rare cases, it is worth using a more complicated
reordering algorithm which has slightly better runtime
performance at the expense of an extra copy of the Jacobian
matrix. Setting Solver::Options::use_postordering to true
enables this tradeoff.
This change also removes Solver::Options::use_block_amd
as an option. All matrices are ordered using their block
structure. The ability to order them by their scalar
sparsity structure has been removed.
Here is what performance on looks like on some BAL problems.
Memory
======
HEAD pre-ordering
16-22106 137957376.0 113516544.0
49-7776 56688640.0 46628864.0
245-198739 1718005760.0 1383550976.0
257-65132 387715072.0 319512576.0
356-226730 2014826496.0 1626087424.0
744-543562 4903358464.0 3957878784.0
1024-110968 968626176.0 822071296.0
Time
====
HEAD pre-ordering
16-22106 3.8 3.7
49-7776 1.9 1.8
245-198739 82.6 81.9
257-65132 14.0 13.4
356-226730 98.8 95.8
744-543562 325.2 301.6
1024-110968 42.1 37.1
Change-Id: I6b2e25f3fed7310f88905386a7898ac94d37467e
Add automatic recursive independent set decomposition.
Clean up the naming and the API for inner iterations.
Change-Id: I3d7d6babb9756842d7367e14b7279d2df98fb724
A non-linear generalization of Ruhe & Wedin's algorithm
for separable non-linear least squares problem. It is implemented
as coordinate descent on an independent subset of the parameter
blocks at the end of every successful Newton step. The resulting
algorithm has much improved convergence at the cost of some
execution time.
Change-Id: I8fdc5edbd0ba1e702c9658b98041b2c2ae705402
Remove the old ordering API, and modify solver_impl.cc
to use the new API everywhere.
In the process also clean up the linear solver instantion
logic in solver_impl.cc a bit too.
Change-Id: Ia66898abc7f622070b184b21fce8cc6140c4cebf
More flexible testing.
Read and parse the certified cost value from the data file.
Remove the ugly hack for computing the certified cost.
Refactored the flags parsing logic
Change-Id: I8f2e6be183b758b2453302fcdc6696bfa0db5eb8
For problems with a small number of variables, but a large
number of residuals, it is sometimes beneficial to use the
Cholesky factorization on the normal equations, instead of
the dense QR factorization of the Jacobian, even though it
is numerically the better thing to do.
Change-Id: I3506b006195754018deec964e6e190b7e8c9ac8f
Refactor some of the perturbation code so that the
normalization and perturbation code can share the
camera decomposition code.
Change-Id: I084064976804a92f9240d8f5e10d1bb23dcb5ff2
Sameer Agarwal