1. Add a version history
2. Update copyright years across the code base
3. Run format_all.sh
4. Update version strings from 2.1.0 to 2.2.0 in the docs and
elsewhere.
Change-Id: I46d8d479d54bd6002d532785e67342106e73c9ac
- TripletSparseMatrix in BlockRandomAccessSparseMatrix is replaced with
BlockSparseMatrix
- BlockSparseMatrix::ToCompressedRowSparseMatrix is performed in a
direct sort-less way
Change-Id: Ib951fda1b9394050e2c47a9721172c5e3c674801
Previously some matrices used Block to keep track of
row/column block sizes and some would just use ints, and
then compute the position of each row and column from it.
By uniformly using Block everywhere, we reduce duplicate
computation and data copies.
I also cleaned up a bunch of c++17 related stuff as I edited
these files.
Change-Id: I4c86b1593fd4c91f9057fbb38314f62f303e0477
* Added CudaCgnrSolver, a new CUDA-accelerated CGNR.
* To use CudaCgnrSolver, the user must select CGNR as the linear_solver
and CUDA_SPARSE as the sparse_linear_algebra_library.
* Updated ConjugateGradientSolver to work with an array of pointers to
scratch to support CudaVectors as scratch.
* Moved CUDA initialization to run in Solver::Solve as needed.
Some performance comparisons on an Ubuntu 20.04 desktop with an
Intel i9-9940X CPU @ 3.30GHz, and an nVidia Quadro RTX 6000,
all configurations run with 24 threads, and 10 iterations.
=================================================
CGNR + CUDA_SPARSE + IDENTITY Preconditioner
problem-1778-993923-pre.txt
=================================================
Cost:
Initial 2.563973e+08
Final 1.724755e+06
Change 2.546725e+08
Minimizer iterations 11
Successful steps 7
Unsuccessful steps 4
Time (in seconds):
Preprocessor 4.020158
Residual only evaluation 1.567092 (10)
Jacobian & residual evaluation 7.847130 (7)
Linear solver 31.688898 (10)
Minimizer 46.834987
Postprocessor 0.353974
Total 51.209120
=================================================
SPARSE_SCHUR (CPU) + SUITE_SPARSE + AMD
problem-1778-993923-pre.txt
=================================================
Cost:
Initial 2.563973e+08
Final 1.651617e+06
Change 2.547457e+08
Minimizer iterations 11
Successful steps 11
Unsuccessful steps 0
Time (in seconds):
Preprocessor 35.812003
Residual only evaluation 1.658980 (10)
Jacobian & residual evaluation 12.218799 (11)
Linear solver 76.409992 (10)
Minimizer 98.809773
Postprocessor 0.372712
Total 134.994489
=================================================
ITERATIVE_SCHUR (CPU) + JACOBI Preconditioner
problem-1778-993923-pre.txt
=================================================
Cost:
Initial 2.563973e+08
Final 1.684447e+06
Change 2.547128e+08
Minimizer iterations 11
Successful steps 8
Unsuccessful steps 3
Time (in seconds):
Preprocessor 15.331614
Residual only evaluation 1.606114 (10)
Jacobian & residual evaluation 8.502166 (8)
Linear solver 351.910080 (10)
Minimizer 368.797327
Postprocessor 0.363536
Total 384.492478
=================================================
CGNR + CUDA_SPARSE + IDENTITY Preconditioner
problem-13682-4456117-pre.txt
=================================================
Cost:
Initial 1.126372e+09
Final 2.269329e+07
Change 1.103678e+09
Minimizer iterations 11
Successful steps 7
Unsuccessful steps 4
Time (in seconds):
Preprocessor 19.140087
Residual only evaluation 8.721920 (10)
Jacobian & residual evaluation 41.955923 (7)
Linear solver 214.121861 (10)
Minimizer 296.636890
Postprocessor 1.971827
Total 317.748804
Change-Id: I3a09f31aa6903f661e91f595afd39d427583e856
1. Convert it from a class to a template function. Where the
template parameter is "DenseVectorType". This allows us
to have a single implementation of Conjugate Gradients
without worrying about where the matrix and the vectors
are stored or what their internal representation is.
For the case of CPU based vectors, we abstract operations
on Eigen vectors using eigen_vector_ops.
2. Introduce ConjugateGradientsLinearOperator which is
templated on DenseVectorType. It is the matrix vector
multiplication abstraction.
3. Port the tests and all usages of ConjugateGradientsSolver
to this new implementation.
4. Introduce Eigen::Vector based RightMultiply and LeftMultiply
methods into LinearOperator which by default delete to the
bare pointer based interfaces.
5. Add an identity preconditioner.
These changes are being made in preparation for adding a CUDA
based CGNR solver.
Change-Id: I9da36dc6c131856dd1a4aa7e645aaf12d25dd79b
covariance.h was using SUITE_SPARSE even when SUITESPARSE
was disabled because it did not have config.h included in it
so it did not see that CERES_NO_SUITESPARSE was defined.
Add more config.h includes to files that are using these
configuration macros.
Change-Id: I6b1d2c2bd9e559de40a6332cd6be85ad4da3377b
Currently, the logic for exporting symbols is rather complicated: when
tests are enabled internal symbols are exported in addition to the
public symbols. Such logic causes several problems. (1) Test binaries
link against a Ceres build that is different from the final release
since fewer optimizations are applied if more symbols are exported. (2)
Also, some toolchains hide symbols by default breaking the existing
logic eventually causing linker errors.
Since internal symbols are not intended to be used outside of the
project, we can compile them into object files and use exactly the same
binary code both for the final build and the tests without relying on
conditionals.
By default, all symbols are now hidden unless annotated as public.
Internal symbols are explicitly marked as not being exported in case
users chose not to hide symbols by default.
Change-Id: I589dd10be2f6f438508783cf99d141af0120057b
Do not define trivial constructors or destructors unless necessary
(e.g., for implementing pimpl) following the rule of zero. Define
virtual base class destructors out-of-line to avoid emitting vtables in
every translation unit.
Change-Id: Iea2d8978e62a8ee5a97b86cbb4e858d56e0fb274
virtual can be ambiguous. Applied changes correspond to clang-tidy fixes
stemming from the modernize-use-override check.
Change-Id: I973afd4680a5df587419777504aeb94467196b89
Like SparseCholesky, the DenseCholesky interface abstracts
away the solution of dense linear systems using Cholesky factorization.
This allows the client code to not worry about the type of dense
linear algebra library being used.
DenseNormalCholeskySolver and DenseSchurComplementSolver code
is considerably simpler as a result.
Change-Id: Ie15f09ee376d5f9a64609e6a55ad83e99c76352a
- Change formatting standard to Cpp11. Main difference is not having
the space between two closing >> for nested templates. We don't
choose c++14, because older versions of clang-format (version 9
and earlier) don't know this value yet, and it doesn't make a
difference in the formatting.
- Apply clang-format to all (non generated) internal source files.
- Manually fix some code sections (clang-format on/off) and c-strings
- Exclude some embedded external files with very different formatting
(gtest/gmock)
- Add script to format all source files
Change-Id: Ic6cea41575ad6e37c9e136dbce176b0d505dc44d
1. Replace CERES_DISALLOW_* with explicitly deleted constructors.
2. Replace use of CERES_ARRAY_SIZE and stack allocated arrays
with std::vector.
3. Move CERES_ALIGN_* macros into manual_constructor.h, which is
the one place they are used and will be deprecated along with that
file.
4. Introduce isnan,isnormal,isinf and isfinite for Jets.
5. Replace IsNormal,IsFinite,IsNaN and IsInfinite with corresponding
c++11 function calls.
Change-Id: I04f33a221aae77d247602150988b6d4aa4efeeab
A Ceres Context holds common global state that can be re-used within
Ceres. The Context current contains a thread pool if compiling with
C++11 threading support. Threads are expensive to create and destroy so
it is good to maintain across multiple Ceres solves.
Tested by compiling with and without TBB support and ran unit tests. Ran
bazel as well.
Change-Id: I82f598dfae642aa0e81a6039dc174608a5e8dbfb
SparseCholesky is an interface to sparse cholesky factorization
routines across sparse linear algebra libraries. Each sparse
linear algebra library is responsible for implementing its own
instance of this interface.
As a result the various places - SparseNormalCholeskySolver,
SparseSchurComplementSolver and VisibilityBasedPreconditioner
are significantly simplified.
Change-Id: I8b465705eae83bba9e1adfffcc741a05c70faf2e
Since Ceres is moving to using GitHub for issues, and the Google
Code URL in the current copyright header will soon become invalid,
update all the headers.
Change-Id: I1fce70375d1bcf098591f07b4d8f01a5c1e0789c
For historical reasons we had a "using namespace std;" in port.h. This
is generally a bad idea. So removing it and along the way doing a bunch
of cpplint cleanup.
Change-Id: Ia125601a55ae62695e247fb0250df4c6f86c46c6
Up till now ITERATIVE_SCHUR evaluates matrix-vector products
between the Schur complement and a vector implicitly by exploiting
the algebraic expression for the Schur complement.
This cost of this evaluation scales with the number of non-zeros
in the Jacobian.
For small to medium sized problems there is a sweet spot where
computing the Schur complement is cheap enough that it is much
more efficient to explicitly compute it and use it for evaluating
the matrix-vector products.
This changes implements support for an explicit Schur complement
in ITERATIVE_SCHUR in combination with the SCHUR_JACOBI preconditioner.
API wise a new bool Solver::Options::use_explicit_schur_complement
has been added.
The implementation extends the SparseSchurComplementSolver to use
Conjugate Gradients.
Example speedup:
use_explicit_schur_complement = false
Time (in seconds):
Preprocessor 0.585
Residual evaluation 0.319
Jacobian evaluation 1.590
Linear solver 25.685
Minimizer 27.990
Postprocessor 0.010
Total 28.585
use_explicit_schur_complement = true
Time (in seconds):
Preprocessor 0.638
Residual evaluation 0.318
Jacobian evaluation 1.507
Linear solver 5.930
Minimizer 8.144
Postprocessor 0.010
Total 8.791
Which indicates an end-to-end speedup of more than 3x, with the linear
solver being sped up by > 4x.
The idea to explore this optimization was inspired by the recent paper:
Mining structure fragments for smart bundle adjustment
L. Carlone, P. Alcantarilla, H. Chiu, K. Zsolt, F. Dellaert
British Machine Vision Conference, 2014
which uses a more complicated algorithm to compute parts of the
Schur complement to speed up the matrix-vector product.
Change-Id: I95324af0ab351faa1600f5204039a1d2a64ae61d
Ordering routines for the Schur complement when using EIGEN_SPARSE.
Also integration into SchurComplementSolver.
Part of this CL is also a refactoring of the block jacobian matrix
construction.
Change-Id: I11d665cc7d4867c64190e6fed1118f4d2e13d59b
The instrumentation revealed that EIGEN_SPARSE can be upto
an order of magnitude slower than CX_SPARSE on some bundle
adjustment problems.
The problem comes down to the quality of AMD ordering that
CXSparse/Eigen implements. It does particularly badly
on the Schur complement. In the CXSparse implementation
we got around this by considering the block sparsity structure
and computing the AMD ordering on it and lifting it to the
full matrix.
This is currently not possible with the release version of
Eigen, as the support for using preordered/natural orderings
is in the master branch but has not been released yet.
Change-Id: I25588d3e723e50606f327db5759f174f58439e29
SPARSE_NORMAL_CHOLESKY and SPARSE_SCHUR can now be used
with EIGEN_SPARSE as the backend.
The performance is not as good as CXSparse. This needs to be
investigated. Is it because the quality of AMD ordering that
we are computing is not as good as the one for CXSparse? This
could be because we are working with the scalar matrix instead
of the block matrix.
Also, the upper/lower triangular story is not completely clear.
Both of these issues will be benchmarked and tackled in the
near future.
Also included in this change is a bunch of cleanup to the
SparseNormalCholeskySolver and SparseSchurComplementSolver
classes around the use of the of defines used to conditionally
compile out parts of the code.
The system_test has been updated to test EIGEN_SPARSE also.
Change-Id: I46a57e9c4c97782696879e0b15cfc7a93fe5496a
A lot of error checking cruft has accumulated over the years
in the various linear solvers. This change makes the error reporting
more robust and consistent across the various solvers.
Preconditioners are not covered by this change and will be the
subject of a future change.
Change-Id: Ibeb2572a1e67758953dde8d12e3abc6d1df9052d
1. Move LinearSolverTerminationType to ceres::internal.
2. Add FATAL_ERROR as a new enum to LinearSolverTerminationType.
3. Pipe SuiteSparse errors via a LinearSolverTerminationType so
to distinguish between fatal and non-fatal errors.
4. Update levenberg marquardt and dogleg strategies to deal
with FATAL_ERROR.
5. Update trust_region_minimizer to terminate when FATAL_ERROR
is encountered.
6. Remove SuiteSparse::SolveCholesky as it screws up the error
handling.
7. Fix all clients calling SuiteSparse to handle the result of
SuiteSparse::Cholesky correctly.
8. Remove fatal failures in SuiteSparse when symbolic factorization
fails.
9. Fix all clients of SuiteSparse to deal with null symbolic factors.
This is a temporary fix to deal with some production problems. A more
extensive cleanup and testing regime will be put in place in a
subsequent CL.
Change-Id: I1f60d539799dd95db7ecc340911e261fa4824f92
Main purpose of this is to make implementation files free from
endless ifdef blocks every time this libraries are needed to be
included. This would hopefully prevent compilation errors in
the future caused by missing ifdef around header include.
This also includes some stubs added to suitesparse/cxsparse
headers to make code even more free from ifdefs.
Change-Id: Ic8554e7df31d8c4751583fe004b99e71b3c9087b
1. Add -DLINE_SEARCH_MINIMIZER to CMake to make the line search
minimizer optional.
2. Better handling of -DSUITESPARSE/-DCXSPARSE in top level cmake
file.
3. Disable code which will never be used if SuiteSparse and/or
CXSparse is not available.
4. Update build docs.
5. Update jni/Android.mk
6. Minor lint cleanup from William Rucklidge.
Change-Id: If60460a858000df82faed7a6bb056dd2bfdde562
Average factorization times for bundle adjustment test problem:
SuiteSparse: 0.2794 s.
CXSparse: 0.4039 s.
CXSparse cached: 0.2399 s.
CXSparse will still be slower, though, because it has to compute
the transpose and J^T * J.
Change-Id: If9cdaa3dd520bee84b56e5fd4953b56a93db6bde
By virtue of the modeling layer in Ceres being block oriented,
all the matrices used by Ceres are also block oriented.
When doing sparse direct factorization of these matrices, the
fill-reducing ordering algorithms can either be run on the
block or the scalar form of these matrices. Running it on the
block form exposes more of the super-nodal structure of the
matrix to the Cholesky factorization routines. This leads to
substantial gains in factorization performance.
This changelist adds support for approximate minimium degree
orderings to be computed on the block structure of the
Schur complement matrix. This affects, SchurComplementSolver
and VisibilityBasedPreconditioner and SparseNormalCholesky
when using SuiteSparse.
A bool, use_block_amd has been added to Solver::Options and
bundle_adjuster.cc has been updated to allow testing with it.
When combined with a multithreaded Schur elimination, speed ups
can be seen quite uniformly across the board. For some problems
this can be dramatic, reducing the factorization time from 70
seconds down to 17 seconds.
Change-Id: I15ebb0afcbc85ada032ec8d179ee3a2f7c8d3e46
1. Added support for CXSparse - SparseNormalCholesky and
SchurComplementSolver support SuiteSparse and CXSparse now.
I am not sure I will add suport for visibility based
preconditioning using CXSparse. Its not a high priority.
2. New enum SparseLinearAlgebraLibraryType which allows the user
to indicate which sparse linear algebra library should be used.
3. Updated tests for SolverImpl and system_test.
4. Build system changes to automatically detect CXSparse and
link to it by default -- just like SuiteSparse.
5. Minor bug fixes dealing in the cmake files and VBP.
6. Changed the order of the system test.
7. Deduped the unsymmetric linear solver test.
Change-Id: I33252a103c87b722ecb7ed7b5f0ae7fd91249244
unnecessarily complexity in the structure of linear solvers and preconditioners.
This is the first step towards cleaning up the Preconditioner interface.
2. Minor tweaks and cleanups to the various linear solvers.
Sameer Agarwal