Eigen provides all the functionality that we need from CXSparse
with a more liberal license.
I will update the documentation in a follow up CL.
Change-Id: I0b9fd8be3c27754cc2986cc0e06595c8b3fdec0b
This was an ill-advised and complicated to interpret option
which offers nothing particularly useful.
Change-Id: Ia7741ed62ef977c96fa52299a884e404bee659ac
With this change, the user can now choose between Approximate Minimum
Degree and Nested Dissection as a fill reducing algorithm when using
a sparse direct factorization based linear solver like SPARSE_NORMAL_CHOLESKY
or SPARSE_SCHUR.
Currenly only SUITE_SPARSE is supported. It requires that
SuiteSparse be compiled with Metis support enabled.
On most problems AMD is still the better choice, but in some cases
like the grid3D dataset from https://lucacarlone.mit.edu/datasets/
the solution time with AMD is 57s and with NESDIS 38 on my M1 Mac.
On some other problems at Google we have observed speedups of 10x,
there is also a corresponding decrease in the total amount of memory
used.
This patch is based on the original work done by NeroBurner in
https://ceres-solver-review.googlesource.com/c/ceres-solver/+/20580
1. Add a new enum to the public api LinearSolverOrderingType and
a setting Solver::Options::linear_solver_ordering_type.
2. TrustRegionPreprocessor had some complicated logic which determined
when linear solvers should reorder their matrices on their own and not
this has been refactored into a more readable function that lives
inside reorder_program.h/cc.
3. Plumbing in reorder_program.cc and trust_region_processor.cc to use
nested dissection.
4. Update bundle_adjuster.cc to use nested dissection.
Change-Id: I388b027934f86c58b4da2b65a4fa5204ea73bf40
1. Fix a stupid error in types.cc
2. Update documentation for Solver::Options::dense_linear_algebra_library_type
3. Add a note to installation.rst to update the installation docs.
4. Mention GPU acceleration in features.rst
Change-Id: Id63202ff090e23bbb211d2ee458559fb8046281d
Currently, the logic for exporting symbols is rather complicated: when
tests are enabled internal symbols are exported in addition to the
public symbols. Such logic causes several problems. (1) Test binaries
link against a Ceres build that is different from the final release
since fewer optimizations are applied if more symbols are exported. (2)
Also, some toolchains hide symbols by default breaking the existing
logic eventually causing linker errors.
Since internal symbols are not intended to be used outside of the
project, we can compile them into object files and use exactly the same
binary code both for the final build and the tests without relying on
conditionals.
By default, all symbols are now hidden unless annotated as public.
Internal symbols are explicitly marked as not being exported in case
users chose not to hide symbols by default.
Change-Id: I589dd10be2f6f438508783cf99d141af0120057b
Add [[deprecate]] notices to everything LocalParameterization
related.
Make sure that Ceres can be compiled without triggering
deprecation warnings.
Update the documentation:
a. Add deprecation notices.
b. Document interaction between LocalParameterization and Manifold
coexisting in the Problem.
c. Add documentation for Manifold(s)
Change-Id: Ie4ad48963c83fded86e533c8c60561af402fbaff
- Also adds documentation of mixed precision solves to Sphinx docs.
- Fix reference to Sphinx theme used (RTD not better).
- Fix NOTE syntax in use_explicit_schur_complement Sphinx docs.
Change-Id: I7bdac0f07eb737f49b05e3fcaa3eebd087355d2d
- ensure all public headers files adhere to clang-format
- preserve one-per-line for enums by adding trailing comma
- preserve include order for en/disable_warning.h
Change-Id: I78dbd0527a294ab2ec5f074fb426e48b20c393e6
Re-organize the section, add some more references and details for
existing preconditioners and add documentation for the SUBSET
precondition.
https://github.com/ceres-solver/ceres-solver/issues/490
Change-Id: I93d0af819c160f5e4ce48b18202f629ddb92ca7b
https://github.com/ceres-solver/ceres-solver/issues/270
Detailed list of changes:
1. Add SUBSET to the PreconditionerType enum.
2. Add Solver::Options::residual_blocks_for_subset_preconditioner
3. Integrate SubsetPreconditioner into the CGNR solver.
4. Add the reordering logic needed for this to TrustRegionPreprocessor.
5. Expect CreateJacobianBlockTranspose to take the starting row block
so that we can work with subparts of the Jacobian matrix.
6. Extend the denoising example to use this preconditioner.
As an illustration of its performance, we consider the performance of
denoising -input ../data/ceres_noisy.pgm --foe_file ../data/5x5.foe
tl;dr
For the same cost,
SPARSE_NORMAL_CHOLESKY - 81s
CGNR + JACOBI - 718s
CGNR + SUBSET - 57s
SPARSE_NORMAL_CHOLESKY
======================
Cost:
Initial 2.317806e+05
Final 2.232323e+04
Change 2.094574e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 2.999746
Residual only evaluation 2.306811 (10)
Jacobian & residual evaluation 7.421727 (10)
Linear solver 65.517273 (10)
Minimizer 78.731011
Postprocessor 0.026079
Total 81.756836
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.573046e-04 <= 1.000000e-03)
CGNR + JACOBI
=============
Cost:
Initial 2.317806e+05
Final 2.232344e+04
Change 2.094572e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 0.648814
Residual only evaluation 2.297607 (10)
Jacobian & residual evaluation 7.327886 (10)
Linear solver 699.601248 (10)
Minimizer 712.419493
Postprocessor 0.024014
Total 713.092321
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.528538e-04 <= 1.000000e-03)
CGNR + SUBSET (random 20% residuals used for the preconditioner)
===============================================================
Cost:
Initial 2.317806e+05
Final 2.232327e+04
Change 2.094574e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 1.472743
Residual only evaluation 2.428315 (10)
Jacobian & residual evaluation 7.367796 (10)
Linear solver 42.585999 (10)
Minimizer 55.664459
Postprocessor 0.024098
Total 57.161301
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.538277e-04 <= 1.000000e-03)
Change-Id: Ifb011408bd53edbb9439b0b7345649a38f999e18
- Accelerate currently does not support dynamic sparsity, whereas the
other sparse linear algebra libraries do (outstanding issue to update)
- Previously we preferred Accelerate to all but SuiteSparse if it was
available, which breaks the dynamic_sparsity_test if SuiteSparse is
*not* available (even if others are) as Accelerate does not support
dynamic sparsity.
Change-Id: Ibc2dd2f14f83cffbecca38097d02bb2188aaaa05
This change only affects the TRUST_REGION minimizer and has no effect
on the LINE_SEARCH minimizer.
This options controls the number of iterations that the line search
algorithm performs. The line search algorithm is as the name implies,
used in the LINE_SEARCH minimizer. It is also used by the TRUST_REGION
minimizer when solving bounds constrained optimization problems.
In some bounds constrained problems, it is enough to project each step
onto the bounds constraints and not perform the line search. This can
have a significant impact on runtime. Setting
Solver::Options::max_num_line_search_step_size_iterations = 0 enables
this functionality.
Changchang Wu (ccwu@google.com) came up with the idea for this
implementation.
https://github.com/ceres-solver/ceres-solver/issues/477
Change-Id: Ifbe0bc5c48eedb2c1231d43cd98e4be7316c0682
Also removed Solver::Summary::num_linear_solver_threads_given
and Solver::Summary::num_linear_solver_threads_used.
Change-Id: I559145ae2e7af597ea06ec03d386645a3a892e9f
1. Replace CERES_DISALLOW_* with explicitly deleted constructors.
2. Replace use of CERES_ARRAY_SIZE and stack allocated arrays
with std::vector.
3. Move CERES_ALIGN_* macros into manual_constructor.h, which is
the one place they are used and will be deprecated along with that
file.
4. Introduce isnan,isnormal,isinf and isfinite for Jets.
5. Replace IsNormal,IsFinite,IsNaN and IsInfinite with corresponding
c++11 function calls.
Change-Id: I04f33a221aae77d247602150988b6d4aa4efeeab
1. Add Solver::Options::use_mixed_precision_solves,
and Solver::Options::max_num_refinement_iterations.
2. Make SparseCholesky::Create return a unique_ptr.
3. SparseCholesky::Create now takes LinearSolver::Options
as an argument.
4. IterativeRefiner's constructor does not require num_cols
as an argument.
5. SparseNormalCholeskySolver now uses a separate rhs vector.
This basic implementation results in a 10% reduction in solver time
and 30% reduction in linear solver memory usage.
Change-Id: I6830f32cae2febf082d2733262eb2c9f0482b0ea
Migrate all Option and Summary structs to use
inline member initialization syntax.
This reduces the amount of code, and collocates the
default values with the documentation for the corresponding
member variable.
Change-Id: I8e6b9ee3b31464699d678667f6166ace5fc137c9
This adds a callback mechanism to for users to get notified just
before jacobian and residual evaluations. This will enable
aggressive caching and sharing of compute between cost functions.
Change-Id: I67993726920218edf71ab9ae70c34c204756c71a
1. Solver::Options::num_threads now controls parallelism in Ceres
Solver. The user specified value of
Solver::Options::num_linear_solver_threads is ignored.
2. If the user specifies Solver::Options::num_linear_solver_threads
and it is different from Solver::Options::num_threads,
a warning is printed.
3. Solver::Summary:num_linear_solver_threads_given and
Solver::Summary::num_linear_solver_threads_used are also
deprecated and are always set to Solver::Summary::num_threads_given
and Solver::Summary::num_threads_used.
Change-Id: I20b9336d9336e400e6f0a15b63857c0c43eb271c
A Ceres Context holds common global state that can be re-used within
Ceres. The Context current contains a thread pool if compiling with
C++11 threading support. Threads are expensive to create and destroy so
it is good to maintain across multiple Ceres solves.
Tested by compiling with and without TBB support and ran unit tests. Ran
bazel as well.
Change-Id: I82f598dfae642aa0e81a6039dc174608a5e8dbfb
1. Replace two maps by 1.
2. Update number of calls and the time for the call at the cost
of a single map lookup.
3. Add Solver::Summary::num_linear_solves.
Fixes https://github.com/ceres-solver/ceres-solver/issues/340
Change-Id: I71eb9be7fb363a8cb066591c4c1761f256c81677
1. Fix a typo in auto_diff_cost_function.h
2. Fix and update Solver::Summary::FullReport() text labels.
3. Add logging of the number of residual and jacobian evaluations
to the full report. The GradientProblemSolver already does this.
Change-Id: I41059af5f0ebe0417accbbc30b0808a4b04b9edb
1. Refactor the python code that generates the template specializations
to remove code duplication.
2. Improved the logic for template specialization selection where
Eigen::Dynamic now serves as a wildcard.
3. Added schur_templates.h/cc which allows querying the set of available
template specializations without instantiating a linear solver.
4. Added Solver::Summary::schur_structre_given and
Solver::Summary::schur_structure_used and expose them in
Solver::Summary::FullReport for better performance debugging.
5. Updated the templates with newer dates and some minor comments cleanup
which lead to the the template specializations to be re-generated.
Change-Id: Iaf3c6f714353597899916c300465da01f151c3de
Solver::Options::numeric_derivative_relative_step_size to
Solver::Options::gradient_check_numeric_derivative_relative_step_size
Change-Id: Ib89ae3f87e588d4aba2a75361770d2cec26f07aa
Accumulate the number of steps of the line search algorithm
and report it as part of Summary::FullReport.
Change-Id: I1de12784009a3e08f2a2c2aff5085d57a3c73828
1. Add answers to a number of FAQs.
2. Add a note to the documentation for NumericDiffCostFunction that
NumericDiffOptions needs to be documented and mentioned.
3. Update the docs for Solver::Options::numeric_derivative_relative_step_size
to indicate that this setting only applies to the gradient checker.
4. Remove deprecated constructors from NumericDiffCostFunction and
DynamicNumericDiffCostFunction.
Change-Id: If8fc011b2a5996dbc2c51268aa477550ed014a1c
Since Ceres is moving to using GitHub for issues, and the Google
Code URL in the current copyright header will soon become invalid,
update all the headers.
Change-Id: I1fce70375d1bcf098591f07b4d8f01a5c1e0789c
For historical reasons we had a "using namespace std;" in port.h. This
is generally a bad idea. So removing it and along the way doing a bunch
of cpplint cleanup.
Change-Id: Ia125601a55ae62695e247fb0250df4c6f86c46c6
- Previously we had no defined default value for
sparse_linear_algebra_library_type in Solver::Options if Ceres
was compiled with no sparse library available. Thus in that case,
the default value (dependent upon the compiler) would indicate that
one was available.
- Now we have an explicit option that means no sparse library is
available, which is now the default value in Solver::Options in this
case.
- Add a warning in CMake when the user disables all sparse libraries.
- Fix typos in trust_region_preprocessor_test:
(SUITE/CX)_SPARSE -> (SUITE/CX)SPARSE that induced failures when
no sparse libraries were available.
Change-Id: I869c399a12d42bfc44220cbb25ce6d6dd80236bd
- We now compute & report the cumulative time spent performing the
following tasks as part of a line search:
- Evaluation of the univariate cost function value & gradient.
- Minimization of the interpolating polynomial.
- Total time spent performing line searches.
- This information is now reported for all minimizers, although only in
the case of a constrained problem for the TR minimizer.
- Remove LineSearch::Function abstraction in place of using
LineSearchFunction implementation directly, and remove virtual
functions from LineSearchFunction.
-- LineSearch::Function added an unnecessary level of abstraction since
the user always had to create a LineSearchFunction anyway to use a
Ceres Evaluator, and it added an unncessary virtual function call.
Change-Id: Ia4e1921d78f351ae119875aa97a3ea5e8b5d9877
Up till now ITERATIVE_SCHUR evaluates matrix-vector products
between the Schur complement and a vector implicitly by exploiting
the algebraic expression for the Schur complement.
This cost of this evaluation scales with the number of non-zeros
in the Jacobian.
For small to medium sized problems there is a sweet spot where
computing the Schur complement is cheap enough that it is much
more efficient to explicitly compute it and use it for evaluating
the matrix-vector products.
This changes implements support for an explicit Schur complement
in ITERATIVE_SCHUR in combination with the SCHUR_JACOBI preconditioner.
API wise a new bool Solver::Options::use_explicit_schur_complement
has been added.
The implementation extends the SparseSchurComplementSolver to use
Conjugate Gradients.
Example speedup:
use_explicit_schur_complement = false
Time (in seconds):
Preprocessor 0.585
Residual evaluation 0.319
Jacobian evaluation 1.590
Linear solver 25.685
Minimizer 27.990
Postprocessor 0.010
Total 28.585
use_explicit_schur_complement = true
Time (in seconds):
Preprocessor 0.638
Residual evaluation 0.318
Jacobian evaluation 1.507
Linear solver 5.930
Minimizer 8.144
Postprocessor 0.010
Total 8.791
Which indicates an end-to-end speedup of more than 3x, with the linear
solver being sped up by > 4x.
The idea to explore this optimization was inspired by the recent paper:
Mining structure fragments for smart bundle adjustment
L. Carlone, P. Alcantarilla, H. Chiu, K. Zsolt, F. Dellaert
British Machine Vision Conference, 2014
which uses a more complicated algorithm to compute parts of the
Schur complement to speed up the matrix-vector product.
Change-Id: I95324af0ab351faa1600f5204039a1d2a64ae61d
1. Fix a build breakage in graph_test.
2. Respect Solver::Options::min_num_linear_solver_iterations in
conjugate_gradients_solver.cc
Thanks to Johannes Schönberger for reporting these.
Change-Id: Ib32e3929bf5d92dd576ae5b53d4d88797095136e
Replace SolverImpl with
a. A minimizer specific preprocessor class.
b. A generic Solve function inside solver.cc
c. Presummarize and Postsummarize functions to handle
updates to the summary object.
The existing SolverImpl class was a mixture of the above three
things and was increasingly complicated code to follow. This change,
breaks it into its three separate constituents, with the aims of
better separation of concerns and thus better testability and
reliability.
The call to Solver::Solve() now consists of
1. Presummarize - summarize the given state of the problem and solver
options.
2. Preprocess - Setup everything that is needed to call the minimizer.
This includes, removing redundant parameter and residual blocks,
setting up the reordering for the linear solver, creating the
linear solver, evaluator, inner iteration minimizer etc.
3. Minimize.
4. Post summarize - summarize the result of the preprocessing and the
solve.
Change-Id: I80f35cfc9f2cbf78f1df4aceace27075779d8a3a
SPARSE_NORMAL_CHOLESKY and SPARSE_SCHUR can now be used
with EIGEN_SPARSE as the backend.
The performance is not as good as CXSparse. This needs to be
investigated. Is it because the quality of AMD ordering that
we are computing is not as good as the one for CXSparse? This
could be because we are working with the scalar matrix instead
of the block matrix.
Also, the upper/lower triangular story is not completely clear.
Both of these issues will be benchmarked and tackled in the
near future.
Also included in this change is a bunch of cleanup to the
SparseNormalCholeskySolver and SparseSchurComplementSolver
classes around the use of the of defines used to conditionally
compile out parts of the code.
The system_test has been updated to test EIGEN_SPARSE also.
Change-Id: I46a57e9c4c97782696879e0b15cfc7a93fe5496a
Sameer Agarwal