This change only affects the TRUST_REGION minimizer and has no effect
on the LINE_SEARCH minimizer.
This options controls the number of iterations that the line search
algorithm performs. The line search algorithm is as the name implies,
used in the LINE_SEARCH minimizer. It is also used by the TRUST_REGION
minimizer when solving bounds constrained optimization problems.
In some bounds constrained problems, it is enough to project each step
onto the bounds constraints and not perform the line search. This can
have a significant impact on runtime. Setting
Solver::Options::max_num_line_search_step_size_iterations = 0 enables
this functionality.
Changchang Wu (ccwu@google.com) came up with the idea for this
implementation.
https://github.com/ceres-solver/ceres-solver/issues/477
Change-Id: Ifbe0bc5c48eedb2c1231d43cd98e4be7316c0682
Also removed Solver::Summary::num_linear_solver_threads_given
and Solver::Summary::num_linear_solver_threads_used.
Change-Id: I559145ae2e7af597ea06ec03d386645a3a892e9f
1. Replace CERES_DISALLOW_* with explicitly deleted constructors.
2. Replace use of CERES_ARRAY_SIZE and stack allocated arrays
with std::vector.
3. Move CERES_ALIGN_* macros into manual_constructor.h, which is
the one place they are used and will be deprecated along with that
file.
4. Introduce isnan,isnormal,isinf and isfinite for Jets.
5. Replace IsNormal,IsFinite,IsNaN and IsInfinite with corresponding
c++11 function calls.
Change-Id: I04f33a221aae77d247602150988b6d4aa4efeeab
1. Add Solver::Options::use_mixed_precision_solves,
and Solver::Options::max_num_refinement_iterations.
2. Make SparseCholesky::Create return a unique_ptr.
3. SparseCholesky::Create now takes LinearSolver::Options
as an argument.
4. IterativeRefiner's constructor does not require num_cols
as an argument.
5. SparseNormalCholeskySolver now uses a separate rhs vector.
This basic implementation results in a 10% reduction in solver time
and 30% reduction in linear solver memory usage.
Change-Id: I6830f32cae2febf082d2733262eb2c9f0482b0ea
Migrate all Option and Summary structs to use
inline member initialization syntax.
This reduces the amount of code, and collocates the
default values with the documentation for the corresponding
member variable.
Change-Id: I8e6b9ee3b31464699d678667f6166ace5fc137c9
This adds a callback mechanism to for users to get notified just
before jacobian and residual evaluations. This will enable
aggressive caching and sharing of compute between cost functions.
Change-Id: I67993726920218edf71ab9ae70c34c204756c71a
1. Solver::Options::num_threads now controls parallelism in Ceres
Solver. The user specified value of
Solver::Options::num_linear_solver_threads is ignored.
2. If the user specifies Solver::Options::num_linear_solver_threads
and it is different from Solver::Options::num_threads,
a warning is printed.
3. Solver::Summary:num_linear_solver_threads_given and
Solver::Summary::num_linear_solver_threads_used are also
deprecated and are always set to Solver::Summary::num_threads_given
and Solver::Summary::num_threads_used.
Change-Id: I20b9336d9336e400e6f0a15b63857c0c43eb271c
A Ceres Context holds common global state that can be re-used within
Ceres. The Context current contains a thread pool if compiling with
C++11 threading support. Threads are expensive to create and destroy so
it is good to maintain across multiple Ceres solves.
Tested by compiling with and without TBB support and ran unit tests. Ran
bazel as well.
Change-Id: I82f598dfae642aa0e81a6039dc174608a5e8dbfb
1. Replace two maps by 1.
2. Update number of calls and the time for the call at the cost
of a single map lookup.
3. Add Solver::Summary::num_linear_solves.
Fixes https://github.com/ceres-solver/ceres-solver/issues/340
Change-Id: I71eb9be7fb363a8cb066591c4c1761f256c81677
1. Fix a typo in auto_diff_cost_function.h
2. Fix and update Solver::Summary::FullReport() text labels.
3. Add logging of the number of residual and jacobian evaluations
to the full report. The GradientProblemSolver already does this.
Change-Id: I41059af5f0ebe0417accbbc30b0808a4b04b9edb
1. Refactor the python code that generates the template specializations
to remove code duplication.
2. Improved the logic for template specialization selection where
Eigen::Dynamic now serves as a wildcard.
3. Added schur_templates.h/cc which allows querying the set of available
template specializations without instantiating a linear solver.
4. Added Solver::Summary::schur_structre_given and
Solver::Summary::schur_structure_used and expose them in
Solver::Summary::FullReport for better performance debugging.
5. Updated the templates with newer dates and some minor comments cleanup
which lead to the the template specializations to be re-generated.
Change-Id: Iaf3c6f714353597899916c300465da01f151c3de
Solver::Options::numeric_derivative_relative_step_size to
Solver::Options::gradient_check_numeric_derivative_relative_step_size
Change-Id: Ib89ae3f87e588d4aba2a75361770d2cec26f07aa
Accumulate the number of steps of the line search algorithm
and report it as part of Summary::FullReport.
Change-Id: I1de12784009a3e08f2a2c2aff5085d57a3c73828
1. Add answers to a number of FAQs.
2. Add a note to the documentation for NumericDiffCostFunction that
NumericDiffOptions needs to be documented and mentioned.
3. Update the docs for Solver::Options::numeric_derivative_relative_step_size
to indicate that this setting only applies to the gradient checker.
4. Remove deprecated constructors from NumericDiffCostFunction and
DynamicNumericDiffCostFunction.
Change-Id: If8fc011b2a5996dbc2c51268aa477550ed014a1c
Since Ceres is moving to using GitHub for issues, and the Google
Code URL in the current copyright header will soon become invalid,
update all the headers.
Change-Id: I1fce70375d1bcf098591f07b4d8f01a5c1e0789c
For historical reasons we had a "using namespace std;" in port.h. This
is generally a bad idea. So removing it and along the way doing a bunch
of cpplint cleanup.
Change-Id: Ia125601a55ae62695e247fb0250df4c6f86c46c6
- Previously we had no defined default value for
sparse_linear_algebra_library_type in Solver::Options if Ceres
was compiled with no sparse library available. Thus in that case,
the default value (dependent upon the compiler) would indicate that
one was available.
- Now we have an explicit option that means no sparse library is
available, which is now the default value in Solver::Options in this
case.
- Add a warning in CMake when the user disables all sparse libraries.
- Fix typos in trust_region_preprocessor_test:
(SUITE/CX)_SPARSE -> (SUITE/CX)SPARSE that induced failures when
no sparse libraries were available.
Change-Id: I869c399a12d42bfc44220cbb25ce6d6dd80236bd
- We now compute & report the cumulative time spent performing the
following tasks as part of a line search:
- Evaluation of the univariate cost function value & gradient.
- Minimization of the interpolating polynomial.
- Total time spent performing line searches.
- This information is now reported for all minimizers, although only in
the case of a constrained problem for the TR minimizer.
- Remove LineSearch::Function abstraction in place of using
LineSearchFunction implementation directly, and remove virtual
functions from LineSearchFunction.
-- LineSearch::Function added an unnecessary level of abstraction since
the user always had to create a LineSearchFunction anyway to use a
Ceres Evaluator, and it added an unncessary virtual function call.
Change-Id: Ia4e1921d78f351ae119875aa97a3ea5e8b5d9877
Up till now ITERATIVE_SCHUR evaluates matrix-vector products
between the Schur complement and a vector implicitly by exploiting
the algebraic expression for the Schur complement.
This cost of this evaluation scales with the number of non-zeros
in the Jacobian.
For small to medium sized problems there is a sweet spot where
computing the Schur complement is cheap enough that it is much
more efficient to explicitly compute it and use it for evaluating
the matrix-vector products.
This changes implements support for an explicit Schur complement
in ITERATIVE_SCHUR in combination with the SCHUR_JACOBI preconditioner.
API wise a new bool Solver::Options::use_explicit_schur_complement
has been added.
The implementation extends the SparseSchurComplementSolver to use
Conjugate Gradients.
Example speedup:
use_explicit_schur_complement = false
Time (in seconds):
Preprocessor 0.585
Residual evaluation 0.319
Jacobian evaluation 1.590
Linear solver 25.685
Minimizer 27.990
Postprocessor 0.010
Total 28.585
use_explicit_schur_complement = true
Time (in seconds):
Preprocessor 0.638
Residual evaluation 0.318
Jacobian evaluation 1.507
Linear solver 5.930
Minimizer 8.144
Postprocessor 0.010
Total 8.791
Which indicates an end-to-end speedup of more than 3x, with the linear
solver being sped up by > 4x.
The idea to explore this optimization was inspired by the recent paper:
Mining structure fragments for smart bundle adjustment
L. Carlone, P. Alcantarilla, H. Chiu, K. Zsolt, F. Dellaert
British Machine Vision Conference, 2014
which uses a more complicated algorithm to compute parts of the
Schur complement to speed up the matrix-vector product.
Change-Id: I95324af0ab351faa1600f5204039a1d2a64ae61d
1. Fix a build breakage in graph_test.
2. Respect Solver::Options::min_num_linear_solver_iterations in
conjugate_gradients_solver.cc
Thanks to Johannes Schönberger for reporting these.
Change-Id: Ib32e3929bf5d92dd576ae5b53d4d88797095136e
Replace SolverImpl with
a. A minimizer specific preprocessor class.
b. A generic Solve function inside solver.cc
c. Presummarize and Postsummarize functions to handle
updates to the summary object.
The existing SolverImpl class was a mixture of the above three
things and was increasingly complicated code to follow. This change,
breaks it into its three separate constituents, with the aims of
better separation of concerns and thus better testability and
reliability.
The call to Solver::Solve() now consists of
1. Presummarize - summarize the given state of the problem and solver
options.
2. Preprocess - Setup everything that is needed to call the minimizer.
This includes, removing redundant parameter and residual blocks,
setting up the reordering for the linear solver, creating the
linear solver, evaluator, inner iteration minimizer etc.
3. Minimize.
4. Post summarize - summarize the result of the preprocessing and the
solve.
Change-Id: I80f35cfc9f2cbf78f1df4aceace27075779d8a3a
SPARSE_NORMAL_CHOLESKY and SPARSE_SCHUR can now be used
with EIGEN_SPARSE as the backend.
The performance is not as good as CXSparse. This needs to be
investigated. Is it because the quality of AMD ordering that
we are computing is not as good as the one for CXSparse? This
could be because we are working with the scalar matrix instead
of the block matrix.
Also, the upper/lower triangular story is not completely clear.
Both of these issues will be benchmarked and tackled in the
near future.
Also included in this change is a bunch of cleanup to the
SparseNormalCholeskySolver and SparseSchurComplementSolver
classes around the use of the of defines used to conditionally
compile out parts of the code.
The system_test has been updated to test EIGEN_SPARSE also.
Change-Id: I46a57e9c4c97782696879e0b15cfc7a93fe5496a
The standard sparse normal Cholesky solver assumes a fixed
sparsity pattern which is useful for a large number of problems
presented to Ceres. However, some problems are symbolically dense
but numerically sparse i.e. each residual is a function of a
large number of parameters but at any given state the residual
only depends on a sparse subset of them. For these class of
problems it is faster to re-analyse the sparsity pattern of the
jacobian at each iteration of the non-linear optimisation instead
of including all of the zero entries in the step computation.
The proposed solution adds the dynamic_sparsity option which can
be used with SPARSE_NORMAL_CHOLESKY. A
DynamicCompressedRowSparseMatrix type (which extends
CompressedRowSparseMatrix) has been introduced which allows
dynamic addition and removal of elements. A Finalize method is
provided which then consolidates the matrix so that it can be
used in place of a regular CompressedRowSparseMatrix. An
associated jacobian writer has also been provided.
Changes that were required to make this extension were adding the
SetMaxNumNonZeros method to CompressedRowSparseMatrix and adding
a JacobianFinalizer template parameter to the ProgramEvaluator.
Change-Id: Ia5a8a9523fdae8d5b027bc35e70b4611ec2a8d01
Solver::Options::linear_solver_ordering and
Solver::Options::inner_iteration_ordering
were bare pointers even though Solver::Options took ownership of these
objects.
This lead to buggy user code and the inability to copy Solver::Options
objects around.
With this change, these naked pointers have been replaced by a
shared_ptr object which will managed the lifetime of these objects. This
also leads to simplification of the lifetime handling of these objects
inside the solver.
The Android.mk and Application.mk files have also been updated
to use a newer NDK revision which ships with LLVM's libc++.
Change-Id: I25161fb3ddf737be0b3e5dfd8e7a0039b22548cd
1. Rename SolverTerminationType to TerminationType.
2. Consolidate the enum as
a. CONVERGENCE - subsumes FUNCTION_TOLERANCE, PARAMETER_TOLERANCE and GRADIENT_TOLERANCE
b. NO_CONVERGENCE
c. FAILURE - captures all kinds of failures including DID_NOT_RUN.
d. USER_SUCCESS
e. USER_FAILURE
3. Solver::Summary::error is renamed to be Solver::Summary::message, to both
reduce confusion as well as capture its true meaning.
Change-Id: I27a382e66e67f5a4750d0ee914d941f6b53c326d
The original visibility based preconditioning paper and
implementation only used the canonical views algorithm.
This algorithm for large dense graphs can be particularly
expensive. As its worst case complexity is cubic in size
of the graph.
Further, for many uses the SCHUR_JACOBI preconditioner
was both effective enough while being cheap. It however
suffers from a fatal flaw. If the camera parameter blocks
are split between two or more parameter blocks, e.g,
extrinsics and intrinsics. The preconditioner because
it is block diagonal will not capture the interactions
between them.
Using CLUSTER_JACOBI or CLUSTER_TRIDIAGONAL will fix
this problem but as mentioned above this can be quite
expensive depending on the problem.
This change extends the visibility based preconditioner
to allow for multiple clustering algorithms. And adds
a simple thresholded single linkage clustering algorithm
which allows you to construct versions of CLUSTER_JACOBI
and CLUSTER_TRIDIAGONAL preconditioners that are cheap
to construct and are more effective than SCHUR_JACOBI.
Currently the constants controlling the threshold above
which edges are considered in the single linkage algorithm
are not exposed. This would be done in a future change.
Change-Id: I7ddc36790943f24b19c7f08b10694ae9a822f5c9
1. Update the documentation to be Sphinx friendly.
2. Remove dead fields in Solver::Summary.
a. Solver::Summary::num_eliminate_blocks_given.
b. Solver::Summary::num_eliminate_blocks_used.
Change-Id: I43e0070c88abe3bf285d91e6c7524f3d887deb33
1. When a LAPACK implementation is present, then
DENSE_QR, DENSE_NORMAL_CHOLESKY and DENSE_SCHUR
can use it for doing dense linear algebra operations.
2. The user can switch dense linear algebra libraries
by setting Solver::Options::dense_linear_algebra_library_type.
3. Solver::Options::sparse_linear_algebra_library is now
Solver::Options::sparse_linear_algebra_library_type to be consistent
with all the other enums in Solver::Options.
4. Updated documentation as well as Solver::Summary::FullReport
to reflect these changes.
Change-Id: I5ab930bc15e90906b648bc399b551e6bd5d6498f
lm_max_diagonal -> max_lm_diagonal
lm_min_diagonal -> min_lm_diagonal
linear_solver_max_num_iterations -> max_linear_solver_iterations
linear_solver_min_num_iterations -> min_linear_solver_iterations
This follows the pattern for the other parameters in Solver::Options
where, the max/min is the first word followed by the name of the
parameter.
Change-Id: I0893610fceb6b7983fdb458a65522ba7079596a7
This support was broken due to the TrustRegionMinimizer refactoring.
It is now enabled again, with the responsibilty for dumping the
problem shifted to the individual TrustRegionStrategy.
There is however one wrinkle, which is perhaps an indication of
poor design to start with. The LinearLeastSquaresProblemProto
carries in it num_eliminate_blocks, something which does not
exist anymore. More importantly, the TrustRegionStrategy does not
have access to this quantity anymore.
Dealing with this will be the subject of a future change.
Change-Id: I358adf6a2e386f4940b617bf950d6c7e87d2635d
Sameer Agarwal