Add logic for checking for availability of CUDA as the
dense linear algebra library before allowing the user
to use it.
Change-Id: I0ceafa1052632504b33685bc731366ef6933e518
Currently, the logic for exporting symbols is rather complicated: when
tests are enabled internal symbols are exported in addition to the
public symbols. Such logic causes several problems. (1) Test binaries
link against a Ceres build that is different from the final release
since fewer optimizations are applied if more symbols are exported. (2)
Also, some toolchains hide symbols by default breaking the existing
logic eventually causing linker errors.
Since internal symbols are not intended to be used outside of the
project, we can compile them into object files and use exactly the same
binary code both for the final build and the tests without relying on
conditionals.
By default, all symbols are now hidden unless annotated as public.
Internal symbols are explicitly marked as not being exported in case
users chose not to hide symbols by default.
Change-Id: I589dd10be2f6f438508783cf99d141af0120057b
Applied changes correspond to clang-tidy fixes
stemming from the modernize-use-equals-default check.
Change-Id: I254b0908a76d464131564b637cd0e42a6b03fb5a
VLOG_IF's evaluation order is ambiguous - does it mean
`if (cond) VLOG(lvl)` or `if (VLOG_IS_ON(lvl) && cond) LOG(INFO)`?
In particular, the way it works now is inconsistent with the way the
rest of the LOG macros evaluate their arguments.
Fixing this would be hard, and the macro's behavior would still surprise
some people. Replacing it with an if statement is simple, clear, and unambiguous.
Change-Id: I97a92d17a932c0a5344a1bf98d676308793ba877
- Change formatting standard to Cpp11. Main difference is not having
the space between two closing >> for nested templates. We don't
choose c++14, because older versions of clang-format (version 9
and earlier) don't know this value yet, and it doesn't make a
difference in the formatting.
- Apply clang-format to all (non generated) internal source files.
- Manually fix some code sections (clang-format on/off) and c-strings
- Exclude some embedded external files with very different formatting
(gtest/gmock)
- Add script to format all source files
Change-Id: Ic6cea41575ad6e37c9e136dbce176b0d505dc44d
https://github.com/ceres-solver/ceres-solver/issues/270
Detailed list of changes:
1. Add SUBSET to the PreconditionerType enum.
2. Add Solver::Options::residual_blocks_for_subset_preconditioner
3. Integrate SubsetPreconditioner into the CGNR solver.
4. Add the reordering logic needed for this to TrustRegionPreprocessor.
5. Expect CreateJacobianBlockTranspose to take the starting row block
so that we can work with subparts of the Jacobian matrix.
6. Extend the denoising example to use this preconditioner.
As an illustration of its performance, we consider the performance of
denoising -input ../data/ceres_noisy.pgm --foe_file ../data/5x5.foe
tl;dr
For the same cost,
SPARSE_NORMAL_CHOLESKY - 81s
CGNR + JACOBI - 718s
CGNR + SUBSET - 57s
SPARSE_NORMAL_CHOLESKY
======================
Cost:
Initial 2.317806e+05
Final 2.232323e+04
Change 2.094574e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 2.999746
Residual only evaluation 2.306811 (10)
Jacobian & residual evaluation 7.421727 (10)
Linear solver 65.517273 (10)
Minimizer 78.731011
Postprocessor 0.026079
Total 81.756836
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.573046e-04 <= 1.000000e-03)
CGNR + JACOBI
=============
Cost:
Initial 2.317806e+05
Final 2.232344e+04
Change 2.094572e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 0.648814
Residual only evaluation 2.297607 (10)
Jacobian & residual evaluation 7.327886 (10)
Linear solver 699.601248 (10)
Minimizer 712.419493
Postprocessor 0.024014
Total 713.092321
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.528538e-04 <= 1.000000e-03)
CGNR + SUBSET (random 20% residuals used for the preconditioner)
===============================================================
Cost:
Initial 2.317806e+05
Final 2.232327e+04
Change 2.094574e+05
Minimizer iterations 10
Successful steps 10
Unsuccessful steps 0
Time (in seconds):
Preprocessor 1.472743
Residual only evaluation 2.428315 (10)
Jacobian & residual evaluation 7.367796 (10)
Linear solver 42.585999 (10)
Minimizer 55.664459
Postprocessor 0.024098
Total 57.161301
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.538277e-04 <= 1.000000e-03)
Change-Id: Ifb011408bd53edbb9439b0b7345649a38f999e18
- Accelerate currently does not support dynamic sparsity, whereas the
other sparse linear algebra libraries do (outstanding issue to update)
- Previously we preferred Accelerate to all but SuiteSparse if it was
available, which breaks the dynamic_sparsity_test if SuiteSparse is
*not* available (even if others are) as Accelerate does not support
dynamic sparsity.
Change-Id: Ibc2dd2f14f83cffbecca38097d02bb2188aaaa05
This change only affects the TRUST_REGION minimizer and has no effect
on the LINE_SEARCH minimizer.
This options controls the number of iterations that the line search
algorithm performs. The line search algorithm is as the name implies,
used in the LINE_SEARCH minimizer. It is also used by the TRUST_REGION
minimizer when solving bounds constrained optimization problems.
In some bounds constrained problems, it is enough to project each step
onto the bounds constraints and not perform the line search. This can
have a significant impact on runtime. Setting
Solver::Options::max_num_line_search_step_size_iterations = 0 enables
this functionality.
Changchang Wu (ccwu@google.com) came up with the idea for this
implementation.
https://github.com/ceres-solver/ceres-solver/issues/477
Change-Id: Ifbe0bc5c48eedb2c1231d43cd98e4be7316c0682
This method gives the user the ability to evaluate a given residual
block.
A couple of minor cleanups.
Problem::problem_impl_ -> Problem::impl_
NULL -> nullptr
https://github.com/ceres-solver/ceres-solver/issues/417
Change-Id: I6dd94762c475fa264c387b8c93d516f6e06fe832
When Solver::Options::check_gradients is true, Ceres internally
creates a new ProblemImpl object which wraps each CostFunction
in the user's problem with a GradientCheckingCostFunction.
Doing this also requires creating new ParameterBlock objects,
and when support for upper and lower bounds was added to Ceres,
CreateGradientCheckingProblemImpl should also have been updated
to create a problem with the same parameter bounds. As a result,
if check_gradients is enabled for a bounded problem, it constructs
an unconstrained problem and solves it.
This CL fixes this, by introducing Problem::GetParameterLowerBound,
and Problem::GetParameterUpperBound and using them to create a bounded
problem when checking gradients.
Thanks to @pbeeson for not only reporting this problem, but also
providing a small standalone reproduction which made debugging this
possible.
https://github.com/ceres-solver/ceres-solver/issues/379
Change-Id: Id18eb858a7009bf4fa452a21b925922d13f3249f
CLUSTER_JACOBI and CLUSTER_TRIDIAGONAL preconditioners require a sparse
Cholesky factorization library. Previously this code was hard coded to
use SuiteSparse. Once SparseCholesky abstraction was introduced, it
became possible to use it with Eigen or CXSParse.
Unfortunately the old code path that was enforcing the requirement
of SuiteSparse was not removed.
This change does the following:
1. Remove the SUITE_SPARSE restriction on CLUSTER_TRIDIAGONAL and
CLUSTER_JACOBI
2. Redo the checking code to be ifdef free and more general.
3. Add bundle adjustment tests to test these configurations.
Thanks to Bjorn Piltz for catching and reporting this bug.
Change-Id: I637791f6f8149694b6aa75f6a4b6417398cb9590
Also removed Solver::Summary::num_linear_solver_threads_given
and Solver::Summary::num_linear_solver_threads_used.
Change-Id: I559145ae2e7af597ea06ec03d386645a3a892e9f
Migrate all Option and Summary structs to use
inline member initialization syntax.
This reduces the amount of code, and collocates the
default values with the documentation for the corresponding
member variable.
Change-Id: I8e6b9ee3b31464699d678667f6166ace5fc137c9
This adds a callback mechanism to for users to get notified just
before jacobian and residual evaluations. This will enable
aggressive caching and sharing of compute between cost functions.
Change-Id: I67993726920218edf71ab9ae70c34c204756c71a
1. When Solver::Options::update_state_every_iteration = true,
the StateUpdatingCallback only updates the state on a
successful iteration. This works fine but will not work
if the user provides an EvaluationCallback, because
every call to Evaluator::Evaluate will change the state
visible to the user. This means that on unsuccessful
iterations when the user's IterationCallback is called
the state visible to the user would be the last evaluation
which did not lead to an improved cost. This CL forces
the StateUpdatingCallback to unconditionally update
the user visible state.
2. When the minimizer terminates, we update the user visible
state only if the solution is usable, otherwise the user's
state will remain the same. To maintain this invariant
we will now cache the user's state and make sure we
use it to reset it upon return if the solver is not
successful.
Change-Id: Ic1f8fa6cb10d2753130ea752c70ff3d6b2f1462f
1. Solver::Options::num_threads now controls parallelism in Ceres
Solver. The user specified value of
Solver::Options::num_linear_solver_threads is ignored.
2. If the user specifies Solver::Options::num_linear_solver_threads
and it is different from Solver::Options::num_threads,
a warning is printed.
3. Solver::Summary:num_linear_solver_threads_given and
Solver::Summary::num_linear_solver_threads_used are also
deprecated and are always set to Solver::Summary::num_threads_given
and Solver::Summary::num_threads_used.
Change-Id: I20b9336d9336e400e6f0a15b63857c0c43eb271c
A Ceres Context holds common global state that can be re-used within
Ceres. The Context current contains a thread pool if compiling with
C++11 threading support. Threads are expensive to create and destroy so
it is good to maintain across multiple Ceres solves.
Tested by compiling with and without TBB support and ran unit tests. Ran
bazel as well.
Change-Id: I82f598dfae642aa0e81a6039dc174608a5e8dbfb
1. Replace two maps by 1.
2. Update number of calls and the time for the call at the cost
of a single map lookup.
3. Add Solver::Summary::num_linear_solves.
Fixes https://github.com/ceres-solver/ceres-solver/issues/340
Change-Id: I71eb9be7fb363a8cb066591c4c1761f256c81677
1. Fix a typo in auto_diff_cost_function.h
2. Fix and update Solver::Summary::FullReport() text labels.
3. Add logging of the number of residual and jacobian evaluations
to the full report. The GradientProblemSolver already does this.
Change-Id: I41059af5f0ebe0417accbbc30b0808a4b04b9edb
In case one has a small problem to solve, where
completion is reached within a few milliseconds,
then four decimal places are not enough to accurately
represent the timings of all the separate sub-steps.
Thus, we extend the reported timings to include
six decimal places.
Change-Id: Iaf88a94a1b8896ea7370c75b1de2f05d8671206e
1. Add schur_templates.cc to Android.mk
2. When detecting the Schur structure of the Jacobian,
the check for whether linear solver being used is indeed
of Schur type or not, should use the LinearSolver::Options
struct created and populated by the preprocessor rather than
depending on the value in the input Solver::Options. The reason
is that the preprocessor may change the linear solver type
depending on the lack of a Schur structure in the problem.
Change-Id: I6f018f6817c05d704409181c7b1e25155528ab84
1. Refactor the python code that generates the template specializations
to remove code duplication.
2. Improved the logic for template specialization selection where
Eigen::Dynamic now serves as a wildcard.
3. Added schur_templates.h/cc which allows querying the set of available
template specializations without instantiating a linear solver.
4. Added Solver::Summary::schur_structre_given and
Solver::Summary::schur_structure_used and expose them in
Solver::Summary::FullReport for better performance debugging.
5. Updated the templates with newer dates and some minor comments cleanup
which lead to the the template specializations to be re-generated.
Change-Id: Iaf3c6f714353597899916c300465da01f151c3de
Solver::Options::numeric_derivative_relative_step_size to
Solver::Options::gradient_check_numeric_derivative_relative_step_size
Change-Id: Ib89ae3f87e588d4aba2a75361770d2cec26f07aa
Change the Ceres gradient checking API to make is useful for
unit testing, clean up code duplication and fix interaction between
gradient checking and local parameterizations.
There were two gradient checking implementations, one being used
when using the check_gradients flag in the Solver, the other
being a standalone class. The standalone version was restricted
to cost functions with fixed parameter sizes at compile time, which
is being lifted here. This enables it to be used inside the
GradientCheckingCostFunction as well.
In addition, this installs new hooks in the Solver to ensure
that Solve will fail if any incorrect gradients are detected. This
way, you can set the check_gradient flags to true and detect
errors in an automated way, instead of just printing error information
to the log. The error log is now also returned in the Solver summary
instead of being printed directly. The user can then decide what to
do with it. The existing hooks for user callbacks are used for
this purpose to keep the internal API changes minimal and non-invasive.
The last and biggest change is the way the the interaction between
local parameterizations and the gradient checker works. Before,
local parameterizations would be ignored by the checker. However,
if a cost function does not compute its Jacobian along the null
space of the local parameterization, this wil not have any effect
on the solver, but would result in a gradient checker error.
With this change, the Jacobians are multiplied by the Jacobians
of the respective local parameterization and thus being compared
in the tangent space only.
The typical use case for this are quaternion parameters, where
a cost function will typically assume that the quaternion is
always normalized, skipping the correct computation of the Jacobian
along the normal to save computation cost.
Change-Id: I5e1bb97b8a899436cea25101efe5011b0bb13282
Accumulate the number of steps of the line search algorithm
and report it as part of Summary::FullReport.
Change-Id: I1de12784009a3e08f2a2c2aff5085d57a3c73828
Since Ceres is moving to using GitHub for issues, and the Google
Code URL in the current copyright header will soon become invalid,
update all the headers.
Change-Id: I1fce70375d1bcf098591f07b4d8f01a5c1e0789c
For historical reasons we had a "using namespace std;" in port.h. This
is generally a bad idea. So removing it and along the way doing a bunch
of cpplint cleanup.
Change-Id: Ia125601a55ae62695e247fb0250df4c6f86c46c6
- Previously we had no defined default value for
sparse_linear_algebra_library_type in Solver::Options if Ceres
was compiled with no sparse library available. Thus in that case,
the default value (dependent upon the compiler) would indicate that
one was available.
- Now we have an explicit option that means no sparse library is
available, which is now the default value in Solver::Options in this
case.
- Add a warning in CMake when the user disables all sparse libraries.
- Fix typos in trust_region_preprocessor_test:
(SUITE/CX)_SPARSE -> (SUITE/CX)SPARSE that induced failures when
no sparse libraries were available.
Change-Id: I869c399a12d42bfc44220cbb25ce6d6dd80236bd
- We now compute & report the cumulative time spent performing the
following tasks as part of a line search:
- Evaluation of the univariate cost function value & gradient.
- Minimization of the interpolating polynomial.
- Total time spent performing line searches.
- This information is now reported for all minimizers, although only in
the case of a constrained problem for the TR minimizer.
- Remove LineSearch::Function abstraction in place of using
LineSearchFunction implementation directly, and remove virtual
functions from LineSearchFunction.
-- LineSearch::Function added an unnecessary level of abstraction since
the user always had to create a LineSearchFunction anyway to use a
Ceres Evaluator, and it added an unncessary virtual function call.
Change-Id: Ia4e1921d78f351ae119875aa97a3ea5e8b5d9877
Since the trust region minimizer can use the line search
minimizer when it is solving a box constrained problem,
ensure that the line search options are valid.
Also some minor spacing fixes in the line search code.
Change-Id: Ife04204855cfac389cf980f0a79155d4accc8662
Up till now ITERATIVE_SCHUR evaluates matrix-vector products
between the Schur complement and a vector implicitly by exploiting
the algebraic expression for the Schur complement.
This cost of this evaluation scales with the number of non-zeros
in the Jacobian.
For small to medium sized problems there is a sweet spot where
computing the Schur complement is cheap enough that it is much
more efficient to explicitly compute it and use it for evaluating
the matrix-vector products.
This changes implements support for an explicit Schur complement
in ITERATIVE_SCHUR in combination with the SCHUR_JACOBI preconditioner.
API wise a new bool Solver::Options::use_explicit_schur_complement
has been added.
The implementation extends the SparseSchurComplementSolver to use
Conjugate Gradients.
Example speedup:
use_explicit_schur_complement = false
Time (in seconds):
Preprocessor 0.585
Residual evaluation 0.319
Jacobian evaluation 1.590
Linear solver 25.685
Minimizer 27.990
Postprocessor 0.010
Total 28.585
use_explicit_schur_complement = true
Time (in seconds):
Preprocessor 0.638
Residual evaluation 0.318
Jacobian evaluation 1.507
Linear solver 5.930
Minimizer 8.144
Postprocessor 0.010
Total 8.791
Which indicates an end-to-end speedup of more than 3x, with the linear
solver being sped up by > 4x.
The idea to explore this optimization was inspired by the recent paper:
Mining structure fragments for smart bundle adjustment
L. Carlone, P. Alcantarilla, H. Chiu, K. Zsolt, F. Dellaert
British Machine Vision Conference, 2014
which uses a more complicated algorithm to compute parts of the
Schur complement to speed up the matrix-vector product.
Change-Id: I95324af0ab351faa1600f5204039a1d2a64ae61d
This moves a couple of routines from solver.cc into solver_utils.h/cc
so that they can also be used by the upcoming GradientProblemSolver.
Change-Id: I627b32ad3dc639422aacde78a8e391459d947e99
The header of Summary::FullReport now looks like
Solver Summary (v 1.10.0-suitesparse-cxsparse-lapack-no_openmp)
Original Reduced
Parameter blocks 22122 22122
Parameters 66462 66462
Residual blocks 83718 83718
Residual 167436 167436
Change-Id: Id1b81bbf90ba412d19e2dd3687eeb9d372b72c1b
1. Allow the minimum number of linear solver iterations to be zero.
2. Fix conjugate gradients solver's iteration loop to be sane again.
Change-Id: I8594815fec940c2b30e28eb58ec5d8baacf13dae
Sameer Agarwal