- We now compute & report the cumulative time spent performing the
following tasks as part of a line search:
- Evaluation of the univariate cost function value & gradient.
- Minimization of the interpolating polynomial.
- Total time spent performing line searches.
- This information is now reported for all minimizers, although only in
the case of a constrained problem for the TR minimizer.
- Remove LineSearch::Function abstraction in place of using
LineSearchFunction implementation directly, and remove virtual
functions from LineSearchFunction.
-- LineSearch::Function added an unnecessary level of abstraction since
the user always had to create a LineSearchFunction anyway to use a
Ceres Evaluator, and it added an unncessary virtual function call.
Change-Id: Ia4e1921d78f351ae119875aa97a3ea5e8b5d9877
1. Remove an unused private member in TukeyLoss.
2. The test for RotationMatrixToAngle had an indexing error.
Change-Id: I7decc9448ae0abef53aa435005f739e9e0931e80
Use this function to implement RotationMatrixToAngleAxis.
This simplifies the implementation of RotationMatrixToAngleAxis,
just like Eigen does. It is also autodiff compatible, unlike the
Eigen based version.
Also significantly improve the test coverage of
RotationMatrixToAngleAxis.
Change-Id: Ic192a12fb5de952197ee24b0deedc45f195477f1
Use Eigen's much more complicated conversion routine
when we encounter cases where the angle of rotation is
close to Pi.
Along the way also fix the way angle_axis vectors are
compared by making the matcher more robust.
Thanks to Tobias Strauss for reporting this.
Change-Id: Ia7e65dafad92c48d29d5f3cd22c4d6534789c183
1. Complete restructuring of the documentation to account for
GradientProblemSolver.
2. Update the version history to account for changes since 1.9.0.
3. Add links and document the various examples that ship with ceres.
4. Documentation for GradientProblem GradientProblemSolver.
Change-Id: If3a18f2850cbc98be1bc34435e9ea468785b8b27
Up till now ITERATIVE_SCHUR evaluates matrix-vector products
between the Schur complement and a vector implicitly by exploiting
the algebraic expression for the Schur complement.
This cost of this evaluation scales with the number of non-zeros
in the Jacobian.
For small to medium sized problems there is a sweet spot where
computing the Schur complement is cheap enough that it is much
more efficient to explicitly compute it and use it for evaluating
the matrix-vector products.
This changes implements support for an explicit Schur complement
in ITERATIVE_SCHUR in combination with the SCHUR_JACOBI preconditioner.
API wise a new bool Solver::Options::use_explicit_schur_complement
has been added.
The implementation extends the SparseSchurComplementSolver to use
Conjugate Gradients.
Example speedup:
use_explicit_schur_complement = false
Time (in seconds):
Preprocessor 0.585
Residual evaluation 0.319
Jacobian evaluation 1.590
Linear solver 25.685
Minimizer 27.990
Postprocessor 0.010
Total 28.585
use_explicit_schur_complement = true
Time (in seconds):
Preprocessor 0.638
Residual evaluation 0.318
Jacobian evaluation 1.507
Linear solver 5.930
Minimizer 8.144
Postprocessor 0.010
Total 8.791
Which indicates an end-to-end speedup of more than 3x, with the linear
solver being sped up by > 4x.
The idea to explore this optimization was inspired by the recent paper:
Mining structure fragments for smart bundle adjustment
L. Carlone, P. Alcantarilla, H. Chiu, K. Zsolt, F. Dellaert
British Machine Vision Conference, 2014
which uses a more complicated algorithm to compute parts of the
Schur complement to speed up the matrix-vector product.
Change-Id: I95324af0ab351faa1600f5204039a1d2a64ae61d
The line search minimizer in Ceres does not require that the
problems that is solving is a sum of squares. Over the past
year there have been multiple requests to expose this algorithm
on its own so that it can be used to solve unconstrained
non-linear minimization problems on its own.
With this change, a new optimization problem called
GradientProblem is introduced which is basically a thin
wrapper around a user defined functor that evaluates cost
and gradients (FirstOrderFunction) and an optional LocalParameterization.
Corresponding to it, a GradientProblemSolver and its associated
options and summary structs are introduced too.
An example that uses the new API to find the minimum of Rosenbrock's
function is also added.
Change-Id: I42bf687540da25de991e9bdb00e321239244e8b4
Problem::GetCostFunctionForResidualBlock
Problem::GetLossFunctionForResidualBlock
are added, so that users do not have to maintain this mapping
outside the Problem.
Change-Id: I38356dfa094b2c7eec90651dafeaf3a33c5f5f56
Its API was broken, and its implementation was an unnecessary
layer of abstraction over CostFunctionToFunctor.
Change-Id: I18fc261fc6a3620b51a9eeb4dde0af03d753af69
The code itself was perfectly capable of handling residuals, but there
was an overly strict runtime check that had to be removed.
Thanks to Domink Reitzle for reporting this.
Change-Id: I6a6d000a7c5203dd5945a61b4caeda1b8aeb09c9
If the parameter block size is 1, asking Eigen to create
a row-major matrix triggers a compile time error. Previously
we were handling the case where the number of rows in the
jacobian block was known statically, but the problem is present
when the nummber of rows is dynamic.
This CL fixes this problem.
Thanks to Dominik Reitzle for reporting this.
Change-Id: I99c3eec3558e66ebf4efa51c4dee8ce292ffe0c1
1. Fix a build breakage in graph_test.
2. Respect Solver::Options::min_num_linear_solver_iterations in
conjugate_gradients_solver.cc
Thanks to Johannes Schönberger for reporting these.
Change-Id: Ib32e3929bf5d92dd576ae5b53d4d88797095136e
Replace SolverImpl with
a. A minimizer specific preprocessor class.
b. A generic Solve function inside solver.cc
c. Presummarize and Postsummarize functions to handle
updates to the summary object.
The existing SolverImpl class was a mixture of the above three
things and was increasingly complicated code to follow. This change,
breaks it into its three separate constituents, with the aims of
better separation of concerns and thus better testability and
reliability.
The call to Solver::Solve() now consists of
1. Presummarize - summarize the given state of the problem and solver
options.
2. Preprocess - Setup everything that is needed to call the minimizer.
This includes, removing redundant parameter and residual blocks,
setting up the reordering for the linear solver, creating the
linear solver, evaluator, inner iteration minimizer etc.
3. Minimize.
4. Post summarize - summarize the result of the preprocessing and the
solve.
Change-Id: I80f35cfc9f2cbf78f1df4aceace27075779d8a3a
SPARSE_NORMAL_CHOLESKY and SPARSE_SCHUR can now be used
with EIGEN_SPARSE as the backend.
The performance is not as good as CXSparse. This needs to be
investigated. Is it because the quality of AMD ordering that
we are computing is not as good as the one for CXSparse? This
could be because we are working with the scalar matrix instead
of the block matrix.
Also, the upper/lower triangular story is not completely clear.
Both of these issues will be benchmarked and tackled in the
near future.
Also included in this change is a bunch of cleanup to the
SparseNormalCholeskySolver and SparseSchurComplementSolver
classes around the use of the of defines used to conditionally
compile out parts of the code.
The system_test has been updated to test EIGEN_SPARSE also.
Change-Id: I46a57e9c4c97782696879e0b15cfc7a93fe5496a
Sparse Cholesky factorization is not rank revealing. Therefore
this algorithm cannot reliably tell when the Jacobian matrix is
rank deficient or so poorly conditioned that the covariance matrix
cannot be estimated.
Making things worse, this algorithm works on the normal equations,
which makes the conditioning problem much worse.
This change, deletes the SPARSE_CHOLESKY algorithm in the covariance
estimation code. Also to make the naming consistent, it renames
SPARSE_QR -> SUITE_SPARSE_QR
so that it parallels EIGEN_SPARSE_QR.
Also, since we now have EIGEN_SPARSE_QR, we can default to using
it when SuiteSparse is not available instead of DENSE_SVD, which
generally speaking should only be used by folks who are dealing
with small rank deficient jacobians.
Change-Id: I8b134c7e8a2e86ca374371f185b19f1c3e74349c
For smaller problems Eigen is faster than SuiteSparseQR. This has been
tested with Eigen 3.2.1. Below are detailed timings. Problem 1 is the
smallest and problem 3 is the largest. The timings below are:
mean +- standard deviation.
Problem 1:
Eigen 0.0009218 +- 0.0002755
SuiteSparse 0.001406 +- 0.001610
Problem 2:
Eigen 0.002338 +- 0.001005
SuiteSparse 0.001910 +- 0.0004513
Problem 3:
Eigen 0.005455 +- 0.001759
SuiteSparse 0.002411 +- 0.0004974
Detailed problem descriptions:
Problem 1 size:
Original Reduced
Parameter blocks 533 54
Parameters 368 104
Effective parameters 1201 94
Residual blocks 233 77
Residual 1194 258
Problem 2 size:
Original Reduced
Parameter blocks 573 84
Parameters 1458 184
Effective parameters 1281 164
Residual blocks 263 107
Residual 1314 378
Problem 3 size:
Original Reduced
Parameter blocks 613 114
Parameters 1548 264
Effective parameters 1361 234
Residual blocks 293 137
Residual 1434 498
Change-Id: I884a67e2f728fe2992812148d82ccf5f27864fd7
This replaces the broken CERES_VERSION and CERES_ABI_VERSION
defines with a different set, including integer versions for
MAJOR/MINOR/etc.
This also adds the Ceres version to Solver::FullReport().
Example report from powell:
Ceres Solver v1.10.0 Solve Report
----------------------------------
Original Reduced
Parameter blocks 4 4
Parameters 4 4
Residual blocks 4 4
Residual 4 4
Minimizer TRUST_REGION
Dense linear algebra library EIGEN
Trust region strategy LEVENBERG_MARQUARDT
Given Used
Linear solver DENSE_QR DENSE_QR
Threads 1 1
Linear solver threads 1 1
Cost:
Initial 1.075000e+02
Final 1.791438e-14
Change 1.075000e+02
Minimizer iterations 14
Successful steps 14
Unsuccessful steps 0
Time (in seconds):
Preprocessor 0.001
Residual evaluation 0.000
Jacobian evaluation 0.000
Linear solver 0.000
Minimizer 0.001
Postprocessor 0.000
Total 0.003
Change-Id: I5bf0e8023693e9195276b1f1e881b13121ba1196
Termination: CONVERGENCE (Gradient tolerance reached. Gradient max norm: 3.642190e-11 <= 1.000000e-10)
1. Add the ability to bulk remove elements.
2. Add accessor for elements_to_group.
3. Early exit in Remove if there are no elements.
Change-Id: I3df1f00de05338a42e9907423b674469a022d3bc
- Previously we overwrote the default (empty) config.h in the source
tree with a configured config.h, generated using the current compile
options.
- This was undesirable as it could lead to inadvertant commits of the
generated config.h.
- This patch moves the default config.h to <src>/config/ceres/internal,
separate from the other headers, thus if Ceres is compiled without
CMake this directory will now also have to be included. This
directory is _not_ added to the CMake include directories for Ceres
(thus the default config.h is never used when compiling with CMake).
- When using CMake, the generated config.h is now placed in
<build>/config/ceres/internal, which is in turn added to the include
directories for Ceres when it is compiled, and the resulting config.h
is copied to ceres/internal when installed.
Change-Id: Ib1ba45e66e383ade2ebb08603af9165c1df616f2
- Previously we passed all compile options to Ceres via add_definitions
in CMake. This was fine for private definitions (used only by Ceres)
but required additional work for public definitions to ensure they
were correctly propagated to clients via CMake using
target_compile_definitions() (>= 2.8.11) or add_definitions().
- A drawback to these approaches is that they did not work for chained
dependencies on Ceres, as in if in the users project B <- A <- Ceres,
then although the required Ceres public compile definitions would
be used when compiling A, they would not be propagated to B.
- This patch replaces the addition of compile definitions via
add_definitions() with an autogenerated config.h header which
is installed with Ceres and defines all of the enabled Ceres compile
options.
- This removes the need for the user to propagate any compile
definitions in their projects, and additionally allows post-install
inspect of the options with which Ceres was compiled.
Change-Id: Idbdb6abdad0eb31e7540370e301afe87a07f2260
1. Update version history.
2. Minor changes to the tutorial to reflect the bounds constrained
problem.
3. Added static factory methods to the SnavelyReprojectionError.
4. Removed relative gradient tolerance from types.h as it is
not true anymore.
Change-Id: I8de386e5278a008c84ef2d3290d2c4351417a9f1
This compiler defines shared_ptr in std::tr1 namespace, but
for this <tr1/memory> is to be included. Further, this compiler
also does have <memory> header which confused previous shared
pointer check.
Simplified logic around defines now, so currently we've got:
- CERES_TR1_MEMORY_HEADER defined if <tr1/memory> is to be
used for shared_ptr, otherwise <memory> is to be used.
- CERES_TR1_SHARED_PTR defined if shared_ptr is defined in
std::tr1 namespace, otherwise it's defined in std namespace.
All the shared_ptr checks are now moved to own file FindSharedPtr
which simplifies main CMakeLists.
Change-Id: I558a74793baaa0bd088801910a356be4ef17c31b
The standard sparse normal Cholesky solver assumes a fixed
sparsity pattern which is useful for a large number of problems
presented to Ceres. However, some problems are symbolically dense
but numerically sparse i.e. each residual is a function of a
large number of parameters but at any given state the residual
only depends on a sparse subset of them. For these class of
problems it is faster to re-analyse the sparsity pattern of the
jacobian at each iteration of the non-linear optimisation instead
of including all of the zero entries in the step computation.
The proposed solution adds the dynamic_sparsity option which can
be used with SPARSE_NORMAL_CHOLESKY. A
DynamicCompressedRowSparseMatrix type (which extends
CompressedRowSparseMatrix) has been introduced which allows
dynamic addition and removal of elements. A Finalize method is
provided which then consolidates the matrix so that it can be
used in place of a regular CompressedRowSparseMatrix. An
associated jacobian writer has also been provided.
Changes that were required to make this extension were adding the
SetMaxNumNonZeros method to CompressedRowSparseMatrix and adding
a JacobianFinalizer template parameter to the ProgramEvaluator.
Change-Id: Ia5a8a9523fdae8d5b027bc35e70b4611ec2a8d01
Fix variable names in port.h and fix fpclassify when
using gnustl. This was tested by switching to gnustl
in the JNI build.
Thanks to Carlos Hernandez for suggesting the gnustl fixes.
Change-Id: I690b73caf495ccc79061f45288e416da1604cc72
By default shared_ptr is now assumed to be
in the standard <memory> header and in the
std namespace.
Previously the way the ifdefs were structured if the appropriate
variable was not defined, it would default to <t1/memory>.
The new defaults are more future proof.
Change-Id: If457806191196be2b6425b8289ea7a3488a27445
Solver::Options::linear_solver_ordering and
Solver::Options::inner_iteration_ordering
were bare pointers even though Solver::Options took ownership of these
objects.
This lead to buggy user code and the inability to copy Solver::Options
objects around.
With this change, these naked pointers have been replaced by a
shared_ptr object which will managed the lifetime of these objects. This
also leads to simplification of the lifetime handling of these objects
inside the solver.
The Android.mk and Application.mk files have also been updated
to use a newer NDK revision which ships with LLVM's libc++.
Change-Id: I25161fb3ddf737be0b3e5dfd8e7a0039b22548cd
- Breaking change: Problem::Options::enable_fast_parameter_block_removal
is now Problem::Options::enable_fast_removal, as it now controls
the behaviour for both parameter and residual blocks.
- Previously we did not check that the specified residual block to
remove in RemoveResidualBlock actually represented a valid residual
for the problem.
- This meant that Ceres would die unexpectedly if the user passed an
uninitialised residual_block, or more likely attempted to remove a
residual block that had already been removed automatically after
the user removed a parameter block upon on which it was dependent.
- RemoveResidualBlock now verifies the validity of the given
residual_block to remove. Either by checking against a hash set of
all residuals maintained in ProblemImpl iff enable_fast_removal
is enabled. Or by a full scan of the residual blocks if not.
Change-Id: I9ab178e2f68a74135f0a8e20905b16405c77a62b
- Previously AutoDiffLocalParameterization would internally instantiate
a functor instance whenever one was required. This prohibits the
user passing arguments to the constructor of the functor.
- Now AutoDiffLocalParameterization can take over ownership of an
allocated functor which the user created. This mimics the behaviour
of AutoDiffCostFunction.
Change-Id: I264e1face44ca5d5e71cc20c77cc7654d3f74cc0
These two methods allow the user to associate upper and lower bounds
with individual parameters inside parameter blocks.
Change-Id: I68dc37f20b64408da510ba06b89a4f08df54ddad
Alex Stewart