1. Replace LineSearch::Summary::optimal_step_size with
LineSearch:Summary::optimal_point which is a FunctionSample.
2. Add the actual vector position and vector gradient of the
point in the FunctionSample
3. Use the above two to get rid of an extraneous function evalation
in LineSearchMinimizer.
Runtime performance is almost 2x improved as a result.
Thanks to @svenpilz for reporting this.
https://github.com/ceres-solver/ceres-solver/issues/296
Change-Id: Iebf2db7acecb2c95c9b1683b73cdc5faab78b02e
1. Move FunctionSample to its own .h/.cc files.
2. Migrate LineSearchFunction::Evaluate to use FunctionSample
for input and output.
Change-Id: I8bfb97e1900d95a4686c9621dda5b584458b45c0
We keep track of evaluator call and time statistics
via a hashmap containing magic strings. These strings
need to be consistent across the GradientProblemSolver
and Solver as the LineSearchMinimizer is used by both
of these solvers. Previously they were inconsistent
in a manner that GradientProblemSolver was not getting
information about the evaluation timing, and in the
process of fixing that I made it so that the TrustRegionMinimizer
when solving bounds constrained probelms will access/update
this information correctly.
So while I look for a more elegant solution, this CL
is meant to fix the inconsistency by making sure that the
same magic strings are used everywhere.
Change-Id: I120ca0bd1c2f77fde2db15edd9e33286a49dbae9
1. Fix a bug which was causing the cost and gradient evaluation
time to not be reported.
2. Add the number of times cost and gradients are evaluated to
the Summary object and to the output of FullReport.
Change-Id: Id0703cd2dafbf437f3e537fbdc30ae81d5f4f540
1. Remove unused variable.
2. Make inner_product_computer compatible with older versions of Eigen,
which don't have a named enum for Eigen::Upper/Lower.
Change-Id: I927af297f93fc74f7f4b29b39e400ef2d75edbd4
Remove outer product computation code from CompressedRowSparseMatrix.
In the process also remove the crsb_cols and crsb_rows vectors from
the matrix, which were added to carry the block sparsity of the matrix
so that the outer product could be computed fast.
InnerProductComputer and its reliance on BlockSparseMatrix has
rendered all of this code moot.
Change-Id: If3ee0dc8ad4ff79594fd1eebc15a647c4495d726
Despite its relative size, this is very significant change
to Ceres.
Why
===
Up till now, when the user chose SPARSE_NORMAL_CHOLESKY,
the Jacobian was evaluated in a CompressedRowSparseMatrix,
which was then use to compute the normal equations which were
passed to a sparse linear algebra library for factorization.
The reason to do this was because in the case of SuiteSparse,
we were able to pass the Jacobian matrix directly without
computing the normal equations and SuiteSparse/CHOLMOD did the
normal equation computation.
This turned out to be slow, so Cheng Wang implemented a high
performance version of the matrix-matrix multiply to compute
the normal equations, and all the sparse linear algebra libraries
now are passed the normal equations.
So that raises the question, as to what the best representation
of the Jacobian which is suitable for the normal equation computation.
Turns out BlockSparseMatrix is ideal. It brings two advantages.
1. Jacobian evaluation into a BlockSparseMatrix is considerably
faster when using a BlockSparseMatrix than
CompressedRowSparseMatrix. This is because we save on a bunch
of memory copies.
2. To make the matrix multiplication fast and use the block structure
Cheng Wang had to essentially make the CompressedRowSparseMatrix
carry a bunch of sidecar information about the block sparsity,
essentially making it behave like a BlockSparseMatrix. The resulting
code had fairly complicated indexing and complicated the semantics
of CompressedRowSparseMatrix. The new InnerProductComputer class
does away with all that and once this CL goes in, I will be able to
remove all that code and simplify the semantics of
CompressedRowSparseMatrix.
Changes
=======
1. Use InnerProductComputer in SparseNormalCholeskySolver.
2. Change the evaluator instantiated for SPARSE_NORMAL_CHOLESKY with
static sparsity inside evaluator.cc
3. The former change necessitates that we change ProblemImpl::Evaluate
to create the evaluate it needs on its own, because it was
depending on passing "SPARSE_NORMAL_CHOLESKY" as linear solver type
to the evaluator factor to get an Evaluator which can use
CompressedRowSparseMatrix objects for storing the Jacobian.
4. Update the tests for SparseNormalCholeskySolver.
5. Separate out the tests for DynamicSparseNormalCholeskySolver into its
own file.
Change-Id: I2ef7ef8fbfbb4967d0c1ec2068c1c778248fdf5b
Add a class that given a block sparse matrix m will compute
the product m'*m efficiently.
This code is refactoring and cleanup of the code in
CompressedRowSparseMatrix devoted to computing the inner product.
In that class, the code is mistakenly said to be computing
the outer product. It is also devoted to computing the inner
product of a CompressedRowSparseMatrix with itself.
This code works with BlockSparseMatrix objects instead, which
are simpler to deal with as they are better structured to handle
block sparse matrices.
Change-Id: I920fee1a396bb0fcae9e6f7e46a308c7391d21aa
There was a bug in the trust region preprocessor where no fill
reducing ordering was computed for the case of SPARSE_SCHUR + CX_SPARSE
but this was not signaled to SchurComplementSolver, so it was using
a naive/natural ordering. To fix this two changes are made:
1. TrustRegionProcessor's logic for signaling the ordering to the
linear solver has been re-worked. The surrounding code has also
been re-organized for better readability.
2. In SchurComplementSolver::SolveReducedSystem the row and column
block structure has been added to the CompressedRowSparseMatrix
containing the Schur complement so that block AMD can be used.
As a result of these changes the linear solve time for
problem-744-543562-pre.txt has been brought down from 58 seconds to
35 seconds.
Change-Id: I4d82efce05175260f97b1f925f8a1b4a9d650cae
It appears that interspersing ifdefs with macros causes problems
with Visual Studio. This patch changes the way we condition
the tests for dense linear solvers based on whether LAPACK is
available or not.
Change-Id: I306247496265c3551edad6bc8fcec9d4cf09e68d
1. Break up unsymmetric_linear_solver_test into
a. dense_linear_solver_test which covers DENSE_QR and
DENSE_NORMAL_CHOLESKY.
b. sparse_normal_cholesky_solver_test which covers
SPARSE_NORMAL_CHOLESKY.
2. dense_linear_solver_test has been completely re-written. It now
uses value parameterized tests for better logging. The number of
test problems as been increased to 2. Last but not the least
the actual test of correctness is not based on a golden solution
computed using another linear solver. We now compute the residual
and ensure that it is small.
https://github.com/ceres-solver/ceres-solver/issues/279
Change-Id: I9546a43e8ae85c31b2096a99405e47da326755ee
SchurComplementSolver implements a variant of ITERATIVE_SCHUR
when explicit_schur_complement is set to true. In this case
the SparseCholesky object should not be instantiated. Even
though there is no CPU cost, it can be the case that ITERATIVE_SCHUR
is being used when there are not sparse linear algebra libraries
are available, and this can result in a crash.
Change-Id: I349d5f79201782689b3ab0ccc2c5001804b44c7b
Parametric tests in gunit use tuple, which can be in the std::tr1
or the std namespaces depending on the version of STL one is using.
This change adds conditions the choice of namespace on whether
CXX11 mode is enabled or not.
It is entirely possible that we will have to come back and add
detection for this along the lines of shared_ptr.
Change-Id: I7fc85a32cf9f3f3bf30f86d9ba972ac67c6635fb
The code for creating and updating preconditioners has been pulled
out in into its own function for better readability.
Change-Id: I5335de3b158a8485cf6d052ee37d0d8fd57145e9
The addition of crsb_rows and crsb_cols to CompressedRowSparseMatrix
broke the build for problems with dynamic sparsity.
The fix is to remove unnecessarily filling of row_blocks and col_blocks,
which was triggering a check inside CompressedRowSparseMatrix around
block handling, since block structure makes no sense for matrices with
dynamic sparsity anyways.
Also added a test "dynamic_sparsity_test" based on i
examples/ellipse_approximation.cc
Thanks to Richard Stebbing for reporting this.
Change-Id: Ic1d49e97690ac17e0ea2949772271bd915277d68
SparseCholesky is an interface to sparse cholesky factorization
routines across sparse linear algebra libraries. Each sparse
linear algebra library is responsible for implementing its own
instance of this interface.
As a result the various places - SparseNormalCholeskySolver,
SparseSchurComplementSolver and VisibilityBasedPreconditioner
are significantly simplified.
Change-Id: I8b465705eae83bba9e1adfffcc741a05c70faf2e
1. Convert a CompressedRowSparseMatrix constructor which
takes a TripletSparseMatrix as input into a factory method
which allows the input to be transposed.
2. Move the random matrix creation routine for CompressedRowSparseMatrix
from being a standalone function to a static method.
3. Add a corresponding random matrix generation static method to
TripletSparseMatrix.
4. Add a new constructor to TripletSparseMatrix, which takes as input
the row, col and values arrays.
Change-Id: Iec7b184646818f432a5e6822bea3b2f3128a82aa
- Use target_compile_features() to specify the C++11 dependency for
Ceres if the CXX11 option is enabled and the current CMake version
supports it (>= 3.1). Otherwise fall back onto our existing
target_compile_options() solution if available.
- We prefer the use of target_compile_features() if available as it more
gracefully handles ‘upgrading’ of the C++ standard in client projects
that depend upon Ceres, e.g. if the client requires C++14. The
current solution may fail to produce the expected result in this case
as raised in
https://github.com/ceres-solver/ceres-solver/issues/273.
Change-Id: Ib3cff8d4b9fe93fa6d6b376b4dd53923bb1c4ecc
Now that there is a single piece of code doing the outer product
computation for all three sparse linear algebra backends, move
this code one level up the call stack and there by make the actual
per-library solver code shorter and simpler.
Also fix a minor omission in the outer product computation code
where row/column blocks were not being copied over to the
outer product matrix.
Change-Id: I22a7967bdc659385b741901afefa7af312e676e5
SchurEliminator::Init now takes a bool that tells it whether
it can assume that the diagonal blocks it is inverting can
be assumed to be full rank or not.
This information is then passed onto InvertPSDMatrix.
Change-Id: I26037b6233f2aad5584fed245f631c3959928afe
THIS IS AN API BREAKING CHANGE.
Decouple the algorithm from the sparse linear algebra
library being used to perform the computation.
Before this change
Covariance::AlgorithmType had values
DENSE_SVD
EIGEN_SPARSE_QR
SUITE_SPARSE_QR
This has been replaced by two enums now.
Covariance::Options::sparse_linear_algebra_library_type
which can take values EIGEN_SPARSE, SUITE_SPARSE or CX_SPARSE.
The last one is currently not supported.
And Covariance::Options::algorithm_type takes values
DENSE_SVD
SPARSE_QR
This sets the stage for future extensions of the covariance
computation algorithm.
Also as part of this change, the covariance computation chapter
has been made a top level chapter on its own instead of being
buried deep inside the Solving Non-linear Least Squares problem.
Change-Id: Ibfbf60902d8d17694d9ff585047a5a57d329ab22
SolveLowerTriangularInPlace
SolveLowerTriangularTransposeInPlace
were unused functions which can be removed.
Change-Id: I0fd29c1efae2a0a74666f6e3541473bebc22ae82
This code was currently buried under a bool inside SparseNormalCholeskySolver.
Pulling this out in its own solver makes the code simpler more readable
and more performant in the case of SuiteSparse.
Change-Id: I72379ca9ca162abbb83c12f7ee8ff92bc71e772c
By adding an enum to CompressedRowSparseMatrix, which indicates
whether the matrix is unsymmetric, upper or lower triangular
we are able to improve the readability and fix some minor
bugs in the way some matrix manipulation code was being
called.
Thank to William Rucklidge for this suggestion.
Change-Id: I355c90d11cd5d31f5a25741b0bda4fc4583e9095
Sameer Agarwal