Remove the Eigen temporary by revealing the columnwise nature
of the computation. This also allows us to get rid of the
special case for nrow = 1.
On problem-356-226730-pre.txt with -robustify evaluation times
change from:
Before:
Residual Evaluations 1.015
Jacobian Evaluations 18.313
After:
Residual Evaluations 1.005
Jacobian Evaluations 8.382
To give a sense of the overhead reduction, compare these numbers
when loss functions are disabled.
Residual Evaluations 0.955
Jacobian Evaluations 7.772
So, this is a 17.5x speedup!
The one dimensional specialization was motivated by denoising.cc.
The evaluation times there are essentially unchanged.
Before:
Residual Evaluations 2.774
Jacobian Evaluations 20.178
After:
Residual Evaluations 2.588
Jacobian Evaluations 19.781
Change-Id: Ic0efbaed75fe4489635039f17189ae24b97802c8
This allows CovarianceImpl to be forward declared without
scoped_ptr freaking out.
Thanks to Nima Keivan for reporting this.
Change-Id: Icd5aa766b3aab70246055225231a4b971c6b7b90
Add a Covariance object to the API.
Given a Problem object and a set of parameter block pairs the
Covariance object computes a sparse covariance matrix corresponding
to those block pairs and provides random access to them.
Constant parameter blocks and parameter blocks with local parameterizations
are correctly handled.
Sparse and dense implementations are provided. With the dense implementation
rank deficient Jacobians can also be handled.
Parts of the code are threaded using OpenMP if available.
Change-Id: I5b49583b3d79579df3e0f334c22567acb23ed4ad
This introduces a simple C API for a subset of Ceres. This opens the door to
using languages like Python to call Ceres, since it is much easier to bind to C
than it is to bind to C++. It will mean giving up the native Ceres autodiff.
The implementation in this patch does not attempt to do everything but is only
just enough to get started. Subsequent patches will increase the surface area
of Ceres that is covered by the C API.
Change-Id: Ic51804bac6865e1a2e476553248aabc91dff3409
The schur ordering is used to construct an elimination
ordering for Schur type solvers when the user has not
supplied an elimination ordering.
The ordering algorithm does an ordered traversal of the
sparsity graph of the Hessian. The order in which this is
done used to be determined by the degree of the parameter
blocks with ties broken arbitrarily using the memory address
of the parameter blocks.
This introduced non-determinism in the solver, causing subtle
numerical differences in the value of the solution everytime
the solve was run.
This change introduces ComputeStableSchurOrdering which utilizes
a new function StableIndependentSetOrdering. The latter takes
as input an ordering of the vertices of the graph which is used
to break ties when ordering the vertice by degree. The former
constructs such an ordering by using the order in which the
parameter blocks were added to the Problem.
In this way, as long as the construction of the problem is
deterministic, the schur ordering will always be deterministic
too.
I have chosen not to delete the existing unstable implementations
of these functions as they are used by the inner iteration
minimizer.
Sometime in the near future I will clean up some of the duplicate
code and see if we can move all the code to using a stable ordering.
Change-Id: I8fbfa240d7307a2c3fe9b135f6968aa410d78780
1. CX_SPARSE supports pre-ordering of the jacobian.
2. Add support for constrained approximate minimum degree ordering
for SuiteSparse versions >= 4.2.0
3. Using 2, support for pre-ordering for SPARSE_SCHUR when used
with SUITE_SPARSE.
4. Using 2, support for user orderings in SPARSE_NORMAL_CHOLESKY.
5. Minor cleanups in documentation and code all around.
6. Test update and refactoring.
Change-Id: Ibfe3ac95d59d54ab14d1d60a07f767688070f29f
Uptil now only SuiteSparse when used with SPARSE_SCHUR would use
the block structure of the reduced camera matrix to find a fill-reducing
ordering.
This leads to substantial speedup for some bundle adjustment
problems.
Credit for this technique goes to the authors of g2o. I learned
about it from reading their source code.
Change-Id: I5403efefd4d9552c9c6fc6e02a65498bdf171584
Evaluator now uses custom BLAS for gradient
computations.
Update the evaluator in trust_region_minimizer_test to compute
gradients.
Change-Id: I3f565bc203b47b2b795a0609d67f25775648653c
Sparse Cholesky factorization algorithms use a fill-reducing
ordering to permute the columns of the Jacobian matrix. There
are two ways of doing this.
1. Compute the Jacobian matrix in some order and then have the
factorization algorithm permute the columns of the Jacobian.
2. Compute the Jacobian with its columns already permuted.
The first option incurs a significant memory penalty. The
factorization algorithm has to make a copy of the permuted
Jacobian matrix.
Starting with this change Ceres pre-permutes the columns of the
Jacobian matrix and generally speaking, there is no performance
penalty for doing so.
In some rare cases, it is worth using a more complicated
reordering algorithm which has slightly better runtime
performance at the expense of an extra copy of the Jacobian
matrix. Setting Solver::Options::use_postordering to true
enables this tradeoff.
This change also removes Solver::Options::use_block_amd
as an option. All matrices are ordered using their block
structure. The ability to order them by their scalar
sparsity structure has been removed.
Here is what performance on looks like on some BAL problems.
Memory
======
HEAD pre-ordering
16-22106 137957376.0 113516544.0
49-7776 56688640.0 46628864.0
245-198739 1718005760.0 1383550976.0
257-65132 387715072.0 319512576.0
356-226730 2014826496.0 1626087424.0
744-543562 4903358464.0 3957878784.0
1024-110968 968626176.0 822071296.0
Time
====
HEAD pre-ordering
16-22106 3.8 3.7
49-7776 1.9 1.8
245-198739 82.6 81.9
257-65132 14.0 13.4
356-226730 98.8 95.8
744-543562 325.2 301.6
1024-110968 42.1 37.1
Change-Id: I6b2e25f3fed7310f88905386a7898ac94d37467e
This will allow CHOLMOD to compute the sparse
Cholesky factorization of J'J without making
a permuted copy of it.
Change-Id: I25d0e18f5957ab7fdce15c543234bb2f09db482e
Break up CreateReducedProgram into smaller functions in
preparation for more sophisticated ordering strategies.
Change-Id: Ic3897522574fde770646d747fe383f5dbd7a6619
1. CreateSparseMatrixTransposeView now returns a struct instead
of a pointer.
2. Add AnalyzeCholeskyWithNaturalOrdering.
Change-Id: If27a5502949c3994edd95be0d25ec7a0d1fa1ae1
This was accidentally an int, which was setting the minimum
trust region radius to zero and effectively disabling a convergence
test based on it.
(Thanks to Sergey Sharybin for providing a reproduction for this)
Change-Id: Id0b9e246bcfee074954a5dc6a3a2342adab56c16
This class is used to create local parameterization
with Jacobians computed via automatic differentiation.
To get an auto differentiated local parameterization,
class with a templated operator() (a functor) that
computes
plus_delta = Plus(x, delta);
shall be defined.
Then given such functor, the auto differentiated local
parameterization can be constructed as
LocalParameterization* local_parameterization =
new AutoDiffLocalParameterization<PlusFunctor, 4, 3>;
| |
Global Size ---------------+ |
Local Size -------------------+
See autodiff_local_parameterization.h for more information
and usage example.
Initial implementation by Keir Mierle, finished by self
and integrated into Ceres and covered with unit tests
by Sameer Agarwal.
Change-Id: I1b3e48ae89f81e0cf1f51416c5696e18223f4b21
1. Switch to Eigen's implementation when all dimensions are fixed.
2. Use lazyProduct for eigen matrix-vector product. This brings
eigen's performance on iterative_schur closer to what it used
to be before the last commit. There is however still an
improvement to be had by using the naive implementation when
the matrix and vector have dynamic dimensions.
BENCHMARK
HEAD CHANGE
problem-16-22106-pre.txt
gcc-eigen sparse_schur 0.859 gcc-eigen sparse_schur 0.853
clang-eigen sparse_schur 0.848 clang-eigen sparse_schur 0.850
gcc-blas sparse_schur 0.956 gcc-blas sparse_schur 0.865
clang-blas sparse_schur 0.954 clang-blas sparse_schur 0.858
gcc-eigen iterative_schur 4.656 gcc-eigen iterative_schur 3.271
clang-eigen iterative_schur 4.664 clang-eigen iterative_schur 3.307
gcc-blas iterative_schur 2.598 gcc-blas iterative_schur 2.620
clang-blas iterative_schur 2.554 clang-blas iterative_schur 2.567
problem-49-7776-pre.txt
gcc-eigen sparse_schur 0.477 gcc-eigen sparse_schur 0.472
clang-eigen sparse_schur 0.475 clang-eigen sparse_schur 0.479
gcc-blas sparse_schur 0.521 gcc-blas sparse_schur 0.469
clang-blas sparse_schur 0.508 clang-blas sparse_schur 0.471
gcc-eigen iterative_schur 3.172 gcc-eigen iterative_schur 2.088
clang-eigen iterative_schur 3.161 clang-eigen iterative_schur 2.079
gcc-blas iterative_schur 1.701 gcc-blas iterative_schur 1.720
clang-blas iterative_schur 1.708 clang-blas iterative_schur 1.694
problem-245-198739-pre.txt
gcc-eigen sparse_schur 28.092 gcc-eigen sparse_schur 28.233
clang-eigen sparse_schur 28.148 clang-eigen sparse_schur 28.400
gcc-blas sparse_schur 30.919 gcc-blas sparse_schur 28.110
clang-blas sparse_schur 31.001 clang-blas sparse_schur 28.407
gcc-eigen iterative_schur 63.095 gcc-eigen iterative_schur 43.694
clang-eigen iterative_schur 63.412 clang-eigen iterative_schur 43.473
gcc-blas iterative_schur 33.353 gcc-blas iterative_schur 33.321
clang-blas iterative_schur 33.276 clang-blas iterative_schur 33.278
problem-257-65132-pre.txt
gcc-eigen sparse_schur 3.687 gcc-eigen sparse_schur 3.629
clang-eigen sparse_schur 3.669 clang-eigen sparse_schur 3.652
gcc-blas sparse_schur 3.947 gcc-blas sparse_schur 3.673
clang-blas sparse_schur 3.952 clang-blas sparse_schur 3.678
gcc-eigen iterative_schur 121.512 gcc-eigen iterative_schur 76.833
clang-eigen iterative_schur 123.547 clang-eigen iterative_schur 78.763
gcc-blas iterative_schur 68.334 gcc-blas iterative_schur 68.612
clang-blas iterative_schur 67.793 clang-blas iterative_schur 68.266
Notes:
1. Naive BLAS was a bit worse than eigen on fixed sized matrices. We did not see this
before because of the different inlining thresholds. Fixing this boosted eigen's
performance. Also the disparity between gcc and clang has gone away.
2. SPARSE_SCHUR performance remains the same, since it is only testing static sized
matrices.
3. ITERATIVE_SCHUR performance goes up substantially due to the lazyProduct change,
but even there, since most of the products are dynamic sized, the naive implementation
wins handily.
Change-Id: Idc17f35b9c68aaebb1b2e131adf3af8374a85a4c
With this ITERATIVE_SCHUR with JACOBI preconditioner went down from
280 seconds to 150 seconds on problem-744-543562-pre.txt.
Change-Id: I4f319c1108421e8d59f58654a4c0576ad65df609
1. Add Matrix-Matrix and Matrix-Vector multiply functions.
2. Replace Eigen usage in SchurEliminator with these custom
matrix operations.
3. Save on some memory allocations in ChunkOuterProduct.
4. Replace LDLT with LLT.
As a result on problem-16-22106-pre.txt, the linear solver time
goes down from 1.2s to 0.64s.
Change-Id: I2daa667960e0a1e8834489965a30be31f37fd87f
1. Silence CHOLMOD's indefiniteness warnings.
2. Add a comment about how the error handling in suitesparse.cc
needs to be improved.
3. Move the analysis logging into suitesparse.cc and out of the
three callsites.
Change-Id: Idd396b8ea4bf59fc1ffc7f9fcbbc7b38ed71643c
Use anonymous namespace for this. Also move some surrounding static
function to this anonymous namespace.
Change-Id: Ie235eb7936976563a9db115ec13c59e6e6869b96
1. Add -DLINE_SEARCH_MINIMIZER to CMake to make the line search
minimizer optional.
2. Better handling of -DSUITESPARSE/-DCXSPARSE in top level cmake
file.
3. Disable code which will never be used if SuiteSparse and/or
CXSparse is not available.
4. Update build docs.
5. Update jni/Android.mk
6. Minor lint cleanup from William Rucklidge.
Change-Id: If60460a858000df82faed7a6bb056dd2bfdde562
With -DRESTRICT_SCHUR_SPECIALIZATIONS, now the various
specializations are empty, decreasing build time and
reducing the size of the static library.
Change-Id: I8ec431279741a9a83516a4167c54a364c4608143
1. Introduce new typdefs in eigen.h to allow for column
major matrices.
2. Clean up old unused typedefs, and the aligned typedefs
since they do not actually add any real performance.
3. Made eigen.h conform to the google style guide by removing
the using directives. They were polluting the ceres namespace.
4. Made the template specialization generator work again.
Change-Id: Ic2268c784534b737ebd6e1a043e2a327adaeca37
Sameer Agarwal