Files
ceres-solver/internal/ceres/block_sparse_matrix.h
T
Dmitriy Korchemkin b1fe603305 Parallel right products for partitioned view
Parallel implementations for right-multiply by dense vector for:
 - Partitioned matrix view
 - Block-sparse matrix
 - CRS matrix (non-symmetric only)

When coupled with non-interleaving indexes in parallel for, this
simple aproach provides a reasonable speedup.
For example, in CRS case difference with GPGPU approach reduces
closer to memory throughput ratio for high enough core count.

./bin/spmv_benchmark
-------------------------------------------------------------------
Benchmark                                                      Time
-------------------------------------------------------------------
BM_BlockSparseRightMultiplyAndAccumulateBA/1              28.5   ms
BM_BlockSparseRightMultiplyAndAccumulateBA/2              15.7   ms
BM_BlockSparseRightMultiplyAndAccumulateBA/4               9.01  ms
BM_BlockSparseRightMultiplyAndAccumulateBA/8               5.60  ms
BM_BlockSparseRightMultiplyAndAccumulateBA/16              3.86  ms
BM_BlockSparseRightMultiplyAndAccumulateBA/28              3.84  ms
BM_BlockSparseRightMultiplyAndAccumulateUnstructured/1    23.8   ms
BM_BlockSparseRightMultiplyAndAccumulateUnstructured/2    15.0   ms
BM_BlockSparseRightMultiplyAndAccumulateUnstructured/4     8.01  ms
BM_BlockSparseRightMultiplyAndAccumulateUnstructured/8     4.02  ms
BM_BlockSparseRightMultiplyAndAccumulateUnstructured/16    2.39  ms
BM_BlockSparseRightMultiplyAndAccumulateUnstructured/28    1.68  ms
BM_BlockSparseLeftMultiplyAndAccumulateBA                 30.7   ms
BM_BlockSparseLeftMultiplyAndAccumulateUnstructured       41.5   ms
BM_CRSRightMultiplyAndAccumulateBA/1                      24.1   ms
BM_CRSRightMultiplyAndAccumulateBA/2                      13.6   ms
BM_CRSRightMultiplyAndAccumulateBA/4                       8.70  ms
BM_CRSRightMultiplyAndAccumulateBA/8                       5.34  ms
BM_CRSRightMultiplyAndAccumulateBA/16                      3.99  ms
BM_CRSRightMultiplyAndAccumulateBA/28                      4.00  ms
BM_CRSRightMultiplyAndAccumulateUnstructured/1            21.1   ms
BM_CRSRightMultiplyAndAccumulateUnstructured/2            10.83  ms
BM_CRSRightMultiplyAndAccumulateUnstructured/4             5.88  ms
BM_CRSRightMultiplyAndAccumulateUnstructured/8             3.68  ms
BM_CRSRightMultiplyAndAccumulateUnstructured/16            2.21  ms
BM_CRSRightMultiplyAndAccumulateUnstructured/28            1.71  ms
BM_CRSLeftMultiplyAndAccumulateBA                         23.6   ms
BM_CRSLeftMultiplyAndAccumulateUnstructured               22.5   ms
BM_CudaRightMultiplyAndAccumulateBA                        0.679 ms
BM_CudaRightMultiplyAndAccumulateUnstructured              0.480 ms
BM_CudaLeftMultiplyAndAccumulateBA                         0.774 ms
BM_CudaLeftMultiplyAndAccumulateUnstructured               0.361 ms

./bin/partitioned_matrix_view_benchmark
-----------------------------------------------------------------
Benchmark                                                    Time
-----------------------------------------------------------------
BM_PatitionedViewRightMultiplyAndAccumulateE_Static/1    18.5  ms
BM_PatitionedViewRightMultiplyAndAccumulateE_Static/2    10.7  ms
BM_PatitionedViewRightMultiplyAndAccumulateE_Static/4     6.34 ms
BM_PatitionedViewRightMultiplyAndAccumulateE_Static/8     4.26 ms
BM_PatitionedViewRightMultiplyAndAccumulateE_Static/16    3.86 ms
BM_PatitionedViewRightMultiplyAndAccumulateE_Static/28    3.75 ms
BM_PatitionedViewRightMultiplyAndAccumulateF_Static/1    18.8  ms
BM_PatitionedViewRightMultiplyAndAccumulateF_Static/2    11.9  ms
BM_PatitionedViewRightMultiplyAndAccumulateF_Static/4     6.94 ms
BM_PatitionedViewRightMultiplyAndAccumulateF_Static/8     4.41 ms
BM_PatitionedViewRightMultiplyAndAccumulateF_Static/16    3.63 ms
BM_PatitionedViewRightMultiplyAndAccumulateF_Static/28    3.86 ms

Timings correspond to intel 8176 cpu and 2080ti nvidia gpu,
with OpenMP threading backend.

Change-Id: Idc07d0563103d057ca3c8412de81a7823fe232af
2022-09-30 17:23:13 +03:00

172 lines
6.7 KiB
C++

// Ceres Solver - A fast non-linear least squares minimizer
// Copyright 2022 Google Inc. All rights reserved.
// http://ceres-solver.org/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
//
// * Redistributions of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors may be
// used to endorse or promote products derived from this software without
// specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
// ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
// LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
// CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
// SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
// CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
// ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
// POSSIBILITY OF SUCH DAMAGE.
//
// Author: sameeragarwal@google.com (Sameer Agarwal)
//
// Implementation of the SparseMatrix interface for block sparse
// matrices.
#ifndef CERES_INTERNAL_BLOCK_SPARSE_MATRIX_H_
#define CERES_INTERNAL_BLOCK_SPARSE_MATRIX_H_
#include <memory>
#include <random>
#include "ceres/block_structure.h"
#include "ceres/compressed_row_sparse_matrix.h"
#include "ceres/context_impl.h"
#include "ceres/internal/disable_warnings.h"
#include "ceres/internal/eigen.h"
#include "ceres/internal/export.h"
#include "ceres/sparse_matrix.h"
namespace ceres::internal {
class TripletSparseMatrix;
// This class implements the SparseMatrix interface for storing and
// manipulating block sparse matrices. The block structure is stored
// in the CompressedRowBlockStructure object and one is needed to
// initialize the matrix. For details on how the blocks structure of
// the matrix is stored please see the documentation
//
// internal/ceres/block_structure.h
//
class CERES_NO_EXPORT BlockSparseMatrix final : public SparseMatrix {
public:
// Construct a block sparse matrix with a fully initialized
// CompressedRowBlockStructure objected. The matrix takes over
// ownership of this object and destroys it upon destruction.
//
// TODO(sameeragarwal): Add a function which will validate legal
// CompressedRowBlockStructure objects.
explicit BlockSparseMatrix(CompressedRowBlockStructure* block_structure);
BlockSparseMatrix(const BlockSparseMatrix&) = delete;
void operator=(const BlockSparseMatrix&) = delete;
// Implementation of SparseMatrix interface.
void SetZero() final;
void RightMultiplyAndAccumulate(const double* x, double* y) const final;
void RightMultiplyAndAccumulate(const double* x,
double* y,
ContextImpl* context,
int num_threads) const final;
void LeftMultiplyAndAccumulate(const double* x, double* y) const final;
void SquaredColumnNorm(double* x) const final;
void ScaleColumns(const double* scale) final;
void ToCompressedRowSparseMatrix(CompressedRowSparseMatrix* matrix) const;
void ToDenseMatrix(Matrix* dense_matrix) const final;
void ToTextFile(FILE* file) const final;
// clang-format off
int num_rows() const final { return num_rows_; }
int num_cols() const final { return num_cols_; }
int num_nonzeros() const final { return num_nonzeros_; }
const double* values() const final { return values_.get(); }
double* mutable_values() final { return values_.get(); }
// clang-format on
void ToTripletSparseMatrix(TripletSparseMatrix* matrix) const;
const CompressedRowBlockStructure* block_structure() const;
// Append the contents of m to the bottom of this matrix. m must
// have the same column blocks structure as this matrix.
void AppendRows(const BlockSparseMatrix& m);
// Delete the bottom delta_rows_blocks.
void DeleteRowBlocks(int delta_row_blocks);
static std::unique_ptr<BlockSparseMatrix> CreateDiagonalMatrix(
const double* diagonal, const std::vector<Block>& column_blocks);
struct RandomMatrixOptions {
int num_row_blocks = 0;
int min_row_block_size = 0;
int max_row_block_size = 0;
int num_col_blocks = 0;
int min_col_block_size = 0;
int max_col_block_size = 0;
// 0 < block_density <= 1 is the probability of a block being
// present in the matrix. A given random matrix will not have
// precisely this density.
double block_density = 0.0;
// If col_blocks is non-empty, then the generated random matrix
// has this block structure and the column related options in this
// struct are ignored.
std::vector<Block> col_blocks;
};
// Create a random BlockSparseMatrix whose entries are normally
// distributed and whose structure is determined by
// RandomMatrixOptions.
static std::unique_ptr<BlockSparseMatrix> CreateRandomMatrix(
const RandomMatrixOptions& options, std::mt19937& prng);
private:
int num_rows_;
int num_cols_;
int num_nonzeros_;
int max_num_nonzeros_;
std::unique_ptr<double[]> values_;
std::unique_ptr<CompressedRowBlockStructure> block_structure_;
};
// A number of algorithms like the SchurEliminator do not need
// access to the full BlockSparseMatrix interface. They only
// need read only access to the values array and the block structure.
//
// BlockSparseDataMatrix a struct that carries these two bits of
// information
class CERES_NO_EXPORT BlockSparseMatrixData {
public:
explicit BlockSparseMatrixData(const BlockSparseMatrix& m)
: block_structure_(m.block_structure()), values_(m.values()){};
BlockSparseMatrixData(const CompressedRowBlockStructure* block_structure,
const double* values)
: block_structure_(block_structure), values_(values) {}
const CompressedRowBlockStructure* block_structure() const {
return block_structure_;
}
const double* values() const { return values_; }
private:
const CompressedRowBlockStructure* block_structure_;
const double* values_;
};
} // namespace ceres::internal
#include "ceres/internal/reenable_warnings.h"
#endif // CERES_INTERNAL_BLOCK_SPARSE_MATRIX_H_