Sparse Linear Algebra

Sparse factorizations call SuiteSparse:

FunctionReturnsLibrary
cholesky, ldltCHOLMOD.FactorCHOLMOD
luUMFPACK.UmfpackLUUMFPACK
qrSPQR.QRSparseSPQR
lqSPQR.AdjointQRSparse, the adjoint of qr(A')SPQR

Solving linear systems

For a sparse A and a dense b, A \ b picks a method from the structure of A and returns a dense result. factorize(A) makes the same choice and returns the factorization.

  • Diagonal or triangular: substitution, no factorization.

  • Hermitian (symmetric, if real): cholesky. If that fails, \ uses lu and factorize uses ldlt.

  • Other square: lu.

  • Tall: qr, giving the least squares solution.

  • Wide: lq, giving the minimum-norm solution, as dense \ does. qr(A) \ b instead returns a basic solution, with the free variables zero.

The structure is read from the stored values, so a symmetric matrix does not need a Symmetric wrapper. A' \ b and transpose(A) \ b make the same choices.

julia> factorize(sparse([4.0 1 0; 1 4 1; 0 1 4])) isa SparseArrays.CHOLMOD.Factor
true

julia> A = sparse([1.0 0 1 0; 0 1 0 1]); b = [1.0, 2.0];

julia> A \ b ≈ [0.5, 1.0, 0.5, 1.0]
true

julia> qr(A) \ b ≈ [1.0, 2.0, 0.0, 0.0]
true

Reusing a factorization

To solve several systems with one matrix, factorize once. F \ B takes a vector or a matrix of right-hand sides, and ldiv!(x, F, b) writes into x.

ldiv! allocates scratch space on each call. To avoid that in a loop, create a workspace once and pass it with the workspace keyword: UMFPACK.UmfpackWS(F) for lu, CHOLMOD.CholmodWS(F) for cholesky and ldlt, and SPQR.SpqrWS(F) for qr. This works with F, F' and transpose(F), and in ldiv!(F, b). A workspace grows as needed and can be reused, but not by two calls at once.

For a new matrix with the same sparsity pattern, lu!(F, A2), cholesky!(F, A2) and ldlt!(F, A2) redo only the numerical factorization, reusing the symbolic analysis in F.

julia> A = sparse([2.0 1 0; 0 3 1; 1 0 4]); b = [1.0, 2.0, 3.0];

julia> F = lu(A);

julia> x = similar(b); ldiv!(x, F, b);

julia> lu!(F, 2A);

julia> F \ b ≈ x / 2
true

Extracting the factors

The factorizations permute rows and columns to reduce fill-in, so the factors reproduce a permuted A. Using F.L as if it were a factor of A gives wrong answers. Solve with F \ b where you can.

FactorizationFactorsRelation
luL, U, p, q, Rs (row scaling)F.L * F.U == (F.Rs .* A)[F.p, F.q]
choleskyL, pL * L' == A[F.p, F.p] with L = sparse(F.L)
ldltLD, pL * D * L' == A[F.p, F.p], with D on the diagonal of sparse(F.LD) and the unit triangular L below it
qrQ, R, prow, pcolF.Q * F.R == A[F.prow, F.pcol]
lqL, Q, prow, pcolF.L * F.Q == A[F.prow, F.pcol]

F.:(:) returns all five lu factors at once. The CHOLMOD factors are lazy: they support solves, and sparse(F.L) for cholesky or sparse(F.LD) for ldlt materializes them. F.PtL (P' * L) and F.UP (L' * P) include the permutation, and ldlt adds F.D, F.DU, F.PtLD and F.DUP. The Q of qr is square and is never formed: products with it return dense arrays.

julia> S = sparse([4.0 1 0; 1 4 1; 0 1 4]); b = [1.0, 2.0, 3.0];

julia> C = cholesky(S);

julia> sparse(C.L) * sparse(C.L)' ≈ S[C.p, C.p]
true

julia> C.UP \ (C.PtL \ b) ≈ S \ b
true

Failures

cholesky and ldlt take a Symmetric or Hermitian view, which reads one triangle, or a matrix that is itself symmetric or Hermitian. Any other matrix throws an ArgumentError.

A failed factorization throws a PosDefException (cholesky), a ZeroPivotException (ldlt) or a SingularException (lu). With check = false it returns anyway, and issuccess(F) tells whether it can be used.

julia> N = sparse([1.0 2; 2 1]);

julia> issuccess(cholesky(N; check = false)), issuccess(ldlt(N; check = false))
(false, true)
SparseArrays.CHOLMOD.Factor — Type
CHOLMOD.Factor{Tv,Ti} <: Factorization{Tv}

The Cholesky (LL') or LDL' factorization of a sparse symmetric or Hermitian matrix, returned by cholesky and ldlt. It wraps a pointer to a cholmod_factor struct, which holds the symbolic analysis, the fill-reducing permutation and the numeric factor. Tv is Float64, Float32, ComplexF64 or ComplexF32, and Ti is Int32 or Int64 (only Int32 on 32-bit systems).

With P the permutation matrix of F.p, the factorization is A == P'*L*L'*P for cholesky and A == P'*L*D*L'*P for ldlt. The properties of F are:

PropertyDescription
F.ppermutation Vector, such that L*L' == A[p, p]
F.L, F.UL and L'
F.PtL, F.UPP'*L and L'*P
F.DD (ldlt only)
F.LD, F.DUL*D and D*L' (ldlt only)
F.PtLD, F.DUPP'*L*D and D*L'*P (ldlt only)

Apart from F.p, these are lazy components for use with \, each applying one step of a solve. Only sparse(F.L), and for ldlt sparse(F.LD) (the unit L with D on its diagonal), can be materialized. sparse(F) reconstructs the factorized matrix.

F supports \, ldiv!, det, logdet, logabsdet, diag, issuccess, nnz, copy, CHOLMOD.rcond, refactorization with cholesky! and ldlt!, and the low-rank modifications lowrankdowndate and lowrankupdate.

CHOLMOD owns the memory, which is released by a finalizer. The pointer is null after deserialization, and using such a factorization throws an ArgumentError. ldiv! takes an optional CHOLMOD.CholmodWS to avoid allocating; the refactorizations take a lock internal to F.

Examples

julia> F = ldlt(sparse([4.0 2.0; 2.0 -3.0]));

julia> propertynames(F)
(:L, :U, :PtL, :UP, :D, :LD, :DU, :PtLD, :DUP, :p, :ptr)

julia> F \ [6.0, -1.0] ≈ [1.0, 1.0]
true
source
SparseArrays.CHOLMOD.Sparse — Type
CHOLMOD.Sparse{Tv,Ti} <: AbstractSparseMatrix{Tv,Ti}

A sparse matrix in compressed column form stored in memory allocated by CHOLMOD, wrapping a pointer to a cholmod_sparse struct. Tv is Float64, Float32, ComplexF64 or ComplexF32, and Ti is Int32 or Int64 (only Int32 on 32-bit systems).

Sparse(A) copies a SparseMatrixCSC, or a Symmetric or Hermitian view of one, converting element types CHOLMOD does not support to a floating-point type it does. CHOLMOD records in the struct's stype whether the whole matrix or only one triangle of a symmetric or Hermitian matrix is stored; a plain SparseMatrixCSC that is Hermitian is stored as one triangle. sparse(S) copies back to a SparseMatrixCSC, wrapped in Symmetric or Hermitian when only a triangle is stored, so it is not type stable. Sparse(F) returns the L (or LD) factor of a CHOLMOD.Factor F.

The memory is released by a finalizer. The pointer is null after deserialization, and using such an object throws an ArgumentError.

source
SparseArrays.CHOLMOD.Dense — Type
CHOLMOD.Dense{Tv} <: DenseMatrix{Tv}

A dense matrix stored in memory allocated by CHOLMOD, wrapping a pointer to a cholmod_dense struct. Tv is Float64, Float32, ComplexF64 or ComplexF32. Dense(A) copies a strided vector or matrix A into CHOLMOD storage, and Matrix, Vector and copyto! copy it back into a Julia array. It is the dense right-hand side and solution type of the CHOLMOD solve routines; F \ b with a Julia array b does not need it.

The memory is released by a finalizer. The pointer is null after deserialization, and using such an object throws an ArgumentError.

source
SparseArrays.UMFPACK.UmfpackLU — Type
UMFPACK.UmfpackLU{Tv,Ti} <: Factorization{Tv}

The LU factorization of a sparse matrix computed by UMFPACK, returned by lu. Tv is Float64 or ComplexF64, and Ti is Int32 or Int64 (only Int32 on 32-bit systems). F holds a zero-based copy of the factorized matrix together with UMFPACK's opaque symbolic and numeric objects, which are released by finalizers.

The factors are copied out of UMFPACK on each property access:

PropertyDescription
F.Lunit lower triangular SparseMatrixCSC
F.Uupper triangular SparseMatrixCSC
F.prow permutation Vector
F.qcolumn permutation Vector
F.RsVector of row scaling factors
F.:(:)the tuple (L, U, p, q, Rs), extracted in one call

They satisfy F.L * F.U == (F.Rs .* A)[F.p, F.q].

F supports \, ldiv!, det, logabsdet, issuccess, nnz, adjoint, transpose, UMFPACK.rcond and refactorization with lu!. A serialized UmfpackLU carries the matrix rather than the factors, which are recomputed on first use after deserialization.

ldiv! takes an optional UMFPACK.UmfpackWS to avoid allocating. Calls with F take an internal lock. To solve with the same factorization from several tasks at once, give each task its own copy(F), which shares the factors.

Examples

julia> A = sparse([4.0 1.0 0.0; 1.0 4.0 1.0; 0.0 1.0 4.0]);

julia> F = lu(A);

julia> F.L * F.U ≈ (F.Rs .* A)[F.p, F.q]
true

julia> L, U, p, q, Rs = F.:(:);

julia> L * U ≈ (Rs .* A)[p, q]
true
source
SparseArrays.SPQR.QRSparse — Type
SPQR.QRSparse{Tv,Ti} <: Factorization{Tv}

The QR factorization of a sparse matrix computed by SPQR, returned by qr. SPQR's output is copied into Julia arrays, so F holds no memory owned by the C library. Ti is Int32 or Int64 (only Int32 on 32-bit systems); see qr for the element types.

PropertyDescription
F.Qorthogonal factor, stored as sparse Householder reflectors
F.Rupper trapezoidal SparseMatrixCSC
F.prowrow permutation Vector
F.pcolcolumn permutation Vector

They satisfy F.Q * F.R == A[F.prow, F.pcol], where F.Q is square and only its leading columns enter the product.

F supports \ and ldiv! for least squares and minimum-norm solutions, rank, copy, and F', which is the LQ factorization AdjointQRSparse of A'. ldiv!, with F or F', takes an optional SPQR.SpqrWS to avoid allocating. Solves with one F take an internal lock, so they are safe from several tasks but run one at a time. For parallel solves, give each task its own copy(F).

Examples

julia> A = sparse([1.0 0.0; 1.0 1.0; 0.0 1.0]);

julia> F = qr(A);

julia> propertynames(F)
(:R, :Q, :prow, :pcol)

julia> F.Q * F.R ≈ A[F.prow, F.pcol]
true
source
SparseArrays.SPQR.AdjointQRSparse — Type
SPQR.AdjointQRSparse{Tv}

The LQ factorization of a sparse matrix, returned by lq and by the adjoint of a QRSparse. It is an alias for AdjointFactorization{Tv,<:QRSparse{Tv}}, a lazy wrapper that shares the data of the QR factorization F' of A'.

PropertyDescription
F.Llower trapezoidal SparseMatrixCSC, a copy of F'.R'
F.Qorthogonal factor, the adjoint of F'.Q
F.prowrow permutation Vector
F.pcolcolumn permutation Vector

They satisfy F.L * F.Q == A[F.prow, F.pcol]. F supports \, ldiv! and rank.

source
LinearAlgebra.cholesky — Function
cholesky(A::SparseMatrixCSC; shift = 0.0, check = true, perm = nothing) -> CHOLMOD.Factor

Compute the Cholesky factorization of a sparse positive definite matrix A. A must be a SparseMatrixCSC or a Symmetric/Hermitian view of a SparseMatrixCSC. Note that if A doesn't have the type tag, it must itself be symmetric or Hermitian. If perm is not given, a fill-reducing permutation is used. F = cholesky(A) is most frequently used to solve systems of equations with F\b, but also the methods diag, det, and logdet are defined for F. You can also extract individual factors from F, using F.L. However, since pivoting is on by default, the factorization is internally represented as A == P'*L*L'*P with a permutation matrix P; using just L without accounting for P will give incorrect answers. To include the effects of permutation, it's typically preferable to extract "combined" factors like PtL = F.PtL (the equivalent of P'*L) and LtP = F.UP (the equivalent of L'*P). The complete list of supported factors is :L, :PtL, :UP, :U. The permutation vector is available as F.p, defined such that L*L' == A[p, p],

The L component can be materialized as a sparse matrix using sparse(F.L). Other components cannot be materialized directly, but can be reconstructed from sparse(F.L) and F.p if needed.

When check = true, an error is thrown if the decomposition fails. When check = false, responsibility for checking the decomposition's validity (via issuccess) lies with the user.

Setting the optional shift keyword argument computes the factorization of A+shift*I instead of A. If the perm argument is provided, it should be a permutation of 1:size(A,1) giving the ordering to use (instead of CHOLMOD's default AMD ordering).

See also ldlt for a similar factorization that does not require positive definiteness, but can be significantly slower than cholesky.

Examples

In the following example, the fill-reducing permutation used is [3, 2, 1]. If perm is set to 1:3 to enforce no permutation, the number of nonzero elements in the factor is 6.

julia> A = [2 1 1; 1 2 0; 1 0 2]
3×3 Matrix{Int64}:
 2  1  1
 1  2  0
 1  0  2

julia> C = cholesky(sparse(A))
SparseArrays.CHOLMOD.Factor{Float64, Int64}
type:    LLt
method:  simplicial
maxnnz:  5
nnz:     5
success: true

julia> C.p
3-element Vector{Int64}:
 3
 2
 1

julia> L = sparse(C.L);

julia> Matrix(L)
3×3 Matrix{Float64}:
 1.41421   0.0       0.0
 0.0       1.41421   0.0
 0.707107  0.707107  1.0

julia> L * L' ≈ A[C.p, C.p]
true

julia> P = sparse(1:3, C.p, ones(3))
3×3 SparseMatrixCSC{Float64, Int64} with 3 stored entries:
  ⋅    ⋅   1.0
  ⋅   1.0   ⋅
 1.0   ⋅    ⋅

julia> P' * L * L' * P ≈ A
true

julia> C = cholesky(sparse(A), perm=1:3)
SparseArrays.CHOLMOD.Factor{Float64, Int64}
type:    LLt
method:  simplicial
maxnnz:  6
nnz:     6
success: true

julia> L = sparse(C.L);

julia> Matrix(L)
3×3 Matrix{Float64}:
 1.41421    0.0       0.0
 0.707107   1.22474   0.0
 0.707107  -0.408248  1.1547

julia> L * L' ≈ A
true
Note

This method uses the CHOLMOD[ACM887][DavisHager2009] library from SuiteSparse. CHOLMOD only supports real or complex types in single or double precision. Input matrices not of those element types will be converted to these types as appropriate.

Many other functions from CHOLMOD are wrapped but not exported from the Base.SparseArrays.CHOLMOD module.

source
LinearAlgebra.cholesky! — Function
cholesky!(F::CHOLMOD.Factor, A::SparseMatrixCSC; shift = 0.0, check = true) -> CHOLMOD.Factor

Compute the Cholesky ($LL'$) factorization of A, reusing the symbolic factorization F. A must be a SparseMatrixCSC or a Symmetric/ Hermitian view of a SparseMatrixCSC. Note that if A doesn't have the type tag, it must itself be symmetric or Hermitian.

See also cholesky.

Note

This method uses the CHOLMOD library from SuiteSparse, which only supports real or complex types in single or double precision. Input matrices not of those element types will be converted to these types as appropriate.

source
LinearAlgebra.lowrankupdate — Function
lowrankupdate(F::CHOLMOD.Factor, C::AbstractArray) -> FF::CHOLMOD.Factor

Get an LDLt Factorization of A + C*C' given an LDLt or LLt factorization F of A.

The returned factor is always an LDLt factorization.

Only real factorizations are supported; CHOLMOD cannot update or downdate a complex one.

See also lowrankupdate!, lowrankdowndate, lowrankdowndate!.

source
LinearAlgebra.lowrankdowndate — Function
lowrankdowndate(F::CHOLMOD.Factor, C::AbstractArray) -> FF::CHOLMOD.Factor

Get an LDLt Factorization of A - C*C' given an LDLt or LLt factorization F of A.

The returned factor is always an LDLt factorization.

Only real factorizations are supported; CHOLMOD cannot update or downdate a complex one.

See also lowrankdowndate!, lowrankupdate, lowrankupdate!.

source
SparseArrays.CHOLMOD.lowrankupdowndate! — Function
lowrankupdowndate!(F::CHOLMOD.Factor, C::Sparse, update::Cint)

Update an LDLt or LLt Factorization F of A to a factorization of A ± C*C'.

If sparsity preserving factorization is used, i.e. L*L' == P*A*P' then the new factor will be L*L' == P*A*P' + C'*C

update: Cint(1) for A + CC', Cint(0) for A - CC'

source
LinearAlgebra.ldlt — Function
ldlt(A::SparseMatrixCSC; shift = 0.0, check = true, perm=nothing) -> CHOLMOD.Factor

Compute the $LDL'$ factorization of a sparse matrix A. A must be a SparseMatrixCSC or a Symmetric/Hermitian view of a SparseMatrixCSC. Note that if A doesn't have the type tag, it must itself be symmetric or Hermitian. A fill-reducing permutation is used. F = ldlt(A) is most frequently used to solve systems of equations A*x = b with F\b. The returned factorization object F also supports the methods diag, det, logdet, and inv. You can extract individual factors from F using F.L. However, since pivoting is on by default, the factorization is internally represented as A == P'*L*D*L'*P with a permutation matrix P; using just L without accounting for P will give incorrect answers. To include the effects of permutation, it is typically preferable to extract "combined" factors like PtL = F.PtL (the equivalent of P'*L) and LtP = F.UP (the equivalent of L'*P). The complete list of supported factors is :L, :PtL, :D, :UP, :U, :LD, :DU, :PtLD, :DUP. Each one acts as the matrix its name spells out, so that for instance F.PtL \ b solves with P'*L and F.LD \ b solves with the product L*D. The permutation vector is available as F.p, defined such that L*D*L' == A[p, p].

Of these, only LD can be materialized, with sparse(F.LD). Beware that the matrix it returns is not the product L*D: it is CHOLMOD's packed $LDL'$ factor, which stores L with its unit diagonal overwritten by the diagonal of D. Solving with it is therefore not the same as solving with F.LD. Unpack it as

LD = sparse(F.LD)
D = Diagonal(diag(LD))   # equivalently, Diagonal(diag(F))
L = tril(LD, -1) + I     # unit lower triangular

after which L*D*L' == A[F.p, F.p]. The remaining components cannot be materialized directly, but can be reconstructed from L, D and F.p.

Unlike the related Cholesky factorization, the $LDL'$ factorization does not require A to be positive definite. However, it still requires all leading principal minors to be well-conditioned and will fail if this is not satisfied.

When check = true, an error is thrown if the decomposition fails. When check = false, responsibility for checking the decomposition's validity (via issuccess) lies with the user.

Setting the optional shift keyword argument computes the factorization of A+shift*I instead of A. If the perm argument is provided, it should be a permutation of 1:size(A,1) giving the ordering to use (instead of CHOLMOD's default AMD ordering).

See also cholesky for a factorization that can be significantly faster than ldlt, but requires A to be positive definite.

Note

This method uses the CHOLMOD[ACM887][DavisHager2009] library from SuiteSparse. CHOLMOD only supports real or complex types in single or double precision. Input matrices not of those element types will be converted to these types as appropriate.

Many other functions from CHOLMOD are wrapped but not exported from the Base.SparseArrays.CHOLMOD module.

source
LinearAlgebra.ldlt! — Function
ldlt!(F::CHOLMOD.Factor, A::SparseMatrixCSC; shift = 0.0, check = true) -> CHOLMOD.Factor

Compute the $LDL'$ factorization of A, reusing the symbolic factorization F. A must be a SparseMatrixCSC or a Symmetric/Hermitian view of a SparseMatrixCSC. Note that if A doesn't have the type tag, it must itself be symmetric or Hermitian.

See also ldlt.

Note

This method uses the CHOLMOD library from SuiteSparse, which only supports real or complex types in single or double precision. Input matrices not of those element types will be converted to these types as appropriate.

source
SparseArrays.CHOLMOD.rcond — Function
rcond(F::CHOLMOD.Factor) -> Float64

Return CHOLMOD's rough estimate of the reciprocal condition number of the factorized matrix, computed from the diagonal of the factor alone: the smallest entry of abs.(diag(F)) divided by the largest, squared when F is an LL' factorization so that the result estimates the reciprocal condition number of the factorized matrix rather than of its factor.

This is much cheaper than a norm-based estimate such as cond(A, 1), but also much cruder. For positive definite A it is exact when A is diagonal, and otherwise an upper bound on 1 / cond(A, 2), so it can report a matrix as far better conditioned than it is. Use it to detect a badly conditioned or singular factorization, not to measure conditioning accurately. The LU counterpart is UMFPACK.rcond.

Returns 0 if the matrix is singular or the factor has a zero or NaN on its diagonal, and 1 if the matrix is 1-by-1. NaN is never returned.

Examples

julia> A = sparse(Diagonal([1.0, 2.0, 4.0]));

julia> SparseArrays.CHOLMOD.rcond(cholesky(A))
0.25

julia> SparseArrays.CHOLMOD.rcond(ldlt(A))
0.25

julia> SparseArrays.CHOLMOD.rcond(cholesky(sparse(Diagonal([1.0, 0.0])); check=false))
0.0
source
LinearAlgebra.qr — Function
qr(A::SparseMatrixCSC; tol=_default_tol(A), ordering=ORDERING_DEFAULT) -> QRSparse

Compute the QR factorization of a sparse matrix A. Fill-reducing row and column permutations are used such that F.R = F.Q'*A[F.prow,F.pcol]. The main application of this type is to solve least squares or underdetermined problems with \. The function calls the C library SPQR[ACM933]. With ordering=ORDERING_FIXED, F.pcol is the identity unless A is rank deficient, in which case the columns that SPQR finds dependent are moved to the end.

Note

Solves with the returned QRSparse object take an internal lock, so concurrent solves with one object run one at a time. For parallel solves, create a separate copy of this object for each task with copy(F).

Note

qr(A::SparseMatrixCSC) uses the SPQR library that is part of SuiteSparse, which only works in double precision. For any other element type, qr factorizes a Float64 or ComplexF64 copy of A. For Float16, Float32, ComplexF16 and ComplexF32 the factors are then converted back, so the returned QRSparse has the element type of A but was computed in double precision and needs temporary storage for the double-precision copies of A and of the factors. Integer and other non-floating-point element types return a Float64 factorization, and floating-point types wider than Float64 throw an ArgumentError.

Examples

julia> A = sparse([1,2,3,4], [1,1,2,2], [1.0,1.0,1.0,1.0])
4×2 SparseMatrixCSC{Float64, Int64} with 4 stored entries:
 1.0   ⋅
 1.0   ⋅
  ⋅   1.0
  ⋅   1.0

julia> qr(A)
SparseArrays.SPQR.QRSparse{Float64, Int64}
Q factor:
4×4 SparseArrays.SPQR.QRSparseQ{Float64, Int64}
R factor:
2×2 SparseMatrixCSC{Float64, Int64} with 2 stored entries:
 -1.41421      ⋅
     ⋅     -1.41421
Row permutation:
4-element Vector{Int64}:
 1
 3
 4
 2
Column permutation:
2-element Vector{Int64}:
 1
 2
source
LinearAlgebra.lq — Function
lq(A::SparseMatrixCSC; tol=_default_tol(A'), ordering=ORDERING_DEFAULT) -> AdjointQRSparse

Compute the LQ factorization of a sparse matrix A as the adjoint of the sparse QR factorization of A', that is qr(A')', using SPQR. See qr for the keyword arguments and the sparse Q.

The factorization F satisfies A[F.prow, F.pcol] == F.L * F.Q, where F.L is a lower triangular sparse matrix and F.Q the adjoint of the Q of the QR factorization. F \ b solves the underdetermined system A * x == b for a wide A and returns the minimum-norm solution, as for dense lq. F' is the QR factorization of A', and lq(A') reuses qr(A) without a copy.

Examples

julia> A = sparse([1.0 0 1 0; 0 1 0 1]);

julia> F = lq(A);

julia> F.L * F.Q ≈ A[F.prow, F.pcol]
true

julia> F \ [1.0, 2.0] ≈ Matrix(A) \ [1.0, 2.0]
true
source
Base.:\ — Method
(\)(F::QRSparse, B::StridedVecOrMat)

Solve the least squares problem $\min\|Ax - b\|^2$ or the linear system of equations $Ax=b$ when F is the sparse QR factorization of $A$. A basic solution is returned when the problem is underdetermined; A \ b and factorize(A) \ b instead return the minimum-norm solution through lq, as for dense matrices.

Examples

julia> A = sparse([1,2,4], [1,1,1], [1.0,1.0,1.0], 4, 2)
4×2 SparseMatrixCSC{Float64, Int64} with 3 stored entries:
 1.0  ⋅
 1.0  ⋅
  ⋅   ⋅
 1.0  ⋅

julia> qr(A)\fill(1.0, 4)
2-element Vector{Float64}:
 1.0
 0.0
source
Base.:\ — Method
(\)(F::AdjointFactorization{<:Any,<:QRSparse}, B::StridedVecOrMat)

Solve the underdetermined system $A^*x=b$ when F is the sparse QR factorization of the tall matrix $A$, i.e. F = qr(A) with size(A, 1) >= size(A, 2). The minimum-norm solution is returned; when $A$ is rank deficient, the equations corresponding to the dependent columns of $A$ are dropped, mirroring the basic solution returned by F \ B. Overdetermined systems are not supported here as they would require a factorization of $A^*$ rather than of $A$.

Examples

julia> A = sparse([1,2,3,4,1,2,3,4], [1,1,1,1,2,2,2,2], [1.0,1.0,1.0,1.0,1.0,-1.0,1.0,-1.0])
4×2 SparseMatrixCSC{Float64, Int64} with 8 stored entries:
 1.0   1.0
 1.0  -1.0
 1.0   1.0
 1.0  -1.0

julia> x = qr(A)'\[4.0, 0.0]
4-element Vector{Float64}:
 1.0
 1.0
 1.0
 1.0

julia> A'x
2-element Vector{Float64}:
 4.0
 0.0
source
LinearAlgebra.lu — Function
lu(A::AbstractSparseMatrixCSC; check = true, q = nothing, control = get_umfpack_control()) -> F::UmfpackLU

Compute the LU factorization of a sparse matrix A.

For sparse A with real or complex element type, the return type of F is UmfpackLU{Tv, Ti}, with Tv = Float64 or ComplexF64 respectively and Ti is an integer type (Int32 or Int64).

When check = true, an error is thrown if the decomposition fails. When check = false, responsibility for checking the decomposition's validity (via issuccess) lies with the user.

The column permutation q can either be a permutation vector or nothing. If no permutation vector is provided or q is nothing, UMFPACK's default is used. If the permutation is not zero-based, a zero-based copy is made.

The control vector defaults to the Julia SparseArrays package's default configuration for UMFPACK (NB: this is modified from the UMFPACK defaults to disable iterative refinement), but can be changed by passing a vector of length UMFPACK_CONTROL, see the UMFPACK manual for possible configurations. For example to reenable iterative refinement:

umfpack_control = SparseArrays.UMFPACK.get_umfpack_control(Float64, Int64) # read Julia default configuration for a Float64 sparse matrix
SparseArrays.UMFPACK.show_umf_ctrl(umfpack_control) # optional - display values
umfpack_control[SparseArrays.UMFPACK.JL_UMFPACK_IRSTEP] = 2.0 # reenable iterative refinement (2 is UMFPACK default max iterative refinement steps)

Alu = lu(A; control = umfpack_control)
x = Alu \ b   # solve Ax = b, including UMFPACK iterative refinement

The individual components of the factorization F can be accessed by indexing:

ComponentDescription
LL (lower triangular) part of LU
UU (upper triangular) part of LU
prow permutation Vector
qcolumn permutation Vector
RsVector of scaling factors
:(L,U,p,q,Rs) components

The relation between F and A is

F.L*F.U == (F.Rs .* A)[F.p, F.q]

F further supports the following functions:

See also lu!

Note

lu(A::AbstractSparseMatrixCSC) uses the UMFPACK[ACM832] library that is part of SuiteSparse. As this library only supports sparse matrices with Float64 or ComplexF64 elements, lu converts A into a copy that is of type SparseMatrixCSC{Float64} or SparseMatrixCSC{ComplexF64} as appropriate.

source
LinearAlgebra.lu! — Function
lu!(F::UmfpackLU, A::AbstractSparseMatrixCSC; check=true, reuse_symbolic=true, q=nothing) -> F::UmfpackLU

Compute the LU factorization of a sparse matrix A, reusing the symbolic factorization of an already existing LU factorization stored in F. Unless reuse_symbolic is set to false, the sparse matrix A must have an identical nonzero pattern as the matrix used to create the LU factorization F, otherwise an error is thrown. If the size of A and F differ, all vectors will be resized accordingly.

When check = true, an error is thrown if the decomposition fails. When check = false, responsibility for checking the decomposition's validity (via issuccess) lies with the user.

The column permutation q can either be a permutation vector or nothing. If no permutation vector is provided or q is nothing, UMFPACK's default is used. If the permutation is not zero based, a zero based copy is made.

See also lu

Note

lu!(F::UmfpackLU, A::AbstractSparseMatrixCSC) uses the UMFPACK library that is part of SuiteSparse. As this library only supports sparse matrices with Float64 or ComplexF64 elements, lu! will automatically convert the types to those set by the LU factorization or SparseMatrixCSC{ComplexF64} as appropriate.

Julia 1.5

lu! for UmfpackLU requires at least Julia 1.5.

Examples

julia> A = sparse(Float64[1.0 2.0; 0.0 3.0]);

julia> F = lu(A);

julia> B = sparse(Float64[1.0 1.0; 0.0 1.0]);

julia> lu!(F, B);

julia> F \ ones(2)
2-element Vector{Float64}:
 0.0
 1.0
source
SparseArrays.UMFPACK.rcond — Function
rcond(F::UmfpackLU) -> Float64

Return UMFPACK's rough estimate of the reciprocal condition number of the factorized matrix, computed from the diagonal of the factor alone: the smallest entry of abs.(diag(F.U)) divided by the largest.

This is much cheaper than a norm-based estimate such as cond(A, 1), but also much cruder, and it describes the matrix UMFPACK actually factorized rather than A itself. UMFPACK scales the rows of A before factorizing by default (see F.Rs), so for instance every diagonal matrix reports 1. Unlike the Cholesky-based CHOLMOD.rcond, the value is neither an upper nor a lower bound on 1 / cond(A, 2). Use it to detect a singular or badly pivoted factorization, not to measure conditioning.

Returns 0 if the matrix is singular, and 1 if the matrix is 1-by-1.

Examples

julia> F = lu(sparse([1.0 3.0; 0.0 1.0]));

julia> SparseArrays.UMFPACK.rcond(F)
0.25

julia> minimum(abs, diag(F.U)) / maximum(abs, diag(F.U))
0.25

julia> SparseArrays.UMFPACK.rcond(lu(sparse([1.0 2.0; 0.0 0.0]); check=false))
0.0
source
SparseArrays.UMFPACK.UmfpackWS — Type
UMFPACK.UmfpackWS(F::UmfpackLU)

Scratch space for ldiv!(x, F, b; workspace), which makes repeated solves allocation-free. Without it, ldiv! allocates its scratch space on each call. A workspace grows as needed, so it can be reused with any factorization, but not by two calls at once.

source
SparseArrays.CHOLMOD.CholmodWS — Type
CHOLMOD.CholmodWS(F::CHOLMOD.Factor)

Scratch space for ldiv!(x, F, b; workspace), which makes repeated solves allocation-free. Without it, ldiv! allocates its scratch space on each call. A workspace can be reused with any Factor of the same index type, but not by two calls at once. Its memory is released by a finalizer or by CHOLMOD.free!.

source
SparseArrays.SPQR.SpqrWS — Type
SPQR.SpqrWS(F::QRSparse)

Scratch space for ldiv!(x, F, b; workspace), which makes repeated solves allocation-free. Without it, ldiv! allocates its scratch space on each call. A workspace grows as needed, so it can be reused with any factorization of the same element type, but not by two calls at once.

source
LinearAlgebra.rank — Method
rank(S::SparseMatrixCSC{Tv,Ti}; [tol::Real]) -> Ti

Calculate rank of S by calculating its QR factorization. Values smaller than tol are considered as zero. See SPQR's manual.

source
Base.copy — Method
copy(F::UmfpackLU)::UmfpackLU

A shallow copy of UmfpackLU to use in multithreaded solve applications. This function duplicates the control, info and lock fields.

source
Base.copy — Method
copy(F::QRSparse)

A copy of F for solving in parallel, one copy per task. The copy shares the factors and permutations, which no call modifies, and has its own lock, so its solves never wait for those of F.

source
SparseArrays.LibSuiteSparse.init_suitesparse — Function
LibSuiteSparse.init_suitesparse

Internal function which is used to initialize the SuiteSparse libraries to the correct memory management functions. Any package which directly wraps one of the following SuiteSparse libraries must ensure that this function is called before the use of that library: AMD, CAMD, COLAMD, CCOLAMD, UMFPACK, CXSparse, CHOLMOD, KLU, BTF, LDL, RBio, SPQR, SPEX, and ParU

Notes:

  • Currently this function only sets the memory management functions of SuiteSparse_config, however there are also override functions for printf, hypot, and divcomplex.
  • SuiteSparse_config, and this initialization function, is not a dependency of CSparse, GraphBLAS, or LAGraph.
source

Multithreading and thread safety

Each factorization object has an internal lock, and every call that reads or changes its mutable state (the factors held by the C library, the stored matrix and the status) holds that lock for the whole call. Calls with one factorization from several tasks are therefore safe but run one at a time, even those that only read it. To work in parallel, give every task its own copy of the factorization: a copy shares nothing that any call modifies with the original or with the other copies, so its calls never wait for theirs.

Typecopy(F)Calls serialized by the lock of one F
UMFPACK.UmfpackLUshares the matrix and the symbolic and numeric factors; new control, info and lock\, ldiv!, det, lu!
SPQR.QRSparseshares the factors and permutations, which no call modifies; new lock\ and ldiv!, with F or F'
CHOLMOD.Factorindependent deep copy of the whole factorldiv!, cholesky!, ldlt!

The copies of an UmfpackLU or a QRSparse are cheap, since they share the factors:

using LinearAlgebra, SparseArrays

F = lu(A)                       # or qr(A)
X = similar(B)
Threads.@threads for j in axes(B, 2)
    Fj = copy(F)                # shared factors, own lock
    ldiv!(view(X, :, j), Fj, view(B, :, j))
end

A loop that performs many solves per task should make the copy once per task rather than once per right-hand side. Because the copies of an UmfpackLU share its factors, do not call lu! on the original or on any copy while another task is solving with one of them: refactorization frees the numeric object they all point to.

For CHOLMOD, only ldiv!, cholesky!, ldlt! and the in-place low-rank updates take the lock of the Factor. F \ b does not, so a Factor is not safe to share between tasks when any of them may refactorize or update it. Use a separate copy(F) per task in that case; note that this duplicates the factor's memory.

CHOLMOD and SPQR keep their parameters, statistics and error state in a cholmod_common structure. SparseArrays creates one lazily for each Julia task (and index type) and keeps it in task-local storage, so factorizing different matrices from different tasks needs no coordination.

Tuning the factorizations

The defaults suit most problems. These keywords change them.

cholesky and ldlt

  • perm: a permutation of 1:size(A, 1) to use instead of CHOLMOD's AMD ordering. perm = 1:size(A, 1) disables reordering, which usually increases fill-in.

  • shift: factorize A + shift*I without forming it, for example to regularize a semidefinite matrix.

  • check: see Failures.

julia> A = sparse([2.0 1 1; 1 2 0; 1 0 2]);

julia> nnz(cholesky(A)), nnz(cholesky(A; perm = 1:3))
(5, 6)

julia> B = sparse([1.0 -1; -1 1]);  # singular

julia> issuccess(cholesky(B; check = false)), issuccess(cholesky(B; shift = 1.0))
(false, true)

lu

  • q: an initial column ordering, which UMFPACK may still refine. F.q is the final one.

  • control: UMFPACK's Control array. Get the defaults with SparseArrays.UMFPACK.get_umfpack_control(Tv, Ti) for the element and index types of A, and index it with the one-based constants SparseArrays.UMFPACK.JL_UMFPACK_*, such as JL_UMFPACK_PIVOT_TOLERANCE or JL_UMFPACK_ORDERING. The UMFPACK user guide describes each entry. The factorization keeps a copy, which later solves and lu! use.

Unlike UMFPACK itself, SparseArrays turns iterative refinement off by default. To turn it back on, for example for an ill-conditioned matrix:

control = SparseArrays.UMFPACK.get_umfpack_control(Float64, Int)
control[SparseArrays.UMFPACK.JL_UMFPACK_IRSTEP] = 2  # UMFPACK's default
F = lu(A; control)

SparseArrays.UMFPACK.show_umf_ctrl(F) prints the settings of F, and SparseArrays.UMFPACK.show_umf_info(F) prints UMFPACK's statistics for it, such as fill-in, flop count and a condition estimate.

qr

  • ordering: the fill-reducing column ordering, one of the SparseArrays.SPQR.ORDERING_* constants: DEFAULT (SPQR's choice), FIXED and NATURAL (no fill-reducing ordering), COLAMD, AMD (on A'A), METIS, CHOLMOD, BEST (best of COLAMD, AMD and METIS) and BESTAMD (best of COLAMD and AMD).

  • tol: columns whose norm drops to tol or below are treated as zero, which is how rank deficiency is detected. The default is 20 * (m + n) * eps() * maximum(norm, eachcol(A)). Raise it for noisy data.

rank(F) is the numerical rank for that tol, and rank(A; tol) computes rank(qr(A; tol)).

julia> A = sparse([1.0 1; 1 1; 1 1 + 1e-10]);

julia> rank(qr(A)), rank(qr(A; tol = 1e-8))
(2, 1)

Using a different SuiteSparse build

By default the solvers use the SuiteSparse libraries bundled with Julia. To use another build, such as one with GPU support, point SparseArrays at a directory holding all of libsuitesparseconfig, libamd, libcamd, libcolamd, libccolamd, libcholmod, libspqr and libumfpack, with the bundled file names and the same major SuiteSparse version. Either call SparseArrays.LibSuiteSparse.set_libdir!(dir) before the first solver call, or set JULIA_SUITESPARSE_LIBDIR before starting Julia; set_libdir! wins. Packages that call SuiteSparse_jll directly are not affected.

SparseArrays.LibSuiteSparse.set_libdir! — Function
LibSuiteSparse.set_libdir!(dir)
LibSuiteSparse.set_libdir!(nothing)

Load the SuiteSparse libraries from dir instead of the copies bundled with Julia, or restore the bundled copies with nothing. dir must contain the whole set of libraries (libsuitesparseconfig, libamd, libcamd, libcolamd, libccolamd, libcholmod, libspqr and libumfpack) under the same file names as the bundled ones, built from the same major SuiteSparse version.

The libraries are loaded on first use, so this must be called before the first solver call. It throws once any of them has been loaded. The directory can also be set with the JULIA_SUITESPARSE_LIBDIR environment variable; set_libdir! takes precedence.

source
  • ACM887Chen, Y., Davis, T. A., Hager, W. W., & Rajamanickam, S. (2008). Algorithm 887: CHOLMOD, Supernodal Sparse Cholesky Factorization and Update/Downdate. ACM Trans. Math. Softw., 35(3). doi:10.1145/1391989.1391995
  • DavisHager2009Davis, Timothy A., & Hager, W. W. (2009). Dynamic Supernodes in Sparse Cholesky Update/Downdate and Triangular Solves. ACM Trans. Math. Softw., 35(4). doi:10.1145/1462173.1462176
  • ACM933Foster, L. V., & Davis, T. A. (2013). Algorithm 933: Reliable Calculation of Numerical Rank, Null Space Bases, Pseudoinverse Solutions, and Basic Solutions Using SuitesparseQR. ACM Trans. Math. Softw., 40(1). doi:10.1145/2513109.2513116
  • ACM832Davis, Timothy A. (2004b). Algorithm 832: UMFPACK V4.3—an Unsymmetric-Pattern Multifrontal Method. ACM Trans. Math. Softw., 30(2), 196–199. doi:10.1145/992200.992206