MatrixMarket Package should support CS* layouts

[buddha314]

As documented here, please support the new CS layouts in the MM package.

As discussed in #7440 the MatrixMarket format does not support the new sparse layouts. The loaded matrices therefore have very little analytic support, including transpose.

@bradcray suggests this sort of work-around for starters.

diff --git a/modules/packages/MatrixMarket.chpl b/modules/packages/MatrixMarket.chpl
index 316647c..ff0a8ee 100644
--- a/modules/packages/MatrixMarket.chpl
+++ b/modules/packages/MatrixMarket.chpl
@@ -363,8 +363,10 @@ class MMReader {
        (nrows, ncols, nnz) = read_matrix_info();
      }
 
+     use LayoutCS;
+
      Dtoret = {1..nrows, 1..ncols};
-     var spDom : sparse subdomain(Dtoret);
+     var spDom : sparse subdomain(Dtoret) dmapped CS();
      var toret : [spDom] eltype;
 
      if finfo.mm_types == MMTypes.Real { assert(eltype == real, "expected real, data in file is not real"); }

[bradcray]

The patch above hard-codes MatrixMarket to always generate CSR arrays. That's a sufficiently common case that it might be reasonable, at least in the short term. A cooler solution might be to have the call be parameterized by the sparse domain map that should be used to store the sparse domain/array. E.g., the Block distribution is parameterized by a sparse layout type that it will use for any sparse subdomains created for it. Perhaps the same thing could be done here?

[buddha314]

I attempted to fork only to realize this package has no tests, silly me. So I edited within my local version of Chapel source and got this error

core-test.chpl:13: error: unresolved call '_array(CSArr(real(64),2,int(64),CSDom(2,int(64),_domain(DefaultRectangularDom(2,int(64),false)),true))).T'
core-test.chpl:13: note: unresolved call had id 194599
make: *** [core] Error 1

I added

use MatrixMarket,
    LinearAlgebra.Sparse,
    LayoutCS,
    Time;

to the top of my test file. The next error is when I compiled with --fast

CCM.shape (60563, 569425)
[CSDom(2,int(64),domain(2,int(64),false),true)] real(64)
make: *** [core] Bus error: 10

Trying w/o fast now, will get back.

[bradcray]

The top error was due to the missing LinearAlgebra.Sparse — if a needed routine isn't pointed to, the compiler won't know where to find it.

The bus error is likely the result of transpose not being prepared to deal with square matrices as discussed on #5

[ben-albrecht]

re: LinearAlgebra.Sparse.transpose bug -- this has been fixed in chapel-lang/chapel#7459

re: the suggested patch in the issue description - @buddha314's preliminary timings show that a matrix read is very slow. It might be worth looking into potential CSR-specific optimizations down the road.

[ct-clmsn]

@buddha314 can you submit a patch and PR for review?

[buddha314]

lil ol' me? I'm not sure I'm qualified. Let me turn in the 2 lines Brad suggested.

[ct-clmsn]

@buddha314 you can read/write chapel! you're totally qualified!

[buddha314]

I don't understand the Chapel testing structure and I can't find the docs. Do you have a link to them?

[ct-clmsn]

@buddha314 apologies for the delay. For testing, a couple simple test programs on small (2x4, 4,4) matrices should be sufficient. I'm good if your testing codes do not use the official chapel testing framework. It could be a simple program that makes a matrix, writes it to disk, loads it from disk and does a comparison and prints "success" or "failure" depending on the outcome. In this case, the simpler a solution is the better!