how to specify numa-aware memory allocation with libfork ?

[Frandy]

When scheduling task, numa-aware usually means one thread will consume tasks in its own pool, then get tasks from pool on the same numa node, and get tasks from the neighbour numa node finally.

While there is a situation, for example, we have 8 threads on two numa nodes ( thread 0~3 on numa node 0, and thread 4~7 on numa node 1), and we have 16 jobs.
And each job has some data, and will visit these data when run task. suppose thread 0~3 running jobs 0~7, and thread 4~7 running jobs 8~15, so data of job 0~7 are on numa node 0, and data of job 8~15 on numa node 1.

So there are two questions,

1. when to allocate data for jobs
   // lazy_pool or busy_pool with 8 threads;
   // for loop to allocate data
   for(jobId = 0; jobId < jobCount-1; ++jobId)
   co_await lf::fork[ allocateDataMem](jobId);
   co_await lf::call[allocateDataMem](fianlJobId);

what's the result for allocated data on which numa node ?

2. when to run task for jobs
   for(jobId = 0; jobId < jobCount - 1; ++jobI)
   co_await lf::fork[ runSomeTask](jobId);
   co_await lf::call[ runSomeTask](jobId);

what's the result for task running on which numa node ?

suppose all those task scheduled evenly, and very balance.

[ConorWilliams]

Libfork's lazy_pool and busy_pool schedulers will ensure workers/threads push/pop task form their own queue's until they run out of local work, then they steal work from neighbours which are topologically "close". All this is handled opaquely. The workers will be distributed and pinned across the physical cores at the construction of the thread pools.

[ConorWilliams]

Hi Frandy,

I've been thinking about this, I don't think libfork provides a nice way of handling this, other than explicitly scheduling all the work (which kind of defeats the point of libfork). I will keep this in mind for V4, I'm open to any suggestions of a nice API for this?

[Frandy]

Thanks. Maybe it need one argument to pass numa hint for task.

[Frandy]

Do you know the library [OpenBLAS] (https://github.com/OpenMathLib/OpenBLAS) or BLIS ? They are high performance matrix/vector library and can use different multi-thread backend, such as openmp, native thread. I wonder if they can get benefit from libfork, especially for nested parallel situation.

[ConorWilliams]

I'm familiar with OpenBLAS, first time I've seen BLIS though, I suspect they could benefit if they use nested parallelism, especially with a small grain size. Any idea how hard it is to hack together a new backend for them?

[Frandy]

I'm user of OpenBLAS/BLIS, and tried to learn the detail of dgemm implementation. If just to verify the ideal, and select one blas3 function to try, it is not that hard. But if to add backend of parallel for all function, it will have a lot of work.

[ConorWilliams]

Just make sure when compiling libfork that CMake prints something like this:

 Found hwloc 2.10.0rc3-git in /home/cj/libfork/build/rel/vcpkg_installed/x64-linux/lib/pkgconfig/../../include:hwloc;m;pthread

hwloc is the library that libfork uses to detect the NUMA topology