Some simple graphs characterising a workflow ...

[DavidPCoster]

I don't know if there is any interest in producing simple graphs characterising a workflow (simpler than those already provided). As an example

which also produces a text version

compute.txt

The analysis script does this for compute, transfer and wait.

If there is interest, I could share my code -- but it is really so simple that anybody could reproduce it ...

[LourensVeen]

I'm not sure how far to go with this actually. MUSCLE3 has some built-in graphs, and a Python API for loading profiling data and preprocessing it into easy-to-visualise data. I guess we can keep adding different kinds of visualisations, but eventually it would become a plotting library 😄.

Actually, maybe this would make for a nice example for the documentation?

[DavidPCoster]

I find those graphs easier for getting a quick understanding than

[LourensVeen]

Point taken 😄. Although that last plot comes close, it just has the profiling bug still in it that causes init to look like it's using a lot of CPU, and perhaps it should be sorted by amount of compute. Pie charts are also not the most accurate way of visualising data

The other issue is that a MUSCLE3 simulation is a parallel system, so just looking at the amount of time spent computing could also be misleading, as there could be a component that computes a lot while never actually being the long pole in the tent. So then it depends on whether you're trying to optimise runtime or resource usage.

[LourensVeen]

Thinking about this more, which question are you trying to answer with these graphs? I think we need to get that clear to see if we need to improve the existing charts, add new ones, or something else.

[LourensVeen]

And one last thought for today: how would we do this with hierarchical models? Maybe something like what graphical disk usage tools do? I seem to recall one that used a kind of 2D spatial tree, but searching the web a bit it seems that most of them are using pie charts now. As I recall it looked something like the first graph in this article on HPC machine market share.