ci: Improve fusion-symlink.nf test

[alberto-miranda]

Description

The fusion-symlink.nf fails randomly due to what looks like a race condition. Since it was impossible to reproduce the problem in the a controlled environment, we are slightly modifying the test in an attempt to better diagnose the problem. To this end, this PR
extends the fusion-symlink.nf in two ways:

1. We add a retry protocol with an exponential backoff to the nextflow fs cp [...] commands used for validating the published data. The idea is that if there's indeed a race condition between the publishing and the validation, retrying the validation should eventually succeed. We ensure the test still fails so that we may detect when the potential race condition occurs.
2. If the error is not a publish/validation race condition, we still want to capture as much information from the running fusion instances. To that end we enable fusion's extended telemetry by setting the FUSION_TRACING_DESTINATION environment variable.

[netlify]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	58e607a
🔍 Latest deploy log	https://app.netlify.com/sites/nextflow-docs-staging/deploys/67bf4da00dbeae0008dfe0f6

[alberto-miranda]

Could someone with access double check if there's a .fusion/traces/ directory in any of the workdirs for the fusion-symlink.nf test? I would like to make sure the setting is working before we merge this, just in case...

[pditommaso]

Where are supposed those files?

»  aws s3 ls s3://nextflow-ci/work/53/7f3a251f902c946e3ad794a5642456/
                           PRE .fusion/
2025-02-25 18:39:13          0 
2025-02-25 18:39:25          0 .command.begin
2025-02-25 18:39:25          0 .command.err
2025-02-25 18:39:25          0 .command.log
2025-02-25 18:39:25          0 .command.out
2025-02-25 18:39:14       4156 .command.run
2025-02-25 18:39:14         38 .command.sh
2025-02-25 18:39:25          1 .exitcode
2025-02-25 18:39:26       9551 .fusion.log
2025-02-25 18:39:25          6 data.txt

[alberto-miranda]

Where are supposed those files?

»  aws s3 ls s3://nextflow-ci/work/53/7f3a251f902c946e3ad794a5642456/
                           PRE .fusion/
2025-02-25 18:39:13          0 
2025-02-25 18:39:25          0 .command.begin
2025-02-25 18:39:25          0 .command.err
2025-02-25 18:39:25          0 .command.log
2025-02-25 18:39:25          0 .command.out
2025-02-25 18:39:14       4156 .command.run
2025-02-25 18:39:14         38 .command.sh
2025-02-25 18:39:25          1 .exitcode
2025-02-25 18:39:26       9551 .fusion.log
2025-02-25 18:39:25          6 data.txt

If I configured it correctly, they should be under the .fusion directory shown in your listing. Can you try with:

aws s3 ls s3://nextflow-ci/work/53/7f3a251f902c946e3ad794a5642456/.fusion/traces/

Thanks!