Normalise URL in conda package while using them to generate target image

[munishchouhan]

This PR will add noramliseUrl method, which will do the followings:

1. remove protocol and domain name
2. replace . and / with _

[munishchouhan]

Tested locally:
env:

channels:
  - conda-forge
  - bioconda
dependencies:
  - git=2.47.1
  - tower-cli=0.10.3
  - pip
  - pip:
      # NOTE https://stackoverflow.com/a/24811490
      - "https://github.com/seqeralabs/seqera-kit/archive/dev.zip"
      # TODO - seqera-kit==0.5.1

CMD:

(base) munish.chouhan@Munishs-MacBook-Pro image_name_issue % wave --conda-file conda.yml --name-strategy=imageSuffix --wave-endpoint http://localhost:9090 --platform linux/arm64
wave.eu.ngrok.io/wt/18766da717b2/build/git_tower-cli_pip_seqeralabs_seqera-kit_archive_dev_zip:69c4273b06bbddb2
(base) munish.chouhan@Munishs-MacBook-Pro image_name_issue % docker pull wave.eu.ngrok.io/wt/18766da717b2/build/git_tower-cli_pip_seqeralabs_seqera-kit_archive_dev_zip:69c4273b06bbddb2
69c4273b06bbddb2: Pulling from wt/18766da717b2/build/git_tower-cli_pip_seqeralabs_seqera-kit_archive_dev_zip
6e59cb05818e: Already exists 
dec6b097362e: Already exists 
5f20d5c36a06: Already exists 
4f4fb700ef54: Already exists 
5a47b42d3e8c: Already exists 
ad4a1cea1cef: Already exists 
c0287eaf45ee: Already exists 
b8a1a9b42924: Pull complete 
57d1d9429ba0: Pull complete 
a444aa1775d5: Pull complete 
b9afb7fce560: Pull complete 
2ff27ad0ff34: Pull complete 
c73f4a895bb0: Pull complete 
Digest: sha256:3b2b94538d26f3c6079898f9c10864613ceeae09acf00a195715dbe3b043a424
Status: Downloaded newer image for wave.eu.ngrok.io/wt/18766da717b2/build/git_tower-cli_pip_seqeralabs_seqera-kit_archive_dev_zip:69c4273b06bbddb2
wave.eu.ngrok.io/wt/18766da717b2/build/git_tower-cli_pip_seqeralabs_seqera-kit_archive_dev_zip:69c4273b06bbddb2

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview wave.eu.ngrok.io/wt/18766da717b2/build/git_tower-cli_pip_seqeralabs_seqera-kit_archive_dev_zip:69c4273b06bbddb2

[pditommaso]

Please add a test similar to this

wave/src/test/groovy/io/seqera/wave/util/ContainerHelperTest.groovy

Lines 435 to 496 in 5fe4b4c

	def 'should make request target' () {
	expect:
	ContainerHelper.makeTargetImage(BuildFormat.DOCKER, 'quay.io/org/name', '12345', null, null)
	== 'quay.io/org/name:12345'
	and:
	ContainerHelper.makeTargetImage(BuildFormat.SINGULARITY, 'quay.io/org/name', '12345', null, null)
	== 'oras://quay.io/org/name:12345'
	and:
	def conda = '''\
	dependencies:
	- salmon=1.2.3
	'''
	ContainerHelper.makeTargetImage(BuildFormat.DOCKER, 'quay.io/org/name', '12345', conda, null)
	== 'quay.io/org/name:salmon-1.2.3--12345'
	}
	@Shared def CONDA1 = '''\
	dependencies:
	- samtools=1.0
	'''
	@Shared def CONDA2 = '''\
	dependencies:
	- samtools=1.0
	- bamtools=2.0
	- multiqc=1.15
	'''
	@Shared def CONDA3 = '''\
	dependencies:
	- samtools=1.0
	- bamtools=2.0
	- multiqc=1.15
	- bwa=1.2.3
	- xx
	- yy
	- zz
	'''
	@Shared def PIP1 = '''\
	channels:
	- bioconda
	- conda-forge
	dependencies:
	- pip
	- pip:
	- pandas==2.2.2
	'''.stripIndent(true)
	@Shared def PIP2 = '''\
	channels:
	- bioconda
	- conda-forge
	dependencies:
	- pip
	- pip:
	- pandas==2.2.2
	- numpy=1.0
	'''.stripIndent(true)

[munishchouhan]

I added the tests but with this approach, image names are getting too big after appending the package URLs
I think its better to create a hash of those URL and add it to the image name

@pditommaso please let me know your thoughts

[pditommaso]

Im inclined to remove it at all, @ewels what do you think?

[ewels]

Can't we just leave it to the normal "URI is too long, shorten it" method?

Could also aim to be a bit clever and match GitHub URLs, stripping of the uninformative bits, as GitHub will account for almost all of these in practice I think.

But yeah, I'm ok with falling back to a hash of just the URL if needed.

[pditommaso]

Tend to be ugly

https://github.com/seqeralabs/libseqera/tree/master/wave-utils/src/main/resources/templates/conda

[munishchouhan]

Could also aim to be a bit clever and match GitHub URLs, stripping of the uninformative bits, as GitHub will account for almost all of these in practice I think.

Then we need to have a function specific for github url

[pditommaso]

but it can be any repo, not just github. I'd keep simpler and remove. It's not meant to be an exhaustive list

[munishchouhan]

but it can be any repo, not just github. I'd keep simpler and remove. It's not meant to be an exhaustive list

in current logic, its generic I remove the protocol and domain name and only keep the rest of the part and there is a limit of 5 package names, but since we are not sure what part of the URL is the package name, I am adding all of it after domain name
I think we can totally remove the URL because the hash of the container will make the image name unique.

@ewels if you are ok with not adding the URL in the image name, I will go ahead with it

[munishchouhan]

in my latest commit, I have updated the logic to ignore URLs