Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.4 canfar run #690

Open
wants to merge 408 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
408 commits
Select commit Hold shift + click to select a range
3ab923f
terminal title from within curl script
martinkilbinger Feb 11, 2024
ee5b3ad
Update tiles_P7.txt
martinkilbinger Feb 11, 2024
79cb00a
Merge pull request #9 from martinkilbinger/P7
martinkilbinger Feb 11, 2024
92e4308
merged Dockerfile from docker_image
martinkilbinger Feb 11, 2024
155f2fe
added COSMOS tile list
martinkilbinger Feb 23, 2024
a86b6fd
Merge remote-tracking branch 'origin/develop' into develop
martinkilbinger Feb 23, 2024
1d928d4
monitoring and job handling scripts; small modifs, output; canfar pip…
martinkilbinger Feb 23, 2024
403e8c5
Merge pull request #10 from martinkilbinger/curl
martinkilbinger Feb 23, 2024
c56dae0
Added (back) from v1 exp mccd ini file
martinkilbinger Feb 23, 2024
a6d8b52
combine run script: flag_tile and flag_exp as diff options
martinkilbinger Mar 5, 2024
1a2d47d
mask runer excluding processed files working
martinkilbinger Mar 6, 2024
de82961
combine runs for masks: changed output target dir name
martinkilbinger Mar 6, 2024
33b5b75
Added COSMOS tile list
martinkilbinger Mar 6, 2024
8928c86
summary bug fixed
martinkilbinger Mar 6, 2024
fc5a7df
comment removed
martinkilbinger Mar 6, 2024
e882df3
style
martinkilbinger Mar 6, 2024
7cdcd73
Merge pull request #11 from martinkilbinger/mask
martinkilbinger Mar 6, 2024
fde8fe9
Added DR5 tile list file
martinkilbinger Mar 6, 2024
802a8cf
t branMerge remote-tracking branch 'origin/develop' into develop
martinkilbinger Mar 6, 2024
2fb58eb
Merge remote-tracking branch 'upstream/develop' into develop
martinkilbinger Mar 6, 2024
160c1b4
Merge remote-tracking branch 'origin/develop' into docker_image
martinkilbinger Mar 6, 2024
93e86e9
Delete scripts/sh/init_canfar.sh
martinkilbinger Mar 6, 2024
8b92dbf
Merge pull request #13 from martinkilbinger/docker_image
martinkilbinger Mar 6, 2024
fe1eb3a
Merge remote-tracking branch 'origin/develop' into develop
martinkilbinger Mar 6, 2024
720afe9
Merge remote-tracking branch 'origin/develop' into plot_spectro
martinkilbinger Mar 6, 2024
07a5924
Merge pull request #14 from martinkilbinger/plot_spectro
martinkilbinger Mar 6, 2024
4ce26ba
Merge remote-tracking branch 'origin/develop' into develop
martinkilbinger Mar 6, 2024
a6bf8a5
remove centos ci
martinkilbinger Mar 6, 2024
339f13f
Merge branch 'develop' of github.com:martinkilbinger/shapepipe-1 into…
martinkilbinger Mar 6, 2024
bddc491
repaired plot spectro notebook
martinkilbinger Mar 6, 2024
4dffd47
added doc string
martinkilbinger Mar 6, 2024
41cfd97
Merge branch 'develop' of github.com:martinkilbinger/shapepipe-1 into…
martinkilbinger Mar 6, 2024
5675c06
python example runner: added back mccd and pysap
martinkilbinger Mar 6, 2024
14569b2
corrected typo in doc string
martinkilbinger Mar 6, 2024
b406438
updated canfar pipeline doc
martinkilbinger Mar 13, 2024
fe66831
fixed combined mask input dir
martinkilbinger Mar 13, 2024
282d76b
added parameters to final cat
martinkilbinger Mar 13, 2024
9ac6bb4
summary
martinkilbinger Mar 13, 2024
25ef4e2
Removed rarely used ngmix njob/ngal options; splits computed automati…
martinkilbinger Mar 13, 2024
75fb975
canfar init + run job: removed prev Ms and Mc dirs for job 256
martinkilbinger Mar 13, 2024
376bbf1
merge sep cats: error if ngmix #HDUs < 6
martinkilbinger Mar 13, 2024
06564a1
Merge pull request #15 from martinkilbinger/cleanup_canfar
martinkilbinger Mar 13, 2024
21deebc
env-dev updated to min versions on canfar (docker image SP 1.1)
martinkilbinger Apr 3, 2024
bb15d08
Merge remote-tracking branch 'upstream' into 651-conda-activate-fix-MK
martinkilbinger Apr 4, 2024
1ea5d4a
Merge remote-tracking branch 'origin/develop' into 651-conda-activate…
martinkilbinger Apr 4, 2024
f2a5a06
minor changes
Apr 10, 2024
df90225
Improved error msg for empty process list and -e flag
martinkilbinger May 14, 2024
b7cf7ad
canfar job descr and summary updated
martinkilbinger May 20, 2024
df983c4
fixed err msg
martinkilbinger May 20, 2024
cca5694
config files updated
martinkilbinger May 20, 2024
6b2f498
summary updated
martinkilbinger May 20, 2024
86816d6
spread model: added check on non-positive object size
martinkilbinger May 20, 2024
1e5f83d
no change
martinkilbinger May 20, 2024
182ff22
added checks
martinkilbinger May 20, 2024
9827d29
scripts updated
martinkilbinger May 20, 2024
a872dfd
split exp runner: added checks and err msg for data integrity
martinkilbinger May 21, 2024
d445932
added Mh cfg file
martinkilbinger May 21, 2024
f91cc10
summary v1.2: split_exp using exposures, no longer shdu
martinkilbinger May 21, 2024
4c9f938
Added script to convert pix to world coordinates of PSF measurements
May 28, 2024
66253b1
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
martinkilbinger May 28, 2024
89622db
merge sep module: fixed cat handling if not sufficient HDUs on input
martinkilbinger May 30, 2024
badca69
summary: split Ms and Mh; writing/reading list_exp to file for sped-up
martinkilbinger May 30, 2024
fd6bb21
improved log and error msg
martinkilbinger May 30, 2024
760c351
tile processing: using all split exp inputs, not only last
martinkilbinger May 30, 2024
79abdc8
convert psf script err msg
martinkilbinger May 30, 2024
03e8150
local Mh handling; 256 512 split
martinkilbinger May 30, 2024
01fea37
psf converting from val input to merge stars working (not yet plotting)
martinkilbinger Jun 3, 2024
1af8b49
summary script fixed bitwise & type
martinkilbinger Jun 3, 2024
5468a5c
init canfar script: added DISPLAY to avoid setools qt bug
martinkilbinger Jun 3, 2024
6bf14e0
trying job name
martinkilbinger Jun 3, 2024
da379f1
merge starcat mod: added input HDU number
martinkilbinger Jun 3, 2024
6c4029b
get IDs of running jobs
martinkilbinger Jun 11, 2024
38719f5
merge star cats w/o plotting; psf conv stars
martinkilbinger Jun 11, 2024
e9e76af
curl&init scrips: remove -k kind options, determined automatically ac…
martinkilbinger Jun 11, 2024
af5f1db
minor bug fixes, messages
martinkilbinger Jun 11, 2024
b2ceed2
combine run for psf_conv updated input dir
martinkilbinger Jun 11, 2024
d61e4c5
merge star cat: set SExcat to False, so output in HDU 1
martinkilbinger Jun 11, 2024
e679935
added gfortran to Docker; pytest to env
martinkilbinger Jun 11, 2024
65d4a68
rho stats using shear_psf_leakage
Jun 14, 2024
69d78fb
canfar doc updated
martinkilbinger Jun 14, 2024
2f11260
cleaned up rho stats code
martinkilbinger Jun 14, 2024
a92236b
rho stats testing
martinkilbinger Jun 14, 2024
67a5d44
added -p psf to curl and init scripts (to test\!); -m MH
martinkilbinger Jun 14, 2024
8436d16
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
martinkilbinger Jun 14, 2024
5109bed
cgf file INPUT updated
martinkilbinger Jun 14, 2024
5226a0a
rho stats plots added UNIONS option (shear_psf_leakage)
martinkilbinger Jun 15, 2024
5a3e89a
plot rho stats for converted star cat
martinkilbinger Jun 16, 2024
f5468d7
ran black; pipeline canfar minor update
Jun 19, 2024
a64ff7f
small modifs to make thing smoother
martinkilbinger Jun 25, 2024
124decc
canfar scripts: -m 1 creates local exp headers indep of job
martinkilbinger Jun 26, 2024
2ae3d35
summary check special working
martinkilbinger Jun 26, 2024
9f4e7fb
ngmix: added last check that all 8 HDUs are present
martinkilbinger Jun 26, 2024
fe69e4d
debug messages removed
martinkilbinger Jun 26, 2024
93eb86a
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
martinkilbinger Jun 26, 2024
1c75779
Added GaAP photometry to SEx param file
martinkilbinger Jul 1, 2024
26c1b78
summary specials
martinkilbinger Jul 1, 2024
c0e5435
print msg if empty vignets
martinkilbinger Jul 1, 2024
a7203a3
doc
martinkilbinger Jul 1, 2024
7f842a3
cleaning up and adding doc strings to vignet maker
martinkilbinger Jul 1, 2024
ee4ef4f
added clarifying comments to vignet maker
martinkilbinger Jul 1, 2024
1c8b4cd
cleaned up curl script
martinkilbinger Jul 12, 2024
3210b88
canfar scripts adopted to new env; added debug
martinkilbinger Jul 12, 2024
ae97110
shapepipe_run script adopted to new canfar env
martinkilbinger Jul 12, 2024
ebb40b1
setools matplotlib Agg to avoid display error
martinkilbinger Jul 12, 2024
ca026ef
ngmix: catching missing keys in postage stamp sqlite database files
martinkilbinger Aug 29, 2024
126be96
combin runs script: added tile_detection
martinkilbinger Aug 29, 2024
f60a5f0
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
martinkilbinger Aug 29, 2024
08d7a7c
cleaned up convery pix script
martinkilbinger Oct 8, 2024
8b4619c
summary added special, n_sh as variable
martinkilbinger Oct 8, 2024
e5c8046
debugging curl canfar
martinkilbinger Oct 8, 2024
c8f657a
clarified doc on merging star cat
martinkilbinger Oct 8, 2024
e0eb0f4
Added checks and improved messages for keys in FITS files
martinkilbinger Oct 8, 2024
ba3a0d8
debugging curl canfar
martinkilbinger Oct 8, 2024
ca7f3e5
Pi psfex HDU number changed
martinkilbinger Oct 8, 2024
e219400
cosmetic changes
martinkilbinger Oct 8, 2024
34edaaf
doc canfar: options sp_local=0,1
martinkilbinger Oct 21, 2024
96ec184
split image config more flexible
martinkilbinger Oct 21, 2024
45384c9
init_curl script: allowing for individual -j 2, 8 (sp_local=1)
martinkilbinger Oct 21, 2024
e15e8db
summary: path for candide
Oct 21, 2024
925ae5b
Merge remote-tracking branch 'upstream/develop' into 651-conda-activa…
martinkilbinger Oct 21, 2024
c6be9cc
Merge branch '651-conda-activate-fix-MK' into v1.4
martinkilbinger Oct 21, 2024
f1ed5a8
added vos
martinkilbinger Oct 21, 2024
f79e0bf
Merge remote-tracking branch 'origin/v1.4' into tiles_202402
martinkilbinger Oct 21, 2024
e2ec9fa
Merge remote-tracking branch 'origin/tiles_202402' into v1.4
martinkilbinger Oct 21, 2024
161d3ca
Merge remote-tracking branch 'origin/v1.4' into cosmos
martinkilbinger Oct 21, 2024
7944b96
added python-pysap to env
Oct 21, 2024
f079da6
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
Oct 21, 2024
c3edaad
Merge remote-tracking branch 'origin/v1.4' into wcs_mk
martinkilbinger Oct 21, 2024
296e190
Merge pull request #20 from martinkilbinger/wcs_mk
martinkilbinger Oct 21, 2024
65cbac2
added pandas to yml
Oct 22, 2024
b8e47fa
Merge remote-tracking branch 'origin/v1.4' into v1.4
Oct 22, 2024
43c3192
added scripts to extract used CCD names
martinkilbinger Oct 22, 2024
4103430
Merge remote-tracking branch 'origin/develop' into random
Oct 23, 2024
eeffac2
Merge remote-tracking branch 'origin/v1.4' into random
Oct 23, 2024
1ad1ac6
Merge pull request #21 from martinkilbinger/random
martinkilbinger Oct 23, 2024
72bcc09
common bash functions to separate file to be loaded in bash scripts
martinkilbinger Nov 13, 2024
f49eb3d
Sp config file fixed (?) exp get image runner input
martinkilbinger Nov 13, 2024
9dbf5e9
P8 job 2 param setting
martinkilbinger Nov 13, 2024
3eb3d72
init script: copying to /scratch; improved job 2 handling for P8
martinkilbinger Nov 13, 2024
a80d731
stats job script added debug option
martinkilbinger Nov 13, 2024
6498c68
P8 job 2 stuff
martinkilbinger Nov 13, 2024
6383b7d
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
martinkilbinger Nov 13, 2024
e7f73c2
added fits file check
Nov 13, 2024
53d7f8b
Cail's first v1.4 Dockerfile w/o conda
Nov 13, 2024
5d133cc
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into mk…
Nov 13, 2024
1bc3e07
scripts to get CCDs used in tiles (proxies)
martinkilbinger Nov 14, 2024
320467a
init script: force sym link to home cfis cfg dir
Nov 16, 2024
05457be
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
Nov 16, 2024
e3a9e36
command -> functions.sh
Nov 17, 2024
4eb181b
run_scratch_local: script to run and submit jobs on candide; analogue…
Nov 17, 2024
e6967cd
sp_local as argument for init script
martinkilbinger Nov 22, 2024
24015ba
added check in psfe interp runner if no object is in HDU (rare cases …
martinkilbinger Nov 22, 2024
30cb450
sextractor runner: fixed stderr, stdout order; added to log file to d…
martinkilbinger Nov 22, 2024
ec6718a
summmary added special case of empty FITS file (mask runner)
martinkilbinger Nov 22, 2024
6d0f6d5
added code to .gitignore
martinkilbinger Nov 22, 2024
f503e94
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
martinkilbinger Nov 22, 2024
53b3e7a
add new dockerfile
cailmdaley Nov 22, 2024
36ce5b1
change cd.yml
cailmdaley Nov 22, 2024
c333fad
functions: get_kind_from_job function
martinkilbinger Nov 22, 2024
6728c03
set back conda version
martinkilbinger Nov 22, 2024
bef14bc
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
martinkilbinger Nov 22, 2024
f2f1998
get_kind from init
Nov 22, 2024
0af84f1
try docker ci
cailmdaley Nov 22, 2024
5e20622
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
Nov 22, 2024
0f6d4f4
more complex deploy-image
cailmdaley Nov 22, 2024
1326055
turn off cd on this branch
cailmdaley Nov 22, 2024
4d4ee81
update to latest versions of gh actions, add macos
cailmdaley Nov 22, 2024
6da11f6
added gh action versioning (point to master)
cailmdaley Nov 22, 2024
1454965
fix missing gh action version
cailmdaley Nov 22, 2024
2537b02
remove macos for now, pin major versions
cailmdaley Nov 22, 2024
b5c6bb1
remove context
cailmdaley Nov 22, 2024
cce15b1
revert galsim version
cailmdaley Nov 22, 2024
349882a
add back PyQt5
cailmdaley Nov 22, 2024
3f04d63
bump galsim minor version
cailmdaley Nov 22, 2024
cba0122
remove hardcoded conda python path in shapepipe_run
cailmdaley Nov 22, 2024
f506ec8
added check in mask module for corrupt external flag file
martinkilbinger Nov 24, 2024
dbf1ef8
run scratch: introduced
Nov 24, 2024
fbc621b
Added messaged and error checks if input number patterns are inconsis…
martinkilbinger Nov 26, 2024
c30f292
Fixed error msg
martinkilbinger Nov 26, 2024
6daa778
ngmix: check existing dir to append cat; batch save. Note that with a…
martinkilbinger Nov 28, 2024
0ac9998
use batch save ngmix cfg
martinkilbinger Nov 28, 2024
0f516fd
-f --fix option in init script
martinkilbinger Nov 28, 2024
cc62ab6
added checks for corrupt input files
martinkilbinger Nov 28, 2024
d6f069b
simple scripts added to remove previous duplicate runs
martinkilbinger Nov 28, 2024
6899860
ngmix batch save cfg
martinkilbinger Nov 28, 2024
21f5fb3
curl and init scripts: improved debug output
martinkilbinger Nov 29, 2024
0d5e3cf
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into hdf5
martinkilbinger Nov 29, 2024
ec48f7c
Merge pull request #22 from martinkilbinger/hdf5
martinkilbinger Nov 29, 2024
28fcd36
improved Fix; get_number_objects; mh_local handling
martinkilbinger Dec 1, 2024
91f226a
switch to PC sextractor package name
cailmdaley Dec 5, 2024
9c96ef6
link to exp script: added shdu option -s
martinkilbinger Dec 6, 2024
c9deb41
summary: added ngmix special for incomplete cat
martinkilbinger Dec 6, 2024
e190ebe
combine runs script added shdu
martinkilbinger Dec 6, 2024
214ba37
canfar job scripts: fix, test options; better debug output
martinkilbinger Dec 6, 2024
23abff3
vignetmaker added file read test
martinkilbinger Dec 6, 2024
3eaf9a7
candide run script added 128
Dec 6, 2024
7bb830d
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
Dec 6, 2024
2cb0519
added code to dead with missing spread model input
martinkilbinger Dec 9, 2024
eb6831d
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
martinkilbinger Dec 9, 2024
d53fa84
canfar scripts improve debug
martinkilbinger Dec 9, 2024
df1dce3
Merge pull request #23 from cailmdaley/v1.4_docker
martinkilbinger Dec 10, 2024
2e6cba6
create hdf5: -l flag
martinkilbinger Dec 10, 2024
b29cb94
summary
martinkilbinger Dec 10, 2024
42f587d
canfar scripts debug msg
martinkilbinger Dec 10, 2024
32f6963
no spread model config file
martinkilbinger Dec 10, 2024
54af384
get_special-script
martinkilbinger Dec 10, 2024
bbc547d
Update deploy-image.yml
martinkilbinger Dec 10, 2024
e21ead3
hdf5 create script improved, working well now
martinkilbinger Dec 12, 2024
c55c27e
fixed bug in stats script (not counting Pending jobs everywhere); std…
martinkilbinger Dec 12, 2024
c3e9b27
Warning! Changed get_images_runner input!
martinkilbinger Dec 12, 2024
5ebbfba
remove SPREAD_MODEL params from final cat (some runs are without)
martinkilbinger Dec 18, 2024
3b1b72e
--sm option (SPREAD_MODEL on/off)
martinkilbinger Dec 18, 2024
73dda61
Uz Git input changed batk
martinkilbinger Dec 18, 2024
ad2a6c4
added single_ID options
martinkilbinger Dec 18, 2024
4c3acf1
script changes
martinkilbinger Dec 18, 2024
79fd70e
added script to prepare patch for sp_val run
martinkilbinger Dec 29, 2024
9c8063c
updated for for new create final hdf4 script
martinkilbinger Dec 29, 2024
c2512b8
canfar run scripts: added -R RAM option; fixed bugs in communicating …
martinkilbinger Dec 29, 2024
bc8eeeb
cleaned up create final hdf5 script
martinkilbinger Dec 29, 2024
4bea0e2
comments
martinkilbinger Dec 29, 2024
19b819b
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
martinkilbinger Dec 29, 2024
444bd89
Update summary_params_pre_v2.py
martinkilbinger Dec 29, 2024
70c043d
Update summary_params_pre_v2.py
martinkilbinger Dec 29, 2024
4e20b37
Merge pull request #24 from martinkilbinger/exp_list
martinkilbinger Dec 29, 2024
b6ebc56
Merge remote-tracking branch 'origin/v1.4' into HEAD
martinkilbinger Jan 2, 2025
95fd7c7
create final cat: added doc strings; canfar scripts: added -B, -V wor…
martinkilbinger Jan 2, 2025
d27b52c
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
martinkilbinger Jan 2, 2025
68a33f2
black
martinkilbinger Jan 9, 2025
67f6033
fixed conda activation; better job handling (-j 64); deal with previo…
martinkilbinger Jan 9, 2025
e4cfec4
try/except for opening and reading images in vignet runner
martinkilbinger Jan 9, 2025
9b04e5a
canfar_doc
martinkilbinger Jan 11, 2025
f22f2cc
Merge remote-tracking branch 'origin/v1.4' into mk_v1.4
martinkilbinger Jan 15, 2025
6f23da6
fixed typo from merge
martinkilbinger Jan 15, 2025
d04c4ea
fixed typo from merge
martinkilbinger Jan 15, 2025
60ca438
Merge pull request #25 from martinkilbinger/mk_v1.4
martinkilbinger Jan 15, 2025
0027593
remove obsolete file with all tile IDS (for HST N)
martinkilbinger Jan 15, 2025
0639b66
remove MKDEBUG comment
martinkilbinger Jan 15, 2025
ea3f941
Ran black on all py files
martinkilbinger Jan 15, 2025
61f193c
fixed bug in init run excl script
Jan 15, 2025
13e4bcc
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
Jan 15, 2025
a448166
added script summary_tiles to characterise found and missing tiles
martinkilbinger Jan 16, 2025
028e589
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
Jan 17, 2025
ddaa88e
new input_cat_type keyword in star cat header
martinkilbinger Jan 19, 2025
b395bab
spread model runner optional
martinkilbinger Jan 19, 2025
8ff1e2a
spread model runner optional
martinkilbinger Jan 19, 2025
07987aa
Merge remote-tracking branch 'origin/v1.4' into v1.4
martinkilbinger Jan 19, 2025
d2c2b7a
added P8 to summary tiles
martinkilbinger Jan 24, 2025
985a1cb
make_cat: check on n_obj
martinkilbinger Jan 24, 2025
aff85a2
create final, version was lost from v1.4
martinkilbinger Jan 24, 2025
27175b8
fixed missing \ in sh script
martinkilbinger Jan 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions .github/workflows/ci-dev.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: CI

on:
pull_request:
branches:
- develop

jobs:

test-full:
name: Full Test Suite
runs-on: ${{ matrix.os }}

strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
python-version: [3.8]

steps:
- uses: actions/checkout@v2

- name: Install Linux dependencies
if: runner.os == 'Linux'
run: sudo apt-get install libopenblas-dev

- name: Install macOS Dependencies
shell: bash -l {0}
if: runner.os == 'macOS'
run: |
brew tap sfarrens/sf
brew install bigmac libomp

- name: Set up conda
uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
python-version: ${{ matrix.python-version }}
auto-activate-base: true

- name: Install package
shell: bash -l {0}
run: |
./install_shapepipe --env-dev --develop

- name: Run tests
shell: bash -l {0}
run: |
conda activate shapepipe-dev
python setup.py test
shapepipe_run -c example/config.ini
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ on:
branches:
- main
- master
- develop

jobs:

Expand Down
41 changes: 41 additions & 0 deletions .github/workflows/deploy-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Create and publish a Docker image

on:
push:
branches: ['master']

env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}

jobs:
build-and-push-image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Checkout repository
uses: actions/checkout@v3

- name: Log in to the Container registry
uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

- name: Build and push Docker image
uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,4 @@ dmypy.json
# Ignore example output
*shapepipe_run_*
*shapepipe_runs*
code
57 changes: 57 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
FROM continuumio/miniconda3

LABEL Description="ShapePipe Docker Image"
ENV SHELL /bin/bash

ARG CC=gcc-9
ARG CXX=g++-9

# gcc < 10 is required to compile ww
ENV CC=gcc-9
ENV CXX=g++-9

RUN apt-get update --allow-releaseinfo-change && \
apt-get update && \
apt-get upgrade -y && \
apt-get install apt-utils -y && \
apt-get install make -y && \
apt-get install automake -y && \
apt-get install autoconf -y && \
apt-get install gcc-9 g++-9 -y && \
apt-get install gfortran -y && \
apt-get install locales -y && \
apt-get install libgl1-mesa-glx -y && \
apt-get install xterm -y && \
apt-get install cmake protobuf-compiler -y && \
apt-get install libtool libtool-bin libtool-doc -y && \
apt-get install libfftw3-bin libfftw3-dev -y && \
apt-get install libatlas-base-dev liblapack-dev libblas-dev -y && \
apt-get install vim -y && \
apt-get install locate -y && \
apt-get install curl -y && \
apt-get install acl -y && \
apt-get install sssd -y && \
apt-get clean

ADD nsswitch.conf /etc/

RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && \
locale-gen
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

SHELL ["/bin/bash", "--login", "-c"]

COPY ./environment.yml ./
COPY install_shapepipe README.rst setup.py setup.cfg ./
RUN touch ./README.md

RUN conda update -n base -c defaults conda -c defaults
RUN conda env create --file environment.yml

COPY shapepipe ./shapepipe
COPY scripts ./scripts

RUN source activate shapepipe
#RUN pip install jupyter
9 changes: 9 additions & 0 deletions auxdir/CFIS/tiles_202402/tiles_COSMOS.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
299.183
300.183
301.183
299.184
300.184
301.184
299.185
300.185
301.185
Loading