Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.0 #13

Draft
wants to merge 142 commits into
base: main
Choose a base branch
from
Draft

v1.0 #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
142 commits
Select commit Hold shift + click to select a range
bbac86c
Renamed test script
martinkilbinger Sep 1, 2023
25d7b30
remove duplicate fct; black formatting
martinkilbinger Sep 2, 2023
cd9e266
debugging apha matrix
martinkilbinger Sep 6, 2023
d64f0fc
Merge remote-tracking branch 'origin/sa_scale' into sa_scale
martinkilbinger Sep 6, 2023
8c5275d
alpha matrix trace
martinkilbinger Sep 13, 2023
f2ad6ea
added option to save xi_ab and xi_bb corr files
martinkilbinger Sep 20, 2023
3e61bf2
Added lock file
martinkilbinger Sep 20, 2023
6f21a37
resetted
martinkilbinger Sep 20, 2023
1d19cbc
Merge remote-tracking branch 'origin/develop' into spin
martinkilbinger Sep 20, 2023
5c0f22d
alpha: fast computation option
martinkilbinger Sep 22, 2023
3968a6c
Clara's email added
martinkilbinger Sep 25, 2023
f8a9638
scale-dep alpha spin matrix testing
martinkilbinger Sep 27, 2023
0d60907
leakage_scale py script updated from Run dir
martinkilbinger Sep 27, 2023
702fac4
spin-consistent alpha(theta)
martinkilbinger Oct 1, 2023
4679875
Added notebook to test weighting for scalar alpha(theta)
martinkilbinger Oct 19, 2023
c12e749
leakage_scale script back to basics
martinkilbinger Oct 21, 2023
aab956c
added notebook package
martinkilbinger Oct 21, 2023
e224407
galaxy weights as option
martinkilbinger Oct 21, 2023
a9d2b5d
leakage scale spin script added
martinkilbinger Oct 21, 2023
38abd62
added poetry.lock
martinkilbinger Oct 21, 2023
8d19c87
Merge pull request #10 from martinkilbinger/spin
martinkilbinger Oct 21, 2023
80e1fd2
leakage scale spin before uncertainties plots
martinkilbinger Oct 24, 2023
257ed2f
uncertainties package to propagate errors
martinkilbinger Nov 8, 2023
da442b6
Changes on asus
martinkilbinger Nov 8, 2023
70a3e89
Merge branch 'develop' into sa_scale
martinkilbinger Nov 8, 2023
76ef7ac
Merge pull request #11 from martinkilbinger/sa_scale
martinkilbinger Nov 8, 2023
b962d28
Merge branch 'develop' into a12
martinkilbinger Nov 8, 2023
df2e395
Merge pull request #12 from martinkilbinger/a12
martinkilbinger Nov 8, 2023
1e64284
Add a module to handle the different treecorr catalogs and a class to…
Nov 14, 2023
cceeef5
Add tau stat class and a test script to check everything works correctly
sachaguer Nov 15, 2023
b427947
Small bug fixes
sachaguer Nov 15, 2023
6deeea8
obj-wise leakage options mix and lin/quad
martinkilbinger Nov 15, 2023
e2c7c0a
Add modules to plot rho and tau stats for different catalogs allowing…
sachaguer Nov 15, 2023
e4adc6b
Add a class to run MCMC analysis and fit alpha, beta and eta for diff…
sachaguer Nov 16, 2023
adaa335
object-wise leakge plots: ground-truth parameter value
martinkilbinger Nov 18, 2023
7193a85
leakage_object: download patch catalogue (to be done for v1)
martinkilbinger Nov 18, 2023
ac6e70c
Merge pull request #1 from martinkilbinger/alpha_ij
martinkilbinger Nov 18, 2023
6e35170
Add tests for the MCMC fit of the parameter and plot the systematic e…
sachaguer Nov 21, 2023
b023bdf
Small bug fixes
sachaguer Nov 21, 2023
5d83497
Small bug fixes and updates
sachaguer Nov 22, 2023
de87ade
Implemented weighted average mean subtraction and small fixes.
sachaguer Nov 29, 2023
69d0393
Merge pull request #13 from sachaguer/sacha_rho_stat
sachaguer Dec 1, 2023
a93ac8c
Removed title from leakage plot
martinkilbinger Dec 1, 2023
f9cb4b0
Merge remote-tracking branch 'origin/develop' into develop
martinkilbinger Dec 1, 2023
b18fbdf
Fixed a mistake in the creation of patches between the different cata…
sachaguer Dec 6, 2023
0a3afd0
Merge pull request #2 from martinkilbinger/cleanup
martinkilbinger Dec 10, 2023
888468d
v0.1.6
martinkilbinger Dec 11, 2023
a7b75e4
Merge pull request #3 from martinkilbinger/v0.1.6
martinkilbinger Dec 11, 2023
b53076b
Add debiasing option when running inference.
sachaguer Dec 11, 2023
9df99f0
Merge branch 'develop' of github.com:CosmoStat/shear_psf_leakage into…
martinkilbinger Dec 11, 2023
d2f37c8
legend outside; minor bug fixes
martinkilbinger Dec 11, 2023
2ca8f5e
added lenspack
martinkilbinger Dec 11, 2023
8517585
xi_psf_sys individual terms
martinkilbinger Dec 12, 2023
e9f034b
added class function plot_xi_psf_sys_terms
martinkilbinger Dec 12, 2023
a3d2e81
tau stats w model plot: set fixed y limits
martinkilbinger Dec 13, 2023
aec3d67
Merge pull request #14 from sachaguer/sacha_rho_stat
martinkilbinger Dec 13, 2023
c86b347
Merge remote-tracking branch 'origin/develop' into plotting
martinkilbinger Dec 13, 2023
ab15eb1
Merge pull request #4 from martinkilbinger/plotting
sachaguer Dec 13, 2023
6366b45
Update README.md
martinkilbinger Dec 15, 2023
2af2e86
rho stats: added check on angular scales with tau stats
martinkilbinger Jan 24, 2024
d772f64
Merge remote-tracking branch 'upstream/develop' into auto_PSF
martinkilbinger Jan 24, 2024
c215a58
removed misleading comment
martinkilbinger Jan 25, 2024
f240f7f
added best-fit par to class
martinkilbinger Jan 25, 2024
a107585
removed error propa functions, now done with uncertainties
martinkilbinger Jan 25, 2024
7e533ef
numpy, cs_util 0.0.7 working
martinkilbinger Jan 25, 2024
a5d720a
new nb/py script autocorr psf
martinkilbinger Jan 25, 2024
b9985e0
leakage autocorr PSF; cs_util local
martinkilbinger Jan 26, 2024
3dd2d61
Added HDU number to arg for rho/tau stats
martinkilbinger Feb 25, 2024
f31ec3d
updated library versions
martinkilbinger Feb 25, 2024
6cb6d07
Add Least Square estimate of the parameters alpha, beta and eta in th…
sachaguer Mar 8, 2024
88c6fd8
Merge pull request #15 from sachaguer/sacha_rho_stat
martinkilbinger Mar 8, 2024
ef9acab
poetry lock change
martinkilbinger Mar 9, 2024
c8b7bd7
Added ground truth to obj-wise leakage; plot styles improved
martinkilbinger Mar 9, 2024
f222ea9
added notebook for autocorr leakage (propto alpha^2)
martinkilbinger Mar 9, 2024
66a1db4
Merge remote-tracking branch 'origin/develop' into develop
martinkilbinger Mar 9, 2024
3fdebf8
json output non-zero
martinkilbinger Mar 11, 2024
4ad24fb
Merge remote-tracking branch 'origin/develop' into style
martinkilbinger Mar 11, 2024
81c455e
Merge branch 'json' into style
martinkilbinger Mar 11, 2024
47220f2
Merge pull request #16 from martinkilbinger/style
martinkilbinger Mar 11, 2024
b854bcf
object-wise leakage: read best-fit parameters from json
martinkilbinger Mar 14, 2024
d1e8e19
Merge remote-tracking branch 'origin/rho_tau_hdu' into json
martinkilbinger Mar 15, 2024
3713cb1
Add a script to compute the covariance of tau statistics analytically.
Mar 29, 2024
30e2ab9
versions updated; cs_util 0.1.0
martinkilbinger Apr 4, 2024
5b5b596
Saving best fit to pickle (instead of json, did not work)
martinkilbinger Apr 4, 2024
cb0ed0c
using args.py from cs_util 0.1.0
martinkilbinger Apr 4, 2024
91209f3
using args.py from cs_util 0.1.0
martinkilbinger Apr 4, 2024
6edfd62
Fixed import
martinkilbinger Apr 4, 2024
358879a
poetry lock
martinkilbinger Apr 4, 2024
3fa43d9
Merge branch 'json' of github.com:martinkilbinger/shear_psf_leakage i…
martinkilbinger Apr 4, 2024
8a15317
run object: added mix and order as arguments
martinkilbinger Apr 7, 2024
d0a9d78
Add analytical covariance for the tau-statistics
sachaguer Apr 16, 2024
c55c231
Add parameters to the class CovTauTh to handle catalogs with differen…
sachaguer Apr 19, 2024
ee48254
Remove unecessary functions in CovTauTh.
sachaguer Apr 30, 2024
023b480
Merge pull request #18 from sachaguer/develop
martinkilbinger Apr 30, 2024
d10934a
Merge remote-tracking branch 'origin/develop' into develop
May 2, 2024
e52e945
Merge remote-tracking branch 'origin/develop' into json
May 2, 2024
6a7aa0a
PSF leakageL: split fit from plot
May 5, 2024
18b772f
scale check between rho and tau
May 5, 2024
9bc0d45
PSF leakage split fit and plot
May 5, 2024
0b663ef
no change
May 5, 2024
b33bb50
Bug fixes
sachaguer May 6, 2024
7935092
Fixed plot 2d argumets (verbose, mix, order)
May 6, 2024
7369340
updated to cs_util 0.1
May 6, 2024
11954d6
Modified the scale of integration to avoid regions where interpolatio…
sachaguer May 6, 2024
2f5a12c
Add calibration of ellipticities for DES.
sachaguer May 6, 2024
d125c80
clean-up; fixed typos
May 6, 2024
02776c1
sourcery fix
May 6, 2024
0d4a323
Merge pull request #19 from sachaguer/develop
sachaguer May 7, 2024
6186365
Merge pull request #6 from martinkilbinger/develop
martinkilbinger May 7, 2024
36eb8c6
Bug correction
sachaguer May 7, 2024
ce9ce66
Merge pull request #21 from sachaguer/develop
sachaguer May 7, 2024
2be9b71
Merge remote-tracking branch 'origin/develop' into develop
May 8, 2024
862d857
Merge remote-tracking branch 'origin/develop' into json
May 8, 2024
1bdad5f
updated to cs_util 0.1
May 8, 2024
2414cc4
version 0.2; in pyproject only
May 8, 2024
e725e86
scale--dep leakage script: updated notation
May 11, 2024
d9bc789
optics
May 11, 2024
3c0c68c
added gsl for (pre-installed) pyccd
May 11, 2024
498cffe
cleaned up leakage scale spin script
May 11, 2024
212945e
scale-dep spin leakage: plots for SPv1.3
May 13, 2024
cc70fe4
Merge branch 'develop' into json
martinkilbinger May 13, 2024
13d1258
Merge pull request #5 from martinkilbinger/json
martinkilbinger May 13, 2024
6036f8e
Merge remote-tracking branch 'upstream/develop' into develop
May 13, 2024
6583e9a
tolerance angular scales back to 0.001
May 13, 2024
dab6435
pyccl 3.0
martinkilbinger May 27, 2024
4ea881d
lock
martinkilbinger May 27, 2024
f1f61be
Merge remote-tracking branch 'upstream/develop' into citation
Jun 10, 2024
af0f0e0
Merge remote-tracking branch 'upstream/develop' into auto_PSF
Jun 10, 2024
57a27cd
Merge pull request #9 from martinkilbinger/auto_PSF
martinkilbinger Jun 10, 2024
31f95bc
preparing v0.2.0
Jun 10, 2024
8a0c0f1
Merge pull request #10 from martinkilbinger/v0.2.0_prep
martinkilbinger Jun 10, 2024
37b6478
Rho plots: keyword inside -> each
Sep 2, 2024
324a627
Merge pull request #11 from martinkilbinger/mk
martinkilbinger Sep 4, 2024
162c054
Update rho_tau_cov with correct computations of the cosmic variance t…
sachaguer Sep 27, 2024
9cbae2e
Merge pull request #24 from CosmoStat/develop
martinkilbinger Oct 18, 2024
2772b43
Merge remote-tracking branch 'upstream/develop' into citation
Oct 18, 2024
896bc32
Merge pull request #25 from martinkilbinger/citation
martinkilbinger Oct 18, 2024
facc4d2
Add boolean param to constructors to compute rho and tau-stats withou…
sachaguer Dec 27, 2024
24c1cfd
Merge pull request #23 from sachaguer/develop
sachaguer Jan 6, 2025
4814838
Modified read_shear_cat to save memory.
sachaguer Jan 24, 2025
cbc094e
Removed unnecessary lines of code.
sachaguer Jan 24, 2025
6417109
Merge pull request #26 from sachaguer/develop
martinkilbinger Jan 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,6 @@ notebooks/leakage_object
notebooks/unions_shapepipe_extended_2022_W3_v1.0.3.fits
__pycache__
__pycache__/*
shear_psf_leakage/__pycache__/*
*/__pycache__/*
build
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,14 @@ pypi.org,
pip install shear_psf_leakage
```

Alternatively, you can use
Alternatively, you can use TBD

## Authors

Martin Kilbinger <martin.kilbinger@cea.fr>

Clara Bonini <mail>
Clara Bonini

Axel Guinot <mail>
Sacha Guerrini

Axel Guinot
206 changes: 206 additions & 0 deletions notebooks/leakage_autocorr_psf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,206 @@
# ---
# jupyter:
# jupytext:
# formats: ipynb,py:light
# text_representation:
# extension: .py
# format_name: light
# format_version: '1.5'
# jupytext_version: 1.16.1
# kernelspec:
# display_name: shear_psf_leakage
# language: python
# name: shear_psf_leakage
# ---

# # PSF leakage: PSF auto-correlation contributions
#
# Martin Kilbinger <martin.kilbinger@cea.fr>

# %reload_ext autoreload
# %autoreload 2

# +
import os
import matplotlib.pylab as plt
import numpy as np

from astropy import units

from uncertainties import ufloat
from uncertainties import unumpy

from cs_util import canfar
from cs_util import plots as cs_plots

import shear_psf_leakage.run_object as run_object
import shear_psf_leakage.run_scale as run_scale
from shear_psf_leakage import leakage
# -

# ## Set input parameters

params_in = {}

# ### Paths

# +
# Input galaxy shear catalogue
params_in["input_path_shear"] = "unions_shapepipe_extended_2022_v1.0.fits"

# Output directory
params_in["output_dir"] = "leakage_PSF_autocorr"
# -

# ### Job control

# Compute (spin-preserving) PSF ellipticity leakage
params_in["PSF_leakage"] = True

# ### Other parameters

# +
# PSF ellipticity column names
params_in["e1_PSF_col"] = "e1_PSF"
params_in["e2_PSF_col"] = "e2_PSF"

# Set verbose output
params_in["verbose"] = True
# -

# ### Retrieve test catalogue from VOspace if not yet downloade

vos_dir = "vos:cfis/XXXX/"
canfar.download(
f"{vos_dir}/{params_in['input_path_shear']}",
params_in["input_path_shear"],
verbose=params_in["verbose"],
)

# ## Compute leakage

# Create leakage instance
obj_object = run_object.LeakageObject()

# Set instance parameters, copy from above
for key in params_in:
obj_object._params[key] = params_in[key]

# Run commands as given in LeakageObject.run()

# +
# Check parameter validity
obj_object.check_params()

# Update parameters (here: strings to list)
obj_object.update_params()

# Prepare output directory
obj_object.prepare_output()
# -

# Read input catalogue
obj_object.read_data()

# Object-by-object spin-consistent PSF leakage
mix = True
order = "lin"
obj_object.PSF_leakage(mix=mix, order=order)

p_dp = {}
for p in obj_object.par_best_fit:
p_dp[p] = ufloat(obj_object.par_best_fit[p].value, obj_object.par_best_fit[p].stderr)
print(p_dp)

s_ds = leakage.param_order2spin(p_dp, order, mix)
print(s_ds)

# Get spin elements
x0 = s_ds["x0"]
x4 = s_ds["x4"]
y4 = s_ds["y4"]
y0 = s_ds["y0"]


# Create scale-dependent leakage instance
obj_scale = run_scale.LeakageScale()

# Set instance parameters, copy from above
for key in params_in:
obj_scale._params[key] = params_in[key]


obj_scale._params["input_path_PSF"] = "unions_shapepipe_psf_2022_v1.0.2.fits"
obj_scale._params["dndz_path"] = "dndz_SP_A.txt"

# ### Run

# Check parameter validity
obj_scale.check_params()

# Prepare output directory and stats file
obj_scale.prepare_output()

# Prepare input
obj_scale.read_data()

# #### Compute correlation function and alpha matrices
# The following command calls `treecorr` to compute auto- and cross-correlation functions.
# This can take a few minutes.

obj_scale.compute_corr_gp_pp_alpha_matrix()

obj_scale.alpha_matrix()

#### For comparison: scalar alpha leakage
obj_scale.compute_corr_gp_pp_alpha()
obj_scale.do_alpha()


# Plot terms
theta_arcmin = obj_scale.get_theta()

term_1 = (
(x0 ** 2 + x4 **2 + y0 **2 + y4 ** 2)
* (obj_scale.xi_pp_m[0][0] + obj_scale.xi_pp_m[1][1])
)
term_2 = 4 * (x0 * y4 - x4 * y0) * obj_scale.xi_pp_m[0][1]

theta = theta_arcmin * units.arcmin
xi_theo_p, xi_theo_m = run_scale.get_theo_xi(theta, obj_scale._params["dndz_path"])

# +
y = [
unumpy.nominal_values(term_1),
-unumpy.nominal_values(term_2),
xi_theo_p,
]
dy = [
unumpy.std_devs(term_1),
unumpy.std_devs(term_2),
[np.nan] * len(xi_theo_p),
]
x = [theta_arcmin] * len(y)

title = "PSF auto-correlation additive terms"
xlabel = r"$\theta$ [arcmin]"
ylabel = "terms"
out_path = f"{obj_scale._params['output_dir']}/auto_corr_terms.png"
labels = ["t1" , "-t2", r"$\xi_p$"]

cs_plots.plot_data_1d(
x,
y,
dy,
title,
xlabel,
ylabel,
out_path,
labels=labels,
shift_x=True,
xlog=True,
ylog=True,
)
# -


82 changes: 25 additions & 57 deletions notebooks/leakage_object.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,81 +15,48 @@

# # PSF leakage: object-wise estimation
#
# Demonstration notebook of shear_psf_leakage.run_object.LeakageObject() class.
# Demonstration notebook of shear_psf_leakage.run_object.LeakageObject class.
#
# Martin Kilbinger <martin.kilbinger@cea.fr>

# +
import os
import matplotlib
matplotlib.use("agg")

import os, sys
import matplotlib.pylab as plt

from cs_util import canfar
from cs_util import args

import shear_psf_leakage.run_object as run
from shear_psf_leakage import leakage

# -

# ## Set input parameters

params_in = {}

# ### Paths

# +
# Input galaxy shear catalogue
params_in["input_path_shear"] = "unions_shapepipe_extended_2022_W3_v1.0.3.fits"

# Output directory
params_in["output_dir"] = "leakage_object"
# -

# ### Job control

# +
# Compute (spin-preserving) PSF ellipticity leakage
params_in["PSF_leakage"] = True

# Compute leakage with other parameters
params_in["obs_leakage"] = True

# Other input parameters
params_in["cols"] = "RA Dec e1_PSF fwhm_PSF w mag snr"

# Ratio between two input columns
params_in["cols_ratio"] = "mag_snr"
# -

# ### Other parameters

# +
# PSF ellipticity column names
params_in["e1_PSF_col"] = "e1_PSF"
params_in["e2_PSF_col"] = "e2_PSF"

# Set verbose output
params_in["verbose"] = True
# -

# ### Retrieve test catalogue from VOspace if not yet downloade

vos_dir = "vos:cfis/XXXX/"
canfar.download(
f"{vos_dir}/{params_in['input_path_shear']}",
params_in["input_path_shear"],
verbose=params_in["verbose"],
)

# ## Compute leakage

# Create leakage instance
obj = run.LeakageObject()

# Set instance parameters, copy from above
for key in params_in:
obj._params[key] = params_in[key]
params_upd = args.read_param_script("params_object.py", obj._params, verbose=True)
for key in params_upd:
obj._params[key] = params_upd[key]

#for key in obj._params:
#print(key, obj._params[key])

# +
#vos_dir = f"vos:cfis/weak_lensing/DataReleases/v1.0/ShapePipe/{patch}"
#canfar.download(
#f"{vos_dir}/{obj_params['input_path_shear']}",
#obj._params["input_path_shear"],
#verbose=obj._params["verbose"],
#)
# -

# ### Run
# There are two options to run the leakage compuations:
# There are two options to run the leakage computations:
# 1. Run all at once with single class routine.
# 2. Execute individual steps.

Expand All @@ -116,7 +83,8 @@

if obj._params["PSF_leakage"]:
# Object-by-object spin-consistent PSF leakage
obj.PSF_leakage()
for order in ("lin", "quad"):
obj.PSF_leakage(mix=True, order=order)

if obj._params["obs_leakage"]:
# Object-by-object spin-consistent PSF leakage
Expand Down
Loading