Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Io artis reader #2973

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Conversation

wkerzendorf
Copy link
Member

New IO ARTIS reader. Making way for more changes down the line.

- Enhance ARTIS model reading with more detailed isotope and elemental mass fraction handling
- Add `to_geometry()` method to convert ArtisModelData to HomologousRadial1DGeometry
- Improve documentation in Jupyter notebook example
- Update mass fraction reading to support normalization and multi-index
- Add index and column names for better data clarity
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@tardis-bot
Copy link
Contributor

tardis-bot commented Feb 15, 2025

*beep* *bop*
Hi human,
I ran ruff on the latest commit (a370960).
Here are the outputs produced.
Results can also be downloaded as artifacts here.
Summarised output:

41	W291  	[*] Trailing whitespace
12	E999  	[ ] SyntaxError: Expected a statement
12	W293  	[*] Blank line contains whitespace
7	I001  	[*] Import block is un-sorted or un-formatted
6	E902  	[ ] stream did not contain valid UTF-8
5	INP001	[ ] File `tardis/analysis/opacities.py` is part of an implicit namespace package. Add an `__init__.py`.
3	F401  	[*] `tardis.io.model.readers.artis.read_artis_density` imported but unused
2	PIE808	[*] Unnecessary `start` argument in `range`
2	RET505	[ ] Unnecessary `else` after `return` statement
2	E712  	[*] Avoid equality comparisons to `False`; use `if not ...:` for false checks
2	W292  	[*] No newline at end of file
2	F811  	[ ] Redefinition of unused `monkeysession` from line 6
1	PT009 	[ ] Use a regular `assert` instead of unittest-style `assert_`
1	E402  	[ ] Module level import not at top of file
1	E703  	[*] Statement ends with an unnecessary semicolon
1	W605  	[*] Invalid escape sequence: `\s`
1	F631  	[ ] Assert test is a non-empty tuple, which is always `True`

Complete output(might be large):

.github/workflows/benchmarks.yml:3:4: E999 SyntaxError: Expected an expression
.github/workflows/benchmarks.yml:20:5: W291 [*] Trailing whitespace
.github/workflows/benchmarks.yml:91:49: W291 [*] Trailing whitespace
.github/workflows/benchmarks.yml:121:81: W291 [*] Trailing whitespace
.github/workflows/benchmarks.yml:122:54: W291 [*] Trailing whitespace
.github/workflows/benchmarks.yml:209:1: W293 [*] Blank line contains whitespace
.github/workflows/benchmarks.yml:228:22: W291 [*] Trailing whitespace
.github/workflows/benchmarks.yml:229:170: W291 [*] Trailing whitespace
.github/workflows/benchmarks.yml:231:84: W291 [*] Trailing whitespace
.github/workflows/benchmarks.yml:248:1: W293 [*] Blank line contains whitespace
.github/workflows/build-docs.yml:7:4: E999 SyntaxError: Expected an expression
.github/workflows/lfs-cache.yml:1:12: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
.github/workflows/lfs-cache.yml:38:1: W293 [*] Blank line contains whitespace
.github/workflows/lfs-cache.yml:50:1: W293 [*] Blank line contains whitespace
.github/workflows/lfs-cache.yml:58:1: W293 [*] Blank line contains whitespace
.github/workflows/lfs-cache.yml:63:1: W293 [*] Blank line contains whitespace
.github/workflows/lfs-cache.yml:68:1: W293 [*] Blank line contains whitespace
.github/workflows/lfs-cache.yml:78:1: W293 [*] Blank line contains whitespace
.github/workflows/tests.yml:7:4: E999 SyntaxError: Expected an expression
docs/index.rst:1:1: E999 SyntaxError: Expected a statement
docs/index.rst:21:51: W291 [*] Trailing whitespace
docs/index.rst:43:76: W291 [*] Trailing whitespace
docs/index.rst:44:64: W291 [*] Trailing whitespace
docs/index.rst:45:71: W291 [*] Trailing whitespace
docs/index.rst:46:69: W291 [*] Trailing whitespace
docs/index.rst:47:68: W291 [*] Trailing whitespace
docs/index.rst:66:1: W293 [*] Blank line contains whitespace
docs/index.rst:79:1: W293 [*] Blank line contains whitespace
docs/index.rst:103:1: W293 [*] Blank line contains whitespace
docs/io/grid/how_to_TardisGridTutorial.ipynb:cell 2:1:1: I001 [*] Import block is un-sorted or un-formatted
docs/io/grid/how_to_TardisGridTutorial.ipynb:cell 7:2:74: W291 [*] Trailing whitespace
docs/io/grid/how_to_TardisGridTutorial.ipynb:cell 14:1:47: W291 [*] Trailing whitespace
docs/io/grid/how_to_TardisGridTutorial.ipynb:cell 14:4:46: E703 [*] Statement ends with an unnecessary semicolon
docs/io/grid/how_to_TardisGridTutorial.ipynb:cell 15:2:55: W291 [*] Trailing whitespace
docs/io/images/custom_abundance_widget.png:1:1: E902 stream did not contain valid UTF-8
docs/io/images/energy_level_widget_options.gif:1:1: E902 stream did not contain valid UTF-8
docs/io/images/line_info_widget_demo.gif:1:1: E902 stream did not contain valid UTF-8
docs/io/images/packet_diagram.jpg:1:1: E902 stream did not contain valid UTF-8
docs/io/images/sdec_ply_interaction.gif:1:1: E902 stream did not contain valid UTF-8
docs/io/images/shell_info_widget_demo.gif:1:1: E902 stream did not contain valid UTF-8
docs/io/model/data/artis_abundances.dat:1:1: E999 SyntaxError: Unexpected indentation
docs/io/model/data/artis_model.dat:1:1: E999 SyntaxError: Unexpected indentation
docs/io/model/index.rst:1:1: E999 SyntaxError: Expected an expression
docs/io/model/index.rst:12:34: W292 [*] No newline at end of file
docs/io/visualization/demo_observed_spectrum.dat:1:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
docs/io/visualization/how_to_generating_widgets.ipynb:cell 5:1:1: I001 [*] Import block is un-sorted or un-formatted
docs/io/visualization/how_to_generating_widgets.ipynb:cell 5:3:5: F401 [*] `tardis.visualization.shell_info_from_hdf` imported but unused
docs/io/visualization/how_to_sdec_plot.ipynb:cell 2:2:1: I001 [*] Import block is un-sorted or un-formatted
docs/io/visualization/how_to_sdec_plot.ipynb:cell 29:1:1: I001 [*] Import block is un-sorted or un-formatted
docs/io/visualization/index.rst:1:1: E999 SyntaxError: Expected a statement
docs/io/visualization/index.rst:33:68: W292 [*] No newline at end of file
docs/io/visualization/tardis_example.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
docs/io/visualization/tutorial_convergence_plot.ipynb:cell 9:4:28: W291 [*] Trailing whitespace
docs/io/visualization/tutorial_convergence_plot.ipynb:cell 12:29:1: W293 [*] Blank line contains whitespace
docs/io/visualization/tutorial_convergence_plot.ipynb:cell 13:1:1: I001 [*] Import block is un-sorted or un-formatted
docs/io/visualization/tutorial_montecarlo_packet_visualization.ipynb:cell 5:1:1: I001 [*] Import block is un-sorted or un-formatted
docs/io/visualization/using_widgets.rst:1:1: E999 SyntaxError: Expected an expression
docs/io/visualization/using_widgets.rst:7:83: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:15:69: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:23:76: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:24:41: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:26:77: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:29:79: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:39:75: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:40:80: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:41:78: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:42:75: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:43:75: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:51:77: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:52:79: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:53:73: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:54:71: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:62:74: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:63:76: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:71:80: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:72:79: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:80:78: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:81:77: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:82:77: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:109:48: W291 [*] Trailing whitespace
docs/io/visualization/using_widgets.rst:161:77: W291 [*] Trailing whitespace
tardis/analysis/opacities.py:1:1: INP001 File `tardis/analysis/opacities.py` is part of an implicit namespace package. Add an `__init__.py`.
tardis/grid/tests/test_grid.py:1:1: INP001 File `tardis/grid/tests/test_grid.py` is part of an implicit namespace package. Add an `__init__.py`.
tardis/io/model/readers/artis.py:119:5: RET505 Unnecessary `else` after `return` statement
tardis/io/model/readers/artis.py:144:34: W605 [*] Invalid escape sequence: `\s`
tardis/io/model/readers/artis.py:197:1: E402 Module level import not at top of file
tardis/io/model/readers/tests/test_arepo_parser.py:1:1: INP001 File `tardis/io/model/readers/tests/test_arepo_parser.py` is part of an implicit namespace package. Add an `__init__.py`.
tardis/io/model/readers/tests/test_artis_readers.py:1:1: INP001 File `tardis/io/model/readers/tests/test_artis_readers.py` is part of an implicit namespace package. Add an `__init__.py`.
tardis/io/model/readers/tests/test_artis_readers.py:43:5: PT009 Use a regular `assert` instead of unittest-style `assert_`
tardis/io/model/readers/tests/test_artis_readers.py:61:5: F631 Assert test is a non-empty tuple, which is always `True`
tardis/io/model/readers/tests/test_ascii_readers.py:1:1: INP001 File `tardis/io/model/readers/tests/test_ascii_readers.py` is part of an implicit namespace package. Add an `__init__.py`.
tardis/io/tests/test_model_reader.py:6:43: F401 [*] `tardis.io.model.readers.artis.read_artis_density` imported but unused
tardis/transport/montecarlo/packet_source.py:246:9: RET505 Unnecessary `else` after `return` statement
tardis/transport/montecarlo/tests/conftest.py:14:5: F811 Redefinition of unused `opacity_state_initialize` from line 7
tardis/visualization/tools/tests/test_convergence_plot.py:3:1: I001 [*] Import block is un-sorted or un-formatted
tardis/visualization/tools/tests/test_convergence_plot.py:6:36: F401 [*] `tardis.tests.test_util.monkeysession` imported but unused
tardis/visualization/tools/tests/test_convergence_plot.py:87:24: PIE808 [*] Unnecessary `start` argument in `range`
tardis/visualization/tools/tests/test_convergence_plot.py:111:22: PIE808 [*] Unnecessary `start` argument in `range`
tardis/visualization/tools/tests/test_convergence_plot.py:198:9: E712 Avoid equality comparisons to `False`; use `if not ...:` for false checks
tardis/visualization/tools/tests/test_convergence_plot.py:215:9: E712 Avoid equality comparisons to `False`; use `if not ...:` for false checks
tardis/visualization/tools/tests/test_convergence_plot.py:220:40: F811 Redefinition of unused `monkeysession` from line 6
Found 101 errors.
[*] 69 fixable with the `--fix` option (2 hidden fixes can be enabled with the `--unsafe-fixes` option).

@tardis-bot
Copy link
Contributor

*beep* *bop*

Hi, human.

The docs workflow has succeeded ✔️

Click here to see your results.

- Remove unnecessary slice indexing on mean density array
- Ensure full density array is preserved when reading ARTIS model files
@tardis-bot
Copy link
Contributor

tardis-bot commented Feb 15, 2025

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (13c9a31) and the latest commit (a370960).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

Significantly changed benchmarks:

All benchmarks:

Benchmarks that have stayed the same:

| Change   | Before [6e790c66] <release-2025.02.23>   | After [a370960e]    | Ratio   | Benchmark (Parameter)                                                                                                               |
|----------|------------------------------------------|---------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------|
|          | 21.5±5μs                                 | 29.9±8μs            | ~1.39   | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_last_interaction_tracker_list |
|          | 45.4±30μs                                | 59.5±40μs           | ~1.31   | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_scatter                                          |
|          | 2.16±1μs                                 | 2.42±1μs            | ~1.12   | transport_montecarlo_estimators_radfield_estimator_calcs.BenchmarkMontecarloMontecarloNumbaPacket.time_update_line_estimators       |
|          | 3.76±0.5μs                               | 3.38±0.5μs          | ~0.90   | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_bad_vpacket                                       |
|          | 601±100ns                                | 511±100ns           | ~0.85   | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation                                      |
|          | 6.44±1μs                                 | 7.08±1μs            | 1.10    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket                                           |
|          | 601±100ns                                | 651±200ns           | 1.08    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation                              |
|          | 1.71±0.03ms                              | 1.85±0.01ms         | 1.08    | transport_montecarlo_main_loop.BenchmarkTransportMontecarloMontecarloMainLoop.time_montecarlo_main_loop                             |
|          | 3.53±0.4μs                               | 3.63±0.5μs          | 1.03    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_within_shell                              |
|          | 710±1ns                                  | 725±0.8ns           | 1.02    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_thomson_scatter                                       |
|          | 40.5±0.1s                                | 40.3±0.07s          | 1.00    | run_tardis.BenchmarkRunTardis.time_run_tardis                                                                                       |
|          | 1.08±0m                                  | 1.08±0m             | 1.00    | run_tardis.BenchmarkRunTardis.time_run_tardis_rpacket_tracking                                                                      |
|          | 2.11±0m                                  | 2.12±0m             | 1.00    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_FormalIntegrator_functions                                 |
|          | 7.54±2μs                                 | 7.55±2μs            | 1.00    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_volley                                    |
|          | 206±0.2ns                                | 203±0.3ns           | 0.99    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_intensity_black_body                                       |
|          | 1.20±0μs                                 | 1.19±0μs            | 0.99    | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_boundary                |
|          | 66.4±0.2ms                               | 65.8±0.2ms          | 0.99    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_rpacket_trackers_to_dataframe                  |
|          | 1.56±0.4μs                               | 1.52±0.3μs          | 0.97    | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_line                    |
|          | 40.4±0.2μs                               | 39.0±0.2μs          | 0.97    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_tracker_list                  |
|          | 45.9±30μs                                | 44.3±20μs           | 0.96    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_emission                                         |
|          | 2.89±0.4ms                               | 2.76±0.4ms          | 0.96    | transport_montecarlo_single_packet_loop.BenchmarkTransportMontecarloSinglePacketLoop.time_single_packet_loop                        |
|          | 4.55±0.02ms                              | 4.29±0.02ms         | 0.94    | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('macroatom')                                   |
|          | 3.45±0.04ms                              | 3.23±0.01ms         | 0.94    | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('scatter')                                     |
|          | 602±100ns                                | 552±100ns           | 0.92    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation                                |

If you want to see the graph of the results, you can check it here

@andrewfullard andrewfullard marked this pull request as draft February 17, 2025 15:41
@andrewfullard andrewfullard self-requested a review February 17, 2025 15:42
mean_density=mean_density,
mass_fractions=mass_fractions,
)
from tardis.model.geometry.radial1d import HomologousRadial1DGeometry
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import at end of file?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to avoid circular import - do you want me to fix?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please

@wkerzendorf wkerzendorf marked this pull request as ready for review February 22, 2025 20:54
@wkerzendorf
Copy link
Member Author

The to_composition does not exist but that will need to wait for the composition rework.

Copy link

codecov bot commented Feb 23, 2025

Codecov Report

Attention: Patch coverage is 97.22222% with 2 lines in your changes missing coverage. Please review.

Project coverage is 69.26%. Comparing base (304154a) to head (a370960).
Report is 6 commits behind head on master.

Files with missing lines Patch % Lines
tardis/io/model/readers/artis.py 94.73% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2973      +/-   ##
==========================================
- Coverage   69.54%   69.26%   -0.28%     
==========================================
  Files         228      229       +1     
  Lines       16429    16487      +58     
==========================================
- Hits        11425    11420       -5     
- Misses       5004     5067      +63     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants