Skip to content

Commit

Permalink
Add Dataset Functionality
Browse files Browse the repository at this point in the history
Change data directory to datsets, implement utility function for loading datasets with Pandas
  • Loading branch information
astroDimitrios committed May 1, 2021
1 parent e7b085d commit 3c22118
Show file tree
Hide file tree
Showing 91 changed files with 8,199 additions and 191 deletions.
2 changes: 1 addition & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
include src/astroedu/interactives/*
include src/astroedu/data/*
include src/astroedu/datasets/*
29 changes: 17 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,26 +8,19 @@ This package is in alpha.

## Installation

**astroedu** requires:

- Numpy
- Matplotlib
- Pandas
- Jupyterlab
- IPywidgets
- Fire
- importlib-resources

To install **astroedu** run
```
>>> pip install astroedu
```
(build not implemented yet!)
then run
```
>>> astroedu build
```
which creates the configuration file **config.ini**.

**config.ini** stores the location of your documents folder.
This is where an astroedu folder for interactive activites will me made. (not yet implemented)

## Current Functionality

### Interactives
Expand All @@ -41,4 +34,16 @@ Jupyter lab arguments can be passed after the interactive name for instance:

```
>>> astroedu interactive wiens_law --port 9999
```
```

### Datasets

To use a data set import the utitlity function ```load_data``` from the datasets module:
```
from astroedu.datasets import load_data
```
Then you can load a data set by passing its name as a string to ```load_data```.
```
planets = load_data('planets')
```
The function returns a Pandas dataframe.
71 changes: 51 additions & 20 deletions build/lib/astroedu/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,46 +5,77 @@
import fire

from astroedu.__build__ import get_astroedu_path
from astroedu.__build__ import build_path_config

commands = ['build', 'interactive']
astroedu_path = get_astroedu_path()
config_file_name = '/config.ini'
config_file_path = astroedu_path + config_file_name

def main():
''' Entry point for package
'''
fire.Fire(startup)

def load_astroedu_config():
''' Loads the config.ini file
Returns:
The config dictionary
Example:
>>> astroedu_config = load_astroedu_config()
'''
if not is_resource('astroedu', 'config.ini'):
raise FileExistsError('No config.ini file found. Did you forget to run: \n astroedu build')
config = ConfigParser()
config.read(config_file_path)
return config['astroedu-config']

commands = ['interactive']
astroedu_path = get_astroedu_path()
config_file_name = '/config.ini'
config_file_path = astroedu_path + config_file_name
def _load_interactive(*args):
''' Loads an interactive notebook
Called from startup() only
'''
interactive_name = args[1]
if not is_resource('astroedu.interactives', interactive_name+'.ipynb'):
raise FileExistsError(f'No interactive named {interactive_name} found.')
with files('astroedu').joinpath('interactives', interactive_name+'.ipynb') as p:
call = ['jupyter', 'lab']
call_args = list(args[2:]) if len(args) > 2 else []
run_interactive = call + [str(p)] + call_args
print(f'Loading interactive {interactive_name}!')
subprocess.run(run_interactive)

def startup(*args):
''' Startup function called by Fire cli function main()
Based on arguments passed to astroedu call
- builds config.ini file
- launches interactive notebook
Args:
args[0] -- string, name of astroedu command to call
Example:
Loads the wiens_law interactive and launches jupyterlab
>>> astroedu interactive wiens_law
Arguments passed after the interactive name go to jupyter
>>> astroedu interactive wiens_law --port 9999
To build the config.ini file
>>> astroedu build
'''
if not args:
raise ValueError('No arguments passed to astroedu call.')
command = args[0]
if command not in commands:
raise ValueError('Command not recognised.')
elif len(args) == 1:
raise ValueError(f'No arguments passed to {command}')

if command == 'build':
from astroedu.__build__ import build_path_config
elif command == 'build':
build_path_config()
elif len(args) == 1: # build doesn't need extras
raise ValueError(f'No arguments passed to {command}')
elif command == 'interactive':
interactive_name = args[1]
# interactive_path = path('astroedu.interactives', interactive_name+'.ipynb')
# interactive_path = './interactives/'+interactive_name+'.ipynb'
if not is_resource('astroedu.interactives', interactive_name+'.ipynb'):
raise FileExistsError(f'No interactive named {interactive_name} found.')
with files('astroedu').joinpath('interactives', interactive_name+'.ipynb') as p:
call = ['jupyter', 'lab']
call_args = list(args[2:]) if len(args) > 2 else []
run_interactive = call + [str(p)] + call_args
print(f'Loading interactive {interactive_name}!')
subprocess.run(run_interactive)
_load_interactive(*args)

if __name__ == '__main__':
fire.Fire(startup)
68 changes: 68 additions & 0 deletions build/lib/astroedu/datasets/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Data

Description of the data files kept in this directory.

To keep this readme short you can find column descriptions in the csv's associated **.txt** file or linked below. (not all are complete)

## mass16Abundant.csv

Atomic data (mass, binding energy etc of isotopes) from the Atomic Mass Data Center ([AMDC](http://amdc.impcas.ac.cn/filel.html)).
**mass16Abundant.csv** is the same but with only the most abundant isotope of each element.

## atmospheres.csv

Data from Rauf KM et al. (2015) Comparing the Atmospheric Compositions of All Planets and Giant Moons in Solar System [https://www.longdom.org/open-access/comparing-the-atmospheric-compositions-of-all-planets-and-giant-moons-in-solar-system-2332-2519-1000136.pdf](https://www.longdom.org/open-access/comparing-the-atmospheric-compositions-of-all-planets-and-giant-moons-in-solar-system-2332-2519-1000136.pdf)

Contains percentage compositions for different gases in the atmospheres of the planets + Pluto.

## earth_chem_crust.csv

Chemical composition of the crust.
Taken from: CRC Handbook of Chemistry and Physics, 97th Edition (2016-2017)

## earth_chem_interior.csv

The chemical compostion of the Earth.
Only some elements included. Some Oxygen data is missing. Values in %.

Rows for: Bulk, Core, Lower Mantle, Upper Mantle, and the Crust.

Data taken from:
Don L. Anderson (1998) Theory of the Earth, Chapter 8, [https://authors.library.caltech.edu/25018/9/TOE08.pdf](https://authors.library.caltech.edu/25018/9/TOE08.pdf)
Claude J. Allegre et al (1995) The chemical composition of the Earth, [https://www.researchgate.net/publication/222035431_The_Chemical-Composition_of_the_Earth](https://www.researchgate.net/publication/222035431_The_Chemical-Composition_of_the_Earth)
Frederick K. Lutgens and Edward J. Tarbuck (2000) Essentials of Geology 7th Edition

## exobase.csv

From A. Garcia Munoz et al - Upper Atmospheres and Ionospheres of Planets and Satellites [https://arxiv.org/ftp/arxiv/papers/1712/1712.02994.pdf](https://arxiv.org/ftp/arxiv/papers/1712/1712.02994.pdf)

Contains data on the exobase (bottom layer of the exosphere) for the planets and Titan.
Note the temperatures for Mercury are average surface temperatures since it has no appreciable atmosphere.

## exoplanets.csv

NASA exoplanet archive data.
[https://exoplanetarchive.ipac.caltech.edu/docs/API_exoplanet_columns.html](https://exoplanetarchive.ipac.caltech.edu/docs/API_exoplanet_columns.html)

## exoplanets_microlensing.csv

NASA exoplanet archive data. Exoplanets discovered using microlensing.
[https://exoplanetarchive.ipac.caltech.edu/docs/API_microlensing.html](https://exoplanetarchive.ipac.caltech.edu/docs/API_microlensing.html)

## geotherm.csv

Geothermal and pressure gradient for the Earth. Taken from [here - https://www.geo.arizona.edu/xtal/geos306/fall06-10.htm](https://www.geo.arizona.edu/xtal/geos306/fall06-10.htm) and adapted. Original is in geotherm_original.csv.

## int_std_atm.csv

Data to construct the international standard atmosphere model.
Taken from the ISA wikipedia page [here](https://en.wikipedia.org/wiki/International_Standard_Atmosphere).

## planets.csv

Planetary data (including our Moon, the Galilean moons, and Pluto/Charon) taken from NASA Planetary factsheet [here](https://nssdc.gsfc.nasa.gov/planetary/factsheet/) and [here - https://www.noao.edu/jagi/sepo/education/plansat/table.html](https://www.noao.edu/jagi/sepo/education/plansat/table.html).

## structure.csv

Layers in planetary interiors and their thicknesses.
Taken from [here - https://github.com/eleanorlutz/topography_atlas_of_space](https://github.com/eleanorlutz/topography_atlas_of_space) and adapted (see other sources in the file).
43 changes: 43 additions & 0 deletions build/lib/astroedu/datasets/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
'''
astroedu.datasets - utilities for loading datasets
'''

from importlib_resources import files, is_resource
import pandas as pd

help_text = '''Pass the name of a dataset to load_data as a string.\n
For instance to load the planets dataset run load_data('planets').\n
The returned value is a Pandas dataframe.\n
To see all available datasets go to https://github.com/astroDimitrios/astroedu/tree/main/src/astroedu/datasets'''

def load_data(data, info=False):
''' Loads a dataset as a Pandas dataframe
Special case data == 'help' the help_text is printed to console
All datasets visible at:
https://github.com/astroDimitrios/astroedu/tree/main/src/astroedu/datasets
Args:
data -- string, name of the csv file to load (without extension)
info -- boolean, default False - if True prints a short description
of the data before loading the dataframe
Returns:
A pandas dataframe
Example:
>>> planets = load_data('planets')
>>> load_data('help')
'''
if data == 'help':
print(help_text)
return
data_csv = data+'.csv'
if not is_resource('astroedu.datasets', data_csv):
raise FileExistsError(f'No data file named {data_csv} found.')
with files('astroedu.datasets').joinpath(data_csv) as p:
if info:
with files('astroedu.datasets').joinpath(data+'.txt') as i:
InfoHandler = open(str(i), 'r')
print(InfoHandler.read())
return pd.read_csv(str(p))
File renamed without changes.
5 changes: 5 additions & 0 deletions build/lib/astroedu/datasets/atmospheres.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
atmospheres.csv
Data from Rauf KM et al. (2015)
Atmospheric compositions of all Planets and Giant Moons in the Solar System
Percentage compositions for the different gases and Pluto is included
https://www.longdom.org/open-access/comparing-the-atmospheric-compositions-of-all-planets-and-giant-moons-in-solar-system-2332-2519-1000136.pdf
File renamed without changes.
3 changes: 3 additions & 0 deletions build/lib/astroedu/datasets/earth_chem_crust.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
earth_chem_crust.csv
Chemical composition of the crust.
Taken from: CRC Handbook of Chemistry and Physics, 97th Edition (2016-2017)
File renamed without changes.
10 changes: 10 additions & 0 deletions build/lib/astroedu/datasets/earth_chem_interior.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
earth_chem_interior.csv
The chemical compostion of the Earth.
Only some elements included. Some Oxygen data is missing. Values in %.
Rows for: Bulk, Core, Lower Mantle, Upper Mantle, and the Crust.
Data taken from:
Don L. Anderson (1998) Theory of the Earth, Chapter 8
https://authors.library.caltech.edu/25018/9/TOE08.pdf
Claude J. Allegre et al (1995) The chemical composition of the Earth
https://www.researchgate.net/publication/222035431_The_Chemical-Composition_of_the_Earth
Frederick K. Lutgens and Edward J. Tarbuck (2000) Essentials of Geology 7th Edition
File renamed without changes.
15 changes: 15 additions & 0 deletions build/lib/astroedu/datasets/exobase.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
exobase.csv
From A. Garcia Munoz et al - Upper Atmospheres and Ionospheres of Planets and Satellites
https://arxiv.org/ftp/arxiv/papers/1712/1712.02994.pdf

Contains data on the exobase (bottom layer of the exosphere) for the planets and Titan.

object - name of the object
exobase alt - altitude in the atmosphere of the exobase (km)
exobase alt range - if the value in the paper was given as a range exobase alt is the center value
and this column contains the range either side of possible values (similar to a +- error but I hesitate to call it that) in km
exobase temp high - max temp of the exobase in K (daytime)
exobase temp low - min temp of the exobase in K (nightime) - sometimes this is the same as the high value
esc vel - the escape velocity of the planet in km/s

Note the temperatures for Mercury are average surface temperatures since it has no appreciable atmosphere.
File renamed without changes.
12 changes: 12 additions & 0 deletions build/lib/astroedu/datasets/geotherm.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
geotherm.csv
Data taken from - Professor Bob Downs, University of Arizona
https://www.geo.arizona.edu/xtal/geos306/fall06-10.htm

Used to construct/plot the Earth's geothermal gradient.
r - depth (km)
r*- distance from the center of the Earth (km)
T - temperature at r (K)

I have altered the T profile from the original to be consistent with the D" layer depth.
In the original file the core mantle boundary temp change occurred at too low a depth.
The original data is in geotherm_original.csv. The original also has pressure vs depth data.
File renamed without changes.
11 changes: 11 additions & 0 deletions build/lib/astroedu/datasets/geotherm_original.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
geotherm_original.csv
Data taken from - Professor Bob Downs, University of Arizona
https://www.geo.arizona.edu/xtal/geos306/fall06-10.htm

Used to construct/plot the Earth's geothermal gradient.
r - depth (km)
r*- distance from the center of the Earth (km)
T - temperature at r (K)

This original data also has pressure vs depth data.
I have adapted the data in geotherm.csv.
File renamed without changes.
12 changes: 12 additions & 0 deletions build/lib/astroedu/datasets/int_std_atm.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
int_std_atm.csv
Taken from the ISA wikipedia page.
https://en.wikipedia.org/wiki/International_Standard_Atmosphere

layer - is the layer number 0 being closest to the surface
level_name - is the layer name
h - is the geopotential altitude in m
z - is the geometric altitude in m
lapse_rate - is the lapse rate in K/km of geopotential altitude
T - is the base temperature for that layer
P - is the base pressure for that layer
rho - is the base density for that layer
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
jupiter_moons.csv
Data from: https://pds-rings.seti.org/jupiter/jupiter_tables.html

Most masses are missing
Expand Down
Loading

0 comments on commit 3c22118

Please sign in to comment.