Merge pull request #213 from elfi-dev/dev

Version 0.6.1

Author: Jarno Lintusaari

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,16 +1,9 @@
 #### Summary:
+Please provide a short summary here
 
-#### Intended Effect:
-
-#### How to Verify:
-
-#### Side Effects:
-
-#### Successful inference in current Notebook examples:
-
-#### Documentation:
-
-#### Reviewer Suggestions: 
+#### Please make sure
+- You have updated the CHANGELOG.rst
+- You have updated the documentation (if applicable)
 
 #### Copyright and Licensing
 

CHANGELOG.rst

@@ -1,6 +1,22 @@
 Changelog
 ==========
 
+0.6.1 (2017-07-21)
+------------------
+
+- Fix elfi.Prior and NoneType error #203
+- Fix a bug preventing the reuse of ArrayPool data with a new inference
+- Added pickling for OutputPool:s
+- Added OutputPool.open to read a closed pool from disk
+- Refactored Sample and SmcSample classes
+- Added elfi.new_model method
+- Made elfi.set_client method to accept clients as strings for easier client switching
+- Fixed a bug in NpyArray that would lead to an inconsistent state if multiple
+  simultaneous instances were opened.
+- Added the ability to move the pool data folder
+- Sample.summary is now a method instead of a property
+- SmcSample methods takes the keyword argument 'all' to show results of all populations
+- Added a section about iterative advancing to documentation
 
 0.6 (2017-07-03)
 ----------------

Makefile

@@ -69,23 +69,23 @@ docs: ## generate Sphinx HTML documentation, including API docs
 	$(MAKE) -C docs html
 	# $(BROWSER) docs/_build/html/index.html
 
-CONTENT_URL := http://research.cs.aalto.fi/pml/software/elfi/docs/0.6/
+CONTENT_URL := http://research.cs.aalto.fi/pml/software/elfi/docs/0.6.1/
 
 notebook-docs: ## Conver notebooks to rst docs. Assumes you have them in `notebooks` directory.
 	jupyter nbconvert --to rst ../notebooks/quickstart.ipynb --output-dir docs
-	sed -i '' 's|\(quickstart_files/quickstart.*\.\)|'${CONTENT_URL}'\1|g' docs/quickstart.rst
+	sed -i 's|\(quickstart_files/quickstart.*\.\)|'${CONTENT_URL}'\1|g' docs/quickstart.rst
 
 	jupyter nbconvert --to rst ../notebooks/tutorial.ipynb --output-dir docs/usage
-	sed -i '' 's|\(tutorial_files/tutorial.*\.\)|'${CONTENT_URL}usage/'\1|g' docs/usage/tutorial.rst
+	sed -i 's|\(tutorial_files/tutorial.*\.\)|'${CONTENT_URL}usage/'\1|g' docs/usage/tutorial.rst
 
 	jupyter nbconvert --to rst ../notebooks/BOLFI.ipynb --output-dir docs/usage
-	sed -i '' 's|\(BOLFI_files/BOLFI.*\.\)|'${CONTENT_URL}usage/'\1|g' docs/usage/BOLFI.rst
+	sed -i 's|\(BOLFI_files/BOLFI.*\.\)|'${CONTENT_URL}usage/'\1|g' docs/usage/BOLFI.rst
 
 	jupyter nbconvert --to rst ../notebooks/parallelization.ipynb --output-dir docs/usage
-	sed -i '' 's|\(parallelization_files/parallelization.*\.\)|'${CONTENT_URL}usage/'\1|g' docs/usage/parallelization.rst
+	sed -i 's|\(parallelization_files/parallelization.*\.\)|'${CONTENT_URL}usage/'\1|g' docs/usage/parallelization.rst
 
 	jupyter nbconvert --to rst ../notebooks/non_python_operations.ipynb --output-dir docs/usage --output=external
-	sed -i '' 's|\(external_files/external.*\.\)|'${CONTENT_URL}usage/'\1|g' docs/usage/external.rst
+	sed -i 's|\(external_files/external.*\.\)|'${CONTENT_URL}usage/'\1|g' docs/usage/external.rst
 
 # release: clean ## package and upload a release
 # 	python setup.py sdist upload

README.md

@@ -1,4 +1,4 @@
-**Version 0.6 released!** See the CHANGELOG and [notebooks](https://github.com/elfi-dev/notebooks).
+**Version 0.6.1 released!** See the CHANGELOG and [notebooks](https://github.com/elfi-dev/notebooks).
 
 ELFI - Engine for Likelihood-Free Inference
 ===========================================
@@ -35,7 +35,7 @@ is preferable.
 Installation
 ------------
 
-ELFI requires and is tested with Python 3.5-3.6. You can install ELFI by typing in your terminal:
+ELFI requires Python 3.5 or greater. You can install ELFI by typing in your terminal:
 
 ```
 pip install elfi
@@ -78,5 +78,4 @@ Resolving these may sometimes go wrong:
 - If you receive an error about `yaml.load`, install `pyyaml`.
 - On OS X with Anaconda virtual environment say `conda install python.app` and then use 
 `pythonw` instead of `python`.
-- Note that ELFI currently supports Python 3.5-3.6 only, although 3.x may work as well, 
-so try `pip3 install elfi`.
+- Note that ELFI requires Python 3.5 or greater so try `pip3 install elfi`.

docs/api.rst

@@ -31,6 +31,7 @@ Below is the API for creating generative models.
 .. currentmodule:: elfi.model.elfi_model
 
 .. autosummary::
+   elfi.new_model
    elfi.get_current_model
    elfi.set_current_model
 
@@ -146,6 +147,8 @@ Modelling API classes
 
 .. currentmodule:: elfi.model.elfi_model
 
+.. automethod:: elfi.new_model
+
 .. automethod:: elfi.get_current_model
 
 .. automethod:: elfi.set_current_model

docs/developer/architecture.rst

@@ -7,14 +7,16 @@ e.g. the inference methods or the data storages. This information is aimed for d
 and is not essential for using ELFI. We assume the reader is quite familiar with Python
 and has perhaps already read some of ELFI's source code.
 
-The low level representation of the ELFI model is a ``networkx.DiGraph`` with nodes
-represented as Python dictionaries that are called node state dictionaries. This
-representation is held in ``ElfiModel.source_net``. Before the ELFI model can be ran, it
-needs to be compiled and loaded with data (e.g. observed data, precomputed data, batch
-index, batch size etc). The compilation and loading of data is the responsibility of the
-``Client`` implementation and makes it possible in essence to translate ``ElfiModel`` to
-any kind of computational backend. Finally the class ``Executor`` is responsible for
-running the compiled and loaded model and producing the outputs of the nodes.
+The low level representation of the ELFI model is a ``networkx.DiGraph`` with node names
+as the nodes. The representation of the node is stored to the corresponding attribute
+dictionary of the ``networkx.DiGraph``. We call this attribute dictionary the node *state*
+dictionary. The ``networkx.DiGraph`` representation can be found from
+``ElfiModel.source_net``. Before the ELFI model can be ran, it needs to be compiled and
+loaded with data (e.g. observed data, precomputed data, batch index, batch size etc). The
+compilation and loading of data is the responsibility of the ``Client`` implementation and
+makes it possible in essence to translate ``ElfiModel`` to any kind of computational
+backend. Finally the class ``Executor`` is responsible for running the compiled and loaded
+model and producing the outputs of the nodes.
 
 A user typically creates this low level representation by working with subclasses of
 ``NodeReference``. These are easy to use UI classes of ELFI such as the ``elfi.Simulator`` or

docs/installation.rst

@@ -73,13 +73,14 @@ Or download the development `tarball`_:
 Note that for development it is recommended to base your work on the `dev` branch instead
 of `master`.
 
-Once you have a copy of the source, you can install it with:
+Once you have a copy of the source, go to the folder and type:
 
 .. code-block:: console
 
    pip install -e .
 
-This will install ELFI along with its default requirements.
+This will install ELFI along with its default requirements. Note that the dot in the end
+means the current folder.
 
 .. _Github repo: https://github.com/elfi-dev/elfi
 .. _tarball: https://github.com/elfi-dev/elfi/tarball/dev

docs/quickstart.rst

@@ -7,15 +7,15 @@ First ensure you have
 Python 3.5 (or greater) and ELFI. After installation you can start using
 ELFI:
 
-.. code:: ipython3
+.. code:: python
 
     import elfi
 
 ELFI includes an easy to use generative modeling syntax, where the
 generative model is specified as a directed acyclic graph (DAG). Let’s
 create two prior nodes:
 
-.. code:: ipython3
+.. code:: python
 
     mu = elfi.Prior('uniform', -2, 4)
     sigma = elfi.Prior('uniform', 1, 4)
@@ -30,7 +30,7 @@ summary statistics for the data. As an example, lets define the
 simulator as 30 draws from a Gaussian distribution with a given mean and
 standard deviation. Let's use mean and variance as our summaries:
 
-.. code:: ipython3
+.. code:: python
 
     import scipy.stats as ss
     import numpy as np
@@ -48,7 +48,7 @@ standard deviation. Let's use mean and variance as our summaries:
 Let’s now assume we have some observed data ``y0`` (here we just create
 some with the simulator):
 
-.. code:: ipython3
+.. code:: python
 
     # Set the generating parameters that we will try to infer
     mean0 = 1
@@ -73,7 +73,7 @@ Now we have all the components needed. Let’s complete our model by
 adding the simulator, the observed data, summaries and a distance to our
 model:
 
-.. code:: ipython3
+.. code:: python
 
     # Add the simulator node and observed data to the model
     sim = elfi.Simulator(simulator, mu, sigma, observed=y0)
@@ -85,23 +85,30 @@ model:
     # Specify distance as euclidean between summary vectors (S1, S2) from simulated and
     # observed data
     d = elfi.Distance('euclidean', S1, S2)
-    
+
+If you have ``graphviz`` installed to your system, you can also
+visualize the model:
+
+.. code:: python
+
     # Plot the complete model (requires graphviz)
     elfi.draw(d)
 
 
 
 
-.. image:: http://research.cs.aalto.fi/pml/software/elfi/docs/0.6/quickstart_files/quickstart_9_0.svg
+.. image:: http://research.cs.aalto.fi/pml/software/elfi/docs/0.6.1/quickstart_files/quickstart_11_0.svg
+
 
 
+.. Note:: The automatic naming of nodes may not work in all environments e.g. in interactive Python shells. You can alternatively provide a name argument for the nodes, e.g. ``S1 = elfi.Summary(mean, sim, name='S1')``.
 
 We can try to infer the true generating parameters ``mean0`` and
 ``std0`` above with any of ELFI’s inference methods. Let’s use ABC
 Rejection sampling and sample 1000 samples from the approximate
 posterior using threshold value 0.5:
 
-.. code:: ipython3
+.. code:: python
 
     rej = elfi.Rejection(d, batch_size=10000, seed=30052017)
     res = rej.sample(1000, threshold=.5)
@@ -111,22 +118,22 @@ posterior using threshold value 0.5:
 .. parsed-literal::
 
     Method: Rejection
-    Number of posterior samples: 1000
+    Number of samples: 1000
     Number of simulations: 120000
     Threshold: 0.492
-    Posterior means: mu: 0.748, sigma: 3.1
+    Sample means: mu: 0.748, sigma: 3.1
     
 
 
 Let's plot also the marginal distributions for the parameters:
 
-.. code:: ipython3
+.. code:: python
 
     import matplotlib.pyplot as plt
     res.plot_marginals()
     plt.show()
 
 
 
-.. image:: http://research.cs.aalto.fi/pml/software/elfi/docs/0.6/quickstart_files/quickstart_13_0.png
+.. image:: http://research.cs.aalto.fi/pml/software/elfi/docs/0.6.1/quickstart_files/quickstart_16_0.png