Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simple Monte Carlo: First Attempt #112

Open
wants to merge 30 commits into
base: wl
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
b515ab1
now ignore calib directory as well
davidthomas5412 Oct 27, 2016
42a63c9
Merge branch 'Issue_103_Python_Data_Fetching' into wl
davidthomas5412 Oct 27, 2016
4e4390a
now ignore calib directory as well
davidthomas5412 Oct 27, 2016
b699c9f
Merge branch 'Issue_103_Python_Data_Fetching' into wl
davidthomas5412 Oct 27, 2016
782d6a3
switch to MIT License, update setup.py authors and their emails
davidthomas5412 Oct 28, 2016
b62957f
reran notebooks
davidthomas5412 Oct 27, 2016
eba2a58
remove astropy table/column/io dependencies, wcs dependency remains
davidthomas5412 Oct 27, 2016
1b2394d
i) remove astropy table/column/io dependencies, wcs dependency remains
davidthomas5412 Oct 27, 2016
949547b
removing dead notebooks; added SimpleMonteCarlo to index
davidthomas5412 Oct 31, 2016
83faed1
Merge branch 'issue_100_dead_notebooks_and_scripts' into wl
davidthomas5412 Oct 31, 2016
433be19
small corrections
davidthomas5412 Nov 1, 2016
4374333
Merge branch 'wl' of github.com:davidthomas5412/Pangloss into david
drphilmarshall Nov 2, 2016
48189de
Working through demos, fixing bugs. Made demo_data_files a dict, tidi…
drphilmarshall Nov 2, 2016
97eb30d
Testing ForegroundGalaxyOverlay demo: runs, but catalog plot has issues.
drphilmarshall Nov 2, 2016
fec33ee
Trying and failing to run notebooks - lensing not working correctly.
drphilmarshall Nov 2, 2016
e5399e2
Ignore pip install develop products
drphilmarshall Nov 2, 2016
d9edaf7
Debugging shearprediction notebook
drphilmarshall Nov 2, 2016
0667e0b
reverted back to hardcoded constants to avoid relying on string keys …
davidthomas5412 Nov 6, 2016
69dfac6
fixing bug
davidthomas5412 Nov 6, 2016
f0b59a6
adding astropy back to project dependencies, removing independent pyfits
davidthomas5412 Nov 6, 2016
19aa172
updated notebooks, plotting etc fails at same places as before
davidthomas5412 Nov 6, 2016
2f9066c
reran LensByHalos and WLMapsDemo notebooks
davidthomas5412 Nov 6, 2016
bacdab8
foreground constructor has optional stellar_mass_threshold, made slig…
davidthomas5412 Nov 11, 2016
f25c0a6
reran WLMapsDemo.ipynb
davidthomas5412 Nov 11, 2016
39afc68
adding static example() constructor to kappamap and shearmap for conv…
davidthomas5412 Nov 13, 2016
f6e5758
halo mass likelihood computation for simple monte carlo inference
davidthomas5412 Nov 13, 2016
06ee2dd
update SimpleMonteCarlo with latest inference - will need to be clean…
davidthomas5412 Nov 14, 2016
fd80f07
fixed pandas sequence issue
davidthomas5412 Dec 19, 2016
b1c0e82
only nonzero masses
davidthomas5412 Dec 20, 2016
7f51313
optional stellar mass threshold
davidthomas5412 Dec 20, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,7 @@ calib/Millennium
calib/SHMR
*ipynb_checkpoints*
data
calib
calib
build
dist
pangloss.egg-info
55 changes: 35 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,22 @@
# Pangloss

Pangloss is software module for reconstructing all the mass within a light cone
Pangloss is a software module for reconstructing all the mass within a light cone
through the Universe. Understanding complex mass distributions like
this is important for accurate time delay lens cosmography, and also for
accurate lens magnification estimation. It aspires to use all available
data (including weakly-lensed galaxy shapes) in an attempt to make the best of all mass maps. However, this
accurate lens magnification estimation - but more generally, we are interested in the inference of cosmological hyper-parameters in a hierarchical model that, in principle, allows us to ask different questions about galaxies and large scale structure than we can with summary statistics alone.

We aspire to use all available
data (including optical photometry and weakly-lensed galaxy shapes) in an attempt to make the best of all mass maps. However, this
problem is sufficiently difficult that we are mostly spending our time
cultivating a garden of toy models and baby steps.

If you are interested in the ongoing investigation of this interesting
problem, you should:
* Read our first paper, [_"Reconstructing the lensing mass in the universe from photometric catalogue data;"_](http://arxiv.org/abs/1303.6564)
* [Browse the wiki](https://github.com/drphilmarshall/Pangloss/wiki)
* [Browse the wiki](https://github.com/drphilmarshall/Pangloss/wiki)
to see what our current plans are;
* [Fork the code](https://github.com/drphilmarshall/Pangloss) and play around with it;
* Contact us! [Phil](mailto:dr.phil.marshall@gmail.com) and [Tom](mailto:[email protected]) are keen to hear from other researchers in the field who would like to coordinate and/or collaborate.
* [Contact us!](https://github.com/drphilmarshall/Pangloss/issues) We are keen to hear from other researchers in the field who would like to coordinate and/or collaborate.

## License

Expand All @@ -23,23 +25,36 @@ Pangloss is distributed under the MIT License.
When reporting on your use of Pangloss, we do ask that you include:
* A citation to: **[Collett et al. (2013), MNRAS, 432, 679, arxiv/1303.6564](http://adsabs.harvard.edu/abs/2013MNRAS.432..679C)**
* An acknowledgment including a link to this repo, such as:
_"This work made use of the Pangloss code, written by Tom Collett and
Phil Marshall, which is freely available at https://github.com/drphilmarshall/Pangloss."_
_"This work made use of the Pangloss code, which is freely available at https://github.com/drphilmarshall/Pangloss."_

## Installation

The setup scripts is setup.py and the project can be setup via

The Pangloss package requirements can be found in `requirements.txt`.
```
pip install -r requirements.txt
```
You can install the Pangloss package itself with
```
pip install git+https://github.com/drphilmarshall/pangloss.git@wl

the package requirements can be found in requirements.txt.
```
To help develop it, you can fork and clone the Pangloss repository, and install it with
```
python setup.py develop
```

## Example use

### Analyzing Weak Lensing Data

Take a look at the demonstration notebooks in the [demos](demos/index.ipynb) folder.


### Reproducing Collett et al 2013

Right now, there are three main scripts that carry out Pangloss's
functions:

* **[Drill.py](https://github.com/drphilmarshall/Pangloss/blob/master/Drill.py)**:
* **[Drill.py](https://github.com/drphilmarshall/Pangloss/blob/master/Drill.py)**:
from an input catalog of galaxies (either real or
simulated), drill out a narrow lightcone (or set of lightcones) centred
on some sky position of interest, and save the resulting smaller
Expand All @@ -63,7 +78,7 @@ we're working towards relaxing this.
They all take, as their sole input, the same configuration file, an
[example of which is given here](https://github.com/drphilmarshall/Pangloss/blob/master/pangloss/example_catalog.txt).

In the calib directory we include the means to obtain the halo catalog
In the calib directory we include the means to obtain the halo catalog
for a 1x1 square degree patch of Millennium Simulation sky, and its
associated ray traced convergence map, for making the calibration
lightcones. A small mock observed galaxy catalog is included in the
Expand All @@ -84,8 +99,8 @@ You should then be able to execute the following example analysis:
python Drill.py example.config
python Reconstruct.py example.config
python Calibrate.py example.config
Analysing from start to finish will take some time. Be patient!

Analysing from start to finish will take some time. Be patient!
(Drill ~2 mins, Reconstruct ~10 mins)

For more details of what the Pangloss scripts are doing, [start reading the code here.](https://github.com/drphilmarshall/Pangloss/wiki/Code-description)
Expand All @@ -96,9 +111,9 @@ Also, check out Tom's flowchart that describes the process of data simulation an

## PDFs based on overdensities

The [original
Pangloss](https://github.com/drphilmarshall/Pangloss) has been
modified so that it can be used to find convergence and
magnification pdfs for any given group of fields based on
comparing overdensities of the observed fields with simulation
The [original
Pangloss](https://github.com/drphilmarshall/Pangloss) has been
modified so that it can be used to find convergence and
magnification pdfs for any given group of fields based on
comparing overdensities of the observed fields with simulation
lightcones.
4 changes: 2 additions & 2 deletions bin/Drill.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,10 +130,10 @@ def Drill(argv):
count = 0
Ncones = Nc/Ncalcats

for i,catalog in enumerate(calcats):
for i, catalog in enumerate(calcats):

print "Drill: Reading in calibration catalog from "+catalog+"..."
table = pangloss.read_hilbert_catalog(catalog,experiment)
table = pangloss.read_hilbert_catalog(catalog, experiment)

if units == 'deg':
table['nRA'] = -table['nRA'] * pangloss.deg2rad
Expand Down
11 changes: 2 additions & 9 deletions bin/Magnifier.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@
import sys

import matplotlib.pyplot as plt
from astropy.io import ascii

import pangloss


Expand Down Expand Up @@ -195,11 +193,9 @@ def Magnifier(argv):

numpy.savetxt(CALIB_DIR+"/lc_density.txt", lc_dens)

print 'Mean overdensity in all fields = %.3f (this should =1)' % numpy.mean(lc_density)
print 'Lightcone overdensities saved to file'
print pangloss.dashedline
del lc_dens
del lc_density

# ==============================================================
# Sample all lightcones to make the pdfs
Expand Down Expand Up @@ -398,12 +394,9 @@ def Magnifier(argv):
means.append(sub_mean)
fieldname.append(field_name[i])

meanmu_table = numpy.array([fieldname, means]).T
ascii.write(meanmu_table, CALIB_DIR+"/"+EXP_NAME+"_table_meanmu.txt", names=['#field','mean_mu'])
print " Mean mu of all the fields = ",numpy.mean(means)
print "Magnifier: saved PDFs to",outputfile

print " Mean mu of all the fields = ",numpy.mean(means)
print "Magnifier: saved PDFs to",outputfile

print pangloss.doubledashedline

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Expand Down
Loading