Merge pull request #100 from astrolabsoftware/0.2.2

Bump version: 0.2.1 --> 0.2.2

CHANGELOG.md

@@ -1,3 +1,12 @@
+## 0.2.2
+
+- Add the conversion from FITS file (single HDU) to parquet ([PR](https://github.com/astrolabsoftware/spark3D/pull/92))
+- Tools to perform live 3D RDD visualisation (in progress) ([PR](https://github.com/astrolabsoftware/spark3D/pull/93))
+- Several updates to the Travis CI  ([PR](https://github.com/astrolabsoftware/spark3D/pull/96))
+- Add example dealing with collapse function in pyspark ([PR](https://github.com/astrolabsoftware/spark3D/pull/97))
+- Deploy pyspark3d with pip ([PR](https://github.com/astrolabsoftware/spark3D/pull/98))
+- Clarify the use of get_spark_session and load_user_conf (for tests only!) ([PR](https://github.com/astrolabsoftware/spark3D/pull/99))
+
 ## 0.2.1
 
 - pyspark3d contains all the features of spark3D ([partitioning](https://github.com/astrolabsoftware/spark3D/pull/89), [operators](https://github.com/astrolabsoftware/spark3D/pull/90))

README.md

@@ -11,9 +11,9 @@
 - [07/2018] **New location**: spark3D is an official project of [AstroLab Software](https://astrolabsoftware.github.io/)!
 - [07/2018] **Release**: version 0.1.3, 0.1.4, 0.1.5
 - [08/2018] **Release**: version 0.2.0, 0.2.1 (pyspark3d)
-- [09/2018] **Release**: version 0.2.2 (pip installation)
+- [09/2018] **Release**: version 0.2.2
 
-<p align="center"><img width="500" src="https://github.com/astrolabsoftware/spark3D/raw/master/pic/spark3d_lib_0.2.1.png"/>
+<p align="center"><img width="500" src="https://github.com/astrolabsoftware/spark3D/raw/master/pic/spark3d_lib_0.2.2.png"/>
 </p>
 
 ## Installation and tutorials
@@ -23,7 +23,7 @@
 You can link spark3D to your project (either spark-shell or spark-submit) by specifying the coordinates:
 
 ```
-spark-submit --packages "com.github.astrolabsoftware:spark3d_2.11:0.2.1"
+spark-submit --packages "com.github.astrolabsoftware:spark3d_2.11:0.2.2"
 ```
 
 ### Python

build.sbt

@@ -19,7 +19,7 @@ import xerial.sbt.Sonatype._
 lazy val root = (project in file(".")).
  settings(
    inThisBuild(List(
-     version      := "0.2.1"
+     version      := "0.2.2"
    )),
    // Name of the application
    name := "spark3D",

docs/01_installation.md

@@ -20,7 +20,7 @@ another version, feel free to contact us. In addition to Spark, the library has
 You can link spark3D to your project (either `spark-shell` or `spark-submit`) by specifying the coordinates:
 
 ```bash
-toto:~$ spark-submit --packages "com.github.astrolabsoftware:spark3d_2.11:0.2.1" <...>
+toto:~$ spark-submit --packages "com.github.astrolabsoftware:spark3d_2.11:0.2.2" <...>
 ```
 
 It might not contain the latest features though (see *Building from source*).
@@ -69,7 +69,7 @@ result on the screen, plus details of the coverage at
 First produce a jar of the spark3D library, and then launch a spark-shell by specifying the external dependencies:
 
 ```bash
-toto:~$ JARS="target/scala-2.11/spark3d_2.11-0.2.1.jar,lib/jhealpix.jar"
+toto:~$ JARS="target/scala-2.11/spark3d_2.11-0.2.2.jar,lib/jhealpix.jar"
 toto:~$ PACKAGES="com.github.astrolabsoftware:spark-fits_2.11:0.6.0"
 toto:~$ spark-shell --jars $JARS --packages $PACKAGES
 ```
@@ -83,7 +83,7 @@ scala> // etc...
 Note that if you make a fat jar (that is building with `sbt ++${SCALA_VERSION} assembly` and not `sbt ++${SCALA_VERSION} package`), you do not need to specify external dependencies as they are already included in the resulting jar:
 
 ```bash
-toto:~$ FATJARS="target/scala-2.11/spark3D-assembly-0.2.1.jar"
+toto:~$ FATJARS="target/scala-2.11/spark3D-assembly-0.2.2.jar"
 toto:~$ spark-shell --jars $FATJARS
 ```
 
@@ -188,7 +188,7 @@ and then launch a pyspark shell:
 
 ```bash
 toto:~$ PYSPARK_DRIVER_PYTHON=ipython pyspark \
-  --jars /path/to/target/scala-2.11/spark3D-assembly-0.2.1.jar
+  --jars /path/to/target/scala-2.11/spark3D-assembly-0.2.2.jar
 ```
 
 You should be able to import objects:

docs/_pages/home.md

@@ -8,7 +8,7 @@ header:
   cta_url: "/docs/installation/"
   caption:
 intro:
-  - excerpt: '<p><font size="6">Spark extension for processing large-scale 3D data sets: Astrophysics, High Energy Physics, Meteorology, ...</font></p><br /><a href="https://github.com/astrolabsoftware/spark3D/releases/tag/0.2.1">Latest release v0.2.1</a>'
+  - excerpt: '<p><font size="6">Spark extension for processing large-scale 3D data sets: Astrophysics, High Energy Physics, Meteorology, ...</font></p><br /><a href="https://github.com/astrolabsoftware/spark3D/releases/tag/0.2.2">Latest release v0.2.2</a>'
 excerpt: '{::nomarkdown}<iframe style="display: inline-block;" src="https://ghbtns.com/github-btn.html?user=astrolabsoftware&repo=spark3D&type=star&count=true&size=large" frameborder="0" scrolling="0" width="160px" height="30px"></iframe> <iframe style="display: inline-block;" src="https://ghbtns.com/github-btn.html?user=astrolabsoftware&repo=spark3D&type=fork&count=true&size=large" frameborder="0" scrolling="0" width="158px" height="30px"></iframe>{:/nomarkdown}'
 feature_row:
   - image_path:

examples/jupyter/CrossMatch.ipynb

@@ -124,7 +124,7 @@
     "%AddDeps org.swinglabs swingx 1.6.1\n",
     "\n",
     "// Add the spark3d JAR. To generate it, run `sbt ++2.11.8 package` at the root of the package\n",
-    "%AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/target/scala-2.11/spark3d_2.11-0.1.5.jar\n",
+    "%AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/target/scala-2.11/spark3d_2.11-0.2.2.jar\n",
     "\n",
     "// Add healpix JAR\n",
     "%AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/lib/jhealpix.jar"

examples/jupyter/onion_partitioning.ipynb

@@ -75,7 +75,7 @@
     "%AddDeps org.swinglabs swingx 1.6.1\n",
     "\n",
     "// Add the spark3d JAR. To generate it, run `sbt ++2.11.8 package` at the root of the package\n",
-    "%AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/target/scala-2.11/spark3d_2.11-0.1.5.jar\n",
+    "%AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/target/scala-2.11/spark3d_2.11-0.2.2.jar\n",
     "\n",
     "// Add healpix JAR\n",
     "%AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/lib/jhealpix.jar"

examples/jupyter/windowQuery.ipynb

@@ -77,7 +77,7 @@
     "%AddDeps org.swinglabs swingx 1.6.1\n",
     "\n",
     "// Add the spark3d JAR. To generate it, run `sbt ++2.11.8` package at the root of the package\n",
-    "%AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/target/scala-2.11/spark3D-assembly-0.2.0.jar\n",
+    "%AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/target/scala-2.11/spark3D-assembly-0.2.2.jar\n",
     "\n",
     "// // Add healpix JAR\n",
     "// %AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/lib/jhealpix.jar"

pyspark3d/version.py

@@ -1,4 +1,4 @@
 # pyspark3d and spark3D have the same version number
-__version__ = "0.2.1"
+__version__ = "0.2.2"
 __scala_version__ = "2.11"
 __scala_version_all__ = "2.11.8"

release.sh

@@ -1,6 +1,7 @@
 #!/bin/bash
 
 ## TODO before bumping version:
+##  - Make a branch with the name of the version
 ##  - run sbt clean
 ##  - update build.sbt
 ##  - update website (01_installation.md, picture of the package, _pages/home.md, )
@@ -9,6 +10,9 @@
 ##  - update the CHANGELOG.md
 ##  - update the runners
 ##  - update the notebook, examples
+##  - run test suites (or alternatively look at the PR)
+##  - merge this branch into master
+##  - goto master and pull change
 ## THEN:
 
 ## GitHub

runners/benchmark_knn.sh

@@ -18,7 +18,7 @@ SBT_VERSION=2.11.8
 SBT_VERSION_SPARK=2.11
 
 ## Package version
-VERSION=0.2.1
+VERSION=0.2.2
 
 # Package it
 sbt ++${SBT_VERSION} package

runners/launch_pyspark.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 
 # Run `sbt 'set test in assembly := {}' ++2.11.8 assembly` to get the JAR
-JARS=../target/scala-2.11/spark3D-assembly-0.2.1.jar
+JARS=../target/scala-2.11/spark3D-assembly-0.2.2.jar
 
 # Octree - no part
  spark-submit --jars $JARS \

runners/run_knn.sh

@@ -18,7 +18,7 @@ SBT_VERSION=2.11.8
 SBT_VERSION_SPARK=2.11
 
 ## Package version
-VERSION=0.2.1
+VERSION=0.2.2
 
 # Package it
 sbt ++${SBT_VERSION} package

runners/run_knnGeo.sh

@@ -18,7 +18,7 @@ SBT_VERSION=2.11.8
 SBT_VERSION_SPARK=2.11
 
 ## Package version
-VERSION=0.2.1
+VERSION=0.2.2
 
 # Package it
 sbt ++${SBT_VERSION} package

runners/run_part.sh

@@ -18,7 +18,7 @@ SBT_VERSION=2.11.8
 SBT_VERSION_SPARK=2.11
 
 ## Package version
-VERSION=0.2.1
+VERSION=0.2.2
 
 # Package it
 sbt ++${SBT_VERSION} package

runners/run_scala.sh

@@ -18,7 +18,7 @@ SBT_VERSION=2.11.8
 SBT_VERSION_SPARK=2.11
 
 ## Package version
-VERSION=0.2.1
+VERSION=0.2.2
 
 # Package it
 sbt ++${SBT_VERSION} package

runners/run_shuffle.sh

@@ -18,7 +18,7 @@ SBT_VERSION=2.11.8
 SBT_VERSION_SPARK=2.11
 
 ## Package version
-VERSION=0.2.1
+VERSION=0.2.2
 
 # Package it
 sbt ++${SBT_VERSION} package

runners/run_xmatch.sh

@@ -18,7 +18,7 @@ SBT_VERSION=2.11.8
 SBT_VERSION_SPARK=2.11
 
 ## Package version
-VERSION=0.2.1
+VERSION=0.2.2
 
 # Package it
 sbt ++${SBT_VERSION} package

runners/run_xmatch_cluster.sh

@@ -18,7 +18,7 @@ SBT_VERSION=2.11.8
 SBT_VERSION_SPARK=2.11
 
 ## Package version
-VERSION=0.2.1
+VERSION=0.2.2
 
 # Package it
 sbt ++${SBT_VERSION} package

runners/run_xmatch_geo.sh

@@ -18,7 +18,7 @@ SBT_VERSION=2.11.8
 SBT_VERSION_SPARK=2.11
 
 ## Package version
-VERSION=0.2.1
+VERSION=0.2.2
 
 # Package it
 sbt ++${SBT_VERSION} package