Merge pull request #138 from UW-GDA/egagli-patch-50

add demo nb to get snotel data from github hosted csvs

book/modules/08_Vector_TimeSeries_SNOTEL/08_SNOTEL_download_csvs.ipynb

@@ -0,0 +1,319 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "ad0998b7-a501-4f9f-a8c4-2ef01a23fb76",
+   "metadata": {},
+   "source": [
+    "# 08 Demo: Using snotel_ccss_stations to get SNOTEL data\n",
+    "\n",
+    "UW Geospatial Data Analysis  \n",
+    "CEE467/CEWA567  \n",
+    "Eric Gagliano  "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5106c82f-297d-46bb-a1c0-0818bfb134e3",
+   "metadata": {},
+   "source": [
+    "**Thanks for checking out this notebook! My hope is that this repository makes it easier to retrieve daily SNOTEL and CCSS data without having to do clunky downloads and conversions. Snow depth / SWE / PRCPSA are in meters, temperatures are in celsius. The only required packages you'll need are geopandas and pandas, the rest of the imports are for applications :)**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b3959e4b-74c7-426f-b903-449ebfff068e",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "import pandas as pd\n",
+    "import geopandas as gpd\n",
+    "import numpy as np\n",
+    "import matplotlib.pyplot as plt\n",
+    "import datetime\n",
+    "import tqdm\n",
+    "import contextily as ctx"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0ec527a0-41b1-449e-8a59-8d456073ceca",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## View all SNOTEL & CCSS stations\n",
+    "- the [SNOwpack TELemetry (SNOTEL) network](https://www.nrcs.usda.gov/wps/portal/wcc/home/aboutUs/monitoringPrograms/automatedSnowMonitoring/) includes over 800 automated weather stations in the Western U.S. for mountain snowpack observation\n",
+    "- the [CCSS program](https://water.ca.gov/Programs/Flood-Management/Flood-Data/Snow-Surveys) manages a network of 130 automated snow sensors located in the Szierra Nevada and Shasta-Trinity Mountains"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dad815f9-c694-4772-b9e8-cdb83f0c9d69",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "### Read the geojson stored at https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/all_stations.geojson\n",
+    "- the daily recurring github action should regularly update the `endDate` column \n",
+    "- set the index of the geodataframe to the code column\n",
+    "- let's only look at sites for which we have data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f7c60689-178f-42a2-b122-eacee67f5bcb",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "all_stations_gdf = gpd.read_file('https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/all_stations.geojson').set_index('code')\n",
+    "all_stations_gdf = all_stations_gdf[all_stations_gdf['csvData']==True]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "adc6b061-823d-44c3-ae81-f6527a23aedd",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "all_stations_gdf"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "43621e35-4326-4cf9-a221-f5f8ea614cbe",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "### Use geopandas `GeoDataFrame.explore()` on the `all_stations_gdf` geodataframe to interactively view the stations \n",
+    "- color by network: red is SNOTEL, blue is CCSS."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1ef6caa1-e841-4e46-8989-afb95af787b5",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "all_stations_gdf.astype(dict(beginDate=str, endDate=str)).explore(column='network',cmap='bwr')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "aaf197e5-7fd4-4c39-8b67-315de019f343",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Read a singular CSV: *Paradise, WA*\n",
+    "- check out information about the [SNOTEL station near Mt. Rainier at Paradise, WA](https://wcc.sc.egov.usda.gov/nwcc/site?sitenum=679)\n",
+    "- cool plots available at the [Northwest River Forecast Center website](https://www.nwrfc.noaa.gov/snow/snowplot.cgi?AFSW1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1cbbc606-69e5-4898-b9bc-6bfff881215a",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "### Place a station code (which you can find in this interactive plot, or by other means) in the url: https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/data/{station_id}.csv\n",
+    "- for SNOTEL stations, this will be of the form {unique number}_{two letter state abbreviation}_SNTL (e.g. 679_WA_SNTL).   \n",
+    "- for CCSS stations, this will be a three letter code (e.g. BLK).   \n",
+    "- use `pd.read_csv()` with `index_col='datetime'` and `parse_dates=True` so we interpret the datetime column as pandas datetime objects"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "82c6bd2e-3d27-4908-ab24-6e61772f4e8a",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "station_id = '679_WA_SNTL'\n",
+    "paradise_snotel = pd.read_csv(f'https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/data/{station_id}.csv',index_col='datetime', parse_dates=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "33bb24d7-44f9-49a5-b728-312495160f50",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "paradise_snotel"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cc267180-8371-46da-a99d-be42c5837ffc",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "### Try a simple plot of snow depth and SWE\n",
+    "- select the column of interest and use pandas built in `Series.plot()`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cf9f9160-7bbb-4424-b58c-97412e814cb9",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "f,ax=plt.subplots(figsize=(12,5))\n",
+    "\n",
+    "paradise_snotel['SNWD'].plot(ax=ax,label='snow depth')\n",
+    "paradise_snotel['WTEQ'].plot(ax=ax,label='snow water equivalent')\n",
+    "\n",
+    "ax.set_xlim(pd.to_datetime(['2017-10-01','2018-09-30']))\n",
+    "\n",
+    "ax.grid()\n",
+    "ax.legend()\n",
+    "\n",
+    "ax.set_xlabel('time')\n",
+    "ax.set_ylabel('snow depth / SWE [meters]')\n",
+    "ax.set_title('Snow depth and SWE at Paradise, WA \\n(water year 2018)')\n",
+    "\n",
+    "f.tight_layout()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "72fcc194-9e9c-4000-bea1-b3a0a9520604",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Read a variable from multiple CSVs by looping over a subset of the geodataframe: *CCSS stations*\n",
+    "- the Sierra Nevada [received a historic amount of snow in 2023](https://www.nps.gov/articles/000/sien-sierranevadamonitor-spring2023.htm)\n",
+    "- let's explore the magnitude of this season by comparing to the median snow pack"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9c4696ca-83e1-42e2-994b-679f8d18efcb",
+   "metadata": {},
+   "source": [
+    "### As before, create a list of the stations we are interested in, loop through and add data to a dictionary with the station code as the key, then read into pandas using `pd.DataFrame.from_dict()`\n",
+    "- create a geodataframe `ccss_stations_gdf` of only CCSS stations from `all_stations_gdf` by creating an index where network equals CCSS\n",
+    "- loop through the CCSS stations and create a dataframe `ccss_stations_snwd_df`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7d817742-e0f9-43c2-91cd-bbf90ae4ea3b",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "ccss_stations_gdf = all_stations_gdf[all_stations_gdf['network']=='CCSS']"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0603ad1a-fd87-4623-ad13-40f9296c1d92",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "ccss_stations_gdf"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d6688f84-bb5e-45a7-8138-771d40c1cfac",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "%%time \n",
+    "station_dict = {}\n",
+    "\n",
+    "for station in tqdm.tqdm(ccss_stations_gdf.index):\n",
+    "    try:\n",
+    "        tmp = pd.read_csv(f'https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/data/{station}.csv',index_col='datetime',parse_dates=True)['SNWD']\n",
+    "        station_dict[station] = tmp\n",
+    "    except:\n",
+    "        print(f'failed to retrieve {station}')\n",
+    "\n",
+    "ccss_stations_snwd_df = pd.DataFrame.from_dict(station_dict).dropna(how='all')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0c4a5224-3312-4e8e-8582-484d5049acca",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "ccss_stations_snwd_df"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1d56714b-89c2-4ff7-a0fa-bae9124426e8",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cb09c66b-b74e-41d4-8c6c-ba6a12d3c4e0",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.7"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}