Combined Data Hub with Electricity Market and Coronavirus Case Data in U.S.

 

This data hub, COVID-EMDA (Coronavirus Disease - Electricity Market Data Aggregation), is orginally designed to track the potential impacts of COVID-19 on the electricity markets in the United States of America, and could also contribute to relevant interdisciplinary researches.\

Features

• Overall, this data hub contains the coronavirus case, weather, generation mix, load and price data for all the existing U.S. electricity marketplaces (CAISO, MISO, ISO-NE, NYISO, PJM, SPP , ERCOT) and some typical cities in these markets (San Francisco, Boston, New York City, Philadelphia, Kansas City, Houston).
• All the coronavirus case, weather and electricity market data are carefully verified and coordinated to match the geological scale.
• Daily load profile, hourly generation mix and day-ahead price are recorded and tidied in a consistent, compact and ready-to-use data format that makes it easy for cross-market analysis.
• Historical data dating back to 2017 are included as a time-series benchmark.

Suggested Reference

• A paper containing comprehensive introduction of this data hub and preliminary analysis is available at: arXiv and EnerarXiv
• Suggested reference for publication associated with this data hub:
  G. Ruan, D. Wu, X. Zheng, S. Sivaranjani, H. Zhong, C. Kang, M. A. Dahleh and L. Xie, A Cross-Domain Approach to Analyzing the Short-Run Impact of COVID-19 on the U.S. Electricity Sector , arXiv preprint: https://arxiv.org/abs/2005.06631, 2020.

Navigation

This github repository is structured as follows.

  COVID-EMDA
  |
  ├── data_source/
  │    ├── caiso/
  │    ├── miso/
  │    ├── isone/
  │    ├── nyiso/
  │    ├── pjm/
  │    ├── spp/
  │    ├── ercot/
  │    ├── covid/
  │    ├── weather/
  │    ├── README.md
  ├── data_release/
  │    ├── caiso/
  │    ├── miso/
  │    ├── isone/
  │    ├── nyiso/
  │    ├── pjm/
  │    ├── spp/
  │    ├── ercot/
  │    ├── README.md
  ├── startup/
  ├── parser/
  ├── README.md

All the data source files are archived in folder date_source/, the cleaned and processed data are stored in folder data_release. The supporting team will provide daily updates to capture the latest sitation. All these files are properly collected by location. The file naming rule is: MARKET_AREA_CATEGORY.csv, e.g. nyiso_nyc_load.csv. This is a dataset of load profile in New York City, from 2017 to present.

Readers can turn to folder startup/ for information about basic applications. For the need of extension, we provide the parser codes, which can be found in folder parser/.

Data Source

This data hub contains three major sources: U.S. electricity market data, COVID-19 cases and weather data. For some categories, multiple data sources are carefully gathered to ensure accuracy.

• Electricity Market Data:
  CAISO,   MISO,   ISO-NE,   NYISO,   PJM,   SPP,   ERCOT,   EIA,   EnergyOnline.
• COVID-19 Case Data:
  John Hopkins CSSE.
• Weather Data:
  Iowa State Univ IEM.

Additional Dataset

• Cellphone Dataset:
  Description: This dataset aggregates and processes the real-time GPS location of cellphone users by Census Block Group. It includes social distancing data and patterns of visits to Point of Interests (POIs). To obtain the access, please click the link below and apply for SafeGraph's permission which is totally free.
  Link: Cellphone Dataset from SafeGraph

• Night Time Light Satellite Dataset:
  Description: This dataset includes the raw satellite image taken at night time in each area.
  Link: NTL Dataset from NASA

Contact Us

Please contact our emails:   Guangchun Ruan,   Le Xie.

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Contributors

This project is a collaboration of our group members with the supervision of Prof. Le Xie at Texas A&M University:

Emails (from left):
@ Guangchun Ruan @ Dongqi Wu @ Xiangtian Zheng @ Sivaranjani Seetharaman @ Suraj Abolaji Alimi @ Jiahan Wu @ Le Xie

Please also check our Group Website for the detailed bio of each group member.