nhi-pylib - helpful python scripts used in Natural Hazards and Impacts

A collection of helpful scripts to support processing and analysis. Predominantly, these are for teh Atmospheric Hazards team.

Instructions

Clone this repo to your local machine. I have previously installed this to a _lib_ folder in a workspace folder, but can be anywhere as long as you have write permissions:

git clone https://github.com/GeoscienceAustralia/nhi-pylib lib

Add the nhi-pylib path to your PYTHONPATH user environment variable. This is done by going to the Windows Settings and searching for "Edit environment variables for your account". Add or edit the PYTHONPATH variable to include the location of the nhi-pylib directory. Make sure to append :bash:`python` to the end of the path. If you have installed the nhi-pylib repo at C:\Users\uname\nhi-pylib, the environment variable must be set to C:\Users\uname\nhi-pylib\python.

What's inside

• dtutils.py: a bunch of datetime manipulation functions
• files.py: file manipulation functions, including configuration files, log files, file sizes, file versions
• ftpclient.py: A wrapper around the standard ftplib library to establish "smart" ftp put/get commands
• ftpscriptrunner.py: Read and parse FTP commands from a text file and execute them using ftpclient.py
• sftpclient.py: A wrapper around pysftp, a secure FTP implementation - similar to ftpclient.py
• sftpscriptrunner.py: Same as ftpscriptrunner.py, but for secure FTP connections
• metutils.py: Meteorological functions (conversion between different variables & units, coriolis, etc.)
• parallel.py: base functions that set up a parallel processing environment
• process.py: Provides functions to control processing of files. Records details of each file processed in a text file, which can then be looked up at a later time to determine if a file has previously been processed.

There's a couple of others, but not really working as intended, or have been superceded.

There's no test suite for this codebase. Errors may not be well handled. No guarantee that it will work out of the box. Use at your own risk.

Requirements

• gitpython
• pysftp
• tendo