INSTALL

====================== COG INSTALLATION INSTRUCTIONS ======================================


>>> PRE-REQUISITES

*-nix OS, including Cent-OS, linux and MacOSX
Python 2.7
Python virtualenv (pip install virtualenv)
Postgres (only if using Postgres back-end)


>>> FIRST INITIAL SETUP (ONE TIME ONLY)

Create a parent directory under which all CoG-related software (Python, CoG installation, CoG configuration)
will be installed. The directory can be located anywhere on the system. The user executing the installation
must have write permissions to this directory. In this instructions, we will use /usr/local/cog as the CoG root directory.

export COG_DIR=/usr/local/cog

At the end of the installation process, the $COG_DIR will contain the following sub-directories:

$COG_DIR/
	|
	|--> cog_config/ : site-specific configuration and media files
	|--> cog_install/ : CoG source distribution
	|--> venv/: Python virtula environment (Python executable and all needed dependencies) 
	
It is recommended to setup the necessary env variables in your ~/.bash_profile or ~/.bashrc file 
(whatever is parsed by your system at login):

export COG_DIR=/usr/local/cog
export COG_CONFIG_DIR=$COG_DIR/cog_config
export COG_INSTALL_DIR=$COG_DIR/cog_install

Create all the necessary directories, if not existing already:

mkdir $COG_DIR
mkdir $COG_CONFIG_DIR
mkdir $COG_INSTALL_DIR

Also create a Python "virtual environment" - 
a directory where to install a specific distribution of Python and all its dependencies needed by CoG:

o cd $COG_DIR
o virtualenv venv

will create the sub-directory venv with:
- copy of python executable in ./bin/python
- location for installation of additional needed packages in ./lib/python2.7/site-packages


>>> INSTALLATION OR UPGRADE

1) Activate the CoG Python virtual environment:

o cd $COG_DIR
o source venv/bin/activate
o which python
will show that the python installation used is located into the venv sub-directory

Note: to later de-activate the Python virtual environment:
o deactivate

2) Checkout the CoG software stack

First time installation only: checkout CoG from the GitHub repository:

o cd $COG_DIR
o git clone https://github.com/EarthSystemCoG/COG cog_install
or:
o git clone git@github.com:EarthSystemCoG/COG.git cog_install

Every time (first time installation or upgrade): checkout a specific CoG tag or branch:
o cd $COG_INSTALL_DIR
o git checkout -b devel origin/devel
or for a specific tag:
o git checkout v2.6.2

3) Install and configure CoG

o cd $COG_INSTALL_DIR
o python setup.py install

will install all necessary CoG dependencies under the CoG python installation in the location
$COG_DIR/venv/lib/python2.7/site-packages

To configure CoG WITHOUT an ESGF node:
o python setup.py setup_cog

or to install CoG as backed-up by an ESGF node:
o python setup.py setup_cog --esgf=true

will create the site configuration file under $COG_CONFIG_DIR/cog_settings.cfg
will create a site_media directory under $COG_CONFIG_DIR/site_media
will copy all CoG system media to $COG_INSTALL_DIR/static
	
if --esgf=false:
	- will create a startup sqllite database under $COG_CONFIG_DIR/django.data
	- if installing for the first time, it will create a "TestProject" and make that the CoG home project
	- if installing for the first time, it will create a super-user "admin" with password "changeit"
	
if --esgf=true:
	- will create a CoG database onto the existing ESGF Postgres installation
	- if installing for the first time, it will create a "TestProject" and make that the CoG home project
	- if installing for the first time, it will create a super-user "admin" with password "changeit" 
	  and openid 'https://<hostname>/esgf-idp/openid/admin (or the first available openid if that is taken already)
	  
will also update the local site name as specified in the cog_settings.cfg file
will also update the list of available CoG peer sites from the file cog/management/commands/sites.xml

o rm -rf $COG_PYTHON/lib/python2.7/site-packages/cog*

will remove CoG from the python site-packages library,
to avoid conflicts with the source installation in $COG_DIR/cog_install


>>> TESTING

Run CoG with the Django development server:

o cd $COG_INSTALL_DIR
o python manage.py runserver
will run CoG on localhost and port 8000

NOTE: to run CoG with a different hostname and port:
python manage.py runserver <hostname>:<port>

o access the top level URL at http://localhost:8000
o login as administrator:
	- if --esgf=true, use the standard login URL: http://<hostname>/login/ with the "admin" openid and password mentioned above
	- if --esgf=false, use the hidden local login: http://<hostname>/login2/ with username "admin" and the password above
	
o immediately change the "admin" password by clicking on "My Profile" -> "Change Password" 
  or directly accessing the URL "http://<hostname>/password/update/
  Note: to fully test the password change, you need to log out of CoG AND quit the browser and log-in as 'admin' again

o also change all other "admin" data to match the local administrator user, 
especially the email address where all site-level notifications will be sent

o start experimenting with creating other projects and registering new users.

o test the email notification when new projects are created


>>> POST INSTALLATION CONFIGURATION

o Review the file $COG_CONFIG_DIR/cog_settings.cfg and modify all values to reflect the
site specific configuration. Values set in this file will not be overridden by
future upgrades. Refer to the file resources/config/cog_settings.cfg (included with the CoG distribution)
for description and example values of the configuration parameters.

o If running the CoG application under a user other than the one executing the installation, 
the directory $COG_CONFIG_DIR/site_media must be owned by that user. For example:

cd $COG_CONFIG_DIR
chown -R apache:apache site_media


o If for any reason the name or domain of the current site needs to change, 
the new values should be entered in cog_settings.cfg, then run the command:

cd $COG_INSTALL_DIR
python manage.py init_site


o Setup federation with other CoG instances:

The list of possible CoG peers is seeded from the file cog/management/commands/sites.xml that is distributed with the CoG application.

In order to load the most up-to-date list of peers into the local database, run the following django management command:

cd $COG_INSTALL_DIR
python manage.py sync_sites

This command will insert any missing sites in the local database (and update their names if they have changed), but will not activate the sites by default.
Note that the same command can also be passed the '--delete' option to delete stale sites that are found in the local database, but are missing from the sites.xml file:

python manage.py sync_sites --delete

Once the sites are listed in the local database, they can be enabled by logging in as a site administrator and clicking 
on the 'Configure Peers' link in the 'Admin' menu in the lower left navigation bar.

The local CoG application can be instructed to load the most recent list of projects from the other federated instances in one of three ways:

a) By visiting the page "Configure Peers" and clicking on the lower link "Synchronize with Peer CoGs Now "
(or directly invoking the URL "http://<your cog domain name>/share/sync/projects/")

b) By invoking the following django management command:

cd $COG_INSTALL_DIR
python manage.py sync_projects

c) By setting up a cron job that executes the same command above. For example:

crontab -l
0 * * * * source ~/.bashrc; python /usr/local/cog/cog_install/manage.py sync_projects >> /tmp/crontab.log 2>&1

will parse the user environment, and run the command at minute 0 of every hour, appending the output to file.

NOTES: 
	- projects will be updated only from peer sites that are "enabled". 
	- IMPORTANT: a site will only share projects correctly if its 'Site' domain name equals its real domain.
	  If its site domain has been recently updated, CoG must be restarted for that change to take effect, otherwise
	  the other CoGs won't see its projects
	- a site will only share those projects that are both 'active' and 'public'
	
	
NOTE: if a CoG peer site is disabled through the admin interface:
	- Its projects won't show up in the projects browser (for none of the tabs)
	- Its projects will not be selectable as peers/parent/children in the project configuration panel
	- Its data cart won't show up in the current data cart page
	- The site will not be queried for user projects updates


o To restore the CoG database from backup:

Stop CoG, then:
- if using postgres:
psql -U postgres -p 5432 cogdb < cogdb_2014-03-12.sql

- if using sqllite:
simply copy the older django.data file to the DATABASE_PATH location 

Finally restart CoG.

	
o Setup cron jobs to execute periodic administration tasks

a) Setup a cron job to periodically update the projects around the federation.
An example such script is provided as scripts/sync_projects.sh - which will need to be customized to your environment.

For example, to synchronize the projects every hour and append the log to a file:

0 * * * * /usr/local/cog/cog_install/resources/scripts/sync_projects.sh  >> /tmp/crontab.log 2>&1


b) Setup a cron job to backup the postgres database content every night. 

An example script is provided as 'scripts/cog_backup.sh' - which will need to be customized to your specific installation.

For example, to backup the database every day at midnight:

0 0 * * * /usr/local/cog/cog_install/scripts/cog_backup.sh

NOTE: to run as cron, the postgres database password must be located in file ~/.pgpass with the format:
host_name:port:database_name:database_user:database_password

Also must set the file permissions as follows:

chmod 0600 ~/.pgpass

o Optionally, customize the header and footer

The default header and footer can be overridden by placing a custom version with the same name under the
root level directory <COG_CONFIG_DIR>/mytemplates/cog/common/<file_name>, where by default
COG_CONFIG_DIR=/usr/local/cog (or COG_CONFIG_DIR can be set through an environment variable available to the CoG web container).

For example, to override the template /cog/templates/cog/common/cog_header_custom.html and /cog/templates/common/cog_footer_custom.html
which come with the distribution, create the two custom files:

<COG_CONFIG_DIR>/mytemplates/cog/common/cog_header_custom.html
<COG_CONFIG_DIR>/mytemplates/cog/common/cog_footer_custom.html

You will note that the path (/cog/templates/cog/common/<filename> to the default file is longer than the path (/cog/common/<filename>
to the file that overrides the default. This is because Django already has within its known paths the first two directories (/cog/templates).

Images and other media should be located under the directory <COG_CONFIG_DIR>/mymedia/ and referenced from within the custom override
as "/mymedia/<image name>". For example, the file located at /usr/local/cog/cog_config/mymedia/nasa.jpeg can be loaded by any template
as src="/mymedia/nasa.jpeg" (please note the first leading '/').

The directories resources/noaa/mytemplates and resources/noaa/mymedia contain some examples of custom header, footer and associated media.