WIP: Creates templates for ECS and EC2 testing

[morris25]

In response to #66

[iamed2]

Can you explain this in detail please?

[nicoleepp]

Can get rid of extra blank lines

[morris25]

Currently, we have functions in AWSCredentials.jl that work specifically for retrieving credentials from EC2 instances and containers running through ECS. As it stands, these functions are untested.

The ec2_test.yml creates an EC2 instance along with logging and AWS CodeDeploy, which should make hooking it up to travis easier. The appspec.yml and runtests.sh are both part of the codedeploy setup.

ecs_test.yml provides the means to launch an ECS Container through AWSBatch, and contains a job description that should run the AWSCore tests.

Both these templates can be launched as nested stacks through awscore_test.yml using the aws cloudformation package command. However, as they are less likely to be frequently used they are disabled by default, meaning the unpackaged template should still work as usual without the nested stacks.

[iamed2]

This proposed CodeDeploy setup should be detailed and demonstrated before anything is added to this repo. This approach creates a lot more complexity in the test infrastructure and needs to be fully evaluated. Alternatives should also be considered. One I can think of would be creating the EC2/ECS compute resources on demand and running tests on them using addprocs and an appropriate cluster manager.

One thing to note: the cost for running a t3.nano for a whole year is less than $50.

ecs_test.yml provides the means to launch an ECS Container through AWSBatch, and contains a job description that should run the AWSCore tests.

Why Batch and not ECS directly?

However, as they are less likely to be frequently used they are disabled by default

Huh? Whatever is done for EC2/ECS credential testing should be run every time.

[morris25]

Huh? Whatever is done for EC2/ECS credential testing should be run every time

For our purposes yes, but others who aren't working directly on the credentials-related code will probably not want to spin up the additional nested stacks for local testing. It's probably easier for them not to have to worry about packaging the templates. Any automated deployment will be set up to test ECS/EC2 every time.

Why Batch and not ECS directly?

ECS seems to be set up to run continuous services. We could launch the tests as a task, but ECS doesn't provide very good output for whether a task completed successfully or not. It expects it to run forever, and so just keeps track of whether it's running or not. AWS Batch runs through an ECS cluster, and provides better output as to success or failure.

[morris25]

After some fiddling I've got a minimal example of using CodeDeploy using the ec2_test stack here.
Here it is working: https://travis-ci.org/morris25/Example.jl/jobs/502654717
Here it is not-so-working: https://travis-ci.org/morris25/Example.jl/jobs/502669069
I'll note that the deployment output doesn't give very much information about failures. So if we use this approach, we would probably need to add something to retrieve the logs.

CodeDeploy has the ability to grab code straight from github, which simplifies the setup a little bit. The AWS User for the branch was created with AWSCodeDeployDeployerAccess and has now been disabled. In future revisions the permissions would be added to the PublicCIUser.

Alternatives should also be considered. One I can think of would be creating the EC2/ECS compute resources on demand and running tests on them using addprocs and an appropriate cluster manager

If you can point me to an appropriate cluster manager (preferably not using AWSCore), I'd be happy to give that approach a shot.

[iamed2]

The built-in SSH cluster manager combined with awscli would work without AWSCore