Skip to content

Latest commit

 

History

History
225 lines (180 loc) · 9.04 KB

README.md

File metadata and controls

225 lines (180 loc) · 9.04 KB
Raw

A synchronous task runner for AWS Fargate on Amazon ECS

CircleCI

pottava/ecs-task-runner

Supported tags and respective Dockerfile links:
・latest (versions/3.0/Dockerfile)
・3.0 (versions/3.0/Dockerfile)
・2.3 (versions/2.3/Dockerfile)
・1 (versions/1.2/Dockerfile)

Description

This is a synchronous task runner for AWS Fargate. It runs a docker container on Fargate and waits for its done. Then it returns its standard output logs from CloudWatch Logs. All resources we need are created temporarily and remove them after the task finished.

Installation

curl (macOS):

$ curl -Lo ecs-task-runner https://github.com/pottava/ecs-task-runner/releases/download/3.0/ecs-task-runner_darwin_amd64 \
    && chmod +x ecs-task-runner

curl (Linux):

$ curl -Lo ecs-task-runner https://github.com/pottava/ecs-task-runner/releases/download/3.0/ecs-task-runner_linux_amd64 \
    && chmod +x ecs-task-runner

go:

go get github.com/pottava/ecs-task-runner/...

docker:

docker pull pottava/ecs-task-runner

Parameters

Common parameters:

Environment Variables Argument Description Required Default
AWS_ACCESS_KEY_ID access-key, a AWS access key for API access
AWS_SECRET_ACCESS_KEY secret-key, s AWS secret key for API access
AWS_DEFAULT_REGION region, r AWS region for API access us-east-1
AWS_PROFILE profile AWS profile for API access default
AWS_ASSUME_ROLE assume-role IAM Role ARN to be assumed
AWS_MFA_SERIAL_NUMBER mfa-serial-num A serial number of MFA device
AWS_MFA_TOKEN mfa-token A token for MFA
ECS_CLUSTER cluster, c Amazon ECS cluster name
EXEC_ROLE_NAME exec-role-name Name of an execution role ecs-task-runner
TASK_TIMEOUT timeout, t Timeout minutes for the task 30
EXTENDED_OUTPUT extended-output True: meta data also returns false

for the run command:

Environment Variables Argument Description Required Default
DOCKER_IMAGE Docker image to be run on ECS *
FARGATE_SPOT spot True: fargate spot will be used false
FORCE_ECR force-ecr, f True: you can use shortened name false
ENTRYPOINT entrypoint Override ENTRYPOINT of the image
COMMAND command Override CMD of the image
PORT port, p Publish ports
ENVIRONMENT environment, e Add ENV to the container
USER docker-user The user inside the container
LABEL label, l Add LABEL to the container
SUBNETS subnets Fargate's Subnets
SECURITY_GROUPS security-groups Fargate's SecurityGroups
TASKDEF_FAMILY taskdef-family ECS Task Definition family name ecs-task-runner
TASK_ROLE task-role-arn ARN of an IAM Role for the task
CPU cpu Requested vCPU to run Fargate 256
MEMORY memory Requested memory to run Fargate 512
NUMBER number, n Number of tasks 1
PRIVATE_REGISTRY_USER user PrivateRegistry Username
PRIVATE_REGISTRY_PASSWORD password PrivateRegistry Password
KMS_CUSTOMKEY_ID kms-key-id KMS custom key ID for SecretsManager
ASSIGN_PUBLIC_IP assign-pub-ip True: Assigns public IP true
READONLY_ROOOTFS readonly-rootfs Make the root file system read-only false
ASYNC async True: Does not wait for the job done false

for the stop command:

Environment Variables Argument Description Required Default
REQUEST_ID Resources ID to be stopped *
TASK_ARN task-arn Task ARNs to be stopped

Samples

$ export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
$ export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
$ ecs-task-runner run alpine --entrypoint env
{
  "container-1": [
    "2018-09-23T11:42:01+09:00: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "2018-09-23T11:42:01+09:00: HOSTNAME=ip-172-31-40-206.us-east-1.compute.internal",
    "2018-09-23T11:42:01+09:00: AWS_DEFAULT_REGION=ap-northeast-1",
    "2018-09-23T11:42:01+09:00: AWS_REGION=ap-northeast-1",
    "2018-09-23T11:42:01+09:00: HOME=/root"
  ]
}

Using an assume role with MFA token

$ unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY
$
$ export AWS_PROFILE=project-alpha-dev
$ export AWS_ASSUME_ROLE=arn:aws:iam::123456789012:role/rolename
$ export AWS_MFA_SERIAL_NUMBER=arn:aws:iam::123456789012:mfa/mfaname
$ export AWS_MFA_TOKEN=123456
$
$ ecs-task-runner run alpine --entrypoint env
{
  "container-1": [
    "2018-09-23T11:42:01+09:00: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "2018-09-23T11:42:01+09:00: HOSTNAME=ip-172-31-40-206.us-east-1.compute.internal",
    "2018-09-23T11:42:01+09:00: AWS_DEFAULT_REGION=ap-northeast-1",
    "2018-09-23T11:42:01+09:00: AWS_REGION=ap-northeast-1",
    "2018-09-23T11:42:01+09:00: HOME=/root"
  ]
}
$ echo $?
0

This app will return with an exit code of the containers:

$ ecs-task-runner run alpine --entrypoint sh,-c --command "exit 255"
{
  "container-1": []
}
$ echo $?
255

Run a container asynchronously with --async flag:

$ ecs-task-runner run nginx --async -p 80 --security-groups sg-public-80-abcdefg
{
  "RequestID": "ecs-task-runner-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "Tasks": [
    {
      "PublicIP": "xx.xxx.xxx.xx",
      "TaskARN": "arn:aws:ecs:us-east-1:xxxxxxxxxxxx:task/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    }
  ]
}

To stop the asynchronous tasks:

$ ecs-task-runner stop ecs-task-runner-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
{
  "container-1": [
    "2018-09-23T22:34:37+09:00: zzz.zz.z.zzz - - [23/Sep/2018:13:34:37 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0 Safari/537.36\" \"-\""
  ]
}

Usage

With arguments:

ecs-task-runner -a AKIAIOSFODNN7EXAMPLE -s wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY run sample/image

With environment variables:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
ecs-task-runner run sample/image

With ECR shortened image name:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
ecs-task-runner run --force-ecr my-ecr/image

With a private registory:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export PRIVATE_REGISTRY_USER=user
export PRIVATE_REGISTRY_PASSWORD=password
ecs-task-runner run sample/secret

With the docker container:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
docker run --rm -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY pottava/ecs-task-runner sample/image

Troubleshooting

If the command returns non-zero exit code, you can try --extended-output to analyze the cause of failure.

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
ecs-task-runner run sample/image --extended-output