<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ryo NAKAMARU on Medium]]></title>
        <description><![CDATA[Stories by Ryo NAKAMARU on Medium]]></description>
        <link>https://medium.com/@pottava?source=rss-8a2c9e15ef20------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*YPmsCWlHC424J1Xe.jpeg</url>
            <title>Stories by Ryo NAKAMARU on Medium</title>
            <link>https://medium.com/@pottava?source=rss-8a2c9e15ef20------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 17:02:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pottava/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A synchronous task runner using AWS Fargate]]></title>
            <link>https://medium.com/@pottava/a-synchronous-task-runner-using-aws-fargate-35e36382f426?source=rss-8a2c9e15ef20------2</link>
            <guid isPermaLink="false">https://medium.com/p/35e36382f426</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[ci]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[fargate]]></category>
            <dc:creator><![CDATA[Ryo NAKAMARU]]></dc:creator>
            <pubDate>Wed, 09 Jan 2019 10:04:23 GMT</pubDate>
            <atom:updated>2020-01-17T14:52:04.424Z</atom:updated>
            <content:encoded><![CDATA[<p>I started to make a tiny tool which runs CI/CD tasks on AWS Fargate last year.<br>The reasons are followings:</p><ul><li>I’m familiar with docker containerized one-off commands very well.</li><li>Some CI tests, like API integration tests, would be better to be run on the same VPC as services. They often do not have their public endpoints.</li><li>Some CD tasks, like DB migrations, should be executed on the VPC as well for security reasons.</li><li>AWS Fargate is an ideal platform for temporary test executions. It does not disturb the main service cluster and it’s cheap enough!</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*jP-ajU7pdTGzXTshjQm98w.png" /></figure><p>So here is the tool.</p><p><strong>ECS Task Runner : </strong><a href="https://github.com/pottava/ecs-task-runner">https://github.com/pottava/ecs-task-runner</a></p><h3>Samples</h3><p>This is the simplest command.</p><pre>$ ecs-task-runner run \<br>    --access-key AKIAIOSFODNN7EXAMPLE \<br>    --secret-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \<br>    alpine:3.8 --entrypoint env<br>{<br>  &quot;container-1&quot;: [<br>    &quot;2019-01-09T08:36:17+09:00: PATH=/usr/local/sbin:/usr/local/b..<br>    &quot;2019-01-09T08:36:17+09:00: HOSTNAME=ip-172-31-46-139.us-east..<br>    &quot;2019-01-09T08:36:17+09:00: AWS_DEFAULT_REGION=us-east-1&quot;,<br>    ..<br>  ]<br>}</pre><p>It returns synchronously when the ECS task has done. The output logs are collected from CloudWatch Logs and sent to standard out properly.</p><p>It returns with an exit code of containers too. You can specify AWS credentials as environment variables instead of arguments. As you know, these features are necessary for CI/CD pipelines.</p><pre>$ ecs-task-runner run alpine --entrypoint sh,-c --command &quot;exit 255&quot;<br>{<br>  &quot;container-1&quot;: []<br>}<br>$ echo $?<br>255</pre><p>Not only public images but also ECR images and even images on your own private repositories can be used so easily. This tool proceeds <a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html"><strong>this</strong></a> behind the scene for you!</p><pre>$ ecs-task-runner run \<br>    123456789012.dkr.ecr.us-east-1.amazonaws.com/sample/image:1.0</pre><pre>// The force-ecr option helps you to complement the image name<br>$ ecs-task-runner run <strong>--force-ecr</strong> sample/image:1.0</pre><pre>// Using third party private repositories<br>$ export PRIVATE_REGISTRY_USER=user<br>$ export PRIVATE_REGISTRY_PASSWORD=password<br>$ ecs-task-runner run docker.supinf.co/nak/my-secrets:1.0</pre><p>You can also handle your sensitive data, of course!<br>This tool is using<strong> </strong><a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html"><strong>this mechanism</strong></a> internally.</p><pre>$ ecs-task-runner run \<br>    <strong>-e</strong> FOO=arn:aws:ssm:us-east-1:123456789012:parameter/foo \<br>    <strong>-e</strong> BAR=arn:aws:ssm:us-east-1:123456789012:parameter/bar \<br>    alpine --entrypoint env<br>{<br>  &quot;container-1&quot;: [<br>    &quot;2019-01-09T08:36:17+09:00: FOO=foo-secret&quot;,<br>    &quot;2019-01-09T08:36:17+09:00: BAR=bar-secret&quot;,<br>    ..<br>  ]<br>}</pre><pre>// Specify the KMS key ID if you encrypted with your custom key.<br>$ ecs-task-runner run \<br>    <strong>-e</strong> BAZ=arn:aws:ssm:us-east-1:123456789012:parameter/baz \<br>    <strong>--kms-key-id</strong> 0ab1cd23-e4f5-6789-01ab-2345abcd67ef \<br>    alpine --entrypoint env<br>{<br>  &quot;container-1&quot;: [<br>    &quot;2019-01-09T08:36:17+09:00: BAZ=baz-secret&quot;,<br>    ..<br>  ]<br>}</pre><p>ecs-task-runner itself is published as docker images as well.</p><pre>$ docker run --rm \<br>    -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY \<br>    pottava/ecs-task-runner:2.3 sample/image</pre><h3>Features</h3><ul><li>This is a <strong>synchronous</strong> task runner for AWS Fargate.</li><li>It returns an exit code of containers properly.</li><li>You don’t have to consider anything but containers. VPC subnets, security groups or IAM roles, all you have to specify normally will be created temporarily and destroyed finally. (If you specify these, any additional resources won’t be created nor destroyed.)</li><li>It can handle private repos and sensitive data.</li><li>You can even run persistent services, like web servers, with --async option.</li></ul><h3>Recommendations</h3><p>It may be useful to run the following applications on the same VPC as services.</p><ul><li><a href="https://learning.getpostman.com/docs/postman/collection_runs/command_line_integration_with_newman/"><strong>Newman</strong></a><strong> </strong>with Postman for API e2s tests (<a href="https://github.com/supinf/dockerized-tools/blob/master/nodejs/newman/versions/4.1/Dockerfile">Dockerfile</a>)</li><li><a href="https://pptr.dev/"><strong>Puppeteer</strong></a> for Web integration tests (<a href="https://github.com/supinf/dockerized-tools/blob/master/nodejs/puppeteer/versions/1.9/Dockerfile">Dockerfile</a>)</li><li><a href="https://flywaydb.org/"><strong>Flyway</strong></a> for DB migration (<a href="https://github.com/supinf/dockerized-tools/blob/master/java/flyway/versions/5.2/Dockerfile">Dockerfile</a>)</li></ul><p>You don’t have to run klar on the VPC in order to scan ECR images, but it must be worth to run it on Fargate with proper <strong>TaskRole</strong>! Then you are able to forget about how to bring your credentials out of AWS safely.</p><ul><li><a href="https://github.com/optiopay/klar"><strong>Klar</strong></a> with Clair for image vulnerability scanning (<a href="https://github.com/supinf/dockerized-tools/blob/master/golang/klar/versions/2.4/Dockerfile">Dockerfile</a>)</li></ul><p>Thank you for reading my first story!<br>Feel free to contact me if you have any requests or feedback.<br><a href="https://twitter.com/pottava">@pottava</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=35e36382f426" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>