Writing GitHub Actions in Go

I was part of the GitHub Actions beta and have used GitHub Actions to run tests, close stale issues, and automate formerly-manual tasks in my repositories. Over the holiday break, I explored authoring GitHub Actions in languages other than Node.js. This post explores how to write and publish GitHub Actions written in Go, but the principles are largely applicable to any language since the deliverable is a Docker container.

Coding

GitHub Actions has an API of sorts that relies on output in a particular format. For example, the following Go code would set an environment variable named MY_ENVVAR to be available in all future steps with a value of the-value:

fmt.Println("::set-env name=MY_ENVVAR::the-value")

While GitHub publishes an official GitHub Actions SDK for Node.js, other languages currently lack an SDK. Fortunately I spent a few hours and put together an unofficial GitHub Actions SDK for Go that provides a Go-like interface for working with GitHub Actions (documentation).

To use this unofficial GitHub Actions SDK for Go, create a new file named main.go.

package main

import "github.com/sethvargo/go-githubactions"

func main() {
    fruit := githubactions.GetInput("fruit")
    if fruit == "" {
        githubactions.Fatalf("missing input 'fruit'")
    }
    githubactions.AddMask(fruit)
}

This gets an input named fruit (failing if not provided), and then adds a mask for that fruit. Subsequent attempts to log the value of the fruit input will print *** instead.

Next, create an action.yml file, which instructs GitHub how to run and invoke the action. By default, actions are invoked as Node.js, but you can also specify then to run via Docker.

name: My Action
inputs:
  secrets:
    fruit: Name of fruit to mask
    required: true

runs:
  using: docker
  image: Dockerfile

Finally, create a Dockerfile which builds the Go code. You can learn more about Docker and containerization in the Docker getting start documentation.

# Specify the version of Go to use
FROM golang:1.13

# Copy all the files from the host into the container
WORKDIR /src
COPY . .

# Enable Go modules
ENV GO111MODULE=on

# Compile the action
RUN go build -o /bin/action

# Specify the container's entrypoint as the action
ENTRYPOINT ["/bin/action"]

That is all - you just wrote a GitHub Action in Go! The next sections cover how to package and distribute this action.

Packaging

On GitHub, create a new repository named my-action. Then commit and push the actions code to this repository.

$ git init .
$ git remote add origin https://github.com/<username>/my-action
$ git commit -m "Initial commit"
$ git push -u origin master

After you commit and push these changes to the repository, import your action by specifying the name of your repo and a tag in a GitHub Actions Workflow configuration:

# .github/workflows/example.yml
name: Example
on: [push]

jobs:
  my_job:
    runs-on: ubuntu-latest

    steps:
    - name: Mask fruit
      uses: <username>/my-action@master
      with:
        fruit: banana

    - name: Echo fruit
      run: echo "apple, banana, and orange"

GitHub Actions will clone the repository at the provided ref, detect the action.yml file with the Docker instruction, and use the provided Dockerfile to build and run the Docker container.

There are two major drawbacks to this approach:

GitHub Actions must download the base container image on each build. This base container is nearly 1 GB in size (803 MB). This will cause slower builds and may incur additional bandwidth costs.
GitHub Actions must compile the Go code each time. This adds additional time to tbe build and defeats one of Go's major benefits - single static binaries.

The next section explores a different approach to packaging and distributing GitHub Actions written in Go (or any other language).

Packaging - Take 2

The first packaging method relies on GitHub Actions to:

Download the code from our repository
Download the golang base container image (803 MB)
Build the container (go build)
Execute the container as a step

Relying on GitHub Actions to perform the Docker build adds time to our setup. Fortunately GitHub Actions steps can also source directly from a Docker container. This means we can build the container and push it to a container registry in advance. You need to sign up for a free Docker Hub account to push to the registry.

Using the same Dockerfile as the previous step:

# Specify the version of Go to use
FROM golang:1.13

# Copy all the files from the host into the container
WORKDIR /src
COPY . .

# Enable Go modules
ENV GO111MODULE=on

# Compile the action
RUN go build -o /bin/action

# Specify the container's entrypoint as the action
ENTRYPOINT ["/bin/action"]

build the container locally:

$ docker build -t <username>/my-action

and push it to the Docker registry:

$ docker push <username>/my-action

To consume this action, use the docker:// prefix in the action definition:

jobs:
  my_job:
    steps:
    - uses: docker://<username>/my-action:latest

GitHub Actions will now download and use the container directly. Pre-compiling the container saves about 18s on each build, but a large portion of the build is spent downloading the container. This is because the container includes the entire Go runtime and build toolchain. The next section explores how to slim the container for even faster builds.

Packaging - Take 3

The second packaging method relies on GitHub Actions to:

Download our container (819 MB)
Execute the container as a step

Since we cannot eliminate any of these steps, the next best area for optimization is to reduce the size of the Docker container. This image is large because it is based off of the golang base image. This image includes the entire Go runtime and build toolchain. However, once our action is compiled, none of that is necessary. In fact, one of the many reasons people choose Go is because of its ability to produce single static binaries.

We can leverage Docker multi-stage builds to further reduce the size of the resulting container. The first stage uses the large golang base container to build the action. The second stage copies the compiled action into a much smaller base container for publishing. Here is the new Dockerfile, commented inline:

#
# Step 1
#

# Specify the version of Go to use
FROM golang:1.13 AS builder

# Install upx (upx.github.io) to compress the compiled action
RUN apt-get update && apt-get -y install upx

# Turn on Go modules support and disable CGO
ENV GO111MODULE=on CGO_ENABLED=0

# Copy all the files from the host into the container
COPY . .

# Compile the action - the added flags instruct Go to produce a
# standalone binary
RUN go build \
  -a \
  -trimpath \
  -ldflags "-s -w -extldflags '-static'" \
  -installsuffix cgo \
  -tags netgo \
  -o /bin/action \
  .

# Strip any symbols - this is not a library
RUN strip /bin/action

# Compress the compiled action
RUN upx -q -9 /bin/action


# Step 2

# Use the most basic and empty container - this container has no
# runtime, files, shell, libraries, etc.
FROM scratch

# Copy over SSL certificates from the first step - this is required
# if our code makes any outbound SSL connections because it contains
# the root CA bundle.
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

# Copy over the compiled action from the first step
COPY --from=builder /bin/action /bin/action

# Specify the container's entrypoint as the action
ENTRYPOINT ["/bin/action"]

Now build the container locally:

$ docker build -t <username>/my-action

and push it to the Docker registry:

$ docker push <username>/my-action

Only the final container – which includes only the most minimal set of files and our compiled action – is saved and pushed to the registry. By using Docker multi-stage builds, the resulting container image is just 764 KB in size!

Using the container is the same as previous steps:

jobs:
  my_job:
    steps:
    - uses: docker://<username>/my-action:latest

This GitHub Actions step now runs almost instantly!

Pros and Cons

Using a Docker container for GitHub Actions has some benefits and drawbacks. Writing GitHub Actions in a different language and distributing them via a Docker container may not work for everyone.

Pros

Controlled execution environment - if you need to modify the base system to install packages or libraries, the container primitive provides a predictable, reproducible, and customizable environment.
Code in any language - Since the Docker container includes the execution environment, use whatever programming language you prefer – C, Rust, Ruby, Python, Cobol, .NET – a world of possibilities await.
Portable - The container primitive is portable and can be used for more than GitHub Actions. docker pull the container locally and test it out locally.

Cons

Bloat - Crafting a slim Docker container is an art. As shown above, it can be challenging to create a small container. Big containers consume unnecessary bandwidth, eat up storage space, and pose a security risk.
Complexity - Adding the Dockerfile adds an additional layer of indirection between the code and the execution environment. For users who are no familiar with container technologies, this added complexity could be burdensome.
Linux-only - At the time of this writing, Docker-based GitHub Actions are only available on Linux builders. If you want to use a GitHub Action on Windows or Mac OS, you must use Node.js.

About Seth