Create your own registry

docker/registry
linuxserver/fleet

Multi-Stage Builds

Make your docker images smaller by separating concerns
Some of these features may require DOCKER_BUILDKIT=1 env var
You can also use docker buildx to use BuildKit

DRY / Flavours

Parameterize with ARG to make slightly different envs

# or slim, buster, etc.
ARG flavour=alpine

FROM python:3.8-$flavour
...

Dev Containers

For max decoupling, you can create different containers with the same Dockerfile and specify which container to build by specifying --target=$NAME

FROM python:3.8-alpine AS build
...

FROM python:3.8-alpine AS dev
COPY --from=build . .
# install only devtool stuff
...

FROM python:3.8-alpine AS release
COPY --from=build app.py .
# no devtools, just the minimum required to deploy
ENTRYPOINT ["python app.py"]

Concurrent Builds

Instead of building linearly, you can treat build steps as a DAG and build in parallel where possible using multiple COPY statements

...
COPY --from=builder-dep-one   /out .
COPY --from-builder-dep-two   /out .
COPY --from-builder-dep-three /out .
...

Secrets and keys

Do not add secrets with ENV or ARG or keys with COPY as they can be leaked
Instead, mount secrets and keys. ```Dockerfile RUN --mount=type=secret,required,id=aws,target=.aws/credentials \ ./get_model_from_s3.sh

RUN --mount=type=ssh,required \ git clone git@github.com:{org}/{repo} ```

Compose with Multi-Stage

Specify which target you are building from the Dockerfile

yml
# docker-compose.yml
services:
  app:
    build:
      context: ./
      target: ${ENVIRON}  # dev, prod, etc.

Auto-build containers on push (GitHub Actions)

See example here: https://github.com/andrewtruong/autobuildtest

Create 3 files inside the container folder (in this case jax):

# jax/Dockerfile
FROM mambaorg/micromamba:0.11.3

COPY env.yml /root/env.yml
RUN micromamba install -y -n base -f /root/env.yml && \
    micromamba clean -a -y

# jax/build.sh
docker build -t itsandrewtruong/jax:latest -f Dockerfile .

# jax/env.yml (from conda export > env.yml)
name: jax
channels:
  - conda-forge
dependencies:
  - python>=3.7
  - jax
  - jaxlib
  # - cudnn  # not required if using nvidia-docker
  # - cudatoolkit

Define build.yml

# .github/workflows/build.yml
name: Build Containers
on:
push:
  branches:
    - master
jobs:
jax:
  name: Build docker containers
  runs-on: ubuntu-latest
  steps:
    - name: Checkout repo
      uses: actions/checkout@v2

    - name: Set up QEMU  # For ARM support (M1)
      uses: docker/setup-qemu-action@v1

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v1

    - name: Login to DockerHub
      uses: docker/login-action@v1
      with:
        username: $
        password: $

    - name: Build and push
      id: docker_build
      uses: docker/build-push-action@v2
      with:
        context: jax
        push: true
        tags: itsandrewtruong/jax:latest

    - name: Image digest
      run: echo $

Setup Credential Store

https://github.com/docker/docker-credential-helpers/releases

Useful Commands

alias dcu='sudo docker-compose up -d'                   # Start all services in background
alias dcd='sudo docker-compose down --remove-orphans'   # Destroy all services
alias dcp='sudo docker-compose ps'                      # Show port mappings
alias dcr='dcd && dcu'                                  # Restart all services in background

Check logs

sudo docker-compose logs container1 container2

Inspect containers (low level)

sudo docker inspect container

Enter shell in container

sudo docker exec -it container /bin/sh

Run from another directory

docker compose -f /path/to/docker-compose.yml up

Kompose

Convert docker-compose.yml to kubernetes files.
Compose files are easier to read and write imo

Convert `docker-compose.yml`

Optionally create helm chart with -c
Auto-generated services are not guaranteed to work.
- You may need to create ConfigMap and Secrets to fill in what would otherwise go in .env
- Databases should not be used with kompose because StatefulSet is not available atm.

SERVICE_NAME='myapp'
mkdir $SERVICE_NAME

kompose convert -o $SERVICE_NAME -c

Docker