bee42 solutions Logo
☰ ×

Rethink IT

Wir verbessern Deine Systeme mit Begeisterung.

Trainings Consulting

Dockerized the Apple Silicon

Before christmas 2020, a lot of people were working on porting their software to the new Mac Apple M1 Silicon family. More and more people are buying one:) It’s a great adventure, and the new ARM processor is an amazing piece of hardware. For me it’s the best Mac ever. The Rosetta 2 Intel CPU emulation made it possible to bootstrap the ARM based world or better a Multi Architecture world! The new Apple Virtualization Framework allows to create lighter Linux VMs. Docker delivered the first Docker for Desktop M1 Preview on December 10 and since December 17, 2020 Parallels® Desktop 16 for M1 Mac Technical Preview is available.

In my previous blog posts, I have containerized and kuberized the Silicon successfully with the vftool to create Ubuntu VMs. Its a bit hacky but useful! Now it is time to dockerize the Silicon with more comfort.



Install the Docker for Desktop

Setup the Docker for Desktop M1 Preview

Before you can start your Docker Container adventure, you must be an accepted part of the Docker Developer Preview programme :) A really nice and helpful community. Your are welcome.

Find the link in the Docker slack channel or download the Docker for Desktop M1 Preview 7 and let it run.



Docker for Desktop Preference

$ docker version
Client:
 Cloud integration: 1.0.2
 Version:           20.10.0
 API version:       1.41
 Go version:        devel +37a32a1833 Thu Dec 3 21:34:39 2020 +0000
 Git commit:        7287ab3890
 Built:             Wed Dec  9 06:23:12 2020
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.0
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       eeddea2
  Built:            Tue Dec  8 19:01:40 2020
  OS/Arch:          linux/arm64
  Experimental:     true
 containerd:
  Version:          v1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0



Yes, Docker did it: Dockerized the Silicon successfully!

Features are:

  • ARM ready Docker Engine 20.10 and CLI Tools are available
  • Docker for Desktop 3 UI is available
  • Auto update isn’t working
  • There are some trouble with internal host DNS
    • Kubernetes needs this!
  • Filesystem performance can be better
  • QEMU is available
    • Some amd64 binaries do not work!
    • Porting to ARM or fixes are on the way.
  • The new hub-tool is available in Preview 7
  • Works with Big Sur 11.1

Many thanks to the Docker Team for working really well. Great job!

It’s time to move on multi architecture container images

Today the Docker Community is well prepared for this challenge with multiple architectures. Don`t forget that the Hypriot Pirates made this possible. Their HypriotOS on Raspberry PIs at the beginning of 2015 was a big first step to move us in this ARM direction. Many thanks Dieter, Stefan, Govinda, Mathias and Andreas for their passion and Lukas for making Kubernetes available on Raspberry PIs.

Today, most ARM based Linux distributions support Docker and Kubernetes. It’s easy to make containers available on tiny embedded machines, hybrid clustering is container business as usual. See the new simple faasd raspberry PI edition that is based on containerd, runc and CNI. All container images supported by Docker are multi architectures that are available on the Docker Hub. The modern registries support the Image Distribution Specification. However, many open source projects and products lag behind these possibilities. Please accept that AMD64 is not enough anymore. Much more computing on the planet is based on ARM. Microsoft also announced on December 18, 2020 that they are working on a new Surface Chip based on ARM.

I hope this tiny example below motivates you to better create your software with multi arch support. Trust me, getting software up and running on lighter machines is amazing. Smarter solutions with fewer power consumption is the new sustainable challenge for the next generation of IT on planet earth.



Docker Developer Experience at the Silicon

It only takes a few steps to create Multi Arch Images or binaries with golang:

  1. Create a tiny Hello World Golang example
$ cat >hello-world.go <<EOF
package main

import (
	"fmt"
	"runtime"
)

func main() {
	fmt.Println("Hello, world!")
	fmt.Println(
"
                        /¯¯\
                        \__/ 
                /¯¯\/¯¯\/¯¯\
                \__/\__/\__/ 
            /¯¯\/¯¯\/¯¯\/¯¯\/¯¯\
bee silicon \__/\__/\__/\__/\__/ on", runtime.GOOS , "-", runtime.GOARCH )
}
EOF
  1. Create a Multi Stage and Multi Arch Dockerfile to create the image:

The Docker OS supports QEMU and golang can cross compile. Use the Image Builder Pattern to split creation from runtime images.

$ cat >Dockerfile <<EOF
# compile it
FROM golang:1.15 as builder
ARG TARGETOS
ARG TARGETARCH
WORKDIR /go/src/github.com/rossbachp/multiarch-example
COPY hello-world.go .
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH go build -o hello-world .

# assemble the resulting image
FROM alpine:3.12 as prod
ARG BUILD_DATE
ARG BUILD_REVISION
LABEL org.opencontainers.image.authors="Peter Rossbach"
LABEL org.opencontainers.image.source=https://github.com/rossbachp/multiarch-example.git
LABEL org.opencontainers.image.licenses=Apache-2.0
# copy the binary from the `build-stage`
COPY --from=builder /go/src/github.com/rossbachp/multiarch-example/hello-world /bin
CMD /bin/hello-world
LABEL org.opencontainers.image.created="${BUILD_DATE}"
LABEL org.opencontainers.image.revision="${BUILD_REVISION}"
EOF
  1. Setup a Multi Arch builder
$ docker buildx create --name builder
$ docker buildx inspect builder --bootstrap
$ docker buildx use builder
  1. Create the Multi Arch images and push it to the Docker Hub.

Only a single command is required to create the four images, create the references multi arch image manifest and push all five artifacts to the Docker Hub.

$ docker login --username <xxx> --password <yyy>
$ docker --context default buildx build \
	--platform linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6 \
	--push --tag rossbachp/multiarch-example:edge .
  1. Let it run locally
$ docker run --rm -ti \
  rossbachp/multiarch-example:edge
Hello, world!

                        /¯¯\
                        \__/ 
                /¯¯\/¯¯\/¯¯\
                \__/\__/\__/ 
            /¯¯\/¯¯\/¯¯\/¯¯\/¯¯\
bee silicon \__/\__/\__/\__/\__/ on linux - arm64
$ docker inspect --format “{{.Architecture}}” “rossbachp/multiarch-examples:edge”
arm64

You also give the other platform images a chance to run it:

$ docker run --rm -ti --pull always --platform linux/amd64 \
  rossbachp/multiarch-example:edge
Hello, world!

                        /¯¯\
                        \__/ 
                /¯¯\/¯¯\/¯¯\
                \__/\__/\__/ 
            /¯¯\/¯¯\/¯¯\/¯¯\/¯¯\
bee silicon \__/\__/\__/\__/\__/ on linux - amd64
$ docker inspect --format “{{.Architecture}}” “rossbachp/multiarch-examples:edge”
amd64

It’s that simple, give it a try or send me a nicer asciiart version…

          /¯¯\
       /¯¯\__/
    /¯¯\__/¯¯\
    \__/¯¯\__/
/¯¯\/¯¯\__/¯¯\/¯¯\
\__/\__/  \__/\__/ bee42 solutions ...

You can find my project on github:

Automated the buildx workflow with Github Actions

An important next step is to build these images fully automatically. It takes time and uses local resources. Sometimes the QEMU emulation does not work well or it requires testing on real hardware. These setups must be well prepared. Use it in a cloud service and connect your special hardware to external runners. Create your Apple Silicon at Macstadium to test it or better buy one :innocent:

Most people today used Github to manage their source code and make the results available. With Github Actions you can automate easily the image building, testing and deployment.

Create a workflow file .github/workflows/release.yml to release your image. Check out my tiny Hello World project for more workflows to trigger after commit and nightly builds.

  • Only start publishing in Docker Hub workflow after tagging
  • Prepare the repository
    • Get the Docker Hub Access Token
    • Add the Docker Hub credentials to your github projects secrets settings.
    • Choose the right repository name
    • Login to the Docker Hub
  • Activate buildx and QEMU
  • Prepare for image caching
  • Build the multi arch and push it to the Docker Hub
name: Publish Releases to Hub

# When it's time to do a release do a full cross platform build for all supported
# architectures and push all of them to Docker Hub.
# Only trigger on semver shaped tags.
on:
  push:
    tags:
      - "v*.*.*"

jobs:
  buildx:
    runs-on: ubuntu-20.04
    steps:
      -
        name: Checkout
        uses: actions/checkout@v2
      - 
        uses: FranzDiebold/github-env-vars-action@v2
      - 
        name: Prepare
        id: prep
        run: |
          DOCKER_IMAGE=rossbachp/${{ github.event.repository.name }}
          VERSION=edge
          if [[ $GITHUB_REF == refs/tags/* ]]; then
            VERSION=${GITHUB_REF#refs/tags/v}
          fi
          if [ "${{ github.event_name }}" = "schedule" ]; then
            VERSION=nightly
          fi
          TAGS="${DOCKER_IMAGE}:${VERSION}"
          if [[ $VERSION =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
            TAGS="$TAGS,${DOCKER_IMAGE}:latest,${DOCKER_IMAGE}:${{ env.CI_SHA_SHORT }}"
          fi
          echo ::set-output name=tags::${TAGS}
      -
        name: Set up QEMU
        uses: docker/setup-qemu-action@v1
        with:
          platforms: all
      -
        name: Set up Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@v1
        with:
          install: true
      - 
        name: Cache Docker layers
        uses: actions/cache@v2
        with:
          path: /tmp/.buildx-cache
          key: ${{ runner.os }}-buildx-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-buildx-
      -
        name: Builder instance name
        run: echo ${{ steps.buildx.outputs.name }}
      -
        name: Available platforms
        run: echo ${{ steps.buildx.outputs.platforms }}
      -
        name: Login to Docker Hub
        if: github.event_name != 'pull_request'
        uses: docker/login-action@v1
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}  
      -
        name: Build and push
        id: docker_build
        uses: docker/build-push-action@v2
        with:
          builder: ${{ steps.buildx.outputs.name }}
          context: .
          file: ./Dockerfile
          platforms: linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6
          build-args: |
            BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
            BUILD_REVISION=${CI_SHA_SHORT}
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.prep.outputs.tags }}
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache
      - 
        name: Image digest
        run: echo ${{ steps.docker_build.outputs.digest }}

Trigger a new release with following commands:

$ git tag -a v1.0.2
$ git push origin v1.0.2

This example is inspired by this project:

More information on GitHub Actions see the links below:

Time for one more thing!

The Docker Hub CLI: hub-tool

I like the Docker Hub image support, but now a long missing piece is available. The hub-tool is a CLI for the Docker Hub and is available in the Docker-for-Desktop 3 release.

The Docker Desktop M1 Silicon Preview 7 edition includes the hub-tool. You also can grab it from your Mac amd64 installation.

The hub-tool manages your account, organization and all artifact repositories.

# Preview 5 need a intel binary
# DOCKER_HOME=/Applications/Docker.app/Contents/Resources/bin/
# scp <big sur intel docker for desktop 3.0 host>:$DOCKER_HOME/hub-tool $DOCKER_HOME
# sudo ln -s $DOCKER_HOME/hub-tool /usr/local/bin/hub-tool
$ hub-tool --help
A tool to manage your Docker Hub images

Usage:
  hub-tool
  hub-tool [command]

Available Commands:
  account     Manage your account
  help        Help about any command
  login       Login to the Hub
  logout      Logout of the Hub
  org         Manage organizations
  repo        Manage repositories
  tag         Manage tags
  token       Manage Personal Access Tokens
  version     Version information about this tool

Flags:
  -h, --help      help for hub-tool
      --verbose   Print logs
      --version   Display the version of this tool

Use "hub-tool [command] --help" for more information about a command.
  1. Login to your account and browse to your repositories.
$ hub-tool login
$ hub-tool token ls
DESCRIPTION       UUID                                    LAST USED       CREATED     ACTIVE
Github Actions    7b725f9d-5b23-4910-9950-655af709d7fd    12 hours ago    13 hours    true
$ ibrew install jq
$ hub-tool repo ls --format json | jq -r ".[].Name"
rossbachp/multiarch-example
rossbachp/employee-service
rossbachp/docker-on-iot-keynote
rossbachp/tomcat8
rossbachp/apache-tomcat8
...
  1. You can review your multi architecture images setup:
$ hub-tool tag inspect rossbachp/multiarch-example:1.0.2
Name:      docker.io/rossbachp/multiarch-example:1.0.2
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:19559956d9b1db18289238bf9e1fda46474342fe18aaa291c3a55ee0eb6c93a3
           
Manifests: 
  Name:      docker.io/rossbachp/multiarch-example:1.0.2@sha256:cb61504997d79f522ded9c3aab42d0a528fb3484b6a0028689e9757ea6d836f8
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64
             
  Name:      docker.io/rossbachp/multiarch-example:1.0.2@sha256:a96e15d10b292e92d7a339ce2afc5491030d4cb82dd84e38bdad5a2f957a5f33
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64
             
  Name:      docker.io/rossbachp/multiarch-example:1.0.2@sha256:b8ea402a2609e86df978c18309c424c48a0c81aef5445b6944c1794af762f728
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v7
             
  Name:      docker.io/rossbachp/multiarch-example:1.0.2@sha256:55b5477742eae03d41552b423e8a6a338d8185bf9fdd80e157b1f91bddf1a7a0
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v6
  1. Or list all tags for a repository:
$ hub-tool tag ls rossbachp/multiarch-example --format json | jq -r ".[].Name"
rossbachp/multiarch-example:09570901
rossbachp/multiarch-example:edge
rossbachp/multiarch-example:1.0.0
rossbachp/multiarch-example:0d6368664fd0c5b9ba73b29d3f0984f49eb60453
rossbachp/multiarch-example:41b58bb3
rossbachp/multiarch-example:33a0c49c
rossbachp/multiarch-example:7ef709ae
rossbachp/multiarch-example:25917c0c
rossbachp/multiarch-example:b7b6f8c0
rossbachp/multiarch-example:594c69ef
rossbachp/multiarch-example:latest
rossbachp/multiarch-example:1.0.2
  1. Control your rate limits

Docker Hub Rate Limits were enforced on November 2nd. A lot of options exist to deal with the rate limit:

  • No problem, works for my tiny projects.
  • Paid for the Docker Hub support.
  • Use a image mirror or proxy.
  • Monitor the usage and wait a little bit.
  • Migrate to another registry.

With the hub-tool it#s easier to be informed:

$ hub-tool account rate-limiting
Unlimited

Summary

My experience as a developer with multi architectures began in June 1993 with NextSTEP. The simple compilation and execution challenge made me happy. With the new Apple M1 Silicon there is now the same experience for Mac and container developers.

On December 25, 1990 Sir Tim Berners-Lee published the first Nexus World Wide Web browser. Now we live most of the time Always ON and drive communication into a new dimension. I hope we all use this massive information overflow to create a more sustainable world.



Nexus the first WorldWideWeb Browser

It’s time to change a lot. Lower power consumption is possible, also on a Mac. Please, #dezombify your computing. Less is more!

Listen to Holly Cummins' amazing KubeCon keynote:

I hope you had the time to celebrate the 30th birthday of the WWW on December 25, 2020! Hope my old NeXTCube and NeXTStation start again. Belief me or not, this is more of a challenge. Cross your fingers that the new SDCard SCSI disk emulation works well!

I wish you all a Merry Christmas.

Regards and stay tuned,

Peter (pathfinder)

Tip

If you want to be notified of updates to container images, use Docker Image Update Notifier:

DIUN logo

Thank you @solidnerd for sending me the link.

References

"Um unsere Webseite für Sie optimal zu gestalten und fortlaufend verbessern zu können, verwenden wir Cookies. Durch die weitere Nutzung der Webseite stimmen Sie der Verwendung von Cookies zu. Weitere Informationen zu Cookies erhalten Sie in unserer Datenschutzerklärung."