Modified 2020-08-10 by Andrea F. Daniele
A builder node is simply a Docker endpoint accessible through the TCP port 2375
.
Modified 2020-08-10 by Andrea F. Daniele
CI Builder nodes are implemented on AWS EC2 instances. To setup a new builder node, you need to:
Modified 2020-08-10 by Andrea F. Daniele
You can create a new instance by visiting the AWS EC2 Dashboard.
We suggest the following AMIs:
ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-arm64-server-20181120 (ami-01ac7d9c1179d7b74)
ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20191113 (ami-00a208c7cdba991ea)
ci-nodes
RSA keypairModified 2020-08-10 by Andrea F. Daniele
On the Duckietown AWS account, a key-pair called ci-nodes
holds the
public key used to login into any CI-related node.
The private key is neither shared here nor stored on AWS for obvious reasons. If you need it, ask Duckietown administrators.
Modified 2020-08-10 by Andrea F. Daniele
By assigning an AWS Elastic IP to your instance, you make sure that the IP never changes, so that we can configure other tools with static IPs pointing at each builder node.
Modified 2020-08-10 by Andrea F. Daniele
SSH into your newly created node and install Docker.
sudo apt install docker.io
Modified 2020-08-10 by Andrea F. Daniele
Builder nodes will build many Docker images over the course of a few days. If we don’t have a way to keep them clean, they will run out of space in less than a week.
For this reason, we setup two cron jobs. The first one runs every day at 2AM and removes all stopped containers and frees any unused resources (e.g. volumes). The second kicks in 1 hour later, at 3AM and removes all unused images. In both jobs, only resources that are not used for more than 24 hours are freed.
You can setup the two cronjobs by running crontab -e
and pasting the
following lines at the end of the file.
00 02 * * * docker system prune --filter until=24h --force
00 03 * * * docker image prune --filter dangling=true --filter until=24h --force
ubuntu
to the group docker
Modified 2020-08-10 by Andrea F. Daniele
Use the following command to add the user ubuntu
to the group docker
.
This will give your user access to the local Docker engine.
sudo usermod -aG docker ubuntu
You need to log out and back in for the changes to have an effect.
Modified 2020-08-10 by Andrea F. Daniele
The Master node is the one triggering builds on builder nodes. For this to happen, each builder node has to make the Docker engine available on a TCP port.
In order to enable the Docker TCP socket, open the Docker service configuration file.
sudo nano /lib/systemd/system/docker.service
Navigate to the [Service]
section of the file and find the line
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
and replace it with the following
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375 --containerd=/run/containerd/containerd.sock
This will give anybody access to your Docker engine over the internet.
Do not worry about possible intruders, we will configure the EC2 instance to
only accept connections on the port 2375
from the master node.
Save the file and reload (then restart) the Docker service with the following commands:
sudo systemctl daemon-reload
sudo service docker restart
Test that your change had an effect by executing the command
curl http://localhost:2375/version
You should get a JSON string with info about the Docker engine. If you get an error, redo this step.
Modified 2020-08-10 by Andrea F. Daniele
Navigate to the AWS EC2 Dashboard page. Then click on Instances
to the left.
Select the newly created builder node and select
Networking -> Change Security Groups
from the Actions
menu at the top of
the list.
Add the group [in]-Docker-API
to the security groups of your instance.
This allows your instance to accept connections from the Master node.