Docker Compose - Httqm's Docs

configuration options that are applied at build time

version: '3.8'
services:
  myService:
    build:
      context: './path/to/myService-7.3.1/'
    image: myService:7.3.1

other example

When using both build and image: imageName:imageTag, imageName:imageTag refers to the image built by Compose, not to the base image.

env_file (details : 1, 2)

in docker-compose.yml :

  elastic:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.14.0
    stop_grace_period: '5m'
    ports:
      - '9207:9200'
    volumes:
      - 'elastic_data:/usr/share/elasticsearch/data:rw'
      - './logs/elastic:/usr/share/elasticsearch/logs:rw'
    env_file:
      - './env/elastic.env'

in ./env/elastic.env :

The expected format is : NAME=value.

cluster.name='talend7pp'
ingest.geoip.downloader.enabled=false
discovery.type='single-node'
_JAVA_OPTIONS='-Xmx4g -Xms4g'

which gives a file tree like :
. project root

docker-compose.yml

.env

env/

service_1.env

service_2.env

service_n.env
Whatever the location, environment variables will be loaded when running docker-compose up (details).
difference between .env and env/serviceName.env files (source) :
- .env :
  - default file to store environment variables + values
  - found in the same directory than docker-compose.yml
- env/serviceName.env :
  - path must be specified :
    
    in docker-compose.yml : with env_file
    
    in CLI : with --env-file
  - path + file name env/serviceName.env is purely conventional
  - allows storing the environment file anywhere and naming it freely

environment

dictionary format :

environment:
  RACK_ENV: development
  SHOW: 'true'		quotes are necessary around boolean-likes in YAML files because of this and that
  SESSION_SECRET:

array format :

environment:
  - RACK_ENV=development
  - SHOW=true
  - SESSION_SECRET

ports

expose ports :

ports:
  - "hostPort:containerPort"

volumes

short syntax :
```
source:target:mode
```
- source :
  - volume name
  - path on the host side
    - absolute path : (explicit)
    - relative path : relative to the directory containing the docker-compose.yml file
- target : path in the container (i.e. mount point)
- mode :
  - ro
  - rw (default)
long syntax
driver : you may also see code like :
```
volumes:
  my_named_volume:
    driver: local
```
- local :
  - means the corresponding volume will be created on the same host where the container is running. This is the default setting and can be omitted in most cases (sources : 1, 2).
  - accepts options similar to the mount command when running on GNU/Linux (details)

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services. Then, with a single command, you create and start all the services from your configuration.

Setup :

make sure you have installed the Docker engine
As root :
1. find the latest Docker Compose version number
2. edit and run :
  dockerComposeVersion='v2.29.1'; localDockerComposeBinary='/usr/local/bin/docker-compose' urlGithub1="https://github.com/docker/compose/releases/download/$dockerComposeVersion/docker-compose-$(uname -s)-$(uname -m)" urlGithub2="https://raw.githubusercontent.com/docker/cli/master/contrib/completion/bash/docker" curl -L "$urlGithub1" -o "$localDockerComposeBinary" && chmod +x "$localDockerComposeBinary"; curl -L "$urlGithub2" -o /etc/bash_completion.d/docker-compose
3. if you plan to run Docker Compose as a non-root user :
  chmod 755 "$localDockerComposeBinary"
check installation :
docker-compose --version
```
Docker Compose version v2.29.1
```
If you don't get this result :
1. check the generated URLs are not broken :
```
cat << EOF
url1 :	$urlGithub1
url2 :	$urlGithub2
EOF
```
2. depending on versions, the v of dockerComposeVersion's vX.Y.Z is/is not included in the path. Try with/without.

Let's build things with it (source) :

Target : have 4 containers (based on this) : 2 web and 2 DB. We'll use these with Ansible later.

Our working environment (source) :

workDir='./dockerCompose'; mkdir -p "$workDir" && cd "$workDir"

The Dockerfile : ./workDir/Dockerfile (source) :

We'll actually re-use this one.

The Compose file : ./workDir/docker-compose.yml (source) :

version: '3.7'
services:

  ubuntu_web1:
    hostname: web1
    build: .				this service uses the image built from the Dockerfile found in this directory (details)
    container_name: c_ubuntu_web1
    tty: true

  ubuntu_web2:
    hostname: web2
    build: .
    container_name: c_ubuntu_web2
    tty: true

  ubuntu_db1:
    hostname: db1
    build: .
    container_name: c_ubuntu_db1
    tty: true

  ubuntu_db2:
    hostname: db2
    build: .
    container_name: c_ubuntu_db2
    tty: true

Build and run (source) :

docker-compose up

These 11 steps are actually repeated for each configured service :

Building ubuntu_db2
Step 1/11 : FROM ubuntu:16.04
 ---> 5e13f8dd4c1a
Step 2/11 : ARG userName='ansible'
 ---> Using cache
 ---> 7f163145edfd
Step 3/11 : ARG homeDir="/home/$userName"
 ---> Using cache
 ---> 9025b899c1df
Step 4/11 : ARG sshDir="$homeDir/.ssh"
 ---> Using cache
 ---> b32d4b7d1abf
Step 5/11 : ARG authorizedKeysFile="$sshDir/authorized_keys"
 ---> Using cache
 ---> 8477ab5bcac4
Step 6/11 : ARG publicSshKey='./ansible.pub'
 ---> Using cache
 ---> 5b2235eb9fbf
Step 7/11 : RUN apt-get update &&       apt-get install -y iproute2 iputils-ping openssh-server &&      apt-get clean &&        useradd -d "$homeDir" -s /bin/bash -m "$userName" &&    mkdir -p "$sshDir"
 ---> Using cache
 ---> 41cfeaaa7dda
Step 8/11 : COPY "$publicSshKey" "$authorizedKeysFile"		this step doesn't like it when source files are symlinks
 ---> Using cache
 ---> 3dcab2010a2b
Step 9/11 : RUN chown -R "$userName":"$userName" "$sshDir" &&   chmod 700 "$sshDir" &&  chmod 600 "$authorizedKeysFile"
 ---> Using cache
 ---> 1f511354cd72
Step 10/11 : EXPOSE 22
 ---> Using cache
 ---> deb5199624a4
Step 11/11 : CMD [ "sh", "-c", "service ssh start; bash"]
 ---> Using cache
 ---> 3bb266505c4c

Successfully built 3bb266505c4c
Successfully tagged dockercompose_ubuntu_db2:latest
WARNING: Image for service ubuntu_db2 was built because it did not already exist. To rebuild this image you must use docker-compose build or docker-compose up --build.

Then :

Creating c_ubuntu_web1 ... done
Creating c_ubuntu_db1  ... done
Creating c_ubuntu_web2 ... done
Creating c_ubuntu_db2  ... done
Attaching to c_ubuntu_db1, c_ubuntu_web1, c_ubuntu_web2, c_ubuntu_db2
c_ubuntu_db1   |  * Starting OpenBSD Secure Shell server sshd            [ OK ]
c_ubuntu_web1  |  * Starting OpenBSD Secure Shell server sshd            [ OK ]
c_ubuntu_web2  |  * Starting OpenBSD Secure Shell server sshd            [ OK ]
c_ubuntu_db2   |  * Starting OpenBSD Secure Shell server sshd            [ OK ]

(No prompt given at that time, so Ctrl-z + bg to get it back)

docker ps

CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS               NAMES
76f413236d70        dockercompose_ubuntu_db2    "sh -c 'service ssh …"   3 minutes ago      Up 3 minutes       22/tcp              c_ubuntu_db2
240e1505313e        dockercompose_ubuntu_web2   "sh -c 'service ssh …"   3 minutes ago      Up 3 minutes       22/tcp              c_ubuntu_web2
b5b6887d2b02        dockercompose_ubuntu_db1    "sh -c 'service ssh …"   3 minutes ago      Up 3 minutes       22/tcp              c_ubuntu_db1
95f988a02ff2        dockercompose_ubuntu_web1   "sh -c 'service ssh …"   3 minutes ago      Up 3 minutes       22/tcp              c_ubuntu_web1

docker image ls

REPOSITORY                  TAG                 IMAGE ID            CREATED            SIZE
dockercompose_ubuntu_db1    latest              3bb266505c4c        4 minutes ago      205MB
dockercompose_ubuntu_db2    latest              3bb266505c4c        4 minutes ago      205MB
dockercompose_ubuntu_web1   latest              3bb266505c4c        4 minutes ago      205MB
dockercompose_ubuntu_web2   latest              3bb266505c4c        4 minutes ago      205MB

Maybe it was not necessary to build 4 distinct images that are actually identical...

Further steps :

To stop them all :
docker-compose stop
To remove all containers and all images (source) :
docker rm $(docker ps -a -q) && docker rmi $(docker images -q) --force

--force will remove everything, including images that may not be related with your current work.
You may now alter the Dockerfile or the Compose file and rebuild :
docker-compose up --build

To start everything again AND still get the shell prompt back :

docker-compose up -d

Creating c_ubuntu_db1  ... done
Creating c_ubuntu_web1 ... done
Creating c_ubuntu_web2 ... done
Creating c_ubuntu_db2  ... done

Now let's use this "virtualized infrastructure" :

get details about a container :
- docker ps
```
CONTAINER ID	IMAGE				COMMAND			CREATED		STATUS		PORTS	NAMES
76f413236d70	dockercompose_ubuntu_db2	"sh -c 'service ssh …"	2 hours ago	Up 34 minutes	22/tcp	c_ubuntu_db2
```
- its IP address
- its name :
  docker inspect -f '{{json .Config.Hostname}}' 76f413236d70
```
"db2"
```
- IP and name :
  - for a single host :
    docker inspect --format='{{json .Config.Hostname}} {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' 76f413236d70
```
"db2" 172.18.0.5
```
  - for several hosts :
    for containerId in $(docker ps -q); do docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} {{json .Config.Hostname}}' "$containerId"; done | tr -d '"' | sort
```
172.18.0.2 db1
172.18.0.3 web2
172.18.0.4 web1
172.18.0.5 db2
```
  - to build an Ansible inventory file :
    for containerId in $(docker ps -q); do docker inspect --format='{{json .Config.Hostname}} ansible_host={{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$containerId"; done | tr -d '"' | sort
Disable recording of the SSH host key in $HOME/.ssh/known_hosts and avoid useless repeated warnings. Make sure no record matches one of the "hosts" (containers) you just created.
Open an SSH connection to any of these hosts :
ssh -i ansible ansible@$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' myContainer)
make sure Ansible can reach one of these hosts. Considering this snippet of ./inventory :
```
[mysqlGroup]
mysql	ansible_host=172.18.0.2
```
Both commands give the same result :
- ansible mysqlGroup --inventory=./inventory -m ping -u ansible
- ansible all --inventory=./inventory --limit mysql -m ping -u ansible
```
mysql | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
```

This article is about docker-compose CLI flags. For details about setup and usage, see my Docker Compose : setup & usage article.

Flag	Usage
-d --detach	run container in the background and give the shell prompt back
down
up

Docker Compose - Define and run multi-container applications

docker-compose.yml directives

difference between .env and env/serviceName.env files (source) :

Docker Compose : setup & usage