This repo contains solutions for two tasks,
-
Deploy a docker swarm stack that would run a container with
privileged
mode. For more details check here. -
Run a ubuntu based docker container which can inturn run docker inside
(docker in docker mode)
. For more details check here.
.
├── README.md
├── docker-compose.yml
├── task-one
│ ├── Dockerfile
│ ├── README.md
│ ├── arch_diagram.png
│ ├── container-handler.sh
│ └── docker-compose.yml
└── task-two
├── Dockerfile
├── README.md
├── apparmor-profiles
│ ├── audit-all-writes
│ └── deny-all-writes
├── arch_diagram.png
├── container-handler.sh
└── docker-compose.yml
- docker-compose.yml - Used to deploy docker swarm stack with two services (task-one and task-two) in a docker swarm cluster.
- task-one
- Dockerfile - Copies container-handler script into the filesystem of the container and execute it within the container
- README.md - Contains instructions to build image and deploy docker swarm stack in swarm cluster
- arch_diagram.png - Architecture diagram for task one docker swarm stack
- container-handler.sh - Creates two sibiling containers, one with
priviliged
mode enabled. Also waits fordocker stack rm <stack_name>
signal and kills the sibiling whenSIGTERM
is received - docker-compose.yml - It has one service and uses an existing network to deploy
container-handler
container to bring up sibiling containers, one withpriviliged
mode enabled
- task-two
- Dockerfile - Copies apparmor profiles and container-handler script into the filesystem of the container and execute the script within container
- README.md - Contains instructions to build image and deploy docker swarm stack in swarm cluster
- apparmor-profiles
- audit-all-writes - This profile will audit all the writes
(i.e creating dirs/files)
happening within the container and logs it to the kernel log - deny-all-writes - This profile will deny all the writes
(i.e creating dirs/files)
happening within the container and logs it to the kernel log
- audit-all-writes - This profile will audit all the writes
- arch_diagram.png - Architecture diagram for task two docker swarm stack
- container-handler.sh - Creates one sibiling upper containers with
priviliged
mode, updates/installs apparmor packages, copies apparmor profiles to/etc/apparmo.d/
in the upper container, creates two child inner containers with apparmor profiles applied. Also waits fordocker stack rm <stack_name>
signal and kills the sibiling upper container(including child inner containers)
whenSIGTERM
is received - docker-compose.yml - It has one service and uses an existing network to deploy
container-handler
container to bring up sibiling upper container withpriviliged
mode enabled and two child inner containers
In this section, we will be deploying both the tasks task-one and task-two in swarm cluster using a single docker-compose file. We can deploy this stack to any cloud platform or even a local machine (with docker installed). For this deployment I'm going to use AWS Cloud platform.
-
Task one service image - You can either build image using Dockerfile (if you also want to use custom image name and tag, make sure to change the same in docker-compose.yml) or you can use my docker image in pramodhayyappan/kk-task-one-container-handler in dockerhub
# To build image with custom name and tag. By default, it uses the latest tag docker build -f task-one/Dockerfile task-one -t pramodhayyappan/kk-task-one-container-handler:<tag name>
-
Task two service image - You can either build image using Dockerfile (if you also want to use custom image name, make sure to change the same image name in docker-compose.yml) or you can use my docker image in pramodhayyappan/kk-task-two-container-handler in dockerhub
# To build image with custom name and tag. By default, it uses the latest tag docker build -f task-two/Dockerfile task-two -t pramodhayyappan/kk-task-two-container-handler:<tag name>
-
Assuming that you have already set up your AWS Credentials in your local machine, create a Security Group(SG) to allow communication between nodes and make note of the SG name
aws ec2 create-security-group --group-name swarm-cluster-sg --description "swarm cluster security group" --vpc-id <vpc-id>
-
Create ingress rules with the protocols and ports mentioned in the offical doc
aws ec2 authorize-security-group-ingress --group-id <sg name from previous step> --protocol tcp --port 22 --cidr <cidr ip> aws ec2 authorize-security-group-ingress --group-id <sg name from previous step> --protocol tcp --port 2377 --cidr <cidr ip> aws ec2 authorize-security-group-ingress --group-id <sg name from previous step> --protocol tcp --port 7946 --cidr <cidr ip> aws ec2 authorize-security-group-ingress --group-id <sg name from previous step> --protocol udp --port 7946 --cidr <cidr ip> aws ec2 authorize-security-group-ingress --group-id <sg name from previous step> --protocol udp --port 4789 --cidr <cidr ip>
-
Create
user-data.sh
script file with below content#!/bin/bash sudo apt-get update sudo apt-get install \ ca-certificates \ curl \ gnupg \ lsb-release -y sudo mkdir -p /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin git -y sudo groupadd docker sudo usermod -aG docker $USER newgrp docker
-
Create two EC2 instances for Docker Swarm cluster
aws ec2 run-instances --image-id <ami-id> --count 1 --instance-type t2.xlarge --key-name <key-pair-name> --security-group-ids <sg name from first step> --subnet-id <subnet-id> --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=ManagerNode}]' --user-data file://user-data.sh aws ec2 run-instances --image-id <ami-id> --count 1 --instance-type t2.xlarge --key-name <key-pair-name> --security-group-ids <sg name from first step> --subnet-id <subnet-id> --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=WorkerNode1}]' --user-data file://user-data.sh
-
The infrastructure for docker swarm cluster with docker installed is ready
-
Create a swarm cluster by executing the below command in manager node
docker swarm init --advertise-addr <MANAGER-IP>
-
Join worker node to the cluster by using the output of the init command or use
join-token
to generate the join command. The command will look likedocker swarm join-token worker docker swarm join --token <token> <ip>:<port>
-
Clone this repo to the manager node
git clone https://github.com/pa/kodekloud-assignment.git cd kodekloud-assignment
-
Deploy docker stack
docker stack deploy --compose-file docker-compose.yml <stack-name>
-
Some useful commands to list stack, services, container and inspect container
# To list docker stacks docker stack ls # To list services deployed by stack docker stack services <stack-name> # To list docker containers docker ps # to inspect docker container docker inspect <container id or name> # runs a new command on a running container docker exec -it <container id or name> <bash or shell> # to remove a docker stack docker stack rm <stack-name>
Demonstration and Testing of docker stack deployment