Shaddock¶
Shaddock is a container based orchestration and deployment tool that rely on a definition model to describe and manage a distributed multi-service infrastructure.
Documentation: http://shaddock.epheo.eu
Sources: https://github.com/epheo/shaddock
Shaddock is distributed under the terms of the Apache License, Version 2.0. The full terms and conditions of this license are detailed in the LICENSE file.
Installation¶
pip install shaddock
Or, if you have virtualenvwrapper installed:
mkvirtualenv shaddock
pip install shaddock
Installing it from sources:
git clone https://github.com/epheo/shaddock
cd shaddock && sudo pip install .
Using an existing yaml definition model:
git clone https://github.com/epheo/shaddock-openstack
Play with it!
shaddock -f openstack-deployer.yml
(shaddock) ps
(shaddock) build
(shaddock) start
(shaddock) rm mysql001
The Shaddock Definiton Model¶
Your model consist in a set of files that describe your general architecture, It can be a simple Yaml few lines, a more complete Jinja2 templated multi files hierarchy or an integrated Python module that you interface with the rest of your infratsructure.
Defining a service¶
The following examples resume how you can describe a service in a cluster:
You will need at least the following options:
clusters:
- name: simple-service
images: images/testdir
services:
- name: simpleservice
image: testuser/arch_base:latest
The following possibilities are currently implemented:
clusters:
- name: complete-service
images: images/testdir
hosts: !include site01/hosts_dc01.yml
vars: !include site01/test-variables01.yml
services:
- name: test900
image: testuser/arch_base:latest
priority: 900
host: node001-socket
ports:
'4321/tcp': ('0.0.0.0', 4322)
'4325/tcp': ('0.0.0.0', 4325)
'4326/tcp': 4327
'4329/tcp': 4329
volumes:
- '/home/user1/:/mnt/vol2:rw'}
- '/var/www:/mnt/vol1:ro'}
environment:
VAR001: '{{ var_001 }}'
depends-on:
- {host: 127.0.0.1, type: http, get: '/', code: 200, useproxy: False}
command: "echo $VAR001"
Including other files¶
Shaddock allow you to split your definition model into multiple yaml files using the !include notation.
This, conbined with the templating allow you to design very complex architectures without repetitions or loosing in readabality of your model.
clusters:
- name: include-test
vars: !include site01/test-variables01.yml
images: images/testdir
services:
- name: test120
image: testuser/arch_base:latest
priority: 120
environment:
VAR001: '{{ var_001 }}'
command: "env |grep VAR001"
- name: command-test
vars:
foo: bar
images: images/testdir
services:
- name: test121
image: testuser/arch_base:latest
priority: 121
environment:
FOO_BAR: '{{ foo }}'
command: "env |grep FOO_BAR"
Using the scheduler¶
The Shaddock scheduler will ensure that all the requirements you provide are matched before starting a new service.
You can check: * A container status * If a port is open (tcp or udp) * The return code of a http GET
You can also specify the number of retry, the time to wait before 2 checks, and if the check should use the system proxy vars or not.
- {name: nova, status: stopped}
- {name: nova, port: 8774, type: tcp}
- {name: nova, port: 8774, state: down, type: tcp}
- {host: google.com, port: 8774, state: down, type: tcp}
- {name: nova, type: http, get: '/v2.0', port: 5000, code: 200}
- {host: google.com, type: http, get: '/v2.0', port: 5000, code: 200}
- {host: 127.0.0.1, type: http, get: '/', code: 200, useproxy: False }
- {name: nova, sleep: 20} # defaults to 10
- {name: nova, retry: 10} # defaults to 5
clusters:
- name: scheduler-tests
images: images/testdir
services:
- name: test500
image: testuser/arch_base:latest
priority: 500
- name: test501
image: testuser/arch_base:latest
priority: 510
command: 'sleep 100'
- name: test503
image: testuser/arch_base:latest
priority: 520
depends-on:
- {name: test500, status: stopped, retry: 10, sleep: 10}
- {name: test501, status: running}
- name: scheduler-tcp-tests
images: images/testdir
services:
- name: test504
image: testuser/netcat:latest
priority: 550
ports: {'1234/tcp': 1234}
command: ["/bin/bash", "-c", "echo 1| nc -l 1234"]
- name: test520
image: testuser/arch_base:latest
priority: 560
# depends-on:
# - {name: test504, port: 1234}
Managing multiple hosts¶
Shaddock is able to schedule your services on different hosts accros your datacenter. The only prerequirements for a host to be part of a Shaddock cluster is toi have the Docker API installed and listening on a port. You can then configure your hosts in your cluster defintion.
- name: node001-socket
api:
base_url: unix://var/run/docker.sock
version: 1.30
timeout: 50
- name: node002-tcp
url: tcp://127.0.0.1:2376
verion: 1.30
- name: node003-tls
url: tcp://127.0.0.1:2376
tls: False
cert_path: None
key_path: None
cacert_path: None
tls_verify: False
clusters:
- name: host-cluster1
images: images/testdir/
hosts: !include site01/hosts_dc01.yml
services:
- name: service202
image: testuser/arch_base:latest
priority: 210
- name: service203
image: testuser/arch_base:latest
host: node001-socket
priority: 211
- name: host-cluster2
images: images/testdir/
hosts: !include site01/hosts_dc02.yml
services:
- name: service204
image: testuser/arch_base:latest
host: node002-tcp
priority: 212
- name: service205
image: testuser/arch_base:latest
host: node003-tls
priority: 213
Using the templating functionalities¶
The model definition variables {{ your_var }} are templated using Jinja2 before being interpreted by Shaddock. You can define any variables value in the vars: section of a cluster definiton.
refs: http://jinja.pocoo.org/
clusters:
- name: jinja2-tests01
vars: !include site01/test-variables01.yml
images: images/testdir
services: |
{% for service in servicelist %}
- name: clu1-{{service}}
image: testuser/arch_base:latest
priority: 401
environment:
VAR001: {{var_001}}
command: env
{% endfor %}
- name: jinja2-tests02
vars: !include 'site01/test-variables01.yml'
images: images/testdir
services: |
{% for service in largeservicelist %}
- name: clu2-{{service}}
image: testuser/arch_base:latest
priority: 401
environment:
VAR001: {{var_001}}
command: env
{% endfor %}
Usage¶
Using the command line interface¶
usage: shdk [--version] [-v] [--log-file LOG_FILE] [-q] [-h] [--debug]
[-f SHDK_MODEL] [-d SHDK_IMGDIR] [-c SHDK_CLUSTER]
[--docker-version DOCKER_VERSION] [-i DOCKER_URL] [--boot2docker]
[--tls] [--tlscert DOCKER_CERT_PATH] [--tlskey DOCKER_KEY_PATH]
[--tlsverify DOCKER_TLS_VERIFY] [--tlscacert DOCKER_CACERT_PATH]
Shaddock shell:
optional arguments:
--version Show Shaddock's version number and exit.
-v, --verbose Increase verbosity of output. Can be repeated.
--log-file LOG_FILE Specify a file to log output. Disabled by default.
-q, --quiet Suppress output except warnings and errors.
-h, --help Show this help message and exit.
--debug Show tracebacks on errors.
-f SHDK_MODEL, --file SHDK_MODEL
Template file to use.
-d SHDK_IMGDIR, --images-dir SHDK_IMGDIR
Directory to build Docker images from.
-c SHDK_CLUSTER, --cluster SHDK_CLUSTER
The cluster to use (No value is all by default).
--docker-version DOCKER_VERSION
Docker API version number
-i DOCKER_URL, --url DOCKER_URL
Force a specific host url API.
--boot2docker Use Boot2Docker TLS conf.You should first:"eval $(sudo
docker-machine env machine_name)"
--tls Use TLS; implied by tls-verify flags.
--tlscert DOCKER_CERT_PATH
Path to TLS certificate file.
--tlskey DOCKER_KEY_PATH
Path to TLS key file.
--tlsverify DOCKER_TLS_VERIFY
Use TLS and verify the remote.
--tlscacert DOCKER_CACERT_PATH
Trust only remotes providing a certificate signed by
theCA given here.
Commands:
build Build a service
complete print bash completion command
create Create a new container
help print detailed help for another command
logs Display the logs of a container
ps Show a list of Containers.
pull Pull a container from the Docker Repository
restart Restart a container
rm Remove a container
show Show details about a container
start Start a new container
stop Stop a container
Using shaddock in a project¶
import shaddock
Examples¶
Shaddock includes many tests to demonstrate the use of Shaddock. All of the tests are in the ‘tests’ folder. It can be used as a starting point for new projects. The tests also serve as SHDK docs.
Please look in the tests folder for examples of how to define yaml files for use with SHDK. Most of the test examples are snippets of yaml, for a more complete example look at the tests in the ‘tests/model’ folder.
OpenStack¶
https://github.com/epheo/shaddock-openstack.git is a yml definition model example you can use with Shaddock to build, deploy and manage the lifecycle of an OpenStack platform from the upstream git sources.
Alternative configuration and other systems¶
Docker Machine and OS X support¶
Please use the --boot2docker
option and Shaddock will source the
environement variables set by Docker machine.
In order to set those variables you may want to
$(sudo docker-machine env machine_name)"
first.
Run the shaddock shell from a container¶
docker run --rm -i -v shaddock/tests/model/:/model \
--env DOCKER_HOST="https://<your_host>:2376" \
--env TEMPLATE_FILE=/model/service-tests.yml \
-t shaddock/shaddock
Seting up the Docker API to listen on tcp:
Contributing¶
Please use Github for any isse reporting or pull request.
Automated Testing¶
The project have it’s own CI system and use Jenkins to test and merge automaticaly.
You can contact us on freenode in the #shaddock channel.
shaddock style commandments¶
Step 0¶
- Step 1: Read pep8
- Step 2: Read pep8 again
- Step 3: Read on
Read the OpenStack Style Commandments http://docs.openstack.org/developer/hacking/
Informations¶
License¶
Shaddock is distributed under the terms of the Apache License, Version 2.0. The full terms and conditions of this license are detailed in the LICENSE file.
References¶
Docker-py API Documentation: http://docker-py.readthedocs.org/
OpenStack Official Documentation: http://docs.openstack.org/