README.md 1.59 KB
Newer Older
poker-root's avatar
poker-root committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# docker swarm setup for hatespeech-api

The docker setup requires:

- a local docker image registry
- NFS setup for the folder `/shared/`


## Scripts

`start_registry.sh` starts a local docker registry on the bastion node.

`build_and_push_images.sh` rebuilds docker images and pushes them to the local registry.

`start_swarm.sh` starts the containers. Only the API webservice runs on different nodes, the database, reverse proxy, sandman2 run on bastion. The default setting is 4 workers, but the script can be adaped.

`shutdown_swarm.sh` stops the containers.

`setup_workers.sh` is needed initially to integrate nodes into the swarm. The IP address are read from `api_nodes.txt`

## Debugging

Show state of all containers
```
$ docker stack ps hatespeech
```

Log files, e.g.
```
$ docker service logs hatespeech_webservice
```


## Updating Models

The ML model is deployed at `/shared/models`. After updating the models
the webservice containers must be restared using

```
$ docker service update hatespeech_webservice
```
schmittu's avatar
schmittu committed
42
43
44
45


# Add new endpoint / new model

schmittu's avatar
udpates    
schmittu committed
46
47
48
49
1. edit `config/nginx.conf` and add new endpoint (check "blick" example in this file).
2. edit `docker-compose.yml`, also adapt `MODEL_PATH`.
3. `docker stack deploy --compose-file ./docker-compose.yml hatespeech`.
4. `docker service update hatespeech_nginx`.
schmittu's avatar
schmittu committed
50
51
52
53


# Add new node

schmittu's avatar
udpates    
schmittu committed
54
55
56
57
1. create new gpu node from template in science cloud.
2. `./setup_worker.sh <IPADDRESS-GPU_NODE>`.
3. check nodes `docker node ls`.
4. to activate node rescale, e.g. `docker service scale hatespeech_default_webservice=4`. This has to be done even if the scale number is not changed!