Admin Guide Help

Quick Start Guide

Resources links:

Cafe Variome V3 (CV3 for short) is an open-source web application for data discovery on healthcare and medical data, so you can either try out our demo or deploy it onto your server. Here are the basic steps on how:

Prerequisite

To deploy a working copy of CV3, you will need:

  • An HTTP server or load balancer (Apache, Nginx, etc.)

  • Docker, Docker Compose, or a compatible container runtime

  • Active network connection

There is also a copy of example Apache configuration file in the backend repository, which can be used as a reference for setting up the reverse proxy. It is located at resources/cafevariome.conf.

Deploying with docker

We maintain a series of docker images for CV3, including the front end and the backend. They are hosted in our Docker Hub. Note that the other required services, such as vault, KeyCloak or MongoDB, are not included in the image, and you will need to provide and configure them separately.

The main Cafe Variome V3 repository contains multiple docker compose files and a series of configuration files that can be used to quickly deploy the system. Here is the content of the two relevant docker-compose files, and what they do:

services: keycloak: image: quay.io/keycloak/keycloak:23.0 deploy: restart_policy: condition: always delay: 5s window: 120s ports: - '8080:8080' depends_on: - keycloak_db command: start-dev --db mariadb --db-url-host keycloak_db --db-username keycloak_docker --db-password KeyCloakPassword1234 --http-port 8080 environment: DB_VENDOR: mariadb DB_ADDR: keycloak_db DB_PORT: 3306 DB_DATABASE: keycloak DB_USER: keycloak_docker DB_PASSWORD: KeyCloakPassword1234 KEYCLOAK_ADMIN: admin KEYCLOAK_ADMIN_PASSWORD: KeyCloakPassword1234 keycloak_db: image: mariadb:lts-jammy environment: MYSQL_ROOT_PASSWORD: KeyCloakPassword1234 MYSQL_DATABASE: keycloak MYSQL_USER: keycloak_docker MYSQL_PASSWORD: KeyCloakPassword1234 restart: always volumes: - ./data/keycloak/db:/var/lib/mysql mongodb: image: mongo:7.0.11 restart: always ports: - "27017:27017" volumes: - ./data/mongodb/db:/data/db redis: image: redis:7.4 restart: always ports: - '6379:6379' command: redis-server volumes: - ./data/redis:/data vault: image: hashicorp/vault:1.17.2 restart: always ports: - "8200:8200" cap_add: - IPC_LOCK environment: VAULT_LOCAL_CONFIG: '{"storage": {"file": {"path": "/vault/file"}}, "listener": [{"tcp": { "address": "0.0.0.0:8200", "tls_disable": true}}], "default_lease_ttl": "168h", "max_lease_ttl": "720h", "ui": true}' command: server volumes: - ./data/vault/file:/vault/file

This is the docker compose file to start the dependent services, such as MongoDB, Redis, Vault, and KeyCloak. These services are usually managed by the cloud provider if using a cloud, so use of this file may not always be necessary. If using it, this stack needs to be started first, and the credentials have to be created in advance and provided to the services. Refer to the Dependent Service Configuration for more details. Several points worth noting:

  1. The KeyCloak server is configured to start as a dev server in this stack, and the database is set to MariaDB. This is to facilitate the easy debugging and run it with minimal configuration. When using in production environment, do not use the dev mode, instead refer to Keycloak documentation on how to configure it properly.

  2. Remember to change the Keycloak admin password and the database password to something secure.

  3. The vault is using file storage backend and has TLS disabled. This is also for development environment only. When using on production server, it's recommended to use vault with high availability cluster. If this is not an option, you should at least use a production storage backend (like MySQL) and enable TLS.

  4. You may add or modify configurations, use a different version of the image, etc. as you see fit. The provided configuration is a minimal working example. As long as the services have compatible API, the version should not matter.

services: cv3-backend-admin: image: brookeslab/cv3-backend-admin:latest restart: unless-stopped network_mode: "host" depends_on: - cv3-backend-database-manager - cv3-backend-scheduler environment: VAULT_ROLE_ID: <Vault Role ID here> VAULT_SECRET_ID: <Vault Secret ID here> volumes: - ./config/backend_config.json:/app/instance_config.json - ./logs:/app/logs # cv3-backend-cli: # image: brookeslab/cv3-backend-cli:latest # restart: unless-stopped # network_mode: "host" # depends_on: # - cv3-backend-database-manager # - cv3-backend-scheduler # environment: # VAULT_ROLE_ID: <Vault Role ID here> # VAULT_SECRET_ID: <Vault Secret ID here> # volumes: # - ./config/backend_config.json:/app/instance_config.json # - ./logs:/app/logs cv3-backend-database-manager: image: brookeslab/cv3-backend-database-manager:latest restart: unless-stopped network_mode: "host" environment: VAULT_ROLE_ID: <Vault Role ID here> VAULT_SECRET_ID: <Vault Secret ID here> KEYCLOAK_CLIENT_SECRET: <Your Keycloak Client Secret here> ADMIN_EMAIL: demo@cafevariome.org ADMIN_AFFILIATION: CafeVariome volumes: - ./config/backend_config.json:/app/instance_config.json - ./logs:/app/logs # cv3-backend-exporter: # image: brookeslab/cv3-backend-exporter:latest # restart: unless-stopped # network_mode: "host" # depends_on: # - cv3-backend-database-manager # - cv3-backend-scheduler # volumes: # - ./config/backend_config.json:/app/instance_config.json # - ./logs:/app/logs cv3-backend-network: image: brookeslab/cv3-backend-network:latest restart: unless-stopped network_mode: "host" depends_on: - cv3-backend-database-manager - cv3-backend-scheduler environment: VAULT_ROLE_ID: <Vault Role ID here> VAULT_SECRET_ID: <Vault Secret ID here> volumes: - ./config/backend_config.json:/app/instance_config.json - ./logs:/app/logs # cv3-backend-nexus: # image: brookeslab/cv3-backend-nexus:latest # restart: unless-stopped # network_mode: "host" # depends_on: # - cv3-backend-database-manager # - cv3-backend-scheduler # environment: # VAULT_ROLE_ID: <Vault Role ID here> # VAULT_SECRET_ID: <Vault Secret ID here> # volumes: # - ./config/backend_config.json:/app/instance_config.json # - ./logs:/app/logs cv3-backend-query: image: brookeslab/cv3-backend-query:latest restart: unless-stopped network_mode: "host" depends_on: - cv3-backend-database-manager - cv3-backend-scheduler environment: VAULT_ROLE_ID: <Vault Role ID here> VAULT_SECRET_ID: <Vault Secret ID here> volumes: - ./config/backend_config.json:/app/instance_config.json - ./logs:/app/logs cv3-backend-query-compiler: image: brookeslab/cv3-backend-query-compiler:latest restart: unless-stopped network_mode: "host" depends_on: - cv3-backend-database-manager - cv3-backend-scheduler environment: VAULT_ROLE_ID: <Vault Role ID here> VAULT_SECRET_ID: <Vault Secret ID here> volumes: - ./config/backend_config.json:/app/instance_config.json - ./logs:/app/logs cv3-backend-query-meta: image: brookeslab/cv3-backend-query-meta:latest restart: unless-stopped network_mode: "host" depends_on: - cv3-backend-database-manager - cv3-backend-scheduler environment: VAULT_ROLE_ID: <Vault Role ID here> VAULT_SECRET_ID: <Vault Secret ID here> volumes: - ./config/backend_config.json:/app/instance_config.json - ./logs:/app/logs cv3-backend-scheduler: image: brookeslab/cv3-backend-scheduler:latest restart: unless-stopped network_mode: "host" depends_on: - cv3-backend-database-manager environment: VAULT_ROLE_ID: <Vault Role ID here> VAULT_SECRET_ID: <Vault Secret ID here> volumes: - ./config/backend_config.json:/app/instance_config.json - ./logs:/app/logs cv3-frontend-admin: image: brookeslab/cv3-frontend-admin:latest restart: unless-stopped ports: - '5080:80' volumes: - ./config/frontend_admin_config.json:/usr/share/nginx/html/assets/assets/config.json cv3-frontend-query: image: brookeslab/cv3-frontend-query:latest restart: unless-stopped ports: - '5081:80' volumes: - ./config/frontend_query_config.json:/usr/share/nginx/html/assets/assets/config.json cv3-frontend-query-meta: image: brookeslab/cv3-frontend-query-meta:latest restart: unless-stopped ports: - '5082:80' volumes: - ./config/frontend_query_meta_config.json:/usr/share/nginx/html/assets/assets/config.json nginx: image: nginx:mainline-alpine3.18 restart: unless-stopped network_mode: "host" ports: - '18080:80' volumes: - ./config/reverse_proxy.nginx.conf:/etc/nginx/nginx.conf

This is the docker compose file to start the Cafe Variome V3 backend and frontend. It does not contain any dependent services, and you may use it with or without the docker-compose.dependencies.yaml stack, as long as all the services are in place. Several points worth noting:

  1. 3 of the containers are commented out. They are the ones that common deployment would not require. For the active configurations, not all are necessary either. Refer to the document of each component to see what they do, if they are optional, and if you should use it.

  2. The cv3-backend-database-manager container has extra environment variables. These are used to initialize the system without using the CLI utility. If you already have a database initialized, you do not need to supply these variables. After the first run, you may safely remove them.

  3. The cv3-backend-exporter does not require credentials.

  4. All backend containers are started to use host network mode. This is so that they can connect to the services running on localhost. If your databases are running with a different hostname or domain (such as in a cloud environment), it's recommended not to use host network mode. If so, the reverse proxy container no longer needs to be in host network mode either.

Last modified: 31 March 2025