Setting up the application on a new environment

This page will give you the necessary information to deploy the application. This guide is a setup from scratch of the stack on a server.

Requirements

You need SSH access to the server, you can ask the accesses to the RedPencil Team by providing them you public key.

To generate a key:

ssh-keygen -t rsa -b 2048 -C "[e-mail]"

Find your public key:

cat ~/.ssh/id_rsa.pub

Setup the server

Usually, we store our applications to the /data directory. Inside this repository, you should see the following folders letsencrypt and metrics already present. Those are already setup for you and the domain(s) should be already mapped to the server, so you don't have anything to do to generate the certificates or setting up the metrics. If it's not the case, please ask the Redpencil team for support.

• Clone the project: cd /data git clone https://github.com/lblod/app-organization-portal.git

• If it's not a prod environment, rename the directory cloned depending on which environment it is e.g app-organization-portal-acc for an ACC environment, app-organization-portal-dev for DEV, etc.

• Inside the cloned directory, create a new file called docker-compose.override.yml . This file should look like below.

version: '3.4'

services:
  web:
    environment:
      VIRTUAL_HOST: "organisaties.abb.lblod.info"
      LETSENCRYPT_HOST: "organisaties.abb.lblod.info"
      LETSENCRYPT_EMAIL: "[e-mail]"
    networks:
      - proxy
      - default
  frontend:
    image: lblod/frontend-organization-portal:0.3.0
    labels:
      - "logging=true"
    environment: 
     EMBER_OAUTH_API_REDIRECT_URL: "https://organisaties.abb.lblod.info/authorization/callback"
     EMBER_OAUTH_API_KEY: "677af572-2e2f-4e61-ad38-036723bb314c"
     EMBER_OAUTH_API_BASE_URL: "https://authenticatie-ti.vlaanderen.be/op/v1/auth"
     EMBER_OAUTH_API_LOGOUT_URL: "https://authenticatie-ti.vlaanderen.be/op/v1/logout"
     EMBER_OAUTH_API_SCOPE: "openid vo profile abb_organisatieportaal"
    restart: always
  login:
    environment:
      MU_APPLICATION_AUTH_CLIENT_SECRET: "secret"



networks:
  proxy:
    external:
      name: letsencrypt_default

• VIRTUAL_HOST to be changed by the target domain

• LETSENCRYPT_HOSTto be changed by the target domain

• EMBER_OAUTH_API_REDIRECT_URL should look likehttps://target-domain.com/authorization/callback

• EMBER_OAUTH_API_KEY the api key provided by acm/idm for the target domain

• MU_APPLICATION_AUTH_CLIENT_SECRET the secret key provided by acm/idm for the target domain

• frontend: image: lblod/frontend-organization-portal:0.3.0 the version of the frontend. Change it if there is a new version available

• Create a new file called .env and paste the folling to it: COMPOSE_FILE=docker-compose.yml:docker-compose.prod.yml:docker-compose.override.yml

Setup Migrations

We need to populate the database with the data extracted from the excel sheet. This step should usually be done only once. The data should be either on your machine (with the rights to do so), or in a protected approved server. The following commands show how to move the data to the new server in those two scenarios.

The data are in your machine

you must put the files in a directory called private data

• Run the following command (you may have to change the app-organization-portal directory if you don't target the prod):

rsync -e ssh -avz /path-to-data/private-data /private-data [e-mail]:/data/app-organization-portal/config/migrations

The data are in a protected server

• from your local machine, run the following command (you may have to change the app-organization-portal directory if you don't target the prod):

ssh -A [e-mail] rsync -e ssh -avz /path-to-data/private-data [e-mail]:/data/app-organization-portal/config/migrations

Setup Kalliope API

Set up credentials and IP whitelist following the steps documented on https://github.com/lblod/jsonld-delta-service

Start the graph sync after the migrations have run.

• Add this to docker-compose.override.yml:

  delta-producer-pub-graph-maintainer-administrative-units: environment: START_INITIAL_SYNC: 'true' delta-producer-bg-jobs-initiator-administrative-units: environment: START_INITIAL_SYNC: 'true' delta-producer-pub-graph-maintainer-organizations: environment: START_INITIAL_SYNC: 'true' delta-producer-bg-jobs-initiator-organizations: environment: START_INITIAL_SYNC: 'true'

• Recreate the containers with the updated config docker-compose up -d

Sanity check:

• The logs show the task being run right after recreating the containers with the updated config

• After the jobs have run (initial sync can take up to a few mins). This should return uploaded dataset as JSON-LD:

  • GET https://target-domain.com/consolidated

  • Basic-Auth:with the credentials created above, make sure the ip address of client is whitelisted.

Setup consumers

The consumers consume data from various producing sources. By default, they are disabled to avoid unexpected synchronization. To explicitly start the consuming processes (initial sync & delta sync), add the following to the docker-compose.override.yml file:

  mandatarissen-consumer:
    environment:
      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be"
      DCR_DISABLE_INITIAL_SYNC: "false"
  leidinggevenden-consumer:
    environment:
      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be"
      DCR_DISABLE_INITIAL_SYNC: "false"
  worship-services-sensitive-consumer:
    environment:
      DCR_SYNC_BASE_URL: "https://loket.lokaalbestuur.vlaanderen.be"
      DCR_SYNC_LOGIN_ENDPOINT: "https://loket.lokaalbestuur.vlaanderen.be/sync/worship-services-sensitive-deltas/login"
      DCR_SECRET_KEY: "SECRET_KEY_CONFIGURED_IN_THE_PRODUCERS_OVERRIDE"
      DCR_DISABLE_INITIAL_SYNC: "false"