Using Firely Server on Docker

We have created a Docker image for Firely Server, so you can run the server in any environment that supports Docker. For this section we assume you have Docker installed on your system. If you want to install Docker for Windows, please read Installing Docker For Windows for specific installation details.

Getting started

Before you can run Firely Server, you will need to pull the Docker Firely Server container and request a license.

  1. Open your favorite command line tool and execute this command: > docker pull firely/server

  2. Sign up for an evaluation license. You will receive an email with the license file as well as a link to the download page were you can download the Firely Server binaries with the version of your choosing.

  3. Create a working directory for Firely Server and place the license file there.

Warning

If you use Docker, you may want to run multiple instances of Firely Server (e.g. with Kubernetes). Read Running imports with multiple instances for caveats with the Administration endpoint.

Note

The firely/server image runs Firely Server as the non-root user firely, with user id 1000 and group id 2000. The working directory /app is owned by that user. This has a few consequences:

  • Files that you mount into the container, such as the license file and the *.instance.json settings files, must be readable by user id 1000.

  • Folders that Firely Server writes to, such as the folder with the SQLite database files, the log folder and the folder for Bulk Data Export files, must be writable by user id 1000. With a bind mount of a host folder, set its owner or permissions accordingly, for example with chown -R 1000:2000 ./resourcedata.

  • Depending on the container runtime, a non-root user may not be allowed to listen on a port below 1024. Keep Hosting:HttpPort at its default 4080 and map it to another port on the host if needed, for example with -p 80:4080.

  • The image sets COMPlus_EnableDiagnostics=0, which disables the .NET diagnostics port. Tools like dotnet-counters, dotnet-trace and dotnet-dump cannot attach to Firely Server in the container. Use the OpenTelemetry metrics and traces instead.

The Docker image of Firely Server Ingest runs as the same user and also disables the diagnostics port.

Note

Since Firely Server version 4.7.0, the Docker image name has changed from simplifier/vonk to firely/server. The old image name will be maintained for a few months to allow for a smooth transition. When updating to version 4.7.0, you should start to use the new image name. Versions 4.6.2 and older will stay available (only) on ‘simplifier/vonk’.

Running a Docker Firely Server in SQLite mode

The easiest and the default way to run a Docker Firely Server container is to run Firely Server in SQLite repository mode. Note that this is not the most performant mode - see MongoDB and SQL Server options below for that.

Open your command prompt and execute this command: > docker images firely/server

You will get a list that looks like:

../../_images/docker1.png

Navigate to your working directory for Firely Server and run the container with this command:

  • in cmd.exe: docker run -d -p 8080:4080 --name firely.server -v %CD%/firelyserver-license.json:/app/firelyserver-license.json firely/server

  • in Powershell / Bash on macOS: docker run -d -p 8080:4080 --name firely.server -v ${PWD}/firelyserver-license.json:/app/firelyserver-license.json firely/server

If your license file has a different name, use that name instead of firelyserver-license on the left side of the -v parameter in the command above. E.g. -v ${PWD}/my-fancy-license.json:/app/firelyserver-license.json

This will spin up a Firely Server container. It maps the host port 8080 to the container port 4080 with the switch -p 8080:4080. It will give the container the name firely.server with the switch --name firely.server. Furthermore it mounts the local licensefile into the /app directory, which is the directory where Firely Server resides in the container. Finally it will run the container in background mode with the switch -d.

To test whether the container is running correctly, type the command:
> docker ps

../../_images/docker2.png

You can also take a look at the logs for Firely Server with:
> docker logs firely.server

Open a browser and use the address http://localhost:8080/. This will show the landing page of Firely Server.

To stop the container just type:
> docker stop firely.server
And to start it again:
> docker start firely.server
To completely remove the container:
> docker rm firely.server

Adjust settings when running in a Docker container

In the paragraph above we showed how to use -v to mount the license file into the app directory. We can use the same technique to mount a custom settings file or logsettings file. Or a directory to host the SQLite database outside of the container. Or have the logfile written to your working directory. Below is a full example using PowerShell. For creating appsettings.instance.json refer to Adjusting Firely Server settings. For creating logsettings.instance.json refer to Log settings.

appsettings.instance.json

{
   "SQLiteDbOptions": {
      "ConnectionString": "Data Source=./resourcedata/vonkdata.db;Cache=Shared",
      "AutoUpdateDatabase": true
   }
}

logsettings.instance.json

Settings for console and levels not included for brevity.

{
   "Serilog": {
      "WriteTo": [
         {
            "Name": "Async",
            "Args": {
               "configure": [
                  {
                     "Name": "File",
                     "Args": {
                         "path": "./log/vonk.log",
                         "rollingInterval": "Day",
                         "fileSizeLimitBytes": "",
                         "retainedFileCountLimit": "7",
                         "outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} {UserId} {Username} [{Application}] [{Level}] [Machine: {MachineName}] [ReqId: {RequestId}] {Message}{NewLine}{Exception}",
                         "restrictedToMinimumLevel": "Information"
                   }
                }
             ]
           }
         },
      ],
      "Enrich": [ "FromLogContext", "WithMachineName", "WithThreadId" ],
      "Properties": {
         "Application": "Firely Server",
         "Environment": "Default"
      }
   }
}

Powershell

mkdir logs
mkdir resourcedata //do not use 'data' - the administration database is already in that folder in the container
//create the appsettings.instance.json above
//create the logsettings.instance.json above

docker run -d -p 8080:4080 --name firely.server `
-v ${PWD}/firelyserver-license.json:/app/firelyserver-license.json `
-v ${PWD}/appsettings.instance.json:/app/appsettings.instance.json `
-v ${PWD}/logsettings.instance.json:/app/logsettings.instance.json `
-v ${PWD}/resourcedata:/app/resourcedata `
-v ${PWD}/log:/app/log `
firely/server

You should see a vonkdata.db appear in the ./resourcedata folder, and a log file in the ./log folder. From here you can experiment with other settings. You can also easily keep different settings files side-by-side, mapping the one you want to test into the container, e.g. -v ${PWD}/some-weird-settings.json:/app/appsettings.instance.json.

Mounting a folder instead of a file

In some environments, e.g. Azure Webapps, it is not possible to mount an individual file into the Firely Server folder. Instead, you can only mount a whole folder. In that case, refer to Providing settings in a different folder to see how you can provide an alternative folder to read the *.instance.json files from.

Loading additional conformance resources in Firely Server on Docker

If you want to load custom conformance resources in Firely Server, you can use the vonk-import folder, as is described in the section Managing Conformance Resources. When running Firely Server on Docker, it is necessary to create this folder in your working directory, copy the conformance resources into this folder and mount a volume:

docker run -d -p 8080:4080 --name firely.server `
-v ${PWD}/appsettings.instance.json:/app/appsettings.instance.json `
-v ${PWD}/vonk-import:/app/vonk-import `
firely/server

Be sure to mount your appsettings.instance.json as well, and make sure to point the AdministrationImportOptions to the vonk-import folder:

"AdministrationImportOptions": {
      "ImportDirectory": "./vonk-import"
}

By default, Firely server will assume the resources placed in the vonk-import folder are R3 resources. If you want to load R4 or R5 resources, you need to alter the name of your import folder and volume accordingly:

docker run -d -p 8080:4080 --name firely.server `
-v ${PWD}/appsettings.instance.json:/app/appsettings.instance.json `
-v ${PWD}/vonk-import.R4:/app/vonk-import.R4 `
firely/server

You can leave the AdministrationImportOptions in the appsettings.instance.json as is, there is no need to point these settings to a separate vonk-import.R4 or vonk-import.R5 folder.

Mounting your custom plugins folder into the container

In a similar way as described above, you can mount your custom plugins into the container. In your working directory, create a folder ‘plugins’, copy your plugin files to this folder and mount this folder in the docker container. You can add a folder structure inside the plugin folder if you want.

Warning

Do not change the target folder to ‘/app/plugins’ as it will overwrite the existing plugins folder in the docker image.

docker run -d -p 8080:4080 --name firely.server `
-v ${PWD}/firelyserver-license.json:/app/firelyserver-license.json `
-v ${PWD}/plugins:/app/plugins/custom `
firely/server

The server is now accessible on http://localhost:8080/.

Spinning up with a docker-compose file

Another way to spin up a Firely Server container is to use a docker-compose file. The above example can also be established by the following docker-compose-sqlite.yml:

 1services:
 2
 3  vonk-web:
 4    image: firely/server
 5    ports:
 6      - "8080:4080"
 7    environment:
 8      - VONK_Repository=SQLite
 9      - VONK_Administration:Repository=SQLite
10      - VONK_License:LicenseFile=./license/firelyserver-license.json
11    volumes:
12      - .:/app/license

Save the text above to a file in your working directory with the name docker-compose.sqlite.yml and then run the following command:
> docker-compose -f docker-compose.sqlite.yml up -d

If your license file has a different name, use that name instead of firelyserver-license in the text above - but make sure to keep ./license as that maps to a Docker volume inside the container.

../../_images/docker3.png

To stop the container, run:
> docker-compose -f docker-compose.sqlite.yml down

Note

Strictly the settings for VONK_Repository and VONK_Administration are not needed here, since SQLite is the default setting. The settings are included to show where to configure the type of database to use. Much more information on that topic is in the paragraphs below.

Note

If you want to use a healthcheck in your docker-compose file, see Liveness and readiness - $liveness, $readiness.

When using the healthchecks in a docker-compose file, the $ in $liveliness or $readiness must be escaped with another $.

1healthcheck:
2  test: ["CMD", "curl", "-f", "http://localhost:4080/$$readiness"]

Running Docker with a SQL Server container

Firely Server supports multiple repository backends such as SQL Server, SQLite, and MongoDB. This section describes how to spin up a Firely Server container and a SQL Server container. We will use docker-compose to achieve this.

Warning

SQL Server container uses at least 3.25 GB of RAM. Make sure to assign enough memory to the Docker VM if you’re running on Docker for Mac or Windows.

Warning

If you also run the Administration database on SQL Server, please read Retain the import history.

 1services:
 2  vonk-web:
 3    image: firely/server
 4    ports:
 5      - "8080:4080"
 6    depends_on:
 7      - vonk-sqlserver-db
 8    environment:
 9      - VONK_Repository=SQL
10      - VONK_SqlDbOptions:ConnectionString=Initial Catalog=VonkData;Data Source=vonk-sqlserver-db,1433;User ID=sa;Password=SQLServerStrong(!)Password*
11      - VONK_SqlDbOptions:AutoUpdateDatabase=true
12      - VONK_SqlDbOptions:AutoUpdateConnectionString=Initial Catalog=VonkData;Data Source=vonk-sqlserver-db,1433;User ID=sa;Password=SQLServerStrong(!)Password*
13      - VONK_Administration:Repository=SQL
14      - VONK_Administration:SqlDbOptions:ConnectionString=Initial Catalog=VonkAdmin;Data Source=vonk-sqlserver-db,1433;User ID=sa;Password=SQLServerStrong(!)Password*
15      - VONK_Administration:SqlDbOptions:AutoUpdateDatabase=true
16      - VONK_Administration:SqlDbOptions:AutoUpdateConnectionString=Initial Catalog=VonkAdmin;Data Source=vonk-sqlserver-db,1433;User ID=sa;Password=SQLServerStrong(!)Password*
17      - VONK_License:LicenseFile=./license/firelyserver-license.json
18    volumes:
19      - .:/app/license
20
21  vonk-sqlserver-db:
22    image: mcr.microsoft.com/mssql/server
23    ports:
24      - "1433:1433"
25    environment:
26      - ACCEPT_EULA=Y
27      - SA_PASSWORD=SQLServerStrong(!)Password*
28    healthcheck:
29      test: /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P 'SQLServerStrong(!)Password*' -Q 'SELECT 1 FROM VonkData.sys.tables'
30      interval: 1m30s
31      timeout: 10s
32      retries: 3

Save the text above to a file in your working directory with the name docker-compose.mssqlserver.yml. Make sure your Firely Server license file is named firelyserver-license.json and is residing in your working directory (see Getting started on how to obtain the license), not in a subdirectory named license (that is an internal directory inside the container). If your license file has a different name, use that name instead of firelyserver-license in the text above.

Warning

Without specifying MSSQL_PID, the server will run SQL Server Developer Edition. For production, you are responsible for obtaining the appropriate license (and setting the value accordingly).
See https://hub.docker.com/_/microsoft-mssql-server for more information.

Note

The current configuration will use SQL Server 2019. There are other versions of SQL server available, which can be selected by adding a tag to the value of vonk-sqlserver-db:image (e.g. mcr.microsoft.com/mssql/server:2022-latest) in the yml-file.
We have successfully tested 2017-latest, 2019-latest and 2022-latest.

Then use this command to spin up a Firely Server container and SQL container:
> docker-compose -f docker-compose.mssqlserver.yml up -d

Open a browser and use the address http://localhost:8080/. This will show the landing page of Firely Server.

Warning

Wait for about 2 minutes, because it takes a while to fire up the SQL container

Running Docker with a SQL Server on host

Another possibility is to run a Firely Server container with a SQL Server repository on the host. You will need a Microsoft SQL Server running on your host. The version of SQL Server must at least be version 2012.

Warning

If you also run the Administration database on SQL Server, please read Retain the import history.

To run the Firely Server container we will use the following docker-compose file:

 1services:
 2
 3  vonk-web:
 4    image: firely/server
 5    ports:
 6      - "8080:4080"
 7    environment:
 8      - VONK_Repository=SQL
 9      - VONK_SqlDbOptions:ConnectionString=Database=VonkData;Server=my_host\<myInstanceName>;User ID=<myUser>;Password=<myPassword>
10      - VONK_SqlDbOptions:AutoUpdateDatabase=true
11      - VONK_SqlDbOptions:AutoUpdateConnectionString=Database=VonkData;Server=my_host\<myInstanceName>;User ID=<DLLUser>;Password=<myPassword>
12      - VONK_Administration:Repository=SQL
13      - VONK_Administration:SqlDbOptions:ConnectionString=Database=VonkAdmin;Server=my_host\<myInstanceName>;User ID=<myUser>;Password=<myPassword>
14      - VONK_Administration:SqlDbOptions:AutoUpdateDatabase=true
15      - VONK_Administration:SqlDbOptions:AutoUpdateConnectionString=Database=VonkAdmin;Server=my_host\<myInstanceName>;User ID=<DLLUser>;Password=<myPassword>
16      - VONK_License:LicenseFile=./license/firelyserver-license.json
17    volumes:
18      - .:/app/license
19    extra_hosts:
20      - "my_host:192.0.2.1"

Save the text above to a file in your working directory with the name docker-compose.mssqlserver_host.yml. Before we spin up the container we have to adjust the docker-compose.mssqlserver_host.yml:

  • On lines 11, 14, 16 and 19 the connection string to the database server is stated. Change the Server to your database server and instance name.

  • Also change the User ID and Password on lines 11, 14, 16 and 19 to your credentials.

  • Furthermore we have to tell Docker which IP address the host uses. This is done on line 24. In this case the host (named my_host) uses IP address 192.0.2.1. Change this to the appropriate address.

After saving your settings, make sure your Firely Server license file is named firelyserver-license.json and is residing in your working directory (see Getting started on how to obtain the license), not in a subdirectory named license (that is an internal directory inside the container). Or use the name of your license file instead of firelyserver-license in the text above.

You can run the Firely Server container as follows:
> docker-compose -f docker-compose.mssqlserver_host.yml up -d

A database will automatically be created if is not already present on the database server. See this page for an overview of permissions the database user needs for creating the database and/or schema.

Open a browser and use the address http://localhost:8080/. This will show the landing page of Firely Server.

Warning

When you have a firewall installed on your host machine, it can block traffic from your Firely Server container to your host. Provide an inbound rule to allow traffic from the container to the host.

Run Docker with a MongoDB container

This section describes how to spin up a Firely Server container and a MongoDB container using a docker-compose. We assume you already have MongoDB installed.

Warning

If you also run the Administration database on MongoDb, please read Retain the import history.

 1services:
 2
 3  vonk-web:
 4    image: firely/server
 5    environment:
 6      - VONK_Repository=MongoDb
 7      - VONK_MongoDbOptions:ConnectionString=mongodb://vonk-mongo-db/vonkdata
 8      - VONK_MongoDbOptions:EntryCollection=vonkentries
 9      - VONK_Administration:Repository=MongoDb
10      - VONK_Administration:MongoDbOptions:ConnectionString=mongodb://vonk-mongo-db/vonkadmin
11      - VONK_Administration:MongoDbOptions:EntryCollection=vonkentries
12      - VONK_License:LicenseFile=./license/firelyserver-license.json
13    volumes:
14      - .:/app/license
15    ports:
16      - "8080:4080"
17
18  vonk-mongo-db:
19    image: mongo

Save the text above to a file in your working directory with the name docker-compose.mongodb.yml. Make sure your Firely Server license file is named firelyserver-license.json and is residing in your working directory (see Getting started on how to obtain the license), not in a subdirectory named license (that is an internal directory inside the container). If your license file has a different name, use that name instead of firelyserver-license in the text above.

Use this command to spin up a Firely Server container and MongoDB container:
> docker-compose -f docker-compose.mongodb.yml up -d

Open a browser and use the address http://localhost:8080/. This will show the landing page of Firely Server.

Providing license via an environment variable

The examples above demonstrate how to provide a license by mounting a license file from the host’s filesystem to the filesystem within a container. Starting from Firely Server v4.7.0, the license can also be provided as a string via an environment variable named VONK_License:LicenseString. This is meant to simplify deployments of Firely Server within Docker as you don’t have to store the license file on the host’s filesystem anymore.

The value of that variable should contain the same text as the license file but all in one line.

The example below shows how to spin up a Docker container by supplying the license as a variable.

docker run -d \
  -p 8080:4080 \
  --name firely.server \
  -e "VONK_License:LicenseString={ 'LicenseOptions': { 'Kind': 'Production', 'ValidUntil': '2022-10-30', 'Licensee': 'example@fire.ly', 'Plugins': [ ... ] }, 'Signature': '...' }" \
  firely/server

If you use docker-compose, you can specify the variable in you docker-compose file like this:

1services:
2
3  vonk-web:
4    image: firely/server
5    ports:
6      - "8080:4080"
7    environment:
8      - "VONK_License:LicenseString={ 'LicenseOptions': { 'Kind': 'Production', 'ValidUntil': '2022-10-30', 'Licensee': 'example@fire.ly', 'Plugins': [ ... ] }, 'Signature': '...' }"

Starting Firely Server with environment variables

The examples above set environment variables one by one, with -e or in the environment section of a docker-compose file. If you have all variables in a file, e.g. firely-server.env exported by the Guided Setup, you can pass the whole file to Docker instead. See Providing environment variables in a file for the format of the file.

Use --env-file:

docker run -d -p 8080:4080 --name firely.server \
  --env-file ./firely-server.env \
  -v ${PWD}/firelyserver-license.json:/app/firelyserver-license.json \
  firely/server

docker run --env-file takes each line literally: quotes are not removed and $ is not expanded. So do not put quotes around the values.

Restarting after a change to the env file

Docker reads the env file only when the container is created. docker restart restarts the existing container with the old variables, so a changed env file is not applied that way.

To apply a changed firely-server.env, remove the container and run it again with the same command:

docker rm -f firely.server
docker run -d -p 8080:4080 --name firely.server \
  --env-file ./firely-server.env \
  -v ${PWD}/firelyserver-license.json:/app/firelyserver-license.json \
  firely/server

To check which variables the new container got, run docker exec firely.server printenv. Be aware that this prints secrets, such as connection strings, to your console.

Use env_file:

 1services:
 2
 3  vonk-web:
 4    image: firely/server
 5    ports:
 6      - "8080:4080"
 7    env_file:
 8      - ./firely-server.env
 9    volumes:
10      - ./firelyserver-license.json:/app/firelyserver-license.json

You can still add an environment section next to env_file. If a variable is in both, the value in environment wins.

Note

Docker Compose applies variable interpolation to the values in an env_file. If a value contains a $ (e.g. in a password), write it as $$, or put the value in single quotes. Only Docker Compose removes these escapes: docker run --env-file, Kubernetes and the start scripts on the other deployment pages would keep $$ or the quotes as part of the value. So a file with such escapes can only be used with Docker Compose.

Restarting after a change to the env file

Docker reads the env file only when the container is created. docker compose restart restarts the existing container with the old variables, so a changed env file is not applied that way.

To apply a changed firely-server.env, recreate the service:

docker compose -f docker-compose.yml up -d --force-recreate vonk-web

To check which variables the new container got, run docker compose -f docker-compose.yml exec vonk-web printenv. Be aware that this prints secrets, such as connection strings, to your console.

Removing or recreating the container does not remove data that you stored outside of it, such as a mounted resourcedata folder or a SQL Server or MongoDB database. Data stored only inside the container (e.g. a SQLite database that is not on a mounted volume) is lost when the container is removed.