Auditing
Firely Server can log access through the RESTful API for auditing purposes. It has three main features:
Write requests and responses to a separate audit logfile.
Include user id and name from the JWT token (if present) in the audit log lines.
Write the audit information as FHIR AuditEvent resources in the Firely Server Data database.
These features can be enabled by including Vonk.Plugin.Audit in the pipeline.
"PipelineOptions": {
"PluginDirectory": "./plugins",
"Branches": [
{
"Path": "/",
"Include": [
"Vonk.Core",
...
"Vonk.Plugin.Audit"
],
...
},
...
]
}
See Configure the pipeline for more details on pipeline configuration.
At present, you can choose either to enable both file and database logging, or only database logging. To enable only database logging, replace Vonk.Plugin.Audit with Vonk.Plugin.Audit.AuditEventConfiguration. In addition, you can choose to log every call or only transaction batches. When you include a specific configuration class and want to enable username logging, you have to include Vonk.Plugin.Audit.UsernameLoggingConfiguration. Please see Auditing for the available options.
Filtering configuration
You can exclude requests from generating audit logs (both audit log file and audit event logging). This is helpful to reduce clutter in the logs. For example, you could exclude logging for an endpoint that is used for health monitoring of the server. The example below disables audit logging for all GET requests to /Patient and sub resources or operations.
"Audit": {
"ExcludedRequests": [
{
"UrlPath": "/Patient",
"Method": "GET"
},
{
"UrlPath": "/Patient/*",
"Method": "GET"
}
]
},
The UrlPath property is required, but not otherwise checked (e.g. if it points to an existing resource). The wildcard (*) can be used to expand matching in different ways, e.g.:
/Medication* will match /Medication, /MedicationRequest, /MedicationAdministration, etc
/$* will match all system level operations
/*/*/$validate will match all validation operations on all resources
The Method property is optional. If left out, null, empty or given the value *, it will match all HTTP verbs. You can enter multiple verbs, delimited by the | symbol (e.g. GET|POST).
Audit log file configuration
File
Configure where to put the audit log file and the format of its lines in a separate file named audit.logsettings.json. Just like the Firely Server application logging, the audit log also uses Serilog for logging audit events. The audit log settings are controlled in json configuration files called audit.logsettings(.*).json. The files are read in a hierarchy, exactly like the appsettings files are.
Firely Server comes with default settings in audit.logsettings.default.json. You can adjust the way Firely Server logs its information by overriding these settings by either adding an additional file called audit.logsettings.json or audit.logsettings.instance.json, or in audit.logsettings.default.json directly. Alternatively you can control Audit log settings with Environment Variables.
{
"AuditLog": {
"WriteTo": [
{
"Name": "Async",
"Args": {
"configure": [
{
"Name": "File",
"Args": {
"path": "./audit/AuditLog.log",
"rollingInterval": "Day",
"fileSizeLimitBytes": "",
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Application}] [Audit] {RequestResponse} [Machine: {MachineName}] [ReqId: {RequestId}] [IP-Address: {Ip}] [Connection: {ConnectionId}] [UserId: {UserId}] [Username: {Username}] [Path: {Path}] [Parameters: {Parameters}] [Action: {Action}] [Resource: {Resource} Key:{ResourceKey}] [Search results: {SearchResultSummary}] [StatusCode: {StatusCode}] {NewLine}"
}
}
]
}
}
]
}
}
The values that you can set for the File sink Args are:
path: The location where the audit log file should be stored.rollingInterval: When this interval expires, the log system will start a new file. The start datetime of each interval is added to the filename. Valid values areInfinite,Year,Month,Day,Hour,Minute.fileSizeLimitBytes: Limit the size of the log file, which is 1GB by default. When it is full, the log system will start a new file.
The OutputTemplate listed here contains all the properties that can be logged:
Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz: When this was logged, with formatting.Application: Firely ServerRequestResponse: indicates wether the audit event was a request or a response.MachineName: Name of the machine hosting the Firely Server instance. Especially useful when running multiple instances all logging to the same file.RequestId: Unique id of this request, use this to correlate request and response.Ip: IP Address of the client.ConnectionId: Use this to correlate requests from the same client.UserId: User id from the JWT token (if present).Username: User name from the JWT token (if present).Path: Request url.Parameters: The request parameters used.Action: Interaction that was requested (like instance_read or type_search).Resource: Resourcetype involved.ResourceKey: ‘Key’ of the resource involved (if any), consisting of the resourcetype and the id, formatted as “resourcetype/id”.StatusCode: Statuscode of the response at the time of logging (by default ‘-1’ when the request is not handled yet).
For transactions and batches, the audit plugin will write a line for the transaction/batch as a whole and one for every entry in the transaction/batch.
Seq
Because we use Serilog for logging audit events, other Log sinks like Seq are also supported. Seq is a web interface to easily inspect structured logs.
For the Seq sink, you can also specify arguments. One of them is the server URL for your
Seq server:
"WriteTo": [
{
"Name": "Seq",
"Args": { "serverUrl": "http://localhost:5341" }
}
Change
serverUrlto the URL of your Seq server
Audit Log reliability and performance considerations
The default Serilog log sink in audit.logsettings.default.json is a asynchronous wrapper around a File sink, which means that audit log messages are pushed to a background worker thread to be written to the log file on disk. This improves application performance as the writing to the audit log is non-blocking. The async wrapper uses a buffer to collect the messages that need to be logged. For the current Serilog Async implementation, the default memory buffer feeding the worker thread is capped to 10,000 items. If this limit is reached any further log events will be dropped until the buffer is below this limit again. To change the limit you can add bufferSize to the audit logsettings. See Serilog.Sinks.Async for more details.
In normal circumstances the buffer will regularly be flushed to the underlying sink. However, when the buffer limit does get reached the reliability of writing messages is compromised and some messages will get lost while the async wrapper tries to recover. If reliability of the auditing is very important, you might want to consider using a synchronous file sink instead. See the audit.logsettings.default.json for an example of a synchronous File sink configuration.
{
"AuditLog": {
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "./audit/AuditLog.log",
"rollingInterval": "Day",
"fileSizeLimitBytes": "",
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Application}] [Audit] {RequestResponse} [Machine: {MachineName}] [ReqId: {RequestId}] [IP-Address: {Ip}] [Connection: {ConnectionId}] [UserId: {UserId}] [Username: {Username}] [Path: {Path}] [Parameters: {Parameters}] [Action: {Action}] [Resource: {Resource} Key:{ResourceKey}] [Search results: {SearchResultSummary}] [StatusCode: {StatusCode}] {NewLine}"
}
}
]
}
}
The downside is that writing to the audit log is blocking and Firely Server now has to wait on the log to finish before it can continue, which in turn affects performance. You will have to try and test what works best for your use case.
AuditEvent logging
There is no further configuration for AuditEvent logging. If you include it in the pipeline, it will start generating AuditEvent resources, conforming to the IHE Basic Audit Log Patterns (BALP) ImplementationGuide.
Note
AuditEvents will not get generated if your configuration restricts the list of supported FHIR resources and AuditEvent is not included (see Restrict supported resources and SearchParameters).
For transactions and batches the audit plugin will create an AuditEvent for the transaction/batch as a whole and one for every entry in the transaction/batch.
Firely Server does not allow you to update or delete the AuditEvent resources through the RESTful API so the Audit log cannot be tampered with. You can of course still manipulate these resources directly on the database, for instance to offload a surplus of old AuditEvent resources elsewhere. Please Contact us us for details if you want to do this.
The table below contains some elements you can find in the generated AuditEvents and the paths where those elements are located (might differ per FHIR version). The table also includes links to AuditEvent examples.
Note
When the order of an item in an array is shown using a colon syntax (e.g. :requestId, :query), that means the order is not deterministic. You need to examine each item’s type and/or role elements to identify the right item.
Property name |
AuditEvent (R3) |
AuditEvent (R4) |
AuditEvent (R5) |
|---|---|---|---|
MachineName |
|
|
|
Action |
|
|
|
Timestamp |
|
|
|
Status Code |
|
|
|
Application |
|
|
|
IP Address |
|
|
|
Client Id |
|
|
|
TokenIssuer |
|
|
|
Jwt Id |
|
|
|
User Id |
|
|
|
Username |
|
|
|
Path |
|
|
|
Request Id |
|
|
|
Connection |
|
|
|
Search Parameters |
|
|
|
Resource |
|
|
|
Resource Key |
|
|
|
Resource Key (Patient) |
|
|
|
Search Results |
|
|
|
Example (search) |
|||
Example (read) |
|||
Example ($erase operation) |
AuditEvent Integrity
Firely server provides a mechanism to validate the integrity of the AuditEvents. On the one hand, it provides a way to sign the AuditEvent upon creation, and on the other hand, it offers a custom operation to validate the signatures, ensuring that the AuditEvents have not been tampered.
AuditEvent Signature
An AuditEvent Signature is a Provenance FHIR resource which contains a signature of the complete AuditEvent FHIR resource JSON. This Provenance FHIR resource also includes a reference to an AuditEvent FHIR resource from which the signature is created.
Note
AuditEvent Signatures will not get generated if your configuration restricts the list of supported FHIR resources and Provenance is not included (see Restrict supported resources and SearchParameters).
AuditEvent Integrity Validation
The validation of the AuditEvent integrity is done by checking that the associated signature of an AuditEvent still matches the current AuditEvent content.
This verification is an asynchronous operation which is triggered by calling the custom operation $verify-integrity on the AuditEvent type, using
the AuditEvent search parameters (see https://www.hl7.org/fhir/auditevent.html#search) to specify which AuditEvents should be validated. Note that only
AuditEvents created before the call are considered.
For example, the following query will trigger the integrity validation of all AuditEvents created in January 2022.
curl '${BASE_URL}/AuditEvent/$verify-integrity?date=ge2022-01-01&date=le2022-01-31' \
--header 'Prefer: respond-async'
If the request succeeds, the status code should be 202, the body should contain an operation outcome with a single issue of information severity
and the Content-Location header should contain the URL where the status of the operation can be retrieved.
While the operation is still in progress, the status endpoint should return a 202 status code.
In case of failure during the operation, the status endpoint should return a 4xx or 5xx status code with an operation outcome stating the issue(s).
Finally, once the operation is terminated, the status code of the reply should be 200 and the body should contain an operation outcome. If all AuditEvents had a valid signatures, the body should be:
{
"resourceType": "OperationOutcome",
"text":
{
"status": "All Audit Event signatures validated",
"div": "<div xmlns=\"http://www.w3.org/1999/xhtml\">\n <p>All Audit Event signatures validated</p>\n </div>"
},
"issue": [
{
"severity": "information",
"code": "informational",
"details": {
"text": "xx Audit Event processed"
},
}
{
"severity": "information",
"code": "informational",
"details": {
"text": "Transaction time: xxx"
}
},
{
"severity": "information",
"code": "informational",
"details": {
"text": "Original Request: xxx"
}
}
]
}
If some AuditEvents were not valid, in addition to the informational issues listed above, there should be one processing issue (see https://www.hl7.org/fhir/codesystem-issue-type.html#issue-type-processing) per validation error:
{
"severity": "error",
"code": "processing",
"expression": "AuditEvent/event_id",
"details": {
"text": "Signature for the event does not match audit event content"
}
}
Finally, if the number of validation failures is higher than the pre-configured threshold, an additional error should be reported:
{
"severity": "error",
"code": "too-costly",
"details": {
"text": "Process interrupted because too many signature validation errors encountered."
}
}
AuditEvent Integrity Configuration
By default, the signature of the AuditEvent and their verification is disabled. In order to enable it, you have to modify the settings of the server.
First of all, in the PipelineOptions, you need to have “Vonk.Plugin.Audit.Integrity” (or a prefix of it) as part of the plugin pipelines.
As it is listed in the Exclude section by default, you have to remove it from this section:
"PipelineOptions": {
"PluginDirectory": "./plugins",
"Branches": [
...
"Vonk.Plugin.Audit",
...
],
"Exclude": [
"Vonk.Subscriptions.Administration"
]
},
Also, as part of the Administration pipeline, you need to enable the support for the asynchronous tasks as they are used
for the asynchronous processing of the integrity verification operation. This is done by having the Task configuration corresponding
to the database type used for the administration:
{
"Path": "/administration",
"Include": [
...
"Vonk.Repository.Sql.SqlTaskConfiguration",
or
"Vonk.Repository.Sqlite.SqliteTaskConfiguration",
or
"Vonk.Repository.MongoDb.MongoDbTaskConfiguration",
...
]
}
In addition to the pipelines setup, you need to configure properly the Audit section of the settings:
"Audit": {
"AuditEventSignatureEnabled": true, // Default is false
"AuditEventSignatureSecret":
{
"SecretType": "JWKS", // Currently only supported type
// This is an example secret. Generate your own and do not use this example 'Secret' in your configuration!
"Secret": "{'keys':[{'kty':'EC','use':'sig','key_ops':['sign','verify'],'alg':'ES256','kid':'66e56ebf-a8de-4cfe-9710-3f2f44ec262f','crv':'P-256','x':'FO0bvAsRHC-wKMczT4xFPWQXI_fhFzqW2l9WxU29Hdc','y':'MHYht76KAnxHhatfB_BdyIuUtbpkK0g0Wuy5940oei4','d':'Nt1RXXNt6s5ytd88T7YhRePd7BqC4rh5WCOtJxdOzTs'}]}"
},
"AsyncProcessingRepeatPeriod" : 10000,
"InvalidAuditEventProcessingThreshold" : 100,
"AuditEventVerificationBatchSize": 20
},
with:
AuditEventSignatureEnabled must be set to true to enable the signature generation.
AuditEventSignatureSecretspecifies the secret to be used when signing the AuditEvent. Currently, it can only contain a JSON Web Key SetSecret.A JSON Web Key Set (JWKS) is a set of JSON Web Tokens (JWT) keys. The next section details how to generate a JWKS.
Note
Currently only the first key in a JSON Web Key Set is used to create signature of an AuditEvent.
AsyncProcessingRepeatPeriod defines the period in milliseconds for the loop checking if a new integrity validation request is pending.
InvalidAuditEventProcessingThreshold specifies the threshold on the maximum number of invalid AuditEvent signatures. Once this threshold
is reached, the operation is terminated and a specific issue is log in the operation outcome.
AuditEventVerificationBatchSize specifies the batch size when validating the AuditEvent signatures,
expressed as number of AuditEvent to verify in one step. We recommend to to set this value to 500 when using SqlServer or MongoDb as data backend, and
20 when using SQLite.
Note
When using SQLite, setting AuditEventVerificationBatchSize will prevent the validation of AuditEvent signature as SQLite
has a limitation on the query size it supports. Concretely, when the provided value is too large, the $verify-signature`operation
would fail, indicating the following error:
``SqliteException (0x80004005): SQLite Error 1: ‘parser stack overflow’`
Finally, in order to enable the integrity verification, the corresponding custom operations must be listed as part of the
SupportedInteractions.
For that, you have to add the type-level custom operations $verify-integrity and the system-level custom operation $verify-integrity-status, as follows:
"SupportedInteractions": {
"InstanceLevelInteractions": "...",
"TypeLevelInteractions": "..., $verify-integrity",
"WholeSystemInteractions": "..., $verify-integrity-status"
}
JSON Web Key Set generation
The following code snippet in C# is an example how you can generate a JSON Web Key Set.
using CreativeCode.JWK.KeyParts;
using CreativeCode.JWK;
...
private static string CreateJSONWebKeySet()
{
var algorithm = Algorithm.ES256;
var keyUse = PublicKeyUse.Signature;
var keyOperations = new HashSet<KeyOperation>(new[]
{
KeyOperation.ComputeDigitalSignature,
KeyOperation.VerifyDigitalSignature
});
var jwk = new JWK(algorithm, keyUse, keyOperations);
var jwks = new JWKS(new[]{ jwk });
return jwks.Export();
}
Output of CreateJSONWebKeySet should look like this
{"keys":[{"kty":"EC","use":"sig","key_ops":["sign","verify"],"alg":"ES256","kid":"66e56ebf-a8de-4cfe-9710-3f2f44ec262f","crv":"P-256","x":"FO0bvAsRHC-wKMczT4xFPWQXI_fhFzqW2l9WxU29Hdc","y":"MHYht76KAnxHhatfB_BdyIuUtbpkK0g0Wuy5940oei4","d":"Nt1RXXNt6s5ytd88T7YhRePd7BqC4rh5WCOtJxdOzTs"}]}
If you are using Ubuntu linux, you can also install jose command to generate a JSON Web Key Set.
Install Ubuntu package jose
sudo apt install jose -y
Generate JSON Web Key Set
jose jwk gen -i '{"kty":"EC","crv":"P-256","use":"sig","key_ops":["sign","verify"],"alg":"ES256","kid":"yourkeyid"}' -s -o ec.jwk
Note
Replace " with ' in the output to use it as Secret of AuditEventSignatureSecret in Audit plugin configuration,
AuditEvent customization
If you need to include additional information in the standard AuditEvents, you can do that with a custom plugin.
To implement such a plugin, it is helpful to understand how AuditEvents get created in Firely Server. Whenever the server receives an incoming HTTP request, a middleware registered in AuditEventConfiguration first passes it transparently to the downstream handlers. Then, when the original requests get handled, the audit middleware creates another artificial request and passes it down the stream again. This time, the request contains a creation operation with the AuditEvent as a payload. Like any other request in Firely Server, this request can be intercepted and changed using a pre-handler before it continues down the pipeline until CreateOperationMiddleware handles it. The order of the customization plugin should be greater than 3170 and less than 4420.
See an example plugin below. This plugin captures all the token claims from the original request and then includes those claims into the AuditEvent. Note that you need to work with SourceNodes at this level. You can read more about manipulating the SourceNodes here and in the Firely .NET SDK documentation.
[VonkConfiguration(order: 3175)]
public static class AuditEventCustomizationConfiguration
{
public static IServiceCollection ConfigureServices(IServiceCollection services)
{
services.AddScoped<AuditEventCustomizationService>();
return services;
}
public static IApplicationBuilder Configure(IApplicationBuilder builder)
{
builder.OnInteraction(VonkInteraction.all).PreHandleWith<AuditEventCustomizationService>((s, ctx) => s.CaptureOriginalRequestInfo(ctx));
builder.OnInteraction(VonkInteraction.type_create).AndResourceTypes("AuditEvent").PreHandleWith<AuditEventCustomizationService>((s, ctx) => s.AmendAuditEvent(ctx));
return builder;
}
private class AuditEventCustomizationService
{
private ClaimsPrincipal _user;
public void CaptureOriginalRequestInfo(IVonkContext ctx)
{
if (!IsAuditEventCreationRequest(ctx))
{
_user = ctx.HttpContext().User;
}
}
public void AmendAuditEvent(IVonkContext ctx)
{
if (IsAuditEventCreationRequest(ctx) && _user != null)
{
if (ctx.Request.Payload.Success)
{
var payloadResource = ctx.Request.Payload.Resource;
var resource = SourceNode.FromNode(payloadResource);
foreach (var claim in _user.Claims)
{
resource = resource.Add(SourceNode.Node("extension",
SourceNode.Valued("url", $"tokenValue-{claim.Type}"),
SourceNode.Valued("valueString", claim.Value)
));
}
ctx.Request.Payload = new RequestPayload
{
Resource = resource.ToIResource(payloadResource.InformationModel),
StatusCode = ctx.Request.Payload.StatusCode,
Success = true
};
}
}
}
private static bool IsAuditEventCreationRequest(IVonkContext ctx) =>
ctx.Request.Interaction == VonkInteraction.type_create
&& ctx.Arguments.TryGetArgument(ArgumentNames.resourceType, out var arg)
&& arg is {Source: ArgumentSource.Internal};
}
}