Loggernaut-Configurations

Backup Strategies

Loggernaut currently supports two backup strategies for the automated backup of raw logs:

• SFTP-Backup – Storage on an SFTP server

• S3-Backup – Storage in an S3-compatible object storage

Both options offer optional client-side encryption using the AGE encryption method.

SFTP Backup

The backup system automates the transfer of all raw logs previously stored exclusively on the management server to an SFTP server. The detailed configuration of this function is described in the Loggernaut config.json and offers you an efficient method for backing up your data.

Choosing a backup strategy

The strategy parameter determines how Loggernaut handles backups:

Value	Description
local	Logs are stored exclusively locally on Loggernaut.
remove	Logs are deleted after the defined TTL has expired.
sftp	Logs are transferred to an external SFTP server. The sftp option must be configured.

Be careful with SFTP chroot! If the destination is ‘/user/siem/logs’ and the chroot is set to ‘/user’, the actual destination should be ‘/siem/logs’ to avoid potential problems.

SFTP-specific fields

Field	Description	Hints
permissions	Unix-style file permissions (e.g., “0666”). Allows all users read and write access.
remoteDirectory	Directory on the SFTP server where the logs are stored.	If the target directory is, for example, /user/siem/logs and the chroot is set to /user, remoteDirectory must be set to /siem/logs!
keepLocalCopy	If false, all logs outside the TTL are deleted as soon as they have been successfully transferred. If true, a copy remains locally on the logger.

SSH access data (sftp.ssh)

Config-value	Description	Hint
username	Username for the SSH connection to the SFTP server.
password	Password for authentication. Can be left blank if an SSH key is used.	You can leave this field blank if a key is used.
ip	IP address of the SFTP/SSH server.
port	Port for the SSH connection. The default is 22 if left blank.	If you leave this field blank, the default port 22 will be used.
privateKeyPath	Path to the private SSH key. Only required if key-based authentication is used.	If you use a password, you do not need to make any adjustments here.
privateKeyPassphrase	Password for the private key, if it is encrypted.
knownHostsPath	Path to the known_hosts file for verifying the SFTP server. If this field is left blank, the host check is omitted.	If you leave this field blank, no host check will be performed and it will be assumed that the correct server has been specified. It does not have to be the ‘known_hosts’ file created by SSH if only this host is to be allowed, but the format must correspond to that of the “original” file.

1. Add the code to your /opt/enginsight/loggernaut/config.json and adjust the configuration as described above.

   1. Sample configuration (without encryption)

      {
          "api": {...},
          "siem": {...},
          ...,
          "backup": {
              "strategy": "sftp",
              "sftp": {
                  "permissions": "0666",
                  "remoteDirectory": "/logs",
                  "keepLocalCopy": false,
                  "ssh": {
                      "username": "siem",
                      "password": "****",
                      "ip": "****",
                      "port": 22,
                      "privateKeyPath": "/opt/enginsight/loggernaut/ssh/id_ed25519",
                      "privateKeyPassphrase": "*****",
                      "knownHostsPath": "/root/.ssh/known_hosts"
                  }
              }
          }
      }

   2. Sample configuration with AGE encryption Loggernaut currently only supports the age encryption package. Generating an AGE key pair You can generate the required key pair as follows: with AGE-CLI

      age-keygen -o key.txt

      with Loggernaut

      sudo ./loggernaut -generate-age-keys

      This generates a public/private key pair. Only the public key is included in the configuration. The private key must be kept safe so that backups can be decrypted later. Backup with encryption

      {
          "backup": {
          "strategy": "sftp",
          "sftp": {
            "permissions": "0666",
            "remoteDirectory": "/logs",
            "keepLocalCopy": true,
            "ssh": {
              "username": "siem",
              "ip": "192.168.1.100",
              "port": 22,
              "privateKeyPath": "/opt/enginsight/loggernaut/ssh/id_ed25519",
              "knownHostsPath": "/root/.ssh/known_hosts"
            }
          },
          "encryption": {
            "package": "age",
            "publickey": "age1ms3c0gmdxakx3lxzlp422g78lju4pmrse0rp6jl8lpu696yvea7qttzsl0"
          }
        }
      }

Technical information about encryption:

• Currently, only the AGE encryption package is supported.

  • A valid AGE public key always begins with age and contains only characters from [a-z0-9]. Example: age1ms3c0gmdxakx3lxzlp422g78lju4pmrse0rp6jl8lpu696yvea7qttzsl0

All organizations on a SIEM instance use the same public key for encryption. There is no org-specific key configuration, and all backups are stored with the same S3 provider. Decryption is performed using the associated private key, which must be maintained centrally.

1. Finally, restart Loggernaut to apply the configurations. sudo systemctl restart ngs-loggernaut

Customize path for backups

By default, Loggernaut backups are stored on the SIEM management server under the following path: /opt/enginsight/loggernaut/backup/

You can change this via the configuration file: /opt/enginsight/loggernaut/config.json

Example:

{
  "backup_directory": "/EIGENER.PFAD"
}

After changing the backup path, Loggernaut must be restarted. Do this as follows:

sudo systemctl restart ngs-loggernaut

S3-Backup

Use this backup option to automatically and securely store backup files in S3-compatible storage. The following instructions show you how to configure Loggernaut accordingly.

1. Information required from the S3 provider

   Configuration value	Description
   endpoint	The URL of the storage endpoint. This may vary depending on the provider, but often follows the pattern: https://(region).(s3/storage).(hostingprovider).(tld)
   accessKeyId	The API access token for authentication.
   accessKeySecret	The secret key to API access.
   region	The region of the storage service (e.g., eu-central).

2. Before configuration, decide whether you want to encrypt the backups and define a suitable bucket prefix.

   1. Encryption You have the option of encrypting backup files on the client side before uploading. Currently, only the AGE encryption method is supported. Alternatively, encryption can also be disabled.

   2. Define Bucket-prefix Each customer automatically receives their own bucket. The name is derived from the following scheme:

      <BUCKET_PREFIX>-<ORG_ID>

      Example: example-siem-backup-1234abcd When starting, Loggernaut automatically checks whether the prefix used is a valid bucket name.

3. Sample configurations

   1. Backup without encryption

      {
        "backup": {
          "strategy": "s3",
          "s3": {
            "endpoint": "eu-central.storage.s3provider.example",
            "accessKeyId": "XXXXXXXXXXXXXXXX",
            "accessKeySecret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
            "bucketPrefix": "example-siem-backup",
            "region": "eu-central",
            "keepLocalCopy": true
          }
        }
      }

      If no encryption block is specified, the backup will be performed without encryption! Explicitly disable encryption Alternatively, encryption can be explicitly disabled:

      "encryption": {
        "package": "none"
      }

      An empty string is also possible:

      "package": ""

   2. Backup with AGE encryption Loggernaut currently only supports the age encryption package.

      1. Generating an AGE key pair You can generate the required key pair as follows: with AGE-CLI:

         age-keygen -o key.txt

         with Loggernaut:

         ./loggernaut -generate-age-keys

         This generates a public/private key pair. Only the public key is included in the configuration. The private key must be kept safe so that backups can be decrypted later.

      2. Configuration with encryption

         {
           "backup": {
             "strategy": "s3",
             "s3": {
               "endpoint": "eu-central.storage.s3provider.example",
               "accessKeyId": "XXXXXXXXXXXXXXXX",
               "accessKeySecret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
               "bucketPrefix": "example-siem-backup",
               "region": "eu-central",
               "keepLocalCopy": true
             },
             "encryption": {
               "package": "age",
               "publickey": "age1ms3c0gmdxakx3lxzlp422g78lju4pmrse0rp6jl8lpu696yvea7qttzsl0"
             }
           }
         }

Technical information about encryption:

• Currently, only the AGE encryption package is supported.

  • A valid AGE public key always begins with age and contains only characters from [a-z0-9]. Example: age1ms3c0gmdxakx3lxzlp422g78lju4pmrse0rp6jl8lpu696yvea7qttzsl0

All organizations on a SIEM instance use the same public key for encryption. There is no org-specific key configuration, and all backups are stored with the same S3 provider. Decryption is performed using the associated private key, which must be maintained centrally.

Option	Description
keepLocalCopy:true	Specifies whether a local copy of the backup files should also be saved (default: true).
"skipSSL": true	If set to true, HTTPS is disabled. Recommended for internal networks only!

skipSSL should not be enabled when backups are transferred over the public Internet.

Full-Text Search Datalake

The Full-Text Search allows you to search for any text in your logs to find relevant information even faster. You can now search logs specifically for content without having to specify the relevant fields. This now also makes it possible to output cross-field results.

Bedenken Sie bitte, dass die Aktivierung der Full-Text Search die Größe des Indexes stark vergrößern kann! Sein Sie sich im Vorhinein darüber bewusst und stellen Sie ausreichend Ressourcen zur Verfügung.

Activation for existing accounts

As an existing Enginsight user, you must navigate to the file /opt/enginsight/loggernaut/config.json and add the entry "fullTextSearch": true at the JSON root level.

Then restart the loggernaut to apply the changes.

Please note that only logs that are sent to the SIEM after activation can be found using the free text search. Old logs are not searchable!

Deactivation after reinstallation

If you want to deactivate the full-text search function please follow the instructions beneath:

• set the entry "fullTextSearch": false in the config.json.

Backup-Log TTL

The backup log time-to-live (TTL) can be configured separately to control the lifetime of backup logs. Use the following JSON configuration to set the TTL for your organization:

{
    "backup": {
        "strategy": "remove",
        "ttl": {
            "<ORGANIZATIONAL_ID>": <TTL_IN_DAYS>
        }
    }
}

Replace <ORGANIZATIONAL_ID> with the name of your organization and <TTL_IN_DAYS> with the desired duration in days after which the backup logs should be automatically removed.

Sample configuration

This configuration enables precise management of the backup log lifetime according to your requirements.

Below you will find a corresponding sample configuration for the TTL log files:

{
    // Mandatory fields in the configuration (irrelevant for TTL)
    "api": {
        "url": "",
        "accessKeySecret": "",
        "accessKeyId": ""
    },
    "siem": {
        "indecees": [
            ""
        ],
        "basicAuth": {
            "username": "",
            "password": ""
        }
    },
    // TTL for raw backup logs on the management server
    "backup": {
        "strategy": "<remove|sftp>",
        "ttl": {
            "<ORGANIZATIONAL_ID>": 90 // Specification in days
        }
    },
    // TTL of searchable logs in the Solr cluster
    "ttl": {
        "<ORGANIZATIONAL_ID>": 30 // Specification in days
    }
}

Please note that the TTL (Time to Live) only affects the logs on the management server. The logs on the SFTP server are no longer processed by the logger and therefore remain unchanged. Furthermore, the adjustment only applies to the respective organization and must be carried out for each organization.

Please ensure that you adjust the configuration accordingly!

Geosplitting

Geosplitting in a SIEM allows you to use multiple SIEM clusters with on-premise, ensuring data sovereignty and performance by only using a single customer.

Geosplitting allows you to move several sub-organizations into a new cluster if required. You also have the option of creating multiple clusters.

1. Configure the Loggernaut as usual.

2. Add the item "alternatives" to your configuration and implement the following instructions:

"siem": {
    "management": {
      "organisation": "ORGANIZATION"
    },
    "alternatives": [{
      "organisations": ["ORGANIZATION_1", "ORGANIZATION_2"],
      "basicAuth": {
        "username": "****",
        "password": "****"
      },
      "url": "url",
      "numShards": 2,
      "replicationFactor": 1,
      "management": {
        "organisation": "ORGANIZATION"
      }
    }],
    "basicAuth": {
      "username": "*****",
      "password": "*****"
    },
    "url": "URL"
  },

1. Enter the relevant information under the points below:

• "ORGANIZATION" All organizations belonging to the cluster.

• "management" The management organization within the new cluster can view the status of the other organizations. An organization must be selected that also exists under "alternatives". Otherwise, everything corresponds to the normal SIEM setup.

Beachten Sie zwingend, dass keine Organisation doppelt vorkommt, also nicht gleichzeitig in zwei Clustern existiert. Alle Organisationen, die nicht in die Alternative verschoben wurden, verbleiben im Management-Cluster.