Collectors
Last updated
Last updated
Enginsight provides a diverse range of collector types that allow for comprehensive data collection. Within the Enginsight system, there are a total of three main types of collectors: Receiving Collectors, Relationship Collectors, and Integrated Collectors. These different collectors act as Pulsar Agents, actively on a mission to collect valuable data to ensure comprehensive insights into the system landscape.
General collectors act as primary data collection points in Enginsight SIEM. They open ports and receive external logs through those ports. It does not matter where the agent is located in the network - even in isolated networks without external access. The only critical requirement is that the firewall allows data transfer to these ports and the agent has the authority to send this data to the API.
In order for the agent acting as event relays to be able to receive data from the firewall, you must allow it to open a port. To do this, go to "Hosts", click on the host in question to get to the detailed view. Now go to "Settings" in the left sidebar under Miscellaneous, open the "Advanced Settings" and then check "SIEM Collector".
Assign a unique name and write a short description. Next, specify a host that will serve as the receive collector. By default, the "bind address" is set to 0.0.0.0. to receive data from the internal and external network. Alignment to an internal IP address is also possible, to leave external IPs out of the consideration, enter a desired IP for this purpose. Select under "Protocol" between UDP and TCP. Under "Format" you specify with which syslog format the incoming data should be parsed.
It is mandatory that the specifications of your Event Relay match those of your connected firewall. Set them within the firewall settings. If you cannot set the data manually at the firewall, please use the default settings: Port: 514 Format: RFC3164
If your network contains several RFC formats, it is also possible to install several event relays via different ports on one agent. Please note that in this case different ports must be used in order not to negatively influence the function of the affected event relays!
Finally, click "Add Collector" to save your created event relay.
The data transfer between Event Relay and API runs under GZIP Level 9, which results in a data compression ratio of about 20 to 1 to save your resources.
Please note that you must define templates for ESET manually. To do this, follow the instructions below.
In Eset Protect Management, go to "Settings" via "More". Enter the corresponding values under Syslog Server.
Now switch to your notifications. And create separate notifications for each event type.
Under "Basic", proceed as shown in the following diagram. Make sure that the notifications are activated and assign names (these are freely selectable and are only used for the overview). The event type is defined in the top line and the syslog event in the 2nd line:
Switch to "Configuration" and select the appropriate event type under Category.
Now switch to "Distribution" and allow the sending of syslogs.
Then enter the corresponding template for the notification in the "Content" sub-item. Proceed according to the following scheme (event type (line 1) and the syslog template (full line 2)).
Use the log files to isolate relevant security information from the log files of various systems that are not able to send independently via event relays.
Create a new collector via Add collector. Assign a unique name and a short description. The default setting is that the collector should send logs. If you do not want this, deactivate the function by clicking on the button. Then define a host assignment. Decide here between:
Reference: Then define at least one host from which the logs are to be recorded.
or Tags: Then define at least one tag from which to apply.
Finally, define at least one extractor for your log files, create the relevant file paths and add your collector by clicking on the relevant button.
These collectors extract data from connected cloud applications and actively transmit it to the SIEM, expanding the scope of overall data collection.
To connect your Office logs to the Enginsight SIEM, you must first create an API key in your Microsoft Office application. To do this, follow the instructions below.
After you have created your key, the permissions under Microsoft Azure must be set as follows:
Finally, create a collector in your SIEM. Assign a unique name and a short description for the collector. Then specify whether the collector can send logs. Select a host from the drop-down menu under "Host". Enter one or more "Channels" to be monitored and add the "Tenant ID" and the "Client ID". Once you have finally decided on the "Authentication method" (secret or certificate), you can save the changes you have made and add the collector by clicking on the "Save changes" button.
You can find your client ID/tenant ID in Microsoft Azure under: "App registration"-"All applications", then click on the corresponding entry. Now take the client ID from this view and add it accordingly when setting up your collector.
Go to the Atlassian administration via the administrator account. Click on your abbreviation in the top right-hand corner to select the "Manage account" option in the menu that appears. Now select the "Settings" tab in the navigation bar. Once in the view, you can now easily create a new API key by pressing the button.
Then create a collector in Enginsight SIEM, select the relevant host and enter the corresponding tenant ID. Add the collector by saving your settings.
The Host Collectors collect logs directly from the operating system using the already installed agents. This enables seamless data collection that directly accesses existing resources and provides a comprehensive view of system activity.
Assign a relevant name and a short description. Use the button below to specify whether the collector should be able to send logs on its own. Under "Host assignment" you can now specify tags and list all tags below that should count for this collector or you can decide for "Reference" and then specify explicit hosts for which the collector applies.
Select from the default channels which you want to monitor and add more channels via the button with just a few clicks.
Easily integrate Exchange Logs. To do this, find the log name (via your Windows event viewer) of the channel you want, copy the exact name and add it under "Add Custom Channel".
Again, select from the default channels at logLevels.de. Please note that unified logs result in a considerable amount of data and we therefore strongly recommend activating Fault by default.
Select the “Sysmon” module. This is a Microsoft tool that acts as a wrapper and combines a large number of events that may be irrelevant for further analysis for the SIEM and compresses them into a standardized file format. This significantly optimizes the handling of such logs and enables a faster response. To confirm this step, you must agree to the third-party license terms.
When installing Sysmon , note that it is loaded from a third-party source (Microsoft). If you use proxy setups, do not forget to whitelist this URL.
By default, the Apache HTTP server writes its logs to local files. However, in order to collect and process the logs efficiently with Pulsar, the logs must be forwarded via syslog. This guide describes how to configure Apache so that the logs are provided in the appropriate format and can be easily parsed.
Open the configuration file of the Apache HTTP server with a suitable text editor:
Debian-based: /etc/apache2/apache2.conf
RHEL-based: /etc/httpd/conf/httpd.conf
Add the following entries or adapt existing ones:
vhost_combined: Contains virtual hosts, referrer and user agent.
combined: Contains referrer and user agent.
common: Simplest format, without referrer and user agent.
referer/agent: Specialized formats for individual fields.
GlobalLog for Access Logs:
httpd: Is used as syslog.app_name (do not change).
local0.info: Syslog facility and severity for the logs.
vhost_combined:Log format supported by the parser.
ErrorLog for Error logs:
LogLevel: Defines the severity levels (e.g. debug, info, notice, warn, error).
Recommendation: notice.
Save changes Speichern und schließen Sie die Konfigurationsdatei.
Apache neu starten Damit die Änderungen wirksam werden, muss der Apache HTTP Server neu gestartet werden:
Debian-based:
RHEL-based:
Check functionality
Make sure that the logs appear in /var/log/syslog or /var/log/messages, depending on the operating system configuration.
Check that access logs and error logs are forwarded correctly in the specified format.
