Collectors
Last updated
Was this helpful?
Last updated
Was this helpful?
Enginsight provides a diverse range of collector types that allow for comprehensive data collection. Within the Enginsight system, there are a total of three main types of collectors: Receiving Collectors, Relationship Collectors, and Integrated Collectors. These different collectors act as Pulsar Agents, actively on a mission to collect valuable data to ensure comprehensive insights into the system landscape.
General collectors act as primary data collection points in Enginsight SIEM. They open ports and receive external logs through those ports. It does not matter where the agent is located in the network - even in isolated networks without external access. The only critical requirement is that the firewall allows data transfer to these ports and the agent has the authority to send this data to the API.
In order for the agent acting as event relays to be able to receive data from the firewall, you must allow it to open a port. To do this, go to "Hosts", click on the host in question to get to the detailed view. Now go to "Settings" in the left sidebar under Miscellaneous, open the "Advanced Settings" and then check "SIEM Collector".
Assign a unique name and write a short description. Next, specify a host that will serve as the receive collector. By default, the "bind address" is set to 0.0.0.0. to receive data from the internal and external network. Alignment to an internal IP address is also possible, to leave external IPs out of the consideration, enter a desired IP for this purpose. Select under "Protocol" between UDP and TCP. Under "Format" you specify with which syslog format the incoming data should be parsed.
It is mandatory that the specifications of your Event Relay match those of your connected firewall. Set them within the firewall settings. If you cannot set the data manually at the firewall, please use the default settings: Port: 514 Format: RFC3164
If your network contains several RFC formats, it is also possible to install several event relays via different ports on one agent. Please note that in this case different ports must be used in order not to negatively influence the function of the affected event relays!
Finally, click "Add Collector" to save your created event relay.
The data transfer between Event Relay and API runs under GZIP Level 9, which results in a data compression ratio of about 20 to 1 to save your resources.
Please note that you must define templates for ESET manually. To do this, follow the instructions below.
In Eset Protect Management, go to "Settings" via "More". Enter the corresponding values under Syslog Server.
Now switch to your notifications. And create separate notifications for each event type.
Under "Basic", proceed as shown in the following diagram. Make sure that the notifications are activated and assign names (these are freely selectable and are only used for the overview). The event type is defined in the top line and the syslog event in the 2nd line:
Switch to "Configuration" and select the appropriate event type under Category.
Now switch to "Distribution" and allow the sending of syslogs.
Then enter the corresponding template for the notification in the "Content" sub-item. Proceed according to the following scheme (event type (line 1) and the syslog template (full line 2)).
Use the log files to isolate relevant security information from the log files of various systems that are not able to send independently via event relays.
Create a new collector via Add collector. Assign a unique name and a short description. The default setting is that the collector should send logs. If you do not want this, deactivate the function by clicking on the button. Then define a host assignment. Decide here between:
Reference: Then define at least one host from which the logs are to be recorded.
or Tags: Then define at least one tag from which to apply.
Finally, define at least one extractor for your log files, create the relevant file paths and add your collector by clicking on the relevant button.
These collectors extract data from connected cloud applications and actively transmit it to the SIEM, expanding the scope of overall data collection.
To connect your Office logs to the Enginsight SIEM, you must first create an API key in your Microsoft Office application. To do this, follow the instructions below.
After you have created your key, the permissions under Microsoft Azure must be set as follows:
Finally, create a collector in your SIEM. Assign a unique name and a short description for the collector. Then specify whether the collector can send logs. Select a host from the drop-down menu under "Host". Enter one or more "Channels" to be monitored and add the "Tenant ID" and the "Client ID". Once you have finally decided on the "Authentication method" (secret or certificate), you can save the changes you have made and add the collector by clicking on the "Save changes" button.
You can find your client ID/tenant ID in Microsoft Azure under: "App registration"-"All applications", then click on the corresponding entry. Now take the client ID from this view and add it accordingly when setting up your collector.
Go to the Atlassian administration via the administrator account. Click on your abbreviation in the top right-hand corner to select the "Manage account" option in the menu that appears. Now select the "Settings" tab in the navigation bar. Once in the view, you can now easily create a new API key by pressing the button.
Then create a collector in Enginsight SIEM, select the relevant host and enter the corresponding tenant ID. Add the collector by saving your settings.
Open the settings of your Bitdefender application.
Navigate to the menu item “API Key” and activate the “Event Push Service API”.
Follow the instructions to create an API key.
Detailed instructions can be found here: Bitdefender Public API Guide.
Follow the link and release the corresponding IPs in your firewall so as not to affect the functionality of the collector.
Switch to the Enginsight application and call up the menu item: SIEM. Select the API collector Bitdefender GravityZone from the API collector list and click on Add collector in the top right-hand corner.
Enter a unique name and a description for your new Bitdefender new collector.
The option to send logs by the collector in the Enginsight application is enabled by default and is required to integrate Bitdefender GravityZone logs into your SIEM.
Enter the appropriate URL of your Bitdefender application under Endpoint: Global: https://cloud.gravityzone.bitdefender.com Europe: https://cloudgz.gravityzone.bitdefender.com
Make sure that the URL begins with https://, as Bitdefender only accepts connections via HTTPS.
Enter the key previously created in Bitdefender in the corresponding field under API key in the Enginsight application.
Under Event types, select the specific events that are to be sent to your SIEM via the collector. All event types are selected by default. If necessary, remove events that are not required from the list.
Under Hosts, select the host that is to act as a collector from the drop-down list.
Make sure that the selected host is accessible via the Internet and can act as a SIEM collector through appropriate policies or host settings.
The bind address defines the IP address and the port of the host via which it receives the push events from Bitdefender. Make sure that the bind address is configured correctly and that the host is accessible via the network.
The option Local "Bind" with HTTPS is deactivated by default. However, it is required if Bitdefender is to send the logs directly to the host.
“Local “Bind” with HTTPS” activated
Activate the button: “Local ‘Bind’ with HTTPS”
Insert a certificate valid for the bind URL.
Enter the private key.
“Local “Bind” with HTTPS” deactivated
Follow the further instructions.
If you use a proxy between Bitdefender and the host that forwards the communication, the “Local bind with HTTPS” option can remain deactivated. The proxy then secures the communication between Bitdefender and itself. The connection between the proxy and the host can then be unencrypted (without HTTPS) if this is permitted in your infrastructure.
Make sure that the proxy is configured correctly in order to transmit the communication securely and forward the traffic correctly.
The bind URL specifies the full URL (including HTTPS) through which the host receives the encrypted push events from Bitdefender. Make sure that the URL is formatted correctly and that HTTPS is used, as Bitdefender only accepts encrypted connections.
If you cannot insert a direct IP or only a certificate for a new domain, please skip the certificate check with the corresponding field. When using self-signed certificates, it is also necessary to skip the certificate check.
Enter a user name and password for basic authentication. This authentication is only required by Bitdefender.
Save the configuration of the collector by clicking on Add Collector.
The Host Collectors collect logs directly from the operating system using the already installed agents. This enables seamless data collection that directly accesses existing resources and provides a comprehensive view of system activity.
Assign a relevant name and a short description. Use the button below to specify whether the collector should be able to send logs on its own. Under "Host assignment" you can now specify tags and list all tags below that should count for this collector or you can decide for "Reference" and then specify explicit hosts for which the collector applies.
Select from the default channels which you want to monitor and add more channels via the button with just a few clicks.
Easily integrate Exchange Logs. To do this, find the log name (via your Windows event viewer) of the channel you want, copy the exact name and add it under "Add Custom Channel".
Again, select from the default channels at logLevels.de. Please note that unified logs result in a considerable amount of data and we therefore strongly recommend activating Fault by default.
Select the “Sysmon” module. This is a Microsoft tool that acts as a wrapper and combines a large number of events that may be irrelevant for further analysis for the SIEM and compresses them into a standardized file format. This significantly optimizes the handling of such logs and enables a faster response. To confirm this step, you must agree to the third-party license terms.
When installing Sysmon , note that it is loaded from a third-party source (Microsoft). If you use proxy setups, do not forget to whitelist this URL.
By default, the Apache HTTP server writes its logs to local files. However, in order to collect and process the logs efficiently with Pulsar, the logs must be forwarded via syslog. This guide describes how to configure Apache so that the logs are provided in the appropriate format and can be easily parsed.
Open the configuration file of the Apache HTTP server with a suitable text editor:
Debian-based: /etc/apache2/apache2.conf
RHEL-based: /etc/httpd/conf/httpd.conf
Add the following entries or adapt existing ones:
vhost_combined: Contains virtual hosts, referrer and user agent.
combined: Contains referrer and user agent.
common: Simplest format, without referrer and user agent.
referer/agent: Specialized formats for individual fields.
GlobalLog for Access Logs:
httpd: Is used as syslog.app_name (do not change).
local0.info: Syslog facility and severity for the logs.
vhost_combined:Log format supported by the parser.
ErrorLog for Error logs:
LogLevel: Defines the severity levels (e.g. debug, info, notice, warn, error).
Recommendation: notice.
Save changes Speichern und schließen Sie die Konfigurationsdatei.
Apache neu starten Damit die Änderungen wirksam werden, muss der Apache HTTP Server neu gestartet werden:
Debian-based:
RHEL-based:
Check functionality
Make sure that the logs appear in /var/log/syslog or /var/log/messages, depending on the operating system configuration.
Check that access logs and error logs are forwarded correctly in the specified format.
