Data Lake
Last updated
Last updated
The Datalake serves as a central and flexible data source and is thus the basis of the entire SIEM. As a central repository, it brings together all raw data collected by agents and collector relays, indexes and groups it, and then presents it in a normalized form. In this way, the Datalake creates the basis for detecting patterns, anomalies and threats. By intelligently reusing the data, security incidents can be effectively detected, investigations can be conducted, and trends can be analyzed.
With the option to set start and end times, you can customize the temporal capture of your view. The graph below provides you with an overview of the frequency of entries captured in the defined time period.
The layout of the log view can be customized using the left icon on the right. Here you can define how comprehensive the preview of the logs should be and whether field names should be abbreviated. By clicking on the icon on the right side you can easily switch to full screen mode, to leave it, just press the escape key.
The now available content-based full text search is activated by default for all new installations after February 2024. Please note that activating the free text search can greatly increase the size of the index!
Find out here how to activate or deactivate it.
For the meaningful further processing of the numerous data it is crucial to find out the results that are relevant for you. For this purpose, you will find a drop-down menu on the left side. The items Generic, Enginsight, Standard and Event Relay are specified by default in the platform. All other fields are filled with information depending on your integrated systems and are listed product grouped. At the very bottom of the list you can still find the filters of the set up collectors.
Generic
Contains basic information across all types of logs. The difference with more specific log sources is that generic logs are less context specific and thus can cover a wide range of events. Gen fields are uniform templates that can be overlaid on all existing logs, recognize the same information (username, geoip.city, geoip. continent, ...) and standardize.
Enginsight
Contains all event logs captured by the Enginsight components (Defence FIM, IDS and Shield).
Standard
Contains all log information that has been individually typed and assigned by extractors.
Event Relais
The content is related to the previous configuration of the event relay and contains uniform RFC fields.
Below the product-specific tabs, the filters of the set-up collectors can also be found.
Currently we support the following firewall providers:
Barracuda Networks
Citrix Systems
ESET
F5 Networks
Fortinet
G Data CyberDefense
Lancom
PfSense/OPNSense
SonicWall
Sophos
Trend Micro
WatchGuard Technologies
genua GmbH
Furthermore, you can add more filters by clicking a field in the datalake. Under Operators, select whether "Equal", "Unequal", "Exists" or "Not Exist" applies to the selected field in your filter. If necessary, activate the full text search or adjust the value of your field manually.
Event Streams are at the heart of the Data Lake and serve as powerful data channels that enable continuous collection, aggregation and analysis of event data from a wide variety of sources. Continuous, real-time monitoring allows SIEM to quickly and reliably identify patterns and detect anomalies and potential security risks or unusual behavior. Use Event Streams to effectively collect, correlate and analyze event data from multiple sources to protect your IT and proactively respond to security threats. Whether for compliance and reporting issues, real-time monitoring of user activity, or incident response and forensics. Event Streams support you in all these areas and help you strengthen your entire IT security structure in the long term.
Use the filter drop-down menu on the left to efficiently filter searched log entries and reliably create event streams. Clicking on a filter opens a detailed view of the available variations. Right next to each variable is a plus icon indicating that the variable should be included in the results, and a minus icon indicating that results with that specific variation should be excluded. Add filters to display logs with specific information. For example, generate views of successful SSH logins and then create new streams. The list of all currently applied filters can be found above the graphic. Reset all filters by clicking on "create new stream" to create new views. Streams that have already been created or predefined can be accessed via Open Stream. Update your stream via the corresponding button or save your set filters via "Save stream". In this case, assign a unique name so that you can quickly find the stream again later if necessary.
