TL;DR
A SIEM is for detection and alerting. A security data lake is for storing large volumes of security data long-term at a predictable cost. A data pipeline is what cleans, normalizes, and routes data between them, tying storage and analysis into a single cost-effective system.
Most security teams need to have a security data lake and a SIEM working together, each doing different jobs. A security data pipeline like Realm.Security connects SIEMs and data lakes to reduce the cost of ownership and increase the performance of the whole system.
In this article, we draw on a recent webinar conversation between Colin Jermain, VP of Data Science at Realm.Security, and David Sztykman, Chief Architect at Hydrolix, to give you real-world definitions (as well as myths and misconceptions) of SIEMs, data lakes, and data pipelines in 2026. You can watch the full discussion here.
A SIEM Is a Tool for Alerting and Detection
When most security teams think of a "SIEM," they tend to imagine a single place where detection and alerting happen, and where logs are retained and stored.
This is a common misconception. A SIEM, by default, doesn't store anything.
Most people conflate "SIEM" with the entire logging platform behind it, for example, a Splunk instance. But those are two different things. In fact, a SIEM is the detection engine only. The storage and search layer is, or at least can be, a different system.
Learn how to avoid SIEM vendor lock-in.
The SIEM part of your security stack is not dependent on one storage backend system over another and should be connected to different systems, like a database or data lake, depending on what you want it to do.
In 2026, SIEMs have absorbed too many jobs
In the companies we talk to, it's common to find SIEMs doing everything from ingestion, search, alerting, and detection to compliance, storage, and analytics. Using a SIEM as a multitool is an expensive place to be.
The result is almost always the same: SOC teams hit cost constraints with their SIEM and can't bring in all the logs they need. Retention gets cut because it's too expensive, creating blind spots for investigations. When traffic spikes (like during a DDoS attack), SIEM costs can spike too, which is exactly when you can least afford to lose visibility.
Some SIEM vendors overcome the storage cost issue by offering a hot/warm/cold storage model as part of their deployments.
But this kind of data storage tends to result in a very uneven data analysis experience. Due to the challenge of performing normalization, compression, and indexing in a cost-effective way, the end result is often unpredictable query performance, i.e., data from a minute ago behaves differently than data collected six months ago.
A better way to use a SIEM is to give it clearer, smaller, and more relevant data so it can do what it's actually good at: rapid, accurate, and cost-effective detection and alerting.
How? By storing data in a data lake.
A Security Data Lake Is for Long-Term Data Retention
A security data lake is append-only storage designed to receive, index, and search large volumes of data. It is a backend system built for compression and long-term retention, not for transactions or constant modification.
A security data lake lives in cloud storage (Google Cloud Storage, for example), and can use columnar storage, meaning querying specific fields (e.g., just IP addresses) is fast and cheap, but pulling everything with a SELECT * is expensive.
Data lakes remove the cost constraint on storage so CISOs don't have to truncate data after a week or 30 days because storage has become too expensive. They also eliminate hot/warm/cold tier complexity and instead provide one consistent storage layer where query performance stays the same regardless of data age.
A security data lake allows security teams to:
- Keep data costs predictable, even during traffic spikes.
- Search specific IOCs, IP addresses, and file hashes across months of data reliably.
- Perform long-term forensics and threat hunting that's impossible when older data has been deleted or archived to slow/inaccessible storage.
- Manage large volumes of data efficiently and query it in a cost-effective way.
However, there are two important things to remember about security data lakes.
First, a data lake is never a replacement for a SIEM. This is because a data lake doesn't do alerting or detection on its own. It's also not designed for transactional workloads or frequent record modification. This is what a database does.
Second, a data lake is not very useful if the data feeding it is raw and unnormalized. Without a pipeline for cleaning data before it goes into a data lake, it's easy to end up with a massive store of inconsistent, hard-to-query logs (e.g., SRC_IP vs. source_IP vs. src.ip inconsistencies that will make accurate, consistent retrieval very difficult).
This is where a data pipeline comes in. Realm's purpose-built retention layer, Realm Data Haven, is one example: full-fidelity, normalized logs accessible for compliance and forensic investigations at a fraction of SIEM storage cost.
A Data Pipeline Is How Security Data Lakes and SIEMs Work Together
A security data pipeline does five jobs that neither the SIEM nor the data lake handles, but that both depend on for proper functionality:
Redaction
Security logs are full of PII. A pipeline masks or removes it before data reaches any destination, keeping you compliant with GDPR, CCPA, HIPAA, and PCI DSS without manual rules at every endpoint.
Filtering
A large proportion of log ingestion is noise: health checks, keep-alive signals, routine events with no security relevance. A pipeline removes these before they hit your SIEM, cutting ingestion volume and the licensing costs that come with it.
Normalization
A pipeline normalizes data in transit so your SIEM and data lake see a consistent structure regardless of source. A FortiGate firewall might log a field as "proto" while another vendor calls it "transport." A Palo Alto device may strip timezone information from its timestamps, making cross-source correlation unreliable. Common Event Format doesn't solve this either - vendors interpret it differently and map custom fields inconsistently. Without normalization at the pipeline layer, these inconsistencies get baked into everything downstream.
Enrichment
Raw logs tell you what happened. Enrichment tells you what it means: mapping IPs to known assets, tagging privileged accounts, and applying risk scoring. Data arrives at your SIEM already contextualized.
Routing
A pipeline sends data to multiple destinations simultaneously. Security-relevant events go to the SIEM. Everything else goes to the data lake. Sensitive data gets redacted before reaching certain destinations. This is what lets the SIEM and data lake each do their job without being burdened with the other's.
"Garbage in, garbage out" is true for security data. But storing garbage data is expensive, too. A data pipeline stops you from paying for useless analysis in your SIEM and unusable data in your data lake.
When you use a data pipeline, your SIEM runs on data that's relevant to it, while your data lake holds logs you can accurately and consistently query. Realm Flow is the AI-native security data pipeline that makes this work end-to-end.
Let Your SIEM (and Your Security Data Lake) Do Its Best Work
Realm.Security is a vendor-neutral security data pipeline that stops noisy, low-value, and expensive data from driving up your storage costs or reducing your SIEM's performance.
Book a demo of Realm.Security, and we'll walk you through how much you could drive next-gen efficiencies from your SIEM without replacing or rearchitecting your infrastructure.