What 25 Million Alerts Reveal About the Cognitive and Social Failures in Security Operations

When institutions classify security events as ‘low severity’, they create a problem of intentional blindness: SOC analysts must learn what to ignore to create a manageable workload.
A recent Intezer report analyzing more than 25 million enterprise security alerts found that roughly 1% of low-severity events conceal real breaches, approximately one per week per organization. For an average enterprise that sees ~450,000 alerts per year, that is approximately 50 confirmed threats annually. In conventional triage approaches, these events go uninvestigated.
It may seem minor. Fifty events over a year should not require much time to investigate. Yet, if analysts aren’t alerted to them, they hide in plain sight. UIs build confidence in their representation of a problem, even though analysts are aware of the false positive rate.
The failure is often glossed as a detection problem but its source runs much deeper in decision-making. The report claims "human analyst capacity is the bottleneck." This standard interpretation rests on an old assumption: if given the right information, people will make the right decision. This ignores the interactions between security practitioners, their tasks, and their environment.
The detection problem plays out at individual and organizational levels concurrently. At the cognitive layer, detection architectures must be redesigned and calibrate response thresholds against current base-rate evidence. At the social layer, institutions must map relationship structures to understand the human attack surface. But these relationships are invisible to most detection architecture.
But invisibility of attack does not need to equate to inevitability. The gap can be mapped by examining what an organization can observe, how it labels what it observes, whether that knowledge travels beyond its own walls, and whether anyone has named the specific techniques being used against it.
Alert Fatigue: A Cognitive Failure
"Alert fatigue" is a common problem. Its causes are often left uninvestigated creating a black box. Thankfully, its cause is common too. Three cognitive mechanisms provide a basic explanation about how it works and suggest how we can overcome it.
1. Evidence Accumulation
Detection theory distinguishes two factors that alert fatigue conflates: sensitivity and response thresholds. Sensitivity reflects the ability to discriminate true positives from false positives. This is a perceptual problem: whether two signals really look different or whether they are highly confusable. In the case of social engineering, phishing emails from ‘Nigerian Princes’ are very different from legitimate emails, while a competent social engineer can make their attack seem legitimate with a ‘new employee’ imminent deadline pretext.
Response thresholds reflect the evidence required to act. Analysts must decide whether they want to be conservative, and try to catch any possible signal or more liberal and allow some signals to go unexamined. This might be a result of their personality or motivation, but it is also a property of their workplace: if they have too many alerts, they need to be strategic and change their response threshold.
Sensitivity and response threshold are independent human factors. This means that an analyst can have high sensitivity (expertise) while missing every real threat in a category if the threshold excludes it from investigation entirely.
.png)
The 1% miss rate is a threshold failure, rather than a lack of expertise. Low-severity alerts have historically had near-zero true-positive rates. We see this in the same report, with only 2% of location anomalies and impossible-travel alerts being malicious. These false positives are attributed to VPNs, mobile behavior, and overlapping tools creating most of the noise.
Analysts and operations leaders adapted their tradecraft accordingly through individual and organizational learning. Given the feedback a system provides, this is rational. But the feedback loop that would update institutional policy never truly closes, because unexamined alerts never reveal what they contained.
Such base-rate effects on criterion placement are common: when the prior probability of a true positive approaches zero, the optimal threshold shifts to exclude that event class from active observation. Adding analysts or tuning rules will not solve this problem. Adjusting the institutional criterion and redesigning systems can.
This simple model is useful for understanding the basic problem facing analysts, but misses deeper cognitive architecture used to make decisions. Analysts must juggle multiple tasks at the same time.
2. Vigilance Decrement
Decision-making requires finite resources like attention and working memory. Attention depletes over time, referred to as vigilance decrement. The limits of working memory mean that only so much novel information can be kept in mind at any given time. As new information comes in, recent information might drop out.
In conditions where true signals are uncommon, analysts face a wicked environment defined by high-volume, low-hit-rates. An analyst watching a queue climb across a 12-hour shift is not the same detector at hour eleven as at hour one. As they look through record after record requiring sustained attention, their cognitive resources deplete. Information might not make it to a decision process to assess. The decrement is based on physiology, not motivation. Analysts can only work for so long and can only handle so much information. Training cannot compensate for these limits.
3. Automation Bias
To reduce our cognitive load, long-term memories help people integrate information together. If something looks like a familiar pattern, we can respond automatically. Heuristics can quickly sort through signals that look like threats, from those that do not.
When we consider the volume and depleted attention resources, analysts will need to rely on their own mental heuristics. If they have been given tools and learn to depend on them, we create an automation bias: analysts will defer to detection systems. The EDR data supports this possibility. Of 82,000 forensic memory scans, 2,600 contained active infections. Of those, 51% had already been marked "mitigated" by the source EDR vendor.
The "mitigated" label raises the analyst's criterion for reinvestigation, and signals to others that a peer has completed the check. The malware families found running on these declared-clean endpoints (Mimikatz, Cobalt Strike, Meterpreter, StrelaStealer) are the operational workhorses of criminal and state-level operations. If a label declares that a job is done, people will not have the time or resources to investigate further.
Social Failure: What the Cognitive Attack Framework Cannot See
Cybersecurity only works when people and technology are combined effectively. Training that focuses on these cognitive vulnerabilities misses this much larger piece that unlocks them. We must not only be able to classify these events as ‘high severity’, we must understand how attempts on the human attack surface work.

Script Violation
Business transactions follow social scripts: well-learned structured sequences that define an event. Once a script is active, interrupting it can create cognitive and interpersonal friction. Breaking the PayPal script would mean treating a payment notification as suspicious, which can feel irrational and costly when every surface feature is legitimate.
This is likely the process that has led to the report's most significant phishing finding: fewer than 6% of confirmed malicious emails contained attachments. Attackers have migrated from links, to language and the abuse of trusted services such as Vercel, CodePen, OneDrive, and PayPal.
Callback scams are the second most common phishing technique in the dataset. Attackers send an email that reports a charge the recipient did not make and asks them to call a number to dispute it. There is no attachment and no malicious link for a content filter to catch. The attacker exploits a known situational context by invoking a recognizable entity, activating a social script that reduces resistance and promotes compliance. Like other sequential attacks, no single social cue or cognitive anomaly defines the attack. Source credibility is only part of what makes it work.
The Cloudflare Turnstile finding follows the same pattern. Sites using it were more likely to be phishing pages in this dataset, with Google reCAPTCHA as part of legitimate infrastructure. We must recognize that attackers have coopted the social signals of institutional trustworthiness as legitimacy markers and build credible workflows to reinforce these narratives.
Organizational Climate and Culture
Organizations learn through formal communications and informal learning, rumors, and gossip. The formal policies and stable set of shared beliefs about what security behaviors are expected and rewarded define an organization’s security culture. This culture persists over time and can often be difficult to change. Even when leadership changes or new policies are introduced, cultural inertia is often apparent. If alerts or tools like UEBA are deemed to be unreliable, analysts learn to ignore them out of habit.
Organizational climate is the more dynamic layer, defined as the shared perception of current conditions and pressures that determines whether cultural commitments are enacted or overridden. A security culture that values vigilance can be overwritten by episodes of deadline pressure, staffing constraints, or visible leadership indifference. The scale is apparent in this report, by a finding that 60% of alerts go unreviewed across both in-house and MDR operations.
Outsourcing the function does not transfer the climatic pressure that produced the original deficit. Over time, a persistent climate will influence the culture. Indifference to alerts can become the norm, making signal difficult to distinguish from noise. Response thresholds will lower, so will organizational security.
Changing the Signal, Change Your Response
Organizational leaders must consider the viability of their ‘human firewall’. Humans will adapt their behavior to their environment. A small number of employees facing overwhelming alert volumes must pick and choose. Even if they have high levels of expertise that increases their sensitivity, they must change their response threshold. The Intezer report highlights what can come through the cracks.
Organizational leaders need to ensure that they have realistic expectations for their human and technical detection capabilities. The introduction of AI into the SOC represents an important step, but it introduces trade-offs. People will need to monitor the alerts and operations of these agents. Changing the source of the problem can create new issues, especially given that these agents have their own attack surface.
Standard detection architectures and training focus on cognitive anomalies such as unusual urgency, unknown authority, mismatched branding. They are blind to social layer anomalies such as a trusted institutional sender requesting something outside its established social script. These are failure conditions that require different detection approaches that analyze relationships and content, mapping whether communications are consistent with established institutional role relationships.
The two failure conditions require different solutions. For the cognitive layer, we must treat the response criterion on low-severity alerts as an active policy variable, rather than an inherited configuration. Build systematic forensic sampling of EDR “mitigated” verdicts into workflow. AI investigation tools can be used to close the feedback loop that uninvestigated alerts currently prevent from forming.
For the social layer, organizations must build detection architecture that can identify relational and interaction anomalies, whether a sender's request is consistent with their established institutional role. This should be combined with conventional content-based detection. Current SIEM and email gateway architectures were not designed for this. The attackers exploiting it know that. They move seamlessly through security networks that ignore the relational architecture.
What is feature engineering
In practice, feature engineering is both science and a bit of witchcraft. It often involves both iteration and experimentation to uncover hidden patterns and relationships within the data. For instance, a data scientist might transform raw sales data into features such as average purchase value, purchase frequency, or customer lifetime value, which can significantly boost the performance of a churn prediction model. By thoughtfully engineering features, practitioners can provide machine learning models with the most informative inputs, ultimately leading to better accuracy and more robust predictions.
What’s more?
- Incorporate more and more data sources
- Feature engineering platform
What is data engineering
As we mentioned above, feature engineering is certainly a subset of data engineering. It involves the ingestion of data from a source, applying a series of transformations, and making the final result available to be queried by a model for training purposes. You can construct feature engineering pipelines to resemble data engineering pipelines, having schedules, specific source and sink destinations, and availability for querying. However, this configuration would only really apply once you have surpassed the experimentation stage and determined a need for a consistent flow of new feature data.
What is feature engineering

1. Functions
Functionally, there is nothing to differentiate data vs features - data points (link). Where feature engineering and data engineering really differ is in the objectives and motivations for constructing the pipelines. In general, data engineering serves a broader, more unified purpose than feature engineering. Data engineering platforms are constructed to be flexible and universal, ingesting various types and sources of data into a unified storage location where any number of transformations and use cases can be applied. The intent of a well constructed fact table or gold layer in a data lake is to provide a single source of truth that answers many different questions, produces many reports, and can be consumed by many downstream customers.
2. Practise
And in practice, an organization’s data engineering team will be responsible for the curation and maintenance of all data pipelines, not just those that relate to machine learning. These pipelines may power BI dashboards used by C-Suite, auditing reports that feed payroll, or event logs that show a user’s history of actions within the application.
Feature engineering, on the other hand, serves a specific purpose, finding the tailored inputs and columns that will generate the best predictive results for a machine learning model. Data scientists and machine learning engineers are not tasked with developing a universal data model that will ingest all data points throughout an organization, they just need to select, curate, and clean the data needed to power their models.
3. Machine learning
Now, as machine learning teams grow and begin to incorporate more and more data sources into their models, their feature engineering platform may start to resemble a larger data engineering platform in the tools and methodologies they employ. But, the intent is not to establish flexible data models that can be used throughout the organization - it is simply to power their machine learning models.


.png)



