In recent months, a new type of AI model has rapidly emerged, making waves across the tech industry. On September 15, TypeSafe AI introduced Jev, a System One model inspired by Kahneman’s fast thinking approach. Shortly after, OpenAI unveiled its Decisions API during DevDay, launching a public beta on October 6. Cribl followed suit with cribl-decision, an open LoRA adapter, and other companies like Laya are integrating similar innovations into open-source tools.
These decision models operate differently by bypassing text generation. Instead, they evaluate a given state and set of questions to deliver a decision, score, or probability-based response.
Why SOCs Should Embrace AI Decision Models
The introduction of decision models is proving to be a significant development for security operations centers (SOCs). Their ability to facilitate alert triage through three key attributes is particularly advantageous. First, they provide a quantifiable response that can be automated. For example, alerts can be categorized based on probability thresholds—automatically closing those deemed benign and escalating potential incidents.
Additionally, the affordability of models like Jev allows them to be used for all alerts, removing the economic constraint of only addressing high-severity alerts. Every decision becomes an auditable record, enabling easy logging and comparison with actual outcomes.
However, it’s important to note that while models claim not to ‘hallucinate,’ incorrect inputs can still lead to erroneous outputs. Calibration ensures a model’s predictions are statistically sound, but the quality of input data remains crucial.
The Role of Forensic Evidence in Decision Making
When it comes to security investigations, understanding the context of alerts is essential. As Filip Stojkovski describes it, ‘Security Context Engineering’ involves transforming alerts into actionable facts. This process involves gathering detailed evidence, such as process trees, command lines, and forensic analyses, to accurately determine the nature of a threat.
Traditional forensic evidence, like memory scans, reveals critical information about active threats that might otherwise be overlooked. For instance, tools like Mimikatz and Cobalt Strike, which typically reside in memory, can evade file-based detection systems.
The challenge lies in the automatic collection of this data for each alert in a timely manner, which is essential for effective incident response.
Integrating Organizational Knowledge
Beyond forensic evidence, understanding organizational context is vital for effective decision-making. Identity alerts often highlight this gap, as models cannot inherently understand the nuances of user behavior within a specific organization.
Maintaining an up-to-date organizational context, which evolves with changes in assets, users, and tools, is challenging. This dynamic environment requires continuous learning and updating to ensure effective decision-making, a process that involves leveraging past cases and live data to inform future decisions.
While decision models enhance the efficiency and cost-effectiveness of SOC operations, their accuracy depends heavily on the quality of input data and contextual understanding. By focusing on evidence collection and organizational knowledge, SOCs can optimize their use of these models, ensuring that automated decisions align with real-world outcomes.
If you’re interested in building robust security pipelines, consider prioritizing evidence gathering and maintaining comprehensive organizational knowledge. These elements are crucial for leveraging the full potential of AI decision models in security operations.
