Getting the Most of AI in Threat Detection and Hunting

Despite these challenges, AI remains a powerful tool that can significantly enhance threat detection and hunting capabilities. Organisations that embrace AI, investing in its development and deployment, gain a competitive advantage in the cyber security landscape. A key thing to note about AI is that it is rapidly evolving and improving, so challenges that currently exist that make AI inappropriate for certain tasks may soon no longer be a challenge. For example, in the case of hallucinations, GPT4 hallucinates less than GPT3, and GPT5 is likely to hallucinate much less than GPT4 and hopefully gain the ability to say when it does not know the answer.

To maximise the benefits of AI for threat detection and hunting, organisations should consider the following recommendations:

• Start Small: Begin with a small pilot project to assess the benefits and challenges of AI before implementing it across your entire security infrastructure. Simple low-level tasks that are often repeated by analysts, ideally tasks with minimal impact if they go wrong. Set up validation on these tasks and monitor for a period, fine tuning where necessary until you hit a minimum required level of accuracy.
• Choosing the Right Tools: Carefully evaluate and select AI tools that align with your organisation’s specific needs and budget. Be aware that using third party tools carries a risk that when these tools change, any systems you have integrated them into may also need updating. For example, the OpenAI API changed the size of its context window, if you had built prompts that used the full length of the previous context window, these prompts would now return an error message. Monitoring and validation should be used to detect when errors like this are introduced. In addition, AI tools that have direct access to the live internet are essential when creating detection rules that are designed to protect against the most up to date emerging threats.
• Train Your Analysts: Provide comprehensive, and ongoing, training to your security analysts, ensuring they understand AI limitations and can effectively interpret AI-generated alerts.
• Monitor and Evaluate: Continuously monitor the performance of your AI tools, identifying areas for improvement and making necessary adjustments.
• Verify: In its current state, AI should be viewed as a tool to enhance the capabilities of analysts and remove some of the leg work, however, whenever possible humans should be kept in the loop to use and verify the output of AI rather than human analysts being replaced by AI.