![](\"https:\/\/d15shllkswkct0.cloudfront.net\/wp-content\/blogs.dir\/1\/files\/2024\/08\/Black-Hat-2024-2.jpg\")
<\/p>\n\n<\/p>\n
The good news in the cybersecurity world is that wider deployment of artificial intelligence has not yet opened a massive security hole in the world\u2019s computer systems. The bad news: Flaws and vulnerabilities are beginning to appear that could foreshadow major problems ahead.<\/p>\n
That was the prevailing sentiment expressed by numerous research analysts and industry executives at the Black Hat USA<\/a> cybersecurity conference in Las Vegas this week. AI may be relatively new to many threat actors, but the technology industry\u2019s familiarity with AI has provided it with a certain measure of protection\u2026 so far.<\/p>\n \u201cMy view right now, when you think about AI versus AI, is that the defense is winning,\u201d\u00a0Chris Krebs<\/a>, a SentinelOne Inc. executive and former director of the U.S. Cybersecurity and Infrastructure Security Agency, said during an appearance at Black Hat\u2019s inaugural AI Summit<\/a> on Tuesday. \u201cWe are seeing some leading-edge capabilities that are actually working. In the state actor space\u2026 they are still messing around with what works. The defense is still outpacing the offense.\u201d<\/p>\n Some of those leading-edge capabilities involve added support for security analysts. AI can take the pressure off weary personnel in security operations centers who suffer from alert overload. It can also provide operational insight much more rapidly, according to Alex Stamos<\/a>, chief information security officer at SentinelOne and former chief security officer for Facebook and Yahoo.<\/p>\n \u201cThe best defensive use of generative AI is analyst efficiency,\u201d Stamos told SiliconANGLE. \u201c\u2019Show me all of my laptops that have talked to a Russian IP address in the last 24 hours.\u2019 It\u2019s taking normal smart humans and upleveling them into super analysts.\u201d<\/p>\n This notion of leveraging AI to provide critical source data has been taken a step further by Dataminr Inc.<\/a> The 15-year-old company, reported to be preparing<\/a> for an initial public offering, began using AI in 2018 to integrate descriptions of what its global network of sensors was detecting.<\/p>\n In 2022, Datminr was selected<\/a> by the Defense Information Systems Agency to provide social media monitoring information to the White House. The firm\u2019s generative AI capabilities received a demonstration in July when Datminr began to pick up signals from social media posts about issues with Microsoft Windows. The service provided early alerts that summarized the narrative in what emerged as the CrowdStrike global outage.<\/p>\n \u201cGenerative AI has the capability to be tied into predictive AI systems to automatically describe the signals in front of you,\u201d\u00a0Ted Bailey<\/a>, Dataminr\u2019s founder and chief executive, said in a presentation during Black Hat\u2019s AI Summit.<\/p>\n The looming problem is that current spending on AI far outpaces investment in security to protect it. Dave Dewalt<\/a>, chief executive of NightDragon and former CEO of FireEye and McAfee, noted during the AI Summit that there had been $67 billion in AI investments over just the past 12 months.<\/p>\n \u201cTake a guess of how much security investment has gone into this in the same amount of time,\u201d Dewalt said. \u201cAbout $300 million. We have to catch up security to AI. We can\u2019t let that gap be there.\u201d<\/p>\n Gaps are already being exposed by security practitioners in how generative AI models are protected. Researchers from Wiz Inc. presented a report at Black Hat on Wednesday detailing how they breached model repositories in AI as-a-service providers\u00a0Hugging Face<\/a>, Replicate<\/a>\u00a0and SAP<\/a>. They reported their exploits to all three companies, which have since corrected the vulnerabilities disclosed.<\/p>\n \u201cWe were able to get access to millions of public and private AI models\u2026 and we had the ability to interfere with all this data,\u201d said Hillai Ben Sasson<\/a>, a security researcher at Wiz. \u201cThis is confidential data we should not have been able to access.\u201d<\/p>\n Security researchers are finding that because the AI attack surface is relatively new, it will take some time to pinpoint where the protection is needed. Nvidia Corp. has been testing weaknesses in large language model structures and found that retrieval-augmented generation or RAG plugins can be especially vulnerable.<\/p>\n \u201cYou can specifically target models for poisoning people on their results if you have access to a specific RAG store,\u201d said Rich Harang<\/a>, principal security architect at Nvidia. \u201cUnfortunately, this is just how RAG works. Limit the data that you have that RAG applications have access to.\u201d<\/p>\n While the security community is focusing on AI models, there is also growing concern around the data that fuels them. Jennifer Gold<\/a>, head of threat intelligence for the FBI-driven public\/private collaboration New York Metro InfraGard<\/a>, noted that ChatGPT and Copilot data had been found on platforms used in the dark web.<\/p>\n One Singapore-based cybersecurity firm reported that more than 225,000 logs<\/a> containing compromised ChatGPT credentials were available for sale in the underground. That could potentially open new pathways of threats as malicious actors are able to tap into data stores that were not readily available before.<\/p>\n \u201cLots of companies are focused, rightfully so, on threats to models,\u201d\u00a0Steve Stone<\/a>, head of Zero Labs at Rubrik Inc., told SiliconANGLE. \u201cI\u2019m concerned about the data. If organizations are already struggling with their data today\u2026 what happens when they have seven times that data? I am deeply concerned that you have a bunch of threat actors that are able to find a much deeper data surface.\u201d<\/p>\n History is not necessarily on the security community\u2019s side for getting ahead of the curve when a technology wave comes ashore. The world\u2019s embrace of the internet spawned a whole new class of threats and disruptions that is still playing out. Mobile platforms have become prime targets in recent years, with reported deepfake attacks on banking apps<\/a> and other smartphone exploits<\/a>.<\/p>\n \u201cAre we handling AI functionally different than any new technology?\u201d Stone said. \u201cI don\u2019t think we are. We probably have already had a really nasty AI intrusion that we don\u2019t even know about.\u201d<\/p>\n This possibility is drawing renewed attention from regulators. Representatives from several government agencies were part of the speaker lineup at Black Hat this week and several expressed concern that, in the urgency to pile onto the AI bandwagon, businesses are not thoroughly testing deployments.<\/p>\n \u201cI am concerned about ways that people are rushing to get AI products to market without safety and security testing,\u201d said Lisa Einstein<\/a>, newly appointed chief AI officer for CISA. \u201cWe see people not being really clear about the ways that security can be brought in.\u201d<\/p>\n Governments are becoming increasingly more motivated to develop a regulatory framework around AI. IDC has reported<\/a> that 60% of governments worldwide will adopt a risk management approach to framing generative AI policies by 2028.<\/p>\n \u201cIt is going to be a really rich enforcement ecosystem,\u201d said former CISA head Krebs.<\/p>\n Much as AI has accelerated the pace of many applications, it has also contributed to a sense within the security community that the threat landscape is shifting rapidly as well. Previous technology waves of adoption have allowed security researchers time to analyze the data and craft ways to combat escalating attacks. Yet, as Black Hat founder Jeff Moss<\/a> told the gathering in his conference keynote on Wednesday, there is an uneasy feeling that this rapidly advancing wave is going to be very different.<\/p>\n \u201cWe\u2019ve got this giant bucket of other problems that\u2019s making it feel like things are speeding up,\u201d Moss said. \u201cIt just feels like it\u2019s different and it feels like it\u2019s getting different faster.\u201d<\/p>\nUpleveling analysts and describing signals<\/h3>\n
Security flaws in AI repositories<\/h3>\n
Data as a blind spot<\/h3>\n
Photo: Mark Albertson\/SiliconANGLE<\/h5>\n