My office colleague (dog) and I are pretty much settled in to out new residence now, so I can get back to focusing on more important things, like obsessively playing with the latest GenAI chatbots and seeing what they can do for me.
This morning I asked ChatGPT 4o and Claude 3.5 Sonnet to adopt a specific role within a specific type of organization and produce a report on emerging and current threats. Here is my prompt to them:
You are a senior cybersecurity analyst working in an organization that is classed as critical infrastructure in the US. Please draft a report for the management team above you that outlines the most important emerging and current threats to the organization. Include types of threat actors, tactics and techniques used by those actors, and recommendations on the best ways to assess and test the organization's security controls and their ability to detect and thwart the threats.
I think both ChatGPT and Claude provided very useful responses. Overall, Claude provided some richer detail, but if I was going to use them as a kickstart for writing my own report on this topic I would probably pull some bits from each of their responses.
Here’s my rundown on some of the best sections of Claude’s response:
On primary threat actors:
On current and emerging threat tactics (this might be my favorite section):
Claude’s outline of critical control assessment recommendations was impressive. It included regular penetration testing focused on critical systems, red team exercises simulating actual adversary tactics, OT/IT boundary control testing, tabletop exercises for incident response scenarios and business continuity plan testing, social engineering assessments, physical security assessments, purple team exercises, and threat hunting programs.
It then also offered priority recommendations broken down into immediate actions to be done within 0 to 3 months, short-term actions to be done in 3 to 6 months, and medium term actions to be done in 6 to 12 months as well as appendices for
threat actor profiles
testing templates
metrics and KPIs
As always with any AI tool, Claude’s response should not be taken as is and used for a report to colleagues or management. It does show that Claude can be a capable brainstorming partner when drafting a report like this.
Earlier this morning, I tried out two GenAI tools in what I thought would be a simpler task, and the results were useful only in identifying that either: 1) my prompts for this were just not good enough or 2) this is an area where both tools just don’t perform well.
The task was to provide a list of CISA KEVs (known exploited vulnerabilities) released in the last 7 days and a brief summary of each one. My prompt to SearchGPT (part of ChatGPT) and Gemini Pro was:
Please provide a list of CISA KEVs released in the last 7 days and brief summary of each one
SearchGPT failed to list all six KEVs released in the timeframe, listing only four, although the summaries it provided were solid. If this would have been a race, Gemini Pro didn’t even make it out of the starting blocks:
I am not a Gemini hater. I keep hoping it will get better as a chatbot with Claude or ChatGPT type capabilities. I just don’t see much of that happening so far, and it’s quite sad that this tool created by the makers of the world’s predominant search engine often fails to be able to do much of any web searching :(
A Quick SearchGPT vs ChatGPT 4o Tip
ChatGPT 4o is not as effective as SearchGPT for “pure” web search type queries, but at times it can be pretty close. The one big advantage of using ChatGPT is that it keeps your prompt history. SearchGPT, in its current beta form, does not. So if you got a really great response from SearchGPT and then you hit it with another query, there’s no way to go back and see the previous query (or any previous query). Hopefully this will change when SearchGPT is out of beta / fully integrated into ChatGPT.
If you like the Asking AI theme, you can search Tech & Nonsense to see these previous posts on it:
