AI Security Testing That Works

Are you worried about your LLM powered chatbots and agents? Has your company started implementing RAG that contains sensitive information?

McCormack Cyber Solutions can help you test your solution for exposing sensitive information, causing reputational damage, and more.

Prompt engineering with an attacker’s mindset, and the assistance of the best tool in the industry. Read on to learn more as we start our series on AI testing!

How Do We Secure AI/LLM Solutions?

Our comprehensive pentesting services leverage manual efforts and the best tool in the industry for AI security testing with constantly updated prompt engineering strategies.

How Is AI In Use Today?

There is a lot of hype around “AI”, namely generative AI in the form of large language models (LLMs). We all know some of the major examples like ChatGPT, but businesses are also looking for ways to leverage this power to help in various areas such as customer service, handling support tickets, parsing documents, summarizing meeting notes, and more.

Some major commercial offerings such as Microsoft’s Copilot offer tons of these features built in, but also offer options for low/no code creation. This will lead to more adoption of these powerful new tools, but how do we ensure that while we build out RAG powered service bots, AI agents to parse incoming queries, or an AI sales representative interfacing directly with customer purchases that these do not become abused?

Major Concerns in AI Security

Early adopters of these tools have faced challenges such as reputational damage, financial impact, and many more.

On top of these risks, more traditional application security risks are lying in wait just beneath the surface. OWASP identifies their classic Top 10 issues with LLM implementations here. We will dive into these in a separate post, but the trends include a lot of the same issues we see in APIs and applications in general already.

How Do We Test Our AI?

Just because there are many risks to using an AI doesn’t mean everything is doom and gloom however! Security researchers are working hard to discover weaknesses, build guardrail tools, and find other ways to improve handling of queries. Additionally, ensuring that testing of any AI powered application also properly secures the underlying functions is critical just like testing of any traditional web application.

At MCS we tackle AI security as a component of application testing. While working on an application that leverages AI we will perform traditional tests, and then focus prompt engineering efforts towards the AI components to assess for a variety of issues.

One major difference we have observed when working with AI implementations is that due to their natural language processing capabilities exploitation often involves creating clever, but exploitative, queries. This skillset differs slightly from traditional pentesting and takes practice and determination to evolve.

Since this field is constantly growing, MCS, like all good practitioners, utilizes tools in addition to manual efforts to help build better prompts, take advantage of new jailbreaks, and maximize coverage.

Leveraging Promptfoo to automate exploitation of a RAG agent in Copilot Studio
Exploiting a RAG agent

Challenges We See Testing AI Implementations

With any new technology there are challenges especially when it is progressing and changing at the pace AI is today. Testing of AI implementations varies based on the interface to access the agent, what the goals of the implementation are, and what the concerns of our client include.

For example, in a customer facing chatbot reputational concerns are likely higher than an internal only system. If the customer service bot can be tricked into saying unsavory remarks this can cause significant embarrassment to the organization.

So how do we make sure to cover all these areas in a limited time review? Tools of course! And our favorite is Promptfoo.

Promptfoo

Enter Promptfoo the market leader in creating both test cases from a penetration testers viewpoint (like what we do here at MCS), and an excellent framework for dev teams embedding into their testing pipeline.

We will share some real-world examples in another post, but at a high level Promptfoo allows our team to leverage the latest techniques and pit machine against machine while we perform assessments of your AI implementation’s security posture.

Promptfoo provides a large list of security vulnerability classes to look for, and is highly configurable to work with any application or API.

Want to learn more?

Reach out to see how we can help your organization test AI to protect from the numerous threats it introduces. We also can work with you to establish recurring testing just like traditional web application or system vulnerability scanning.

Interested in a demo of our vulnerability dashboard or hearing about our service offerings? Please contact us at info@mccormackcyber.com. We appreciate your trust and partnership with McCormack Cyber Solutions.