In this presentation, the core AI Red Team at Meta will take you on a journey through the story of Red Teaming the Llama 3 Large Language Model. This talk is perfect for anyone eager to delve into the complexity of advanced model Red Teaming and safety, as well as how to perform their own research to find new attacks should attend this talk. We’ll begin by exploring what AI Red Teaming is truly about, before exploring Meta’s process and approaches on the topic. The team will detail our methodology for discovering new risks within complex AI capabilities, how emergent capabilities may breed emergent risks, what types of attacks we’re looking to perform across different model capabilities and how or why the attacks even work. Moreover, we’ll explore insights into which lessons from decades of security expertise can – and cannot – be applied as we venture into a new era of AI trust and safety.

The team will then move on to how we used automation to scale attacks up, our novel approach to multi-turn adversarial AI agents and the systems we built to benchmark safety across a set of different high-risk areas. We also plan to discuss advanced cyber-attacks (both human and automated), Meta’s open benchmark CyberSecEvals and touch on Red Teaming for national security threats presented by state-of-the-art models. For each of these areas we’ll touch on various assessment and measurement challenges, ending on where we see the AI Red Teaming industry gaps, as well as where AI Safety is heading at a rapid pace.

AI assistants like ChatGPT are changing how we interact with technology. But what if someone could read your confidential chats? Imagine awkwardly asking your AI about a strange rash, or to edit an email, only to have that conversation exposed to someone on the net. In this talk we'll unveil a novel side-channel vulnerability in popular AI assistants and demonstrate how it can be used to read encrypted messages sent from AI Assistants.

Before our disclosure, major players like OpenAI, Microsoft, Cloudflare, Quora, and Notion were at risk. We'll reveal the technical details of this exploit and show real-world examples of intercepted conversations. This talk isn't just about the problem – learn how to identify this vulnerability in other AI assistants as well! We'll dissect network traffic, discuss attack models, and explore the far-reaching consequences of this discovery.

Machine learning (ML) pipelines are vulnerable to model backdoors that compromise the integrity of the underlying system. Although many backdoor attacks limit the attack surface to the model, ML models are not standalone objects. Instead, they are artifacts built using a wide range of tools and embedded into pipelines with many interacting components.

In this talk, we introduce incubated ML exploits in which attackers inject model backdoors into ML pipelines using input-handling bugs in ML tools. Using a language-theoretic security (LangSec) framework, we systematically exploited ML model serialization bugs in popular tools to construct backdoors. In the process, we developed malicious artifacts such as polyglot and ambiguous files using ML model files. We also contributed to Fickling, a pickle security tool tailored for ML use cases. Finally, we formulated a set of guidelines for security researchers and ML practitioners. By chaining system security issues and model vulnerabilities, incubated ML exploits emerge as a new class of exploits that highlight the importance of a holistic approach to ML security.

Come ask us anything about the GRT2! There will be a short presentation about how to participate and the objectives of the event and then you can ask us anything about it and ML flaws and vulnerabilities.

AI’ll be watching you will cover attacking an embedded AI on a family of popular security cameras with over 100,000 combined reviews on Amazon. The camera’s embedded AI system is used for on-device person detection, a system that filters notifications based on whether a person is detected. Traditionally the camera would alert the owner if any motion was detected, meaning that an attacker would have to have no motion be detected, but now with the embedded AI making decisions, an attacker needs to only appear not to be human. While this may seem a simple task, dressing up as a giant bush would be noticeable by the people around the attacker, meaning that a successful attack against this system requires the on-camera AI to be tricked while not alerting nearby people to any suspicious disguises.

In this talk we will cover the steps we took to research and gain access to the device in order to perform greybox attacks against its embedded AI. We will demonstrate how we rooted an older version of the device to gain access to how the models were brought to the camera. We will show how the knowledge we gained while reverse engineering let us download the models for any arbitrary device or firmware and, eventually, how we were able to exploit and gain root on the newer, more secure device. We will show the audience our process in which we discovered and reverse-engineered a proprietary model format that we had never seen before. Finally, we will show how, once we understood the model, we were able to perform attacks against both it and the camera.

The purpose of this talk is to raise awareness about the insecurity of embedded AI as well as to demonstrate how known attack techniques can be used on never-before-seen models, showcasing that AI/ML research has truly passed the infant stage and has reached a point where developed methods can be broadly applied.

This talk will focus on the implications of our work defending AI based cybersecurity systems against file format abuse for the design of AI systems for cyber. The audience will learn how the interface between traditional cybersecurity systems and the AI models being integrated into them impacts security. File format abuse enables polyglot files to bypass state-of-the-art malware detection systems (EDR tools) that utilize machine learning in an attempt to catch novel forms of malware. The polyglot file is sent to the wrong model because the embedded file type is not detected. Existing file type, file carving, and polyglot detection tools are insufficient to detect polyglots used by threat actors in the wild. However, we trained a machine learning model capable of detecting all polyglot types in our dataset, which is based on threat actor usage of polyglots in the wild, with over 99.9% accuracy. Content disarm and reconstruct (CDR) tools can also be used to disarm polyglots, but are not effective on all file types.

Christina will speak to the latest MITRE ATLAS community efforts focused on capturing and sharing cross community data on real world AI incidents, expanding the community’s data on vulnerabilities that can arise when using open-source AI models or data, especially for vulnerabilities that fall outside of the scope of CVE/CWE, and developing mitigations to defend against these AI security threats and vulnerabilities.

MITRE ATLAS is a public knowledge base of adversary tactics and techniques based on real-world attack observations and realistic demonstrations from artificial intelligence (AI) red teams and security groups. There are a growing number of vulnerabilities in AI-enabled systems as the incorporation of AI increases the attack surfaces of existing systems beyond those of traditional cyberattacks. We developed ATLAS to raise community awareness and readiness for these unique threats, vulnerabilities, and risks in the broader AI assurance landscape.

As artificial intelligence and machine learning increasingly become the backbone of our cybersecurity infrastructure, we face a new set of ethical challenges that go beyond traditional security concerns. This keynote dives into the critical issues of fairness, transparency, and accountability in AI-driven security systems. We’ll explore the relevance of AI ethics to safety and security testing, especially red teaming efforts. Finally, we’ll discuss the importance of ethical AI development in cybersecurity, emphasizing the need for diverse development teams, rigorous testing for biases, and ongoing audits of AI systems in production. This keynote aims to spark a crucial conversation in the hacker community about our responsibility to ensure that as we push the boundaries of AI in security, we don’t lose sight of the human values and ethical principles that should guide our work.

Large Language Model (LLM) deployment and integration comes with a need for scalable evaluation of how these models respond to adversarial attacks. However, LLM security is a moving target: models produce unpredictable output, are constantly updated, and the potential adversary is highly diverse: anyone with access to the internet and a decent command of natural language. Further, what constitutes a weakness in one context may not be an issue in a different context; one-fits-all guardrails remain theoretical. It is time to rethink what constitutes ``LLM security’’, and pursue a holistic approach to LLM security evaluation, where exploration and discovery of issues are central. To this end, this paper introduces garak (Generative AI Red-teaming and Assessment Kit), a framework which can be used to discover and identify vulnerabilities in a target LLM or dialog system. garak probes an LLM in a structured fashion to discover potential vulnerabilities. The outputs of the framework describe a target model’s weaknesses, contribute to an informed discussion of what composes vulnerabilities in unique contexts, and can inform alignment and policy discussions for LLM deployment.

The hype for integrating artificial intelligence into an enterprise’s daily work has become more prevalent after introducing AI-driven systems that use Retrieval Augmented Generation (RAG), such as Copilot for Microsoft 365. But is the trust in such systems and their control over decision-making processes within enterprises rational? Copilot and other RAG-based systems can be misused to cause dissemination of misinformation that negatively impacts decision-making processes without proper auditing and safeguarding of data available to large language models in RAG-based systems.

This talk will demonstrate such an attack that we have termed ConfusedPilot because of its ability to turn Copilot into a confused deputy. The attack occurs when a malicious document is introduced to the data pool (documents, presentations, other relevant files, etc.) related to a topic affecting the enterprise’s decision-making process. The malicious document contains a combination of corrupt data and malicious strings that suppress the correct documents related to the topic and respond to the user’s query with only the information present within the malicious document. Furthermore, the talk highlights how this attack can persist after deleting content within the malicious document or the document itself. The talk also points to the larger implications of such attacks, highlighting their cascading effect and existing security measures that can be used to reduce the attack’s effectiveness. Our talk sheds light on the current attacks and potential security measures that can shield enterprises from the adverse effects of such attacks on their AI-driven systems.

This panel will explore the critical challenges and opportunities in developing a robust workforce for AI and machine learning (ML) security. As AI systems become increasingly prevalent across industries, the need for skilled professionals who can safeguard these technologies against adversarial attacks and vulnerabilities has never been greater.

A key focus of the discussion will be addressing the significant shortage of practitioners with hands-on experience in securing ML models deployed in real-world adversarial environments. Panelists will examine how this lack of battle-tested expertise impacts the industry’s ability to defend against sophisticated attacks and discuss strategies for cultivating this essential skill set.

Prompt injections are a class of attacks against LLM-powered applications that exploit the inclusion of untrusted user inputs in LLM prompts. We give an overview of two open source frameworks developed by Meta related to understanding and mitigating prompt injection risks:

- our CyberSecEval Prompt Injection benchmarks (evaluations of the propensity of popular LLMs to succumb to prompt injection when used without guardrails),

- as well as PromptGuard (an open-source model for identifying risky inputs to LLM-powered applications, both direct jailbreaks and indirect injections)

Findings of interest:

- Evaluating foundation model vulnerability to indirect prompt injection: LLMs can be trained to have contextual awareness of which parts of the input prompt are coming from a trusted user versus an untrusted third party - in particular via inclusion of a system prompt. We share our benchmark for direct and indirect prompt injection susceptibility of foundational LLMs (across a wide variety of attack strategies) introduced as part of CyberSecEval (an open-source suite of benchmarks for measuring the cybersecurity risks of foundational models). We present the results of these evaluations for currently-popular foundational LLMs. We conclude that model conditioning is not enough to defend against indirect prompt injection risks in most contexts, even with the usage of a system prompt.
- Guardrailing against prompt injection attacks in real applications: We present PromptGuard, a model designed for both the detection of direct jailbreak and indirect injection attacks. We highlight the differences between our models and existing malicious prompt detectors (which largely only address direct prompt injection or jailbreaking risks), and the specific risks that can be prevented by utilizing our guardrail in LLM-powered applications. We also show how the model can be fine-tuned to improve application-specific performance.

The possibility of an altered photo revising history in a convincing way highlights a salient threat of imaging technology. After all, seeing is believing. Or is it? The examples history has preserved make it clear that the observer is more often than not meant to understand that something has changed. Surprisingly, the objectives of photographic manipulation have remained largely the same since the camera first appeared in the 19th century. The old battleworn techniques have simply evolved to keep pace with technological developments. In this talk, we will learn about the history of photographic manipulation, from the invention of the camera to the advent of generative AI. Importantly, we will consider the reception of photo editing and its relationship to the notion of reality, which is more significant than the technologies themselves. Surprisingly, we will discover that creative myth making has found a new medium to embed itself in. This talk is based on Walter Scheirer’s recent book A History of Fake Things on the Internet (Stanford University Press 2023).

For the past few months, I’ve been seeing how far I can push several commercially available GenAI systems past their ethical boundaries. … hint: it’s way too far.

In this talk, I’ll demonstrate how I was able to turn LLMs into a powerful backend for realtime, interactive voice enabled cyber scams. I’ll share my prompting strategy, social engineering tactics, the backend systems used, and show how each of these are working innocently in their own right, but enable massive possibilities for deception and harm when combined (in their current form). I’ll also cover a few key insights gained from this research, including unexpected lessons from both successful and unsuccessful attempts.

Note: this session includes demos of a violent and profane chatbot. Please do not attend if that will be offensive to you.

During the work on my SANS Master's thesis, I realized two things: I am not a developer and ChatGPT makes a pretty good one. Using ChatGPT to write the Python scripts for my research, I started to branch out and use it to write defensive tools such as for identifying unknown assets on the network as a listening service or offensively such as when taking a PLC out of Run mode remotely. If you can think through the process, ChatGPT (or other GenAI) can help you make it a reality. Want to Live off the Land and don't want to download a Python script which might be spotted? Use ChatGPT to convert it to PowerShell on the spot! Receiving error messages from the code it wrote for you? Don't worry - it can fix those issues too! The presentation will walk attendees through prompt creation for two sample coding projects - both with offensive/defensive capabilities, tools that attendees would be able to use back on the job. And, with inspiration, go out and create their own tools!

AI Goat is a deliberately vulnerable AI infrastructure designed to help security enthusiasts and pen-testers understand and exploit AI-specific vulnerabilities based on the OWASP AI Top 10. This arsenal session will demonstrate how to deploy AI Goat, explore various vulnerabilities, and guide participants in exploiting these weaknesses. Attendees will engage hands-on with the tool, gaining practical experience in AI security. Deployment scripts will be open-source and available after the session.

In this talk, we will discuss the strengths and limitations of LLMs for code analysis tasks like code search and code clone detection. We will show when the LLMs make mistakes and what kinds of mistakes they make. For example, we observe that the performance of popular LLMs heavily relies on the well-defined variable and function names, therefore, they will make mistakes when some misleading variable name is given. Anyone interested in exploring the intersection of AI and code security analysis can attend this talk.

From theory to practice: dive into the lessons learned from building and defending an LLM application. This talk offers firsthand insights into the challenges and breakthroughs experienced while developing and securing large language models in real-world settings. We'll explore critical vulnerabilities, innovative defense strategies, and practical tips for enhancing the robustness of AI applications. Join us to gain actionable knowledge that can help you navigate the evolving landscape of AI security with confidence.

How do we use the apparent magic of LLMs to help us threat model? What are the challenges? What works? What doesn’t?

Learn how DARPA's Signature Management using Operational Knowledge and Environments (SMOKE) program aims to develop data-driven tools to automate the planning and execution of emulated threat actor infrastructure needed for red team operations.

Heather Adkins will pull from over 25 years of experience, including responding to major security incidents that impacted national security, to detail how the threat landscape has evolved into what it is today with the introduction of AI. She'll provide lessons learned by the industry in applying AI for security over the years, and explain how AI can be used in arming cyber defenders tasked with protecting the critical infrastructure we rely upon every day.

ARPA-H accelerates better health outcomes for everyone by supporting the development of high-impact solutions to society's most challenging health problems. Join us in discussing why strong cybersecurity security is a critical piece of healthcare innovation and how ARPA-H is enabling this through the AIxCC, DIGIHEALS, and UPGRADE programs.

Open Source Program Offices (OSPOs) are an increasingly adopted approach to establishing and cultivating a culture of contribution. The Digital Service at CMS.gov will share the programs, policies, and projects they’re building to identify and mitigate continuity and security risks in the software supply chain across the Federal Ecosystem.

There are few opportunities to learn how code can be transformed into a visualization project. Tune in as Mark Griffin from UnDaunted shares about how his team took the competitor submissions and translated them into the AIxCC competition experience at DEF CON.

The convergence of Artificial Intelligence (AI) and national security not only fuels international discourse but also inspires narratives within popular culture. Harriet is no stranger to these myths, as an ex-intelligence professional who specialized in applying machine learning to cyber security. In fact, she likes to lean into them. This makes her previous bosses nervous, so she uses pop culture as the lens through which to communicate her insights - and in this talk she utilizes the worlds of Ghost in the Shell, Neuromancer and Mission Impossible.

Through these stories, as well as her own decade of experience working at the intersection of artificial intelligence and cyber security, Harriet discusses the extent to which fears surrounding AI systems are applicable to real life national security settings. From cyber warfare to AI-driven surveillance, she unravels the interplay between hackers, AI, and government agencies. This session is interactive, with demos of how these AI systems actually work under the hood, as well as discussion time. Blur the lines between human and machine, and understand how you can contribute your skills to prevent our own modern day Puppet Master.

Social media have been a decade-long dress rehearsal in online manipulation. AI can create information, make predictions and take decisions that will affect human behaviour, including our behaviours as citizens, workers and consumers. Safeguards are needed, since generative AI will only exacerbate the personal, social and societal harms already caused by data-driven business models.

We examine the centrality of human dignity in tech law and policy and how our mindsets and legal frameworks must be informed by psychological, technological and societal perspectives. Based on insights from market dynamics, marketing techniques, design strategies, and human frailties we demonstrate how information asymmetries have reduced individual agency and the ability to create transparency.

Human dignity is a core value in liberal democracies that must also be reflected in tech policy. Protections are required when businesses interfere with our rights to freedom, property, privacy and non-discrimination. With the digitalisation of the human experience, users have become programmable objects. We cannot rely on regulation alone and need to discuss how we can act to reclaim our dignity.

Homeland security agencies have been rapidly developing AI-powered solutions to enhance their operational effectiveness. As AI-powered systems become more ubiquitous, a greater emphasis must be placed on Cybersecurity. A career police officer, Yeow Boon understands first-hand the myriad ways AI can augment Homeland Security operations. In his current roles as the Deputy Chief Executive (Development) and Chief Information Officer of Singapore’s Home Team Science and Technology Agency (HTX), Yeow Boon oversees the agency's transformation, and by extension, the AI transformation of Singapore’s Home Team. Concurrently, he has to ensure that any risks involved in the increased attack surface associated with the development of AI-powered technologies are mitigated. In his speech, Yeow Boon will highlight the critical role of Cybersecurity in AI-driven homeland security. From the development stage to organisational best practices, he emphasises the importance of keeping Cybersecurity at the forefront of any agency’s AI transformation. During this session, look forward to gaining insights into the latest Cybersecurity threats in AI-driven Homeland Security and how to combat them.

In this presentation, we explore the integration of chatbots and large language models (LLMs) like ChatGPT in cybersecurity. We begin by explaining chatbots and LLMs, their operation, and their relevance to cybersecurity. We'll discuss practical applications on both defensive and offensive sides. Defensively, chatbots can automate tasks such as log parsing, web scraping, and data analysis, and aid in educating team members on complex security concepts. Offensively, chatbots can be employed for social engineering, phishing simulations, and automating attack techniques. Real-world examples demonstrate how ChatGPT supports security engineering by generating Python scripts, creating cybersecurity content, and assisting with complex projects. By the end, you'll understand the potential of chatbots and LLMs in enhancing cybersecurity workflows.

This panel discussion will delve into the critical intersection of artificial intelligence and cybersecurity in the context of the rapidly evolving 5G network environment. By bringing together experts in AI, 5G network security, and AI-driven solutions

OSINT is a great mechanism for the collection of data, but what do you do with all of it once it’s collected? It can take significant time with the human eye to analyze each image or video. Furthermore, you may miss important artifacts in the foreground or background. Computer vision can churn through the plethora of data to pull out the relevant evidence at lightning speed. For the last 5 years we’ve been exploring the use of Azure and AWS for computer vision to rapidly process large sets of image and video data acquisitions for forensic evidence. Through the use of AI we have analyzed thousands of images and videos to perform object detection, facial recognition, OCR, voice to text analysis, and more. In this session we’ll explore the use of cloud platforms to exponentially increase your analysis of uncovering key artifacts to your case using demos and real world examples. We then apply chronolocation analysis to allow an investigator to paint a true-life narrative, for example an individual with a weapon at a location at a particular time. We’ll provide live demos of common scenarios to reveal benefits to processing your data collections in a rapid, efficient, comprehensive, and accurate manner. We’ll then wrap-up the presentation with additional AI computer vision recommendations and resources.

Honeypots are invaluable tools for monitoring internet-wide scans and understanding attackers' techniques. Traditional low-interaction web honeypots use manual methods to emulate various applications or vulnerabilities. Introducing Galah, an LLM-powered web honeypot that mimics diverse applications with a single prompt. This honeypot dynamically crafts relevant HTTP responses, including headers and body content, to various HTTP requests, effectively simulating multiple web applications. In this talk, I will share lessons learned from building and deploying Galah and address two key questions: How do different large language models perform in generating HTTP messages? Does delivering authentic-looking HTTP responses increase attackers’ engagement with the honeypot?

We will begin with an overview of event analysis systems and their challenges. Participants will learn about different types of data sources and logs, the prevalence of false positives, and the difficulty in identifying coordinated attacks. We will set the stage for the live, hands-on demonstration environment where participants can interact and apply what they learn in real-time. Importantly, no prior data science knowledge is required; all tasks will be performed using simple, user-friendly interfaces.

Introduction to MITRE ATT&CK Framework: An essential part of our session is understanding the MITRE ATT&CK framework. We will cover its structure, including Tactics, Techniques, and Procedures (TTPs), and explain why mapping alerts to this framework is crucial for standardizing threat detection and enhancing our capabilities.

Leveraging Open Source AI Tools: Next, we will delve into the open-source AI tools that will be used throughout the session. We will introduce families of algorithms including clustering and community detection, natural language processing with large language models (LLMs), and Markov chains. These tools are designed to be accessible and will be operated through straightforward interfaces. Participants will be guided through setting up a demo environment to follow along and interact with the exercises.

Data Preprocessing and Normalization: Participants will learn how to import and clean data from various sources, normalize data formats, and handle missing data. We will highlight some methods to get rich test data. This step is crucial for ensuring that the subsequent analysis is accurate and reliable. The hands-on exercise will involve preprocessing a sample dataset in real-time, using easy-to-follow steps and intuitive interfaces.

Mapping Alerts to MITRE ATT&CK Techniques: We will demonstrate techniques for mapping SIEM data to MITRE ATT&CK manually and using automated tools. The live demo will include a hands-on exercise where participants will map a sample dataset to MITRE ATT&CK Techniques, using AI to enhance mapping accuracy. All this will be done through simple interfaces that do not require deep technical knowledge.

Clustering Alerts into Contextualized Attack Steps: This section focuses on methods such as clustering and community detection. Participants will learn the criteria for clustering alerts based on temporal, spatial, and technical attributes. They will engage in a hands-on exercise to cluster sample alerts and evaluate the quality and relevance of the clusters, again using user-friendly interfaces.

Building Killchains: Participants will understand the concept and importance of killchains in cybersecurity. We will demonstrate methods for linking attack steps into a cohesive killchain, with a hands-on exercise to create a killchain from clustered data. Participants will analyze killchains to identify patterns and coordinated attacks, all through accessible interfaces.

Generating Actionable Tickets: We will outline the criteria for generating three types of tickets: FP Tickets, Incident Tickets, and Attack Story Tickets. Through a hands-on exercise, participants will generate sample tickets and learn how to ensure each ticket type is comprehensive and actionable. This process will be facilitated through simple interfaces that guide the user step-by-step.

Integrating and Automating the Workflow: Finally, we will discuss integrating this workflow into existing SOC setups and automating the process using scripts and tools. Participants will see how to maintain and update the system, ensuring continuous improvement in threat detection and response. The automation will be demonstrated in a way that requires minimal technical skills.

Q&A and Troubleshooting: The session will conclude with an open floor for questions, addressing common issues, and offering troubleshooting tips. Participants will also receive resources for further learning and support to continue enhancing their skills post-session.

Conclusion: By the end of this interactive session, participants will have hands-on experience using open-source AI tools to enhance their SOC capabilities. They will be able to map alerts to MITRE ATT&CK Techniques, cluster data into meaningful attack steps, and build comprehensive killchains to uncover coordinated attacks. Additionally, they will learn to generate actionable tickets to facilitate immediate response and long-term improvements in their security posture. All of this will be achieved without needing advanced data science knowledge, thanks to the simple and intuitive interfaces provided.

Participants are encouraged to apply these techniques in their own environments and continue exploring the vast capabilities of open-source AI in cybersecurity. The live demo environment setup will provide a practical and engaging way to solidify these concepts and skills.

Given a SIEM loaded with alerts, logs and events from a variety of data sources, your task is to find the coordinated attack in the LOTS of noise of false positives & lone incidents.

Together we will use opensource AI tools to map all of the hetrogenous data on the SIEM to MITRE ATT&CK Techniques, and then Cluster based on a variety of attributes to form contetualized Attack Steps. We will then fuse these attack steps based on timeline, causality and assets involved into killchains to reveal coordinated attacks.

You are required to output the following tickets:

- FP Ticket that has clusters of false positives and tuning advisories & suggestions that should be forwarded to detection engineering to tune.

- Incident Ticket that has remediation & investigation advisories & action playbooks for the contextualized lone-incidents identified.

- Attack Story Ticket that has a correalted set of clusters of alerts & logs revealing a coordinated attack killchain affecting a variety of assets over a stretch of time.

As security researchers, we constantly attempt to stay ahead of the curve, seeking innovative solutions to enhance our offensive security strategies. In recent years, the advent of artificial intelligence (AI) has introduced a new dimension to our efforts, particularly in the realm of bug bounties and pentesting. While significant attention has been given to understanding and mitigating attacks against AI systems, the potential of AI to assist in the offensive security field remains largely unclear.

This talk pretends to dig into the research and development process undertaken to create an AI agent designed to augment the bug bounty and pentesting workflow. Our AI agent is not merely a theoretical concept but a practical tool aimed at enhancing the efficiency and effectiveness of security researchers.

We have conducted extensive research to understand how AI can mimic and enhance human intuition and creativity in identifying vulnerabilities. While this may sound trivial, there is little evidence of this being tested before on generative AI agents. Our work breaks new ground by pushing the boundaries of what AI can achieve in offensive security.

Will AI become an indispensable tool in our arsenal, capable of autonomously identifying and exploiting vulnerabilities? Join us as we explore the possibilities and implications of AI as an offensive assistant in this new era of offensive security.

Jailbreak vulnerabilities in Large Language Models (LLMs), which exploit meticulously crafted prompts to elicit content that violates service guidelines, have captured the attention of research communities. While model owners can defend against individual jailbreak prompts through safety training strategies, this relatively passive approach struggles to handle the broader category of similar jailbreaks. To tackle this issue, we introduce FuzzLLM, an automated fuzzing framework designed to proactively test and discover jailbreak vulnerabilities in LLMs. We utilize templates to capture the structural integrity of a prompt and isolate key features of a jailbreak class as constraints. By integrating different base classes into powerful combo attacks and varying the elements of constraints and prohibited questions, FuzzLLM enables efficient testing with reduced manual effort. Extensive experiments demonstrate FuzzLLM's effectiveness and comprehensiveness in vulnerability discovery across various LLMs.

The Transparency Algorithm (TTA): AI's Answer to Legal Equality is a groundbreaking initiative to dismantle the deep-rooted biases and systemic disparities that have plagued the American legal system for far too long. This revolutionary effort leverages the unparalleled power of artificial intelligence (AI) and machine learning (ML) to shine a glaring light on the injustices that have condemned generations of marginalized communities to a cycle of inequity and despair. Imagine a justice system where your fate is determined not by the content of your character but by the color of your skin, your gender, or your socioeconomic status. This is the grim reality that TTA seeks to eradicate. TTA meticulously analyzes public court case data to expose the hidden biases and discriminatory practices perpetuating injustice. This initiative doesn't just aim to reform; it seeks to revolutionize the very foundation of our legal system, bringing about the transparency and accountability that have long been overdue. At its core, TTA is a powerful tool for democratizing access to justice. Empowering citizens with data-driven insights provides a platform for informed public discourse and advocacy for systemic reform. The time for change is now, and TTA is the catalyst we need to forge a path towards a truly equitable legal landscape.