Casey Ellis on the cybersecurity implications and applications of Generative AI.
Subscribe: Apple • Spotify • Overcast • Google • AntennaPod • Podcast Addict • Amazon • RSS.
Casey Ellis is Founder/Chair/CTO of Bugcrowd, a Crowdsourced Cybersecurity Platform. Bugcrowd recently released “Inside the Mind of a Hacker 2023”, an interesting report that provides insights into the motivations, challenges, and specializations of hackers, as well as security implications of AI. This episode delves into the cybersecurity implications and applications of Generative AI.
Interview highlights – key sections from the video version:
- Section I: Attacking LLMs and other foundation models
- What are teams doing today to secure their custom LLMs
- What are some initial steps that teams should do to harden their custom LLMs
- Key takeaways from Section I (securing LLMs)
- Section II: LLMs for Cyber Offense
- Section III: LLMs for Cyber Defense
- LLMs in next-gen threat assessment tools
- LLMs to help fix bugs and vulnerabilitites in (open source) software
- Section IV: Social, Legal, Ethical implications of Generative AI
- ❛I believe that evaluating potential risks during the design stage is paramount. With technologies like LLM, once they’re trained and you begin to layer additional functionalities on them, it becomes increasingly challenging to backtrack and retrain, especially based on the subsequent assumptions you’ve made. Thus, concepts like malicious training, poisoning, and if there’s a supervisory loop, malicious supervision, should all be considered during design. It’s vital to envision potential threats: What would someone with ill intent do? If I were an adversary, how might I exploit the system I’m creating? While some people have a knack for such foresight, often stemming from a background in cybersecurity like mine and many of those in a bug bounty context, you don’t have to be a self-proclaimed hacker to think critically. It’s crucial to solicit diverse feedback and challenge your assumptions during design.
- A video version of this conversation is available on our YouTube channel.
- What We Can Learn from the FTC’s OpenAI Probe
- Large Language Models in Cybersecurity
- Entity Resolution: Insights and Implications for AI Applications
- Ram Shankar: Securing machine learning applications
- Dan Geer and Andrew Burt: Security and privacy for the disoriented
- Vector Database Primer
- Emil Eifrem: The Future of Graph Databases
- Navigating the Intricacies of LLM Inference & Serving
- Specialized Hardware for AI: Rethinking Assumptions and Implications for the Future
- Brian Raymond: ETL for LLMs
- Jerry Liu: An Open Source Data Framework for LLMs
- Andrew Feldman: The Rise of Custom Foundation Models
One persistent challenge with emerging technologies, as witnessed in fields like IoT and connected cars, is the rush to innovate. When new technology emerges, there’s an innate drive to create rapidly, which often overlooks safety and security. It’s vital to pause and assess potential vulnerabilities or harm and design solutions proactively.
… You know, to me, adversarial or hacker input is about examining the assumptions of a designer and turning them on their head to see the results. Often, designers find it challenging to do this themselves. They’re mainly focused on just making their design function correctly. This is where the importance of collective input shines. External feedback is invaluable and essential. As you mentioned, this feedback can be integrated at any stage in the design process. Once a design is trained and deployed, it’s essential to revisit and ask, ‘What did we overlook? Did we address all potential areas of abuse?’ If vulnerabilities are found, we should revert to the design phase, evaluating the training data and underlying assumptions of the model. This kind of feedback can be incorporated throughout the design process. The more we apply such thinking, the more robust the resulting system becomes. That’s why I strongly advocate for continuous feedback throughout the process.
… I’d like to add just one brief point: treat security as a continuous journey. It’s like a North Star, not a destination. You can’t just finish and consider the job done. Because, adversaries, those looking to exploit systems, will always seek to innovate beyond our defenses. We can’t simply relax once we feel we’ve gotten it right. New challenges will always arise. So, incorporating security as a core aspect of engineering is, I believe, the correct approach.❜
– Casey Ellis, Founder/Chair/CTO of Bugcrowd
Related content:
If you enjoyed this episode, please support our work by encouraging your friends and colleagues to subscribe to our newsletter: