AI's Achilles' Heel: Startups Emerge to Secure the Generative AI Stack

The increasing adoption of generative AI models has brought new security challenges. Startups are emerging to address these challenges by providing solutions in three categories: governance, observability, and security.


  • Generative AI models are increasingly vulnerable to cyberattacks. These attacks can target the models themselves (e.g., model theft) or exploit them to generate malicious content (e.g., phishing emails).
  • There are three main categories of security solutions for generative AI: governance, observability, and security. Governance solutions help organizations understand and manage their AI usage. Observability tools enable organizations to monitor the behavior of their AI models. Security solutions protect models from attacks and misuse.
  • Some of the most promising startups in the generative AI security space include Robust Intelligence, Lakera, Prompt Security, Private AI, Nightfall, Hiddenlayer, Lasso Security, DynamoML, FedML, Tonic, Gretel, Private AI, Kobalt Labs, Protect AI, and Giskard.
  • Menlo Ventures is actively investing in generative AI security startups. They are looking for teams with deep expertise in AI infrastructure, governance, and security.


Related post

AI security

Defend Against Prompt Injection Attacks: Secure Your Large Language Models

Prompt injection attacks can manipulate Large Language Models (LLMs) to perform unintended actions, potentially exposing sensitive data or causing harm. This article outlines three approaches to defend against such attacks, each with its own strengths and weaknesses. This article also introduces NexusGenAI, a platform that simplifies building secure LLM applications.