Other Model Attacks
Learn about how models can be hacked or coerced into revealing private information.
Model security is essentially just cybersecurity for models. It has been demonstrated many times in the past that when attacked in the right way, models can reveal sensitive information about the data they were trained on. This can be a big risk for companies with data that must comply with legislation like HIPAA and GDPR.
The need for model security
In recent times, it has been demonstrated that LLMs (particularly ChatGPT) can occasionally surface an individual's data. Recall the famous Samsung case, in which Samsung employees used ChatGPT for something related to one of their proprietary products, leading to leaked private information elsewhere.
Even in traditional ML domains, models can be attacked (called privacy attacks) to force them to reveal private data. As we’ve seen in other lessons, they can also be attacked to cause nefarious consequences to happen (adversarial ...