...

/

Other Model Attacks

Other Model Attacks

Learn about how models can be hacked or coerced into revealing private information.

Model security is essentially just cybersecurity for models. It has been demonstrated many times in the past that when attacked in the right way, models can reveal sensitive information about the data they were trained on. This can be a big risk for companies with data that must comply with legislation like HIPAA and GDPR.

The need for model security

In recent times, it has been demonstrated that LLMs (particularly ChatGPT) can occasionally surface an individual's data. Recall the famous Samsung case, in which Samsung employees used ChatGPT for something related to one of their proprietary products, leading to leaked private information elsewhere.

Even in traditional ML domains, models can be attacked (called privacy attacks) to force them to reveal private data. As we’ve seen in other lessons, they can also be attacked to cause nefarious consequences to happen (adversarial attacks). In an excellent 2021 paperRigaki, M., Garcia, S. “A Survey of Privacy Attacks in Machine Learning.” 2021., researchers outlined the various ways machine learning algorithms (not just centralized ML, but federated ML too) can be subject to privacy attacks.

The need for model security has never been greater. Adversarial attacks are ...