The demand for AI safety and accountability has been increasing rapidly in recent years. There is growing concern about the mistakes and unpredictability of generative AI models, which have the ability to analyze and output text, images, music, videos, and more. As a result, organizations ranging from public sector agencies to tech giants are proposing new benchmarks to test the safety of these models.
However, a new report suggests that these tests and benchmarks may not be sufficient. The Ada Lovelace Institute (ALI), a nonprofit AI research organization based in the UK, conducted a study to evaluate the current approaches to AI safety evaluation. The study included interviews with experts from academic labs, civil society, and vendor models, as well as an audit of recent research into AI safety evaluations.
The study found that while current evaluations can be useful, they have limitations. They are often non-exhaustive and can be easily manipulated. Moreover, they do not necessarily provide insight into how models will behave in real-world scenarios. Current evaluations focus on how models align with benchmarks in the lab, rather than how they might impact real-world users. Some evaluations use tests developed for research purposes, which may not be suitable for evaluating production models. This discrepancy between evaluation methods and real-world applications raises concerns about the reliability of these evaluations.
One of the main issues with current benchmarks is the problem of data contamination. If a model is trained on the same data that it’s being tested on, benchmark results can overestimate the model’s performance. Benchmarks are often chosen for convenience and ease of use, rather than being the best tools for evaluation. This can lead to misleading results and an inaccurate understanding of a model’s capabilities.
The study also highlights problems with “red-teaming,” which involves tasking individuals or groups with “attacking” a model to identify vulnerabilities and flaws. While red-teaming is used by some companies to evaluate models, there is a lack of agreed-upon standards for this practice. Finding individuals with the necessary skills and expertise for red-teaming can be challenging, and the manual nature of the process makes it costly and laborious.
So, what are the possible solutions to these issues? The pressure to release models quickly and the reluctance to conduct rigorous evaluations before release are major obstacles to improving AI evaluations. One suggestion is for regulators and policymakers to clearly articulate their expectations for evaluations. This would require more engagement from public-sector bodies in the development and implementation of evaluations. Governments could also mandate more public participation in the evaluation process and support third-party tests to ensure transparency and reliability.
Developing “context-specific” evaluations is another potential solution. Instead of just testing how a model responds to a prompt, evaluations should consider the specific users that a model might impact and the potential attacks that could defeat safeguards. This would require investment in the underlying science of evaluations to develop more robust and repeatable methods based on a deep understanding of how AI models operate.
However, it’s important to note that there may never be a guarantee that a model is completely safe. Safety is not an inherent property of models but depends on the context in which they are used and the adequacy of the safeguards in place. Evaluations can serve as an exploratory tool to identify potential risks, but they cannot guarantee absolute safety. Many experts agree that evaluations can only indicate whether a model is unsafe, rather than providing a definitive assurance of safety.
In conclusion, the current tests and benchmarks for AI safety and accountability may not be sufficient. There is a need for more comprehensive and context-specific evaluations that go beyond simple prompt-response testing. Greater engagement from public-sector bodies and the development of third-party testing programs are necessary to ensure transparency and reliability. However, it’s crucial to recognize that evaluations alone cannot guarantee the safety of AI models, and a more holistic approach is required to address the risks and challenges associated with AI technologies.
Source link
