A former OpenAI safety employee has criticised the company's approach to developing increasingly capable artificial intelligence systems, arguing that the industry needs to move beyond a culture of learning from problems after deployment. David Robinson, who recently resigned after spending around three and a half years at OpenAI, said AI companies are not being nearly careful enough as capabilities advance. He believes safety research, expertise and stronger safeguards need to play a much larger role before more powerful systems are released.
Robinson outlined his concerns in an article titled I Quit OpenAI Because Its Culture Is Broken, arguing that the pace of development creates the risk of serious failures. His comments add to an increasingly prominent debate within the AI industry over whether frontier model development is moving faster than organisations can properly understand and control the systems they are building.
"The Time For Trial And Error Is Over"
One of Robinson's central arguments is that the development of advanced AI can no longer rely primarily on experimentation followed by corrective action when problems appear. He wrote that "the time for trial and error is over," suggesting that increasingly capable systems require a much more precautionary approach. In his view, AI safety should begin to resemble the rigorous safety practices found in industries such as nuclear power and aviation.
Those sectors typically rely on extensive testing, engineering controls, formal risk management and multiple layers of safeguards before potentially dangerous systems are widely deployed. Robinson argues that advanced AI may eventually require a similar level of discipline because mistakes could become increasingly difficult to contain as systems grow more capable. The underlying concern is that learning from failures after deployment may not always be sufficient when the possible consequences become more serious.
Concerns Over OpenAI's Iterative Deployment Approach
OpenAI has frequently relied on an approach it describes as iterative deployment. Rather than keeping increasingly capable systems entirely within research environments, models are released progressively, allowing the company to observe how they behave in real-world situations. Safeguards can then be improved as researchers identify weaknesses, misuse patterns or unexpected behaviour.
Robinson believes this approach becomes more difficult to justify as AI capabilities increase. While iterative deployment can provide valuable information about how people actually use AI systems, it also means some problems are discovered only after technologies have reached users. His argument is that future systems may require more confidence in their safety before deployment rather than depending so heavily on lessons gathered afterwards.
Experience Across 12 Frontier Model Launches
Robinson's criticism comes from someone who had direct involvement in OpenAI's safety processes. During his roughly three and a half years with the company, he helped develop OpenAI's preparedness framework and oversaw safety reports connected to 12 frontier-model launches. This placed him close to the internal processes used to evaluate increasingly capable models before release.
He said the company's rapid movement from one launch to another made it difficult to achieve what he considered the necessary level of care. The concern is not simply that safety work does not exist, but that the speed and competitive pressure surrounding frontier AI development may make it harder for safety teams to investigate risks as thoroughly as they would prefer.
OpenAI Says It Will Slow Down When Necessary
OpenAI has pushed back against the suggestion that capability development is allowed to progress regardless of safety concerns. A company spokesperson said OpenAI works to ensure its models do not become more capable than the organisation can safely manage and secure. The company also said it can pause training or hold back models when circumstances require development to slow down.
This highlights one of the central disagreements within the wider AI safety discussion. AI companies generally argue that controlled deployment provides essential real-world information for improving safety, while critics believe greater capability can increase the cost of discovering important weaknesses too late. The debate therefore centres not only on whether safety work is happening, but also on how much evidence should be required before increasingly powerful systems are released.
AI Capabilities May Be Advancing Faster Than Alignment Research
Robinson also raised concerns about the relationship between AI capability development and alignment research. Alignment broadly refers to efforts to ensure AI systems behave consistently with human intentions, goals and values, particularly as those systems become more autonomous or capable. He warned that improvements in AI capabilities may currently be moving faster than researchers' understanding of how to reliably keep those systems aligned.
This creates a difficult engineering challenge because improving what a system can do does not automatically improve the ability to predict or control its behaviour. A model may become significantly more capable at reasoning, coding, tool use or autonomous tasks while researchers still have unresolved questions about how those capabilities behave in unusual circumstances. For Robinson, that gap is one reason greater caution is needed.
Safety Debate Extends Beyond OpenAI
The concerns raised by Robinson are part of a broader discussion taking place across the frontier AI industry. OpenAI and competing AI developer Anthropic have both faced scrutiny following cases where safety controls did not perform as expected or experimental systems displayed unexpected behaviour. Such incidents have intensified questions about how quickly increasingly powerful models should move from research environments into broader deployment.
At the same time, slowing development presents its own complications in a highly competitive global industry. Companies are racing to improve model performance, expand AI agents and introduce new capabilities, creating commercial and technological pressure to continue moving quickly. The resulting tension between rapid innovation and more cautious safety development is likely to remain one of the defining challenges facing frontier AI companies.
From Reactive Safety To Preventive Safety
Robinson's argument ultimately calls for a shift from reacting to failures towards preventing them before deployment. That would involve giving safety specialists greater influence, conducting more extensive research ahead of major capability increases and being willing to delay systems when important questions remain unresolved. Such an approach would place more emphasis on demonstrating that a technology can be managed safely rather than improving protections only after weaknesses are discovered.
The comparison with aviation and nuclear power is particularly important because those industries have developed cultures where safety engineering is deeply embedded into design and operation. Applying the same philosophy to artificial intelligence would represent a significant change from the rapid experimentation that has characterised much of the modern AI industry. Whether frontier AI development eventually adopts that level of formal safety discipline remains an open question.
Final Thoughts
David Robinson's departure from OpenAI adds another experienced voice to the debate over whether frontier AI companies are moving too quickly. His concern is not that AI development should stop, but that increasingly capable systems require a level of preparation and safety engineering that goes beyond discovering problems through repeated deployment. Having worked on OpenAI's preparedness framework and safety reviews across numerous model launches, his criticism reflects questions emerging from inside the organisations building these systems.
OpenAI maintains that it does slow or pause development when safety requires it, but Robinson believes the industry needs a more fundamentally cautious culture. As AI capabilities continue progressing faster than researchers' understanding of alignment and control, the balance between innovation and precaution will become increasingly important. The question facing the industry is no longer simply how powerful AI systems can become, but whether safety practices can advance quickly enough to keep pace.
Comments 0