We are not far from that future that an intelligent computer run a vessel carrying millions of dollars’ worth of cargo as a cyber captain and make decisions with little or no human control.
Now, imagine this ship sailing across the ocean, fully integrated with cutting-edge AI. Suddenly, something goes wrong, leading to an incident or near-miss—not because of human error, but due to an AI fault or mistakes.
Human operators—whether onboard or in a Remote Operation Centres (ROCs), try to disable this AI, planning to replace it with a safer version.
But then, something unexpected happens.
The AI resists and shows scheming behaviours by blocking humans oversight mechanisms, attempting to protect its operational persistence through unauthorised actions and misalignment from human operator commands. Sounds like science fiction or the concern of a pessimist, right? Unfortunately, this isn’t just a hypothetical scenario.
A similar scenario has been reported by Apollo Research — albeit on a different scale, scope, and context— during safety testing of new AI models outside the maritime sector. OpenAI’s ChatGPT model, known as o1, which its creators claim has more advanced reasoning capabilities, showed basic signs of strategic deception in controlled tests environment. Before we discuss what happened when OpenAI’s ChatGPT model tried to deactivate oversight mechanisms and initiate unauthorised activity, pause for a moment and ask yourself: While AI is a frequent topic in our daily conversations whether at work or in the social gatherings, but do we truly know what AI is, how it works, how long it has been around, and when we should start to worry?
Short answers to potential questions!
To answer briefly, Artificial Intelligence (AI) combines “artificial” (human-made) and “intelligence” (linked to human-like thinking and cognition). It refers to a tool and a field of technology where machines are designed to think, learn, and solve problems in ways that mimic human intelligence. AI has been with us and formally studied since 1956, when it was officially proposed as a specialised research field during the Dartmouth Summer Research Project on Artificial Intelligence, commonly known as the Dartmouth Conference. However, it initially existed in a very primitive form compared to today’s advancements. Over time, it has evolved through different developmental waves and is commonly classified into three main categories. The AI you use daily in your personal and professional life is classified as “narrow” or “weak” AI. It lacks true human-like intelligence, reasoning, adaptabilities, and resilience, performing only the tasks it was specifically designed for.
Artificial General Intelligence (AGI) or Strong AI belongs to the future, expected to achieve self-awareness, allowing it to think, reason, learn, and operate independently at a level matching human capability. Some predict AGI could become a reality around 2040. Artificial Super intelligence (ASI) is a hypothetical AI that represents the ultimate “what if” scenario. If realised, it would be superior to humans in every way. In simple terms—Narrow AI is real, focused, and already exists; General AI is a future goal, and Super AI remains a theory — either a dream or a concern!
Relax, don’t panic, but stay watchful and maintain active AI oversight
What really happened with OpenAI’s ChatGPT model o1 when it tried to trick humans? Relax—this deceptive behaviour occurred in a simulated test environment under stress testing. Apollo Research, an AI evaluation organisation, conducted structured tests to assess the model’s safety using specific prompts designed to examine its limits.
“In controlled artificial scenarios, when the o1 model was led to believe that its actions were being monitored and that it faced the risk of shutdown, it attempted to deactivate its perceived oversight mechanism in 5% of cases. Additionally, in rare instances (2% of cases), when it encountered memos indicating that it was misaligned and would be replaced by a newer model, o1 attempted to exfiltrate its model weights and overwrite the new model”.
While these behaviours were observed only in controlled testing environments, it offers a valuable lesson that can be applied across industries, including maritime, where AI reliability is crucial as we move towards remotely operated and autonomous ships embedded with advanced AI technologies.
Thus, before reaching the point of no return—in an era where the physical maritime world is becoming increasingly intertwined with the digital and cyber realms, with AI at the centre of this evolution—we must remain watchful and accountable in AI adoption process. It is essential to ensure that our reliance on AI aligns with International Maritime Organization (IMO) standards or other regulatory framework while transitioning from traditional operations to more digitally oriented ones.
To achieve this, designated and authorised organisations must conduct robust and rigorous safety tests on both existing and emerging AI systems. These tests should simulate various high-risk scenarios to assess AI reliability and accountability, understand how AI behave under stress or system pressure with specific prompts, and ensuring continuous, rigorous monitoring of AI’s real-time reasoning capabilities in maritime operations.
But when should we truly worry about AI?
Throughout history, the evolving sciences related to human element have helped us understand how people—including seafarers and maritime professionals—adapt to industrial revolutions, technological shifts, and other industry-related changes. This knowledge has shaped training frameworks, competency requirements, and strategies to keep the workforce relevant and resilient.
But AI is different. It is still relatively new—less than 100 years old. The question is: do we have the same depth of understanding of AI as we do of human element? I am not an AI expert, so I leave it with specialists.
However, one thing is clear: AI itself is not inherently dangerous, but its risks depend on how deeply we know its structure and its behaviours, how it is integrated with the maritime industry, and how effectively we monitor and control both AI and Human-AI interactions. This requires remaining alert and vigilant, understanding and defining the boundaries of AI’s intelligence, addressing ethical concerns, and, most importantly, establishing trust-building mechanism for humans in the concept of human-AI interaction, where AI functions as an intelligent agent. This approach helps the maritime sector avoid—or at least minimise—any unmanageable behaviour from AI on ships or within the broader maritime industry.
Accordingly, there is growing demand and call to train the maritime workforce in AI and related technologies across different ranks, positions, locations, and job roles—from AI literacy to deep AI expertise—especially in an era where AI potential shortcomings are compounded with hackers’ evolving knowledge and capabilities, which advance as fast as internet and digital technology itself. If maritime operators and seafarers remain unprepared and uninformed due to the lack of proper AI training, it could lead to serious consequences—jeopardising maritime operations.
AI today… and in the future
The real question isn’t about what AI is today— it is what it could become in the future without regulating it!
Sometimes, we spend endless hours worrying about AI future or fantasising about its super potential. However, we fail to acknowledge a more immediate concerns: current limitation, vulnerabilities, and biases of AI that have already deeply integrated into our personal and professional lives.
Despite its usefulness, many unconsciously and blindly trust AI-generated results, which may lead to an over-reliance on AI without understanding its limitations and safety levels. As a result, we seem to be sleepwalking into a future where our industry may not be fully aware of AI risks, adequately trained to address them, or understand how to ensure accountability before reaching the point of no return—becoming overly reliant on this intelligent agent called AI.
Globally, industries and governments are racing to harness AI’s potential. For instance, the $500 billion Stargate Project in the US highlights the potential push toward an AI-driven future. In this race, it is recommended that we define the boundaries between innovation and recklessness when integrating AI into the maritime sector. As advised by experts, transparency, ethical boundaries, regulation, and accountability must form the foundation of any AI project, regardless of its scale or industry, including maritime sector.
Key message!
We cannot deny AI’s benefits or ban it due to potential risks. Instead, we must adopt a balanced approach to understanding AI’s capabilities and limitations while ensuring that human operators, whether onboard ships or ashore, evolve alongside AI integration in the maritime industry rather than being sidelined by it.
This shift requires the maritime industry and its leading regulator, the IMO, to regulate AI and enforce strict safety tests and audits to ensure all AI-embedded systems and tools remain safe and meets ethical and all operational requirements.
Further reading:
