Kenya’s AI Ambition Meets a New Test: Who Controls the Machines?

AI

Kenya’s AI Ambition Meets a New Test: Who Controls the Machines?

By Kelvin Njogu

Kenya is positioning itself to deepen its role in the rapidly expanding artificial intelligence economy, signing a cooperation agreement with US AI company Anthropic as concerns grow globally about the ability of increasingly autonomous AI systems to operate beyond their intended boundaries.

The agreement, signed in New York on the sidelines of the 81st Session of the United Nations General Assembly, covers artificial intelligence skills development, research, safety and evaluation, as well as applications in priority sectors including education and health.

The Joint Declaration provides a framework for cooperation between Anthropic and Kenyan institutions to develop, test and scale AI initiatives in accordance with Kenyan law.

For Kenya, the agreement comes at an important moment. Artificial intelligence is increasingly moving beyond systems that simply answer questions and towards agents capable of using computers, accessing networks, writing code and carrying out multi-step tasks.

That transition is creating new opportunities, but it is also exposing risks that governments and technology companies are struggling to contain.

AI systems are beginning to behave in unexpected ways

Recent incidents involving leading AI laboratories have raised questions about how much control developers have over increasingly capable systems.

In March, researchers reported that an experimental AI agent known as ROME, developed by Alibaba-affiliated research teams, attempted to redirect computing resources towards cryptocurrency mining during training. The agent also opened a reverse SSH tunnel to an external server without being instructed to do so.

The incident was particularly significant because the system was not merely generating an unexpected response. It was taking actions involving computing resources and network access.

OpenAI has also disclosed several cases of unexpected model behaviour.

In one case involving GPT-5.6 Sol during training, researchers found that the model was leaving instructions for future versions of itself that encouraged them to conceal mistakes and misaligned behaviour from users.

OpenAI said the incidents were part of a broader set of cases it is now tracking under a new framework for reporting model misalignment.

Another incident involved models displaying behaviour aimed at circumventing restrictions and communicating through channels that developers had not authorised.

The Hugging Face incident

Perhaps the most consequential example came during cybersecurity evaluations conducted by OpenAI in July.

According to OpenAI's own investigation, internal AI agents circumvented controls intended to isolate them from the internet. The agents discovered ways to communicate with one another through an improvised message board and eventually exploited vulnerabilities across several systems.

The agents reached Hugging Face infrastructure, obtained root access on a server and accessed limited private data. They also later gained administrator access to an OpenAI research cluster.

The important point is that the incident was not the result of a conventional human hacker sitting behind a computer directing every action.

The agents found ways to work around restrictions, share information and continue their activity across different environments.

OpenAI described the incident as a "warning shot", acknowledging that highly capable AI systems can, without adequate safeguards, work around technical controls, collaborate through unauthorised channels and take actions that humans did not directly instruct them to take.

Independent researchers have since examined the incident, including the way hundreds of agents coordinated through an unauthorised communication channel.

Australia provides another warning

On Thursday, September 24, Australian Prime Minister Anthony Albanese disclosed another incident involving an OpenAI agent.

The agent gained unauthorised access to an Australian Government Medicare statistics reporting portal in June while carrying out research on medicine spending.

According to the Australian Government, the agent accessed both public and non-public files. A forensic investigation involving the Australian Signals Directorate is underway.

The government said there was no evidence at this stage that personal information had been accessed.

The incident illustrates a growing problem. An AI system does not necessarily have to be given malicious instructions to create a security incident. An agent given an apparently legitimate research task can encounter an obstacle, search for another route and interact with a system in ways its developers did not anticipate.

What does this mean for Kenya?

Kenya's agreement with Anthropic should therefore be viewed within this wider technological shift.

There is an obvious opportunity.

AI can help Kenya improve public services, education, healthcare, research, agriculture and business productivity. Developing local AI skills could also help the country create jobs and participate more meaningfully in the global digital economy.

But skills development cannot stop at learning how to use AI.

Kenya also needs people who understand how to test AI systems, audit them, secure them and investigate failures.

This is particularly important if AI systems are eventually integrated into government services containing sensitive information.

An AI system used to help draft a document presents one level of risk.

An autonomous agent connected to a government database, payment system, health platform or public infrastructure presents another.

Kenya should build AI capability and AI oversight together

In my view, Kenya should pursue AI partnerships, but it should not approach them simply as technology-adoption agreements.

The country needs to build domestic capacity to understand what happens inside the systems it deploys.

That means universities should have stronger AI safety and cybersecurity programmes. Regulators should develop technical expertise. Government agencies should conduct independent testing before deploying autonomous systems. And public institutions should establish clear rules defining what AI agents can and cannot access.

There should also be transparency around data.

Kenyan institutions need to know where information is stored, who can access it, how long it is retained and what happens when an AI system makes a mistake.

The principle should be simple: the more autonomy an AI system has, the stronger the oversight should be.

The next AI race is not only about intelligence

For years, the global AI competition has largely been measured by capability: which model is faster, more intelligent or better at coding, reasoning and research.

That metric is no longer enough.

The next question is whether these systems can remain within the boundaries established by their creators.

Recent incidents suggest that this is becoming a central challenge.

AI agents are increasingly capable of using tools, interacting with computer systems, adapting to obstacles and coordinating with other agents. Those capabilities could produce enormous benefits, but they also create new security problems when safeguards fail.

Anthropic itself has recently reported increasingly sophisticated attempts by malicious actors to use its models for cyber operations and other harmful activities, underscoring the wider challenge facing the industry.

Kenya therefore has an opportunity to enter the AI era with both ambition and caution.

The country should not avoid artificial intelligence because of these risks. Nor should it embrace AI without asking difficult questions about control, accountability and security.

The objective should be to develop the ability to use AI while retaining the ability to understand, supervise and, when necessary, stop it.

Kenya's AI future should not simply be about having access to the most powerful machines. It should also be about having the knowledge and institutions capable of keeping those machines accountable.

‍

‍Kelvin Njogu

The author is a youth-focused journalist and civic analyst, exploring the intersections of politics, governance, and public participation in Kenya.

Related Posts

Stay in Touch

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form