AI hallucination led to the risk of a US-China military confrontation

The use of artificial intelligence algorithms in US Navy intelligence operations nearly triggered an international conflict. The data processing system mistakenly identified a Chinese civilian ship as a carrier of prohibited nuclear components, putting the military command on the verge of intercepting and inspecting the vessel.

According to a CNN report, the incident occurred due to a critical error in the analytical model, which interpreted incorrect data as evidence of violating international sanctions. The algorithm generated a false report based on incomplete satellite images and distorted metadata, leading to the automatic classification of the ship as a "high-risk object".

During the subsequent investigation, it was found that the model had "hallucinated", filling in the missing details of the ship's design to the level of military specifications. The error was detected by staff officers just minutes before the start of the operation to forcibly stop the ship, which allowed avoiding a direct confrontation with Chinese forces in the disputed region.

This case has become a reason for revising the protocols for using autonomous systems in the Pentagon's intelligence activities. Currently, the military department is introducing mandatory verification of all AI system conclusions by human operators to exclude automatic decision-making based on unreliable data. The incident highlights the risks of introducing generative models into critical security infrastructure, where the cost of an error can lead to an escalation of geopolitical tensions.