Abstract
Maritime stakeholders face significant challenges monitoring threats such as piracy, terrorism, and environmental degradation, yet data science applications in the maritime domain remain limited. This study addresses this gap by leveraging natural language processing (NLP) and large language models (LLMs) to enhance maritime situational awareness. We present Kea, a decision support prototype that automates the processing and interpretation of over 2500 incident reports extracted from 271 monthly Worldwide Threat to Shipping reports published by the U.S. Office of Naval Intelligence (2018–2023), demonstrating end-to-end capabilities including data scraping, abstractive summarisation, severity scoring, and geospatial visualisation. A focus of this research is the comparative implementation of text summarisation using BART and PEGASUS models, alongside a rule-based VADER model for severity scoring. To evaluate outputs without ground-truth summaries, we propose a reference-free framework combining cosine-similarity with a human-in-the-loop survey based on the Five-C criteria (Coherence, Conciseness, Consistency, Construct, Confidence). Survey-based validation with New Zealand Defence Force personnel indicates that PEGASUS produces superior summaries compared to BART, though both are outperformed by ChatGPT-3.5, highlighting scope to explore state-of-the-art models within defence security constraints. Cosine-similarity evaluation yields an average score of 0.73, demonstrating high semantic association with original narratives. The mean VADER compound severity score is -0.32, consistent with a predominance of high-risk events, with longitudinal analysis suggesting gradual improvement from 2018 to 2023. These findings suggest that the Kea pipeline improves scalability and interpretability of maritime risk assessment while highlighting limitations relative to state-of-the-art LLMs, laying a foundation for future data-driven decision-making.