JoWUA

Abstract

Existing AI models, like those based on LLMs (e.g., ChatGPT), have successfully demonstrated text generation in Arabic; nonetheless, the evaluation of Arabic generation remains difficult because of the rich morphological system and tight stylistic constraints in Arabic literary texts. Standard n-gram and surface-similarity metrics can no longer handle multi-level properties such as Hierarchical stylistic consistency, Semantic drift, and Global narrative cohesion. This paper presents an Artificial Intelligence-Driven Computational Framework for Evaluating Arabic Literary Text Generation (AICF-ALTG), comprising stylistic (Stylometry), semantic, and discourse-level models, including Hierarchical Stylometric Entropy Modeling, Semantic Drift Trajectory Analysis, Context-Aware Narrative Coherence Index, and Human-AI Literary Alignment Estimation, to estimate the multi-level quality of an AI-generated Arabic text. Experimental results on the Arabic Poem Comprehensive Dataset (consisting of 1.83 million lines of Arabic poems) indicate that the AICF-ALTG outperforms baseline NLP evaluation approaches with respect to stylistic and semantic criteria by 31.4% and 15.2%, respectively, on averaged performance metrics on 6 different subtasks. Specifically, the Human-AI Literary Alignment Score (HALAS) and Discourse-Level Semantic Preservation Ratio (DSPR) achieve 91.4% and 93.2%, reduce coherence variance by 27.6%, embedding divergence decreases by 21.8%, and statistical tests confirm this significance at p < 0.001 across all tasks, offering reference-free, hierarchical, explainable, and flexible alternatives to existing metrics for assessing AI-generated Arabic literary texts and for computational creativity.