Nov 27 – 28, 2026
Asia/Ho_Chi_Minh timezone

EVALUATING AN AI-ASSISTED WORKFLOW FOR CODING RHETORICAL MOVES IN CORPUS-BASED GENRE ANALYSIS

Not scheduled
20m
Linguistics & Cultural Studies

Speaker

Mrs Minh Phương Lương (Đại học Thương mại)

Description

The increasing use of Generative AI (GenAI) has created new opportunities for supporting qualitative research in applied linguistics, yet its reliability in tasks requiring contextual interpretation remains underexplored. Drawing on the genre frameworks of Swales (1990) and Bhatia (1993), this study evaluates a human-in-the-loop workflow for identifying and coding rhetorical moves and steps in corporate press releases. The corpus comprises 40 technology-sector press releases, including 20 English texts from multinational corporations and 20 Vietnamese texts from domestic companies. Using a standardized zero-shot prompt and Codebook V1.2, NotebookLM assigned preliminary move-step labels to 785 provisionally segmented units and flagged suspected boundary problems. The first researcher coded the texts independently before consulting the AI output, while a second researcher independently coded 152 paired units drawn from ten releases. Coding disagreements were resolved through adjudication, and selected concordances were examined using AntConc. For the 152 double-coded units, human agreement was 94.08% for moves and 92.11% for move-step labels, with Cohen's kappa values of 0.926 and 0.912, respectively. Across 376 directly comparable AI-human units, NotebookLM matched independent R1 labels in 81.65% of move assignments and 64.10% of move-step assignments. Qualitative analysis documented boundary splits and merges and showed that similar expressions could serve different rhetorical functions. The findings highlight the need to assess label agreement separately from functional segmentation. The study offers an auditable workflow that distinguishes preliminary AI suggestions, independent human labels, and adjudicated decisions. It supports cautious GenAI use in corpus-based genre analysis without establishing whole-corpus accuracy or language-specific effects. Potential applications in AI literacy and TESOL require further empirical examination.

Author

Mrs Minh Phương Lương (Đại học Thương mại)

Presentation materials

There are no materials yet.