DA Module 10 From Sound to Evidence
DA Module 12 Processing Speech and Writing
DA Module 11: When AI Writes the Dialogue
⚡️ Critically evaluating generated interaction and designing feasible UDL-aligned role-play
Generative AI can produce a dialogue in seconds, but grammatical fluency does not guarantee conversational plausibility. Generated exchanges may be unnaturally balanced, overexplicit, culturally generic, sequentially tidy, or inconsistent with the participants’ roles. This week introduces Level 4 AI use: task design and critical evaluation. You will prompt an approved tool to create interactional or transactional talk, then code and fact-check the result using Unit III concepts. The transcript becomes raw material, not authoritative evidence. After revision, you will design a feasible ELT role-play aligned with Universal Design for Learning. Your grade depends on how well you evaluate the AI’s limitations, justify changes, anticipate classroom conditions, and create multiple ways for learners to engage, understand the task, and demonstrate pragmatic competence.
Essential understanding(s)
What makes an AI-generated dialogue interactionally plausible and pedagogically usable?
How can UDL principles strengthen a discourse-based role-play without removing authentic communicative challenge?
Guiding Questions
Which speech acts, adjacency pairs, turn-taking practices, and pragmatic features does the generated dialogue represent accurately or inaccurately?
How should implausible sequencing, excessive explicitness, or cultural assumptions be revised?
What learner variability, barriers, and classroom constraints must the role-play address?
What evidence supports the final design choices?
Learning outcomes
Generate a bounded interactional or transactional transcript with an approved GenAI tool and preserve the prompt history.
Critically code and evaluate conversational naturalness, sequence, context, and pragmatic plausibility.
Revise AI output while documenting limitations and reasons for changes.
Design a feasible UDL-aligned ELT role-play with clear objectives, supports, and assessment evidence.
Submit the Unit III assignment and demonstrate conceptual understanding on the Unit III quiz.
Key concepts and vocabulary
| Term | Working definition |
|---|---|
| Generated transcript | Dialogue produced by a generative model and treated as a draft requiring critical evaluation. |
| Plausibility | The degree to which discourse is credible for the specified participants, setting, purpose, and sequence. |
| Hallucination | Confidently generated but unsupported, invented, or contextually false content. |
| Prompt constraint | A condition specifying roles, setting, goals, language level, sequence, or output format. |
| UDL | Universal Design for Learning, a framework for reducing barriers through multiple means of engagement, representation, and action/expression. |
| Learner variability | Expected differences in language proficiency, background knowledge, attention, confidence, sensory access, and preferred participation. |
| Scaffold | Temporary support that enables performance and can be adjusted or removed as competence develops. |
| Role-play | A structured simulation in which learners pursue communicative goals from assigned or chosen roles. |
| Feasibility | The practical fit of an instructional design with time, group size, resources, proficiency, and classroom conditions. |
| Evidence-based rationale | A justification connecting design choices to discourse analysis, learner needs, and credible pedagogical principles. |
Main chapter content
-
Why fluent dialogue can still be poor discourse
Language models are optimized to produce probable continuations, not to participate in the lived situation described by the prompt. Their dialogues often display complete sentences, symmetrical turn lengths, immediate compliance, and explicit statements of intention. Real interaction contains uncertainty, partial understanding, timing problems, relationship work, and locally negotiated meaning.
A generated service encounter may have the customer state every detail in one turn and the employee respond with a comprehensive solution. A generated disagreement may resolve after one polite exchange. A classroom dialogue may give every learner equal participation without showing how turns are allocated. These patterns can be useful as simplified input, but they should not be mistaken for naturalness.
Evaluation begins with the specified context. Are the roles and entitlements plausible? Do first-pair parts receive relevant responses? Are refusals, repairs, and topic shifts designed credibly? Does the language fit the relationship and purpose? The transcript should be tested against the concepts already learned, not judged by intuition alone. -
Prompting for analysable raw material
A strong prompt defines the scene without asking the model to solve the analysis. Specify participants, relationship, setting, purpose, complication, approximate length, and whether the talk is primarily interactional or transactional. You may require at least one repair sequence or indirect request so that the transcript contains relevant phenomena.
Do not ask the model to label its own speech acts as the final coding. It may produce circular or inconsistent explanations. The student’s task is to code independently. Preserve the initial output before editing so the evaluation can show what the model produced.
Prompt iteration should be documented. If a second prompt corrects an unrealistic feature, explain what problem you identified and how the new constraint attempted to address it. Repeated generation without analysis is not critical workflow. Each iteration should be purposeful and limited. -
Critical coding and revision
Apply the Unit III coding scheme to the generated transcript. Identify speech acts, adjacency pairs, preference features, turn allocation, repair, and shifts between interactional and transactional orientations. Then mark places where the sequence is implausible, overexplicit, culturally narrow, or pedagogically misleading.
Revision should preserve useful features and change only what the rationale supports. You might add a delayed refusal, split an overloaded turn, insert a clarification sequence, remove an unnatural explanation, or alter a request to fit the participant relationship. You may also decide that a simplified exchange is appropriate for a lower proficiency level, provided you label it as pedagogical simplification rather than authentic conversation.
Fact-checking includes contextual claims. If the dialogue refers to institutional procedures, cultural practices, or professional roles, verify them or remove them. Models can invent policies and treat stereotypes as norms. The final transcript should be transparent about its generated origin and critical revision. -
Designing a UDL-aligned role-play
The role-play should begin with a discourse objective, such as managing a refusal, repairing a misunderstanding, balancing rapport and task talk, or completing an adjacency-pair sequence. A general objective like “practice speaking” is too broad. Success criteria should describe observable interactional behavior.
Multiple means of engagement can include role choice, relevant scenarios, graduated challenge, and clear purpose. Multiple means of representation can include written role cards, audio models, visual sequence maps, vocabulary supports, and examples of possible strategies without scripting every turn. Multiple means of action and expression can include live performance, audio recording, paired rehearsal, observer coding, or reflective annotation.
UDL does not mean eliminating challenge. Supports should reduce barriers unrelated to the objective while preserving the need to negotiate meaning. If the objective is spontaneous repair, a fully scripted dialogue would remove the target performance. A better scaffold might provide repair stems and a surprise information gap. -
Feasibility, assessment, and submission
A classroom-ready design states time, grouping, materials, roles, preparation, procedure, teacher monitoring, and debriefing. Anticipate group size, uneven proficiency, reluctant participation, noise, and limited technology. Provide an alternative pathway for learners who cannot perform publicly while maintaining equivalent evidence of the objective.
Assessment can combine an observer checklist with transcript excerpts or reflection. Criteria might include recognition of the partner’s action, appropriate response design, repair, turn management, and adaptation to context. Grammar can be included, but it should not overshadow the discourse objective.
The Unit III submission should show the whole chain: prompt, raw AI dialogue, independent coding, critique, revised transcript, role-play plan, UDL options, and evidence-based rationale. The most important product is your evaluation. A polished lesson based on unexamined AI output does not meet the unit objective. -
Stress-testing the role-play
Before final submission, run a paper-based simulation of the role-play. Ask two peers to follow the role cards while a third observes. The observer should record whether the designed complication actually produces the target discourse behavior. If learners can finish without making a request, repairing trouble, or negotiating a refusal, the task is not aligned.
Collect feedback on clarity, realism, cognitive load, time, and access. A role card may contain too much reading for a speaking objective, or the scenario may require cultural knowledge that has not been provided. Revise the supports while preserving the communicative gap. For example, replace a dense paragraph with a visual schedule, but do not script the target sequence.
Also test the assessment criteria. Can an observer reliably identify the evidence? Replace vague criteria such as “speaks naturally” with observable indicators such as recognizes the partner’s refusal, uses a repair strategy after trouble, or closes the transaction appropriately. Document the stress test in the rationale. Feasibility is demonstrated through anticipated classroom performance, not through attractive formatting alone. -
Ethical and transparent use of generated scenarios
Generated dialogues should never be presented to learners as naturally occurring evidence unless they are clearly labeled. Their value lies in controlled design, contrast, and critique. A teacher might pair an AI-generated exchange with an authentic clip and ask learners to identify differences in timing, repair, turn length, or social realism.
Prompt records and revisions also protect transparency. Include the tool used, date, purpose, constraints, raw output, and major edits. Avoid entering identifiable student data or confidential classroom situations into an external model. Construct fictional or generalized scenarios instead.
The teacher remains accountable for cultural representation. If the model produces a stereotype, remove it and document the problem rather than quietly polishing the wording. This makes AI literacy part of discourse literacy: learners and teachers examine who is represented, whose norms are assumed, and which interactional possibilities are absent. The final role-play should be defensible even if the AI tool is unavailable, because its pedagogical value comes from the human analysis and design.
- Once you have completed Week 10 Types of Talk Transcription Task ss.docx, use the following prompt (as is) to compare and contrast any differences: Student LLM Prompt_Revising a Conversation While Preserving Its Original Meaning. Avoid modifying the prompt in any way except copying-and-pasting your transcription at the end of the prompt. If you have any issues or questions about the prompt itself and/or your output, let me know.
- Submit your handout and comparison to View assignment: “Module 10 Handouts: Cooperative Principle ”.