Benjamin L. Stewart

Properties1
tagsresearch

12 min read

From Data to Defensible Themes

A Practical Workflow for Qualitative Classroom Analysis (Thesis Seminar)

  • Attendees

What to participate in the 2nd annual academic week of the PA May 20, 2026 10:00 AM?

  • Participants

BENJAMIN LEE STEWART: đź“· Sent an image | Thesis Seminar 2026 > General | Microsoft Teams

Course Calendar… Course Guidelines…

Thesis Seminar Class Calendar 2026.pdf

1) Start with your research questions (RQs)

  1. Write 2-4 analysis-focused research questions: What patterns? How? Why? Under what conditions?

  2. Decide what counts as “evidence” for each RQ (talk, text, behavior, artifact)

  3. Identify the phenomenon: interaction, feedback, engagement, task performance, beliefs, identity, strategy use, etc.

🖊️

2) Map your dataset (what you actually have)

  • Recorded Interviews

    • interviews / stimulated recall

  • Recorded Classroom Observations

    • classroom audio/video + transcripts

    • field notes / observation protocols

  • Documents or Artifacts (Content/Document Analysis)

    • learner texts (drafts, journals, homework, worksheets) + teacher feedback

    • lesson plans, materials, LMS posts

    • school mission and vision statements, objectives, values, curriculum, etc.

  • Create a dataset table: source, date, participants, lesson/task, file link, notes

    A “dataset table” is basically an inventory (a log) of every piece of data you might analyze, with enough metadata that you can (a) find it again fast, (b) decide whether it’s relevant/usable, and (c) justify what you included/excluded later.

    Below are a few good table designs you can copy depending on your project.

    Option A (most common): One row = one data item / file

    Use this if your main unit is “a thing you collected” (an interview recording, a transcript, a set of student drafts, a lesson observation, etc.).

    Dataset inventory (example columns)

    IDData source typeFile / artifactDateContext (class / lesson)ParticipantsDuration / sizeLanguageUnit(s) availableStatusLocation linkNotes
    INT-01Interview (stimulated recall)Transcript + audio2026-03-02Writing II / Lesson 11.5Student A + instructor42 minEN/ESturns, episodesCleanedDrive/Notion linkIncludes reflection on feedback use
    OBS-03Classroom observationVideo + field notes2026-03-10Thesis Seminar / peer reviewWhole class60 minENepisodesTranscribedlinkAudio noisy in last 10 min
    DOC-07Learner writingDraft 1 + teacher comments2026-03-12Argument essayStudent B780 wordsENparagraphs, commentsAnonymizedlinkContains AI-detection report too

    Why these columns work

    • Data source type lets you filter (interviews vs observations vs documents).

    • Unit(s) available forces you to be explicit about what you can actually code (turns, episodes, paragraphs, feedback comments, etc.).

    • Status (“raw / cleaned / anonymized / ready to code”) prevents chaos later.

    Option B: One row = one session / event (good for classroom work)

    Use this if you collect many things per class meeting (video + audio + field notes + slides + student artifacts) and want them grouped.

    Session log (example)

    Session IDDateCourseLesson topicData capturedParticipantsArtifacts producedCoverage/qualityLinksNotes
    S-052026-03-10Thesis SeminarCoding practicevideo, audio, field notes18 studentsworksheet submissionsgood except 5 min missinglinksseating chart attached
    S-062026-03-12Writing IICounterargumentsaudio, board photos22 studentsDraft paragraphsaudio oklinksstrong group-work segments

    Then, optionally, you make a second table where each row is a file, linked to the session (relation).

    Option C: Two-level design (best practice if you have lots of data)

    This is what many theses end up needing:

    1) Dataset Sessions (one row = one class / one interview appointment)

    Columns: Session ID, date, context, participants, what was collected, links, notes.

    2) Dataset Items (one row = one artifact/file)

    Columns: Item ID, type, session link, participant(s), file link, status, unit of analysis, etc.

    This structure keeps things tidy when your dataset grows.

    Suggested column templates you can paste into Notion quickly

    Minimal (starter)

    • ID

    • Type (interview / observation / document / student text / feedback)

    • Date

    • Participants

    • Context

    • Link

    • Notes

    Better (thesis-ready)

    • ID

    • Type

    • Date

    • Context (course/lesson/task)

    • Participants (and role)

    • Data format (audio/video/text/images)

    • Duration / word count / pages

    • Language(s)

    • Consent / ethics status (if relevant)

    • Anonymized? (checkbox)

    • Transcribed? (checkbox)

    • Unit of analysis (turn/episode/paragraph/comment)

    • Included in analysis? (yes/no + reason)

    • Link(s)

    • Quality / issues

    • Memo (early analytic notes)

    Example “Included in analysis?” logic (very helpful later)

    Add two columns:

    • Included? (checkbox)

    • Exclusion reason (select): “not relevant to RQs”, “missing consent”, “poor audio”, “incomplete artifact”, “duplicate”, etc.

    This gives you an instant audit trail when writing your Methods chapter.

    If you tell me what your main data types are (e.g., classroom audio + field notes + student drafts + teacher feedback), I can show a table tailored to that exact mix (with the best “unit of analysis” column for it).

3) Prepare the data (quality in = quality out)

  1. Determine your unit of analysis: turn, episode, exchange, paragraph, feedback comment, reflective unit, etc.

🖊️

  • Technological Tools for Transcribing Audio to Text

    For students working on a budget, these free and open-source tools provide professional-level results without subscription costs.

    Free Audio-to-Text Transcription

    • Buzz (OpenAI Whisper GUI): This is a free, open-source desktop application that serves as a user-friendly wrapper for OpenAI’s Whisper model. It runs entirely locally on a computer, meaning students don’t need to pay for cloud minutes, and their research data remains private. It handles various audio formats and is excellent at identifying different speakers.

    • Google Docs Voice Typing: For a simple, cloud-based solution, students can use the built-in “Voice Typing” feature in Google Docs. While it is designed for live dictation, students can play their recorded audio through their computer’s speakers (or use a virtual audio cable) to transcribe it into a document in real-time for free.

    • VOSK: This is an open-source speech recognition toolkit that supports over 20 languages. It is particularly useful for students who are technically inclined and want a lightweight, offline tool that doesn’t require high-end hardware.

    Free Qualitative Data Analysis & Coding

    • Taguette: This is the gold standard for free, open-source qualitative research. It allows students to import transcripts, highlight segments, and apply codes (categories) to them. It is straightforward, works on all operating systems, and allows for easy exporting of results into Excel or Word for further analysis.

    • QDA Miner Lite: This is a free version of a popular professional tool. It offers a structured environment for coding text and allows for basic analysis of the frequency and co-occurrence of categories. It is a great way for students to learn the logic of CAQDAS (Computer-Assisted Qualitative Data Analysis Software) without the financial barrier of a full license.

    • Markdown-Based Coding: For a “local-first” approach that emphasizes transparency and long-term data portability, students can use plain-text Markdown files. By using internal links or hashtags (e.g., categoryname) within a Markdown editor, they can create a networked system of codes. This method ensures that their research data is never locked into a proprietary format and can be easily shared or published as part of an open teaching practice.

    Since these tools are free, they often rely on the student’s local hardware. Would you like instructions on how they can set up a local environment for these tools, or should we focus on the most user-friendly web-based workflows?

  • Transcribe (or clean transcripts): consistent speaker labels, turns, timestamps (if needed)

  • Maintain confidentiality: pseudonyms

  • Anonymize (if applicable): names, schools, any identifiers - few students will anonymize their data.

  • Keep an audit trail: what was changed, by whom, when

4) Choose an analytic approach (fit it to your RQs)

  1. Determine your analytic approach.

🖊️

  • Rule of thumb: pick one “main engine” and use others only as support

    • Thematic analysis: patterns of meaning across the dataset

    • Qualitative content analysis: categories + frequency/occurrence with interpretation

    • Interaction/Discourse approaches (if relevant): focus on how talk accomplishes actions

    • Grounded theory-style coding: build concepts/categories from data iteratively

  • Examples Matching Analytic Approaches to RQs

    Reflexive Thematic Analysis (Braun & Clarke style) — student perceptions/experiences

    Good for: interviews, open-ended survey responses, reflective journals.

    Research questions (example set)

    • RQ1: What experiences do EFL students report when receiving teacher written feedback on drafts?

    • RQ2: What meanings do students attach to “useful feedback” vs “confusing feedback”?

    • RQ3: What conditions (task type, time pressure, proficiency) shape how they respond to feedback?

    Why this fit works: RQs ask “what patterns of meaning” and “how students interpret,” which aligns with theme development from rich narrative data.

    Qualitative Content Analysis (directed or conventional) — “what types” + “how often/where”

    Good for: documents you can systematically categorize (LMS posts, reflection entries, teacher comments, discussion boards).

    Research questions (example set)

    • RQ1: What categories of teacher feedback appear most often in student drafts (e.g., grammar, organization, content, citation)?

    • RQ2: How does the distribution of feedback categories change from Draft 1 to Draft 3?

    • RQ3: Which feedback categories are most often followed by successful revision?

    Why this fit works: Content analysis supports clear category systems and can connect categories to counts/trends without becoming purely quantitative.

    Conversation Analysis / Micro-Interaction Analysis — “how talk accomplishes actions”

    Good for: classroom audio/video with detailed transcripts (turn-taking, repair, scaffolding, negotiation).

    Research questions (example set)

    • RQ1: How do teachers prompt student self-correction during speaking tasks?

    • RQ2: What repair sequences (self-repair vs other-repair) occur after teacher feedback?

    • RQ3: Under what interactional conditions do students uptake feedback (immediate incorporation vs resistance vs silence)?

    Why this fit works: CA answers “how” questions about sequential organization of talk, not just “what students think.”

    Discourse Analysis (genre/rhetorical move analysis) — patterns in texts as social action

    Good for: student writing, teacher comments, lesson materials; especially argumentation, literature reviews, abstracts.

    Research questions (example set)

    • RQ1: What rhetorical moves do students use in argumentative introductions (hook, background, thesis, significance)?

    • RQ2: How do students express stance and hedging (e.g., boosters/hedges) in claims and counterclaims?

    • RQ3: How do these discourse patterns differ between higher- and lower-rated essays (or early vs late drafts)?

    Why this fit works: The RQs focus on language choices and rhetorical structure in texts, which discourse/genre analysis targets directly.

    Grounded Theory–style (constructivist) — building an explanatory model/process

    ~~Good for: repeated interviews + observations where you want to explain a process (not just themes).~~

    Research questions (example set)

    • RQ1: What process do novice EFL writers go through when revising after feedback across a semester?

    • RQ2: What factors shape movement from “surface editing” to “meaning-level revision”?

    • RQ3: How do students resolve tensions (time, confidence, teacher expectations, AI tools) during revision?

    ~~Why this fit works: The goal is to develop a model (stages, conditions, strategies, outcomes), which is exactly what grounded theory methods are for.~~

5) Build a codebook (make your coding defensible)

  • Example: 3-level coding (Theme → Category → Specific codes)

    👉 Context (sample study): EFL students revising after teacher written feedback.

    Theme 1: Feedback becomes “usable” when it is clear, actionable, and prioritized

    Category 1.1: Clarity of feedback message

    • Specific codes

      • Unclear meaning of comment (e.g., “awkward” with no explanation)

      • Ambiguous referent (student can’t tell what the teacher is pointing to)

      • Confusing terminology/metalanguage (e.g., “cohesion,” “hedging” not understood)

      • Overgeneral directive (e.g., “improve flow” without guidance)

    Category 1.2: Actionability (what the student can do next)

    • Specific codes

      • Concrete revision suggestion provided (teacher offers an example rewrite)

      • Step-by-step revision instruction (“add claim → add evidence → add explanation”)

      • Resource/tool recommended (link to model paragraph, checklist, rubric)

      • Student identifies a next action (“I need to add a counterargument here”)

    Category 1.3: Prioritization of issues

    • Specific codes

      • Too many issues at once (overload)

      • Student focuses on surface errors first (grammar/spelling before argument)

      • Teacher signals priority (“focus on thesis + evidence first”)

      • Revision plan created (student lists “top 3 fixes”)

    Theme 2: Emotional and identity responses shape uptake of feedback

    Category 2.1: Affective reaction

    • Specific codes

      • Discouragement / demotivation

      • Anxiety about grade or evaluation

      • Relief / increased confidence

      • Frustration with repeated comments

    Category 2.2: Perceived fairness/credibility of feedback

    • Specific codes

      • Trust in teacher expertise

      • Perceived inconsistency (“last time you said the opposite”)

      • Feedback seen as subjective (“it’s just opinion”)

      • Student seeks confirmation (asks peers/tutor if teacher is “right”)

    Category 2.3: Agency and ownership

    • Specific codes

      • Student resists feedback (keeps original wording/idea)

      • Negotiates meaning with teacher (asks clarifying questions)

      • Selective uptake (accepts some comments, rejects others with reasons)

      • Revision framed as “my voice” protection

    Theme 3: Classroom conditions enable or block revision

    Category 3.1: Time and workload constraints

    • Specific codes

      • Insufficient time to revise

      • Revision done last-minute

      • Competing course deadlines

      • Stops revising after “minimum required”

    Category 3.2: Support structures

    • Specific codes

      • Peer review helps interpret feedback

      • Tutoring session clarifies next steps

      • Teacher conference increases uptake

      • Model texts used as scaffold

    Category 3.3: Tool use (including AI)

    • Specific codes

      • Uses grammar tools for surface edits

      • Uses AI to paraphrase (risk of voice loss)

      • Uses AI to generate ideas/outline

      • Avoids tools due to integrity concerns

    One-line “codebook framing” you can paste under section 6)

    Theme = interpretive claim; Category = grouped pattern that supports the claim; Specific code = observable/repeatable label applied to a segment (a sentence/turn/paragraph/comment).

  • For each code include:

    • name

    • 1–2 examples (short excerpts)

6) First-cycle coding (generate codes) - Code what is relevant

  • Use code types that match your topic:

    • process/strategy codes (e.g., self-repair, peer scaffolding)

    • affect/stance (e.g., anxiety, confidence)

    • pedagogical moves (e.g., recast, prompt, meta-linguistic feedback)

    • task interaction (e.g., negotiation of meaning, uptake)

    • In vivo codes (i.e., when participants’ wording is theoretically meaningful)

  • Follow the same coding process across all data sources: transcribed audios of interview, focus groups, classroom observations, etc. , documents, questionnaires, etc.

7) Second-cycle coding (reduce & organize)

  • Merge duplicates; split overly broad codes

  • Cluster codes into categories (what goes together and why)

  • Ask: Which categories actually answer the RQs?

  • Look for:

    • recurring patterns

    • contrasts (high/low proficiency; novice/experienced teachers; task types)

    • conditions (when/where a pattern appears)

8) From categories to themes/claims

  • A theme is an interpretive pattern, not a topic label

  • Test a theme with 3 questions:

    • Is there enough evidence across data sources?

    • Is it clearly bounded (not everything fits)?

    • Does it explain something about the classroom context?

  • Create a theme map: Theme → subthemes → key excerpts → data sources

9) Use analytic memos (your “thinking on paper”)

  • Memo after coding sessions:

    • why you coded an excerpt as X

    • what changed in definitions

    • emerging hunches and rival explanations

    • links to constructs in SLA/applied linguistics (only when warranted)

  • Memos become sentences/paragraphs in your findings chapter

10) Establish trustworthiness (plan, don’t “add later”)

  • Triangulation: compare interviews ↔ observations ↔ artifacts (i.e., documents)

  • Member checking: confirm interpretations (as appropriate)

  • Peer debriefing: explain coding/theme logic to a critical colleague

  • Negative cases: actively search for counterexamples

  • Thick description: context + task + participants to support transferability

  • Audit trail: dataset log, codebook versions, memo log

11) Practical workflow (a simple sequence)

  1. Dataset map + data prep

  2. Codebook v1 + memoing

  3. Full coding + periodic codebook revisions

  4. Category/theme building + negative cases

  5. Triangulation + finalize claims

12) Output checklist (what you should be able to show)

  • A dataset inventory (what, when, who)

  • A codebook with definitions + examples

  • A memo trail documenting decisions

  • A theme map (themes, subthemes, evidence)

  • 2–4 defensible claims that directly answer the RQs

Graph View

Backlinks