Course Description

An interdisciplinary exploration of conversational AI through humanistic and social-scientific lenses. Examination of the technologies behind contemporary conversational interfaces widely deployed across industries and evaluation of their effectiveness, human-likeness, and social implications using frameworks from various intellectual traditions, such as intercultural pragmatics, conversation analysis, and the psychology of joint action.

---

Athena Title

Demyst Conversational AI

---

Grading System

A - F (Traditional)

---

Student learning Outcomes

• Students will be able to describe the technologies behind different types of conversational AI to a nontechnical audience.
• Students will be able to interpret publicly available model documentation (e.g. research blogs, model cards, and data sheets) to identify key details regarding the training and capabilities of conversational AI, and evaluate the extent to which these disclosures are informative about system performance in real-world situations.
• Students will be able to identify major skills required for conversational communication (e.g. turn-taking, repair, common ground management) in both human-human and human-machine interactions across cultures, and describe how AI models are trained to imitate these skills.
• Students will be able to evaluate the extent to which conversational AI systems communicate in humanlike ways by analyzing conversational transcripts, as well as the benefits and risks of deploying AI systems that display or lack humanlike characteristics.
• Students will be able to identify the causes of interactional trouble in human-machine interaction and how they were or were not resolved, and evaluate the interactional burden imposed on human users of different social and cultural backgrounds.
• Students will be able to identify possible instances of interactional harm caused by conversational AI systems and evaluate the extent to which current safeguards reduce such harms in different social and cultural contexts.
• Students will be able to evaluate how the development and deployment of conversational AI is influenced by cultural assumptions such as referentialism, monologism and the planning model, and how these assumptions shape users’ experiences with AI around the world.
• Students will be able to design simple conversational flows using accessible, no-code systems while considering the needs of all potential users from different global contexts and minimizing potential interactional harm.

---

Topical Outline

• I. Introduction to conversational AI: history and classification of conversational AI; processes in human conversation
• II. Processes involved in conversational AI: automatic speech recognition; natural language understanding; dialogue management; natural language generation; text-to-speech synthesis
• III. Technologies behind conversational AI: modular models; retrieval-based models; end-to-end neural models; large language models: pre-training, instruction tuning, reinforcement learning from human feedback, retrieval-augmented generation; model descriptions in model cards
• IV. Datasets behind conversational AI: Datasets for traditional supervised NLP; pre-training data; supervised fine-tuning data; human preference data; data sheets
• V. Traditional evaluation of conversational AI: Turing test; business metrics; static benchmarks; human preference ratings; cultural assumptions behind traditional evaluation; evaluation metrics in model cards
• VI. Understanding language in context: referential forms and entity extraction; entity linking and coreference resolution; ellipsis resolution and query rewriting; referentialism and monologism in the production of AI language
• VII. Managing the local structure of interactions: dialogue acts; intent classification and slot-filling; use of discourse markers; sequence expansion and closing; the planning model vs. situated action
• VIII. Mutual understanding and its breakdown: grounding, feedback and the backchannel; conversational troubles and repair; interactional burden
• IX. Turn-taking and participation: turn-taking cues in human-human and human-machine communication; overlaps and interruptions; silences; embedding of conversational AI in human-human interactions; full-duplex conversational AI models
• X. Managing the global structure of interactions: Opening and closing conversations; memory and common ground; topic management
• XI. Personifying conversational AI: stance, indexicality and persona; linguistic construction of AI sycophancy and its interactional effects; affective stance, emotion, and sentiment analysis; anthropomorphism and the Eliza effect; persona effect; benefits and risks of humanlikeness
• XII. Cultural variation and conversational AI: Linguistic variation and language models; linguacultures; formality and register; indirectness and politeness; multilingual language models and their difficulties; standard language ideology in AI; the digital divide in conversational AI
• XIII. Nonverbal communication, embodiment and social robots: prosody in speech technology; gaze and head movement; gesture recognition
• XIV. Social impacts and ethics: representational balance and fairness; harmlessness, honesty and helpfulness; guardrails; interactional harm and its measurement

---

Institutional Competencies Learning Outcomes

Analytical Thinking

The ability to reason, interpret, analyze, and solve problems from a wide array of authentic contexts.

Critical Thinking

The ability to pursue and comprehensively evaluate information before accepting or establishing a conclusion, decision, or action.

---

Syllabus

Select Faculty / Instructor to download syllabus:   Lai, Ka Yau

---

Public CV

Select a file to download:   Lai, Ka Yau

---