Spoken Language Processing (CS 4706), Spring 2005


MW  2:40-3:55


Mudd 253


Julia Hirschberg

Office Hours: 






Teaching Assistant:

Agus Gravano

Office Hours:

MW 4-5, CEPSR 7LW1



Phone: 212-939-7147


Announcements || Academic Integrity || Description
Resources || Requirements || Syllabus || Readings


This course introduces students to research in spoken language in computational linguistics, aka natural language processing (NLP). We will study the different `meanings' that can be conveyed by the way that speakers produce sentences, techniques for analyzing spoken language, methods of developing speech technologies, and applications of such technologies in the real world, such as text-to-speech systems, speech recognizers, spoken dialogue systems, and detectors for various types of emotional speech.  NB: This course can be counted as a PhD elective in Advanced AI .  It is a requirement for the MS NLP Track.  There are no official prerequisites for this course and no prior knowledge of NLP will be assumed.


Required readings:  Acoustic & Auditory Phonetics by Keith Johnson  and selected chapters from  Speech and Language Processing by Jurafsky and Martin.  Both are available from Morningside Bookstore and will also be on reserve in the library.  Other course readings will be available either on the web or in-class handouts. NB: '*' means that the reading is optional.

 Course Requirements:

Midterm and final; 4 lab homeworks (can be done on your own pc); 1 10m class presentation on a paper relevant to one of the classes.

Academic Integrity:

Copying or paraphrasing someone's work (code included), or permitting your own work to be copied or paraphrased, even if only in part, is not allowed, and will result in an automatic grade of 0 for the entire assignment or exam in which the copying or paraphrasing was done. Your grade should reflect your own work. If you believe you are going to have trouble completing an assignment, please talk to the professor in advance of the due date.



Help using ToBI

Text-to-Song synthesis!


Feb 3




Readings and Assignments


Jan 19

Introduction to the Course



Jan 24

Understanding Speech Variation



Jan 26

Interpreting Speech Variation

Pierrehumbert&Hirschberg '90; Hedbergetal02; Syrdal&Jilka04; Dohertyetal04;


Jan 31

Analyzing the Speech Signal:  Symbolic Approaches

J&M 4-4.2


Feb 2

Analyzing the Speech Signal:  Signal Processing Approaches

Johnson, Ch. 1-2


Feb 7

Representing Speech Variation

Wilson93; ToBI labeling conventions (and see ToBI examples); Baumannetal01


Feb 9

Tools for Speech Analysis

Praat tutorial


Feb 14

Lab:  Learning ToBI Labeling

Assignment 1: Complete the ToBI Labeling Manual Examples


Feb 16

Speech Generation: From Concept and from Text

HLT96-ch5, TTS systems


Feb 21

Modeling Pronunciation

J&M, 4.6


Feb 23

Predicting Accents and Phrasing

J&M, 4.7
Pan99, Sun02, Koehn00, Rambow01


Feb 28

Information Status: Focus and Given/New

*Nakatani99, GBrown83, Bard99, Prince92, Dahan02


Mar 2

 Discourse Structure in TTS Systems

J&M, 18.3, 19.4


Mar 7

Speech Recognition and Understanding

J&M, 7; HLT96-ch1


Mar 9




Mar 21

Speech Disfluencies

*Brennan&Schober99, Hindle83, Hirschberg&Nakatani?;Bear92


Mar 23

Sentence, Speaker, and Topic Segmentation

 Shriberg00, Shriberg98


Mar 28

Identifying Speech Acts

J&M, 19.2-3 (new version)
Jurafsky98, Nickerson&Chu-Carroll99


Mar 30

Spoken Dialogue Systems

J&M, 19 (new version)
Walkeretal97, Goldberg03, Bell&Gustafson00,[pdf] Krahmer01


Apr 4

Turntaking in Spoken Dialogue

J&M, 19.1 (new version)
Turn-taking in Conversational Analysis (follow links)


Apr 6

Corrections and Repairs



Apr 11

Speech Search

Hirschbergetal01, SCANMail demo


Apr 13

Speech Data Mining

Maskeyetal03, Hearst99


Apr 18

Speech Summarization

Barzilay00, Furui02,  Maskey&Hirschberg03


Apr 20

Emotional Speech

Cowie00, Pereira00, Schroeder01, Bosch00, *Burkhardt00, *Ang02


Apr 25

Deceptive Speech



Apr 27

Charismatic Speech



May 2

Summing Up



May 3-5

Study Days



May 6-13



Announcements || Academic Integrity || Description
Links to Resources || Requirements || Syllabus || Text || Thanks