LSA 2017: Intonation and Computation

Time: Mon/Thu 3:30-5:20
Place: TBA

Professor Julia Hirschberg (Office Hours TBA)

Announcements | Academic Integrity | Description
Readings | Resources | Requirements | Syllabus


This course introduces to computational linguistics approaches to the study of intonation.


The major requirement of the course will be to design an experiment to evaluate intonational characteristics of some type of speaker state.


Academic Integrity

Copying or paraphrasing someone's work without attribution is plagiarism and will result in a grade of zero for the assignment. Please see the university policy.

Required texts are available online via links from this syllabus.


  • 50% Class participation

  • 50% Course Project


·         Praat - Praat resources

·         Help using ToBI - ToBI Annotation Environments

·         Text-to-Speech Links and more...

·         Sox - audio file editing





Reading  Assignments

 HW Due Dates and Other Assignments

Jan 18

It's not what you said, it's how you said it



Jan 23 Spoken Dialogue: Human and Machine J&M 24-24.1, 24.8 Project Description assigned.  Make appointments to talk with Prof or TAs about your project.

Jan 25

From Sounds to Language

J&M 7.1-7.3, 7.5


Jan 30

Acoustics of Speech

J&M 7.4


Feb 1

Tools for Speech Analysis

Praat tutorial 1


Download Praat to your laptop if you have one and bring to class with headphones if you have.

Feb 6 Project Elevator Speeches and Lab Visit BVC Contest Project Description due.

Each team gives a 4-5m elevator speech about project in class.

Feb 8

Speech Generation Overview [pdf]

J&M 8 (pp. 249-50, 281-84); TTS-history; Historical examples


Feb 13

Building a TTS System


Rivka Levitan

Project Part 1 (TTS) assigned

Feb 15

Text Normalization

J&M 8.1, Sproatetal01


Feb 20

Modeling Pronunciation

J&M 8.2; Ghoshaletal09


Feb 22

Prosody Modeling

Hirschberg03, J&M 8.3.0-8.3.4, ToBI labeling conventions

Download and listen to all the ToBI examples.


Feb 27

Predicting Prosody from Text

J&M 8.3.4-8.3.7


Feb 29

Information Status: Focus and Given/New

GBrown83, Prince92, Terken&Hirschberg93

Takehome exercises (due in class Mar 7)

Mar 5

Backend Synthesis and TTS Evaluation [pdf]

J&M &M 8.4-5, 8.6 Tokuda35al02

Project Part 1 due

Project Part 2 (ASR) assigned 

(option: Installing Pocket Sphinx on your laptop)

Mar 7



NB:  Please deposit the take-home exercises for this exam in Courseworks before class.

Mar 12-16

Spring Break



Mar 19 Human Speech Perception J&M 10.7  

Mar 21

ASR: Overview

J&M 9-9.2


Mar 26

Building an ASR System

Sphinx home page, examples: C++, Python

Erica Cooper

Mar 28

Context-Free Grammars

J&M Ch 12-12.4


Apr 2

Language Modeling and Grammars

J&M 4-4.4


Apr 4

Pronunciation Modeling

J&M 6-6.5, 9.3-9.5

 Michael Picheny, IBM Research

Apr 9

Features, Training and Decoding

J&M 9.6-9.9


Project Part 2 due

Project 3 (SDS) assigned; Special Addition

Apr 11

SDS System Architectures

J&M 24.2-3


Apr 16

Recovering from Errors in SDS

J&M 24.5, Hirschbergetal04


Apr 18

Turn-Taking in SDS



Apr 23

Entrainment in SDS



Apr 25

SDS Evaluation

J&M 24.4, Walkeretal97

Preliminary Project Demos

Apr 30

Final Exam



May 1-3

Study Days



May 9

Project Demos (1:10-4)


Project Part 3 due; Special Addition


Links to Resources

cf. also resources available from the text homepage

Places to look up definitions and descriptions of terminology:

  1. Oxford Dictionary of Linguistics
  2. Interesting Language Factoids and Non

Other resources

  1. Karen Chung Language and Linguistics links
  2. CatSpeak
  3. Check out Eliza
  4. Appelt and Israel's information extraction tutorial (IJCAI-99).
  5. Framenet.
  6. Ask Jeeves-- a search engine that answers questions in plain English.
  7. Answer Bus -- another Q/A system.
  8. Columbia's NewsBlaster summarizer
  9. Systran machine translation (also in use at Babelfish)
  10. AT&T Labs - Research Finite State Machine Library
  11. Michael Collins' Parser
  12. On-line dictionaries in many languages.
  13. WordNet
  14. Framenet
  15. CoBuildDirect Corpus
  16. AT&T's SCANMail voicemail browsing/search system
  17. DiaLeague 2001 -- includes a link to an online dialogue system demo.
  18. James Allen's Dialogue Modeling for Spoken Language Systems ACL 1997 Tutorial
  19. Festival speech synthesizer demo and links to other TTS systems
  20. Julia Hirschberg's Intonational Variation in Spoken Dialogue Systems tutorial

Julia Hirshberg Portrait

Julia Hirschberg
Professor, Computer Science

Columbia University
Department of Computer Science
1214 Amsterdam Avenue
M/C 0401
450 CS Building
New York, NY 10027

email: julia@cs.columbia.edu
phone: (212) 939-7114

Download CV

Columbia University Department of Computer Science / Fu Foundation School of Engineering & Applied Science
450 Computer Science Building / 1214 Amsterdam Avenue, Mailcode: 0401 / New York, New York 10027-7003
Tel: 1.212.939.7000 / Fax: 1.212.666.0140