Review Questions<br>
----------------------<br>
<br>
IMPORTANT: Please note that there is no guarantee that the questions in the final will be anything like the review questions. These questions are just a sampling of the material and should NOT be considered a substitute for going through the lectures and the textbook.<br>
<br>
n-gram language model<br>
=====================<br>
<br>
-- Why are language models useful in NLP? What are some NLP applications that use n-gram LMs?<br>
-- What is the operative assumption that allows us to build n-gram LMs?<br>
-- How would you compute the maximum likelihood estimate for the bigram probability P(w1,w2)?<br>
-- What are the two problems associated with scaling n-gram language models to higher n?<br>
-- Why is smoothing necessary? Why is it also called discounting?<br>
-- Is Laplace's law an effective smoothing technique? Why/Why not?<br>
-- Derive the expression for the bigram conditional probability under Laplace's law P_LAP(w2|w1).<br>
-- Prove that Good-Turing is actually performing discounting (not a formal proof, just an intuitive one).<br>
-- Given a corpus with N tokens, V types, B unique bigrams and the (r,Nr) distribution from the example in the lecture slides (second slide on page 30 in the pdf), explain how you would go about performing Good-Turing discounting.<br>
-- Is it ok to use the MLE P(w1) as the denominator when computing P_GT(w2|w1)? Why/Why not?<br>
-- Is it a good idea to perform Good-Turing discounting uniformly,i.e., for all r? If not, what possible alternatives do we have?<br>
-- What is the difference between interpolation and backoff (in the context of LMs)?<br>
-- Can I perform backoff with regular MLEs of n-gram probabilities or do I need to do discounting first?<br>
-- Does Katz Backoff yield well-formed probability estimates?<br>
-- What are grammatical zeros? Can Katz Backoff deal with genuine grammatical zeros?<br>
-- What is the other disadvantage of Katz Backoff?<br>
-- Is the unigram model P_{cont} in Kneser Ney smoothing different from a regular MLE unigram model? If so, how? Why is it useful?<br>
-- What is the difference between closed vocabulary and open vocabulary language modeling?<br>
-- What is perplexity? Why is it an appropriate measure for evaluating LMs? Intuitively, can you say if perplexity differs significantly from entropy?<br>
-- Why do closed vocabulary scenarios lead to lower perplexities?<br>
-- Why would testing a newswire-trained LM on blogs yield higher perplexities?<br>
<br>
<br>
WSD questions<br>
============<br>
<br>
-- What is the difference between homonymy and polysemy?<br>
-- What is a synset?<br>
-- What do the edges in WordNet represent?<br>
-- What are hypernyms?<br>
-- What are meronyms?<br>
-- What is the difference between the lexical sample task and the all-words WSD tasks?<br>
-- What are some commonly used features in a WSD task?<br>
-- Give example of an extrinsic evaluation of a WSD system?<br>
-- What is Senseval/SemEval?<br>
-- What is sense-annotated data? Why is it needed?<br>
-- What do knowledge-lean and a knowledge-rich WSD approaches mean?<br>
-- What are supervised, unsupervised, and semi-supervised WSD approaches?<br>
-- Describe Lesk's WSD algorithm?<br>
-- Describe Yarowsky's cotraining algorithm for WSD?<br>
-- What is the difference between word sense disambiguation and word sense discrimination?<br>
-- Give two ways in which the basic Lesk algorithm can be improved.  <br>
-- What are training and test data?<br>
-- Why does more training data give better results?<br>
-- What is the Naive Bayes Algorithm for WSD?<br>
-- What is a decision list?<br>
-- What are collocations?<br>
-- Why is Yarowsky's algorithm appealing?<br>
-- What are its drawbacks?<br>
<br>
<br>
Semantic Distance<br>
===============<br>
<br>
-- What is the difference between semantic similarity and semantic relatedness?<br>
-- Give examples of how semantic distance algorithms are evaluated?<br>
-- Understand the edge-distance, Wu Palmer, Hirst--St. Onge, Resnik, and Jiang Conrath methods.<br>
-- What is the basic idea behind distributional measures of similarity?<br>
-- What is information content?<br>
-- What are the differences between WordNet-based and distributional measures of similarity?<br>
<br>
Clustering<br>
========<br>
<br>
-- What is text clustering? <br>
-- How does partitional clustering work?<br>
-- What are first-order and second-order co-occurrences?<br>
-- What are intra-cluster and inter-cluster distances?<br>
<br>
MT, IR, Speech<br>
==============<br>
For the guest lectures on MT, IR and Speech processing, we will most likely ask you very high-level questions that do not go into any significant technical detail but rather test your understanding of the basic concepts that the lectures touched on.
For example, one such question for MT might ask you to identify and briefly describe the various components of a typical phrase-based SMT system. This is something that Philip covered in great detail in the class.