11-26-2003 Notes

11-26-2003 Notes	IAMTC
Notes from 11-26-2003 Teleconference

Minutes of phone meeting
Wednesday Nov 26
Host: ISI

Minutes by Eduard

Meeting started 10:45 PST = 13:45 EST

1. NSF blurb -- Bonnie
The NSF (Mary Harper) wants us to send them some material about our project. The full blurb is below. We need to find someone to take charge of this. THIS IS DUE DECEMBER 8th!
Action: Bureaucracy committee (David) will write up a blurb and send it around. We will later pirate this for the ACL workshop proposal, etc.

2. HLT-NAACL workshop -- Bonnie
2a. Respondent
Should we start looking around for someone to invite to critique our project at the NAACL workshop? (Christiane?) Several candidates were mentioned: Sergei, Christiane, Martha. The favorites were Martha and Sergei. Lori suggested we invite all.
Action: None; we await ok from workshop committee before proceeding

2b. Format at workshop
It appears Adam Meyers is a little confused about what we want to do, and sent Bonnie many questions.
Action: Bonnie will hold his hand and explain to him what’s planned: 40-minute project description (including details of each site), plus 20-minute commentary by one or two invited people. We will invite the commenters also to submit written notes for the workshop proceedings.

2c.Workshop paper -- Bonnie
We need to write a joint paper for that workshop---the deadline is January 21st.
Action: David will take a first stab at making an outline and incorporating his notes, the workshop proposal written by Bonnie and Ed, etc.

3. Annotation tool update -- Eduard
ISI has been working hard to get a version of the tool ready by Dec 1, and are planning to distribute the tool, first version, starting Monday next week. There are four modules, not yet integrated as much as we’d like:

Dependency parser (either Connexor or Owen’s) – runs at UMD or Columbia
IL0 parse grapher/editing tool TrEd – runs locally as part of tool
Main annotation interface, with display and editing windows – runs locally as part of tool
Omega and interface – runs at ISI

You will notice that Omega now includes (some of) the roles; apparently only up to verbs starting with “d”. ISI will fix.
Action: Further details about the distribution plan for next week will be sent out separately by email.

4. Foreign dependency parsers -- Eduard
This is with an eye toward the future. It appears we have a lot semi-available already:

English: Connexor (UMD) and Owen’s (Columbia). Converter for Connexor into TrEd format to be written by Nizar
French: Keith’s (MITRE) and Connexor (UMD). Converter to be adapted by Nizar and Keith
Spanish: Connexor (UMD)
Arabic: system at UMD. Converter into TrEd format to be written by Nizar
Hindi: system downloadable from UPenn; need to convert Treebank-style output to dependency form; to be investigated by Adwaith and Nizar
Japanese: system at CMU. Converter into TrEd format to be written by Simon
Korean: system known by Owen; Eduard will contact him

5. Deep syntax vs. semantics: what are we annotating?
Steve brought up the question. Various responses. Steve and others listed several examples of differences: proper names, verb-particle constructions, extent of compound nominals, conjunction, etc.
Ed suggested a czar might have to make up the rule in each case. Lori said that it looks like some people have less difficulty using dependency as a starting point for semantics than others. Adwaith suggested automatically parsing up just the NP level; i.e., just ‘chunks’, instead of starting with the whole dependency tree, and then manually stitching the chunks together using the roleset. Various people commented (pro: dependencies are more trustworthy and more intuitive lower down; con: dependency parse does some canonicalization, zero anaphor resolution, etc.).
Action: No definitive conclusion. Plan is to bear this issue in mind during the next week’s annotation using the tool, and thinking about which version would be easier: - start with full dependency parse, which one person edits for all before annotation - start with chunk-level parse and stitch chunks together - start with no parse and do it all manually

6. Annotation exercise: F1E1 sentence 2 -- all
Since time was running out and people had to go, we decided to postpone the discussion of this until next week.
Action: use the annotation tool to (re-)do the second sentence of F1E1.

7. Next week's phone call -- Eduard
Friday December 5, 11:15 – 12:15 EST
CMU is the host (Lori)

Meeting ended 11:55 am PST = 14:55 EST

Notes by Keith

David will write blurb for NSF as well as the outline, Ed and others will pirate this for ACL workshop, etc.

2 suggestions for people to comment on our work during Adam's workshop: Christiane, Sergei
Ed suggested Martha (Palmer)
Several people agreed, and Lori suggested that it would be interesting to hear from all of them

Some potentail issues with Adam's workshop: he is uncomfortable with our proposal, but there seems to have been some miscommunication. We will go with his suggestion of 40 minutes presentation with 20 minutes of structured discussion and critique. This was our intent.
Bonnie will take care of talking to Adam.

Annotation tool update -- Ed

they are working like crazy to get tool ready by end of month. Should be ready to deliver *something* to sites by next week.
we will work with annotation tool to annotate second sentence, and discuss this at the meeting next week.
next week will be a week to set up tool, and play with it, so we are all ready to go, tool-wise, by the end of next week
foreign dependency parsers: we need to either do parse by hand or find parsers: UMD has Connexor for En, Es, Fr. We will look at output of these, to see if it is passable. If not, there are other options for Fr parser that MITRE will look at. Can convert Treebank output to dependency for Hindi, and there is progress on Arabic.

Question: Are we annotating SYNTAX or SEMANTICS? What are annotators going to annotate?
See Steve's e-mail on syntax correction and intercoder reliability.
NN compounds
V-Particle constructions
etc....

Ed suggests that czar for each section can set standard.
Dependencies are part of semantics, but what are we trying to get IA on??
Steve proposes that we have someone correct the parse, and then give that to the annotators.
Adwait and David suggest simply chunking, not assigning syntactic structure, and then annotating dependencies/semantics from that.
We could do two sentences for next week - one with each method.

Next week's phone call: Early -- 11:15 to 12:15. We will discuss annotation process on sentence 2, and questions that came up wrt the parser and to the cleanup and annotation.

Version 5, Wed 26 Nov 2003 19:36:05 [EH] - created Wed 26 Nov 2003 14:48:46 [KJM]