Topics
Note: Happy New Year

1. Committee Reports:
   
   • Bureaucracy
     Nothing not already on Agenda.
     
   • Procedure
     Nothing to report.
     We are far behind, so we need to realign, and plan for training. Should we do 2-3 training sessions / texts before doing the real texts indepedently. It is agreed that the schedule has to be redone.
     We will not have a Friday teleconference next week - it is two days before the ISI meeting - we will discuss new schedule then.
   • Corpora & Data
     Nothing to report. Can browse corpus on wiki.
     
   • Annotation
     Annotators have been working (UMD). Homepage|Documents| Annotation Manual (for English IL0, IL0->IL1) - please add comments.
     Annotators should use instructions on annotator wiki. "Double concept annotation" not reflected in current documentation. Also, there may be issues with this - may not be able to find two concepts. Unless there is NO concept applicable, we will continue to do double concept annotation. Do WordNet annotation, then MK, but only do theta role assignment once.
   • Tools
     Nothing to report. Other things later in agenda.
     
   • Evaluation
     Nizar made some changes to way results are presented. Link is on wiki. You can now see annotations in parallel: structure of tree, concept distributions, ISI annotators added. There are still some desiderata for the eval reports. Others on the Eval committee should suggest metrics. It was suggested that the evaluation committee meet before the ISI meeting to discuss issues and make decisions/suggestions. Ed suggested that we also need a tutorial on the interpretation of the results. We might want something other than "exact match" on concepts.. e.g. "weak match" a certain number of links away. Owen suggests using kappa and alpha. Should factor in number of _possible_ categories. Evaluation committee will meet next Wednesday at 1:30 pm.
     
     
2. Tool Issues
   • "airline<line" - items with multiple source tags (MK & WN) (Simon, Andrew, Lei)
     Some of this has been fixed already. Some concepts came from both MK and WN. No outstanding issues.
   • Offline annotation with Omega: new multi-threaded version should be fast enough - Nizar says that it is much faster now. MITRE has gotten laptops for their annotators, so we might not need offline access to Omega.
   • Tool configuration for annotators with conflicting sw requirements: will discuss with Lei separately
     
     
3. Procedural questions (Simon et al.)
   • Words with no matching concept in one or another ontology See above. If absolutely no concept, leave blank.
   • Pronouns with multiple referents Owen: Don't annotate pronouns. Resolution is not part of current annotation round. David suggests annotating them with dummy concepts. We can do this automatically. They do still select thematic role. Owen will put a relevant note about this in annotation instructions. We are not doing coreference.
   • Pronouns with proper noun referents See previous. Also, don't annotate proper nouns for this week's homework.
   • Dates e.g. Mid-1992. Any temporal expressions. We should put YEAR for "1992", MONTH for "February", PERSON for "John Smith". For "Air France", you tag the 'head, if there is one. IL0 should include combination of proper nouns into single nodes. We might consider all working from same (corrected) IL0 tree, at least for the first 6 months. We won't make a big contribution to NE processing in English, BUT we can make a contribution to crosslingual NE processing. We need a separate section of the manual for named entities. Tiamat may also limit choices in annotation tools for proper names. Content committee will meet during the week, and present suggestions at meeting: Wednesday, 12:30 pm. Likewise, there is a problem with phrasal verbs, and prepositions in general. Decision is very semantic in some cases (e.g. "go through")
     
     
4. Other Weekly Annotation Task Discussion
   • Don't annotate proper nouns, and don't worry too much about phrasal verbs right now
     
     
5. NAACL Workshop Paper
   • Circulated: Bonnie made comments. Content committee will fill in theta grids, authors should fill in sections. Submission date is Jan. 21. 8-page limit. David would us to bring the sections to the meeting to finalize for sumbission.
     
     
6. ACL Workshop update (Ed)
   • Rocky. Workshop not accepted. Suggested we work with Text Meaning and Understanding workshop. They think this is reasonable. They have added one or two topics of specific interest to us. Ed has contacted them - they will do 1-1/2 days. We will have to present a paper. We don't have representation on the committee and we may have panel. Alternatively, we could withdraw. Ed's suggestion is just to present our work there, and then do a COLING workshop or similar.
     
     
7. ISI meeting logistics (Ed)
   • wireless access for everyone if we want it.
   • annotators may join phone call at certain time(s) Sun afternoon and Monday.
   • send agenda items to David / Ed
     
     
8. Phase II Proposal Plan
   • ITIC, ITR, or both. Need to adhere to NSF standards. We should be working on this ASAP. Bonnie will work on this, and send it out by tomorrow. We need to have a NSF-ITR Letter of Intent (2500 characters max.) submitted by January 14 - by each site. Bonnie will draft this and send out. Feb. 24 is NSF-ITR proposal deadline.