Suggestions from Inter-site Annotator Meeting Thursday 1/29/2004

 

  1. Everyone likes new tool but some little things
  2. There is some instabilities of how Tiamat is handling multiple assignments of concepts and roles.  Here are some examples:
    1. WN concepts are turned into MK concepts after role-assignment… this is a big problem since (i) Tiamat would not accept at all since it think WN concepots have not been selected (ii) after a concept is turned from MK to WN, Tiamat doesn’t allow you to edit roles.
    2. Some concepts have {MK/WN}… but they behave the same way as in point b.
    3. Assignment of multiple theta role sets is not currently working properly.
  1. There is no feedback right now in Tiamat.  When a node is retyrned to after being annotated, there is no sense of what was done.  It is as if we are starting from scratch. 
  2. Tiamat allows Accepting only once. (what is the purpose of accept button anyway, can we get rid of it? Shouldn’t every change be logged?)

 

  1. Some Suggestions
    1. Concepts that are assigned MK/WN, should be accepted once for both types of annotations.
    2. Display issue (This is very useful if possible):

                                                               i.      In the big text box where concepts are displayed, can we see at the top all the features of the node being annotated, including POS information and previously assigned concepts/roles.  This should happen every time we click on the word. 

                                                             ii.      Can the greyout of selected concepts be used to show which theta-roles grids have been done or not.

                                                            iii.      Is it possible to always integrate the results from the query with the completed annotations? 

                                                           iv.      When working with a specific file, can its name be shown in the top window label/bar. (for example, TIAMAT 2.0: <file-name>). After a save as, it the bar should be updated.

                                                          v.      Can we do better linewrapping in the window displaying the article.  Instead of wrapping at text box (which cuts through words), wrapping should occur on a white space).

                                                           vi.      Please get rid of the warning message "You have not picked a {WN/MK} concept." If an annotator missed a concept, just assign Dummy Concept to it by default.

                                                          vii.      Can concepts be listed in alphebatical order?  Roles were listed alphabetically an dthen they changed. Why?

c.  When a word that has been seen and annotated before in the same article appears, can a default assignment for concepts be used by copying the same set of concepts from the last annotation attempted? Theta roles will obviously have to be done again.