Research and Teaching Page of William Webber

Hi! I'm a postdoctoral research associate in the e-discovery lab of the iSchool at the University of Maryland, College Park. I am working on NSF IIS Program #1065250, Development and Evaluation of Search Technology for Discovery of Evidence in Civil Litigation, under the direction of Doug Oard and colleagues.

I completed by PhD thesis at the University of Melbourne in 2010, under the supervision of Alistair Moffat and Justin Zobel. My personal page may also be of interest to you.

My research interests include Information Retrieval for Civil Litigation (E-Discovery); Experimental and Evaluation Methodology for Information Retrieval; and Distributed Information Retrieval.

I keep a research blog on evaluation, information retrieval, computer science, and information technology.

PhD Thesis

Publications

• "Approximate Recall Confidence Intervals", William Webber, 31 pages (in submission; arXiv:1202.2880 v2, 2012-05-17) (code and data).
• "Principles for Robust Evaluation Infrastructure", Justin Zobel, William Webber, Mark Sanderson, and Alistair Moffat, Proc. CIKM Data Infrastructures for Supporting Information Retrieval Evaluation, Glasgow, UK, October 2011, 4 pages (paper).
• "Re-examining the Effectiveness of Manual Review", William Webber, Proc. SIGIR Information Retrieval for E-Discovery Workshop, Beijing, China, July 2011, 8 pages (paper).
• "A Similarity Measure for Indefinite Rankings", William Webber, Alistair Moffat, and Justin Zobel, ACM Transactions on Information Systems, Volume 28, Issue 4 (November, 2010), pages 20:1--38 (paper).
• "Assessor Error in Stratified Evaluation", William Webber, Douglas W. Oard, Falk Scholer, and Bruce Hedin, Proc. ACM-CIKM, Toronto, October 2010, page 539--548 (paper).
• "The Effect of Pooling and Evaluation Depth on Metric Stability", William Webber, Alistair Moffat, and Justin Zobel, Proc. EVIA, Tokyo, June 2010, pages 7--15 (paper).
• "Improvements That Don't Add Up: Ad-Hoc Retrieval Results Since 1998", Timothy G. Armstrong, Alistair Moffat, William Webber, and Justin Zobel, Proc. ACM-CIKM, Hong Kong, November 2009, pages 601--610 (paper).
• "Score Adjustment for Correction of Pooling Bias", William Webber and Laurence Park, Proc. ACM-SIGIR, Boston, July 2009, pages 444--451 (paper).
• "Relative Significance is Insufficient: Baselines Matter Too", Timothy G. Armstrong, Justin Zobel, William Webber, and Alistair Moffat, Proc. ACM-SIGIR Workshop on The Future of IR Evaluation, Boston, July 2009, pages 25--26 (poster) (paper).
• "Has Adhoc Retrieval Improved Since 1994?", Timothy G. Armstrong, Alistair Moffat, William Webber, and Justin Zobel. Proc. ACM-SIGIR, Boston, July 2009, pages 692--693 (poster) (poster).
• "EvaluatIR: An Online Tool for Evaluating and Comparing IR Systems", Timothy G. Armstrong, Alistair Moffat, William Webber, and Justin Zobel, Proc. ACM-SIGIR, Boston, July 2009, page 834 (demo) (extended abstract, EvaluatIR.org website).
• "Statistical Power in Retrieval Experimentation", W. Webber, A. Moffat, and J. Zobel, Proc. ACM-CIKM, Napa, USA, October 2008, pages 571--580 (paper).
• "Score Standardization for Inter-Collection Comparison of Retrieval Systems", W. Webber, A. Moffat, and J. Zobel, Proc. ACM-SIGIR, Singapore, July 2008, pages 51--58 (paper).
• "Precision-At-Ten Considered Redundant", W. Webber, A. Moffat, J. Zobel, and T. Sakai, Proc. ACM-SIGIR, Singapore, July 2008, pages 695-696 (poster).
• "Score Standardization for Robust Comparison of Retrieval Systems", W. Webber, A. Moffat, J. Zobel, Proc. 12th Australasian Document Computing Symposium, Melbourne, December 2007, pages 1-8 (paper). Winner, Best Student Paper Prize. (Please see addendum concerning resampling method used in Figure 5.6.)
• "Strategic System Comparisons via Targeted Relevance Judgments", A. Moffat, W. Webber, J. Zobel, Proceedings of the International ACM-SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, July 2007 (paper).
• "Design and Evaluation of a Pipelined Distributed Information Retrieval Architecture", W. Webber, Thesis, Master of Engineering, University of Melbourne, June 2007.
• "A pipelined architecture for distributed text query evaluation", A. Moffat, W. Webber, J. Zobel, R. Baeza-Yates, Information Retrieval, Springer Netherlands, October 2006.
• "Load balancing for term-distributed parallel retrieval", A. Moffat, W. Webber, and J. Zobel, Proceedings of the International ACM-SIGIR Conference on Research and Development in Information Retrieval, Washington, August 2006 (paper).
• "Space-limited ranked query evaluation using adaptive pruning", N. Lester, A. Moffat, W. Webber, and J. Zobel, Proceedings of the WISE Workshop on Information Systems Engineering, M. Kitsuregawa, E. Neuhold, and A.H.H. Ngu (eds), New York, November 2005, pp. 470-477. Full text Bibtex
• "In Search of Reliable Retrieval Experiments", Webber, Moffat. Proc. 10th Australasian Document Computing Symposium, Sydney, December 2005, pages 26-33. Full text Bibtex
• "Melbourne University 2005: Enterprise and Terabyte Tracks", Anh, Webber, Moffat. Proc. 2005 TREC Notebook Proceedings, Gaithersburg, MD, November 2005.
• "RMIT University at TREC 2005: Terabyte and Robust Track", Y. Bernstein, B. Billerbeck, S. Garcia, N. Lester, F. Scholer, J. Zobel, W. Webber.
• "RMIT University at TREC 2004", B. Billerbeck, A. Cannane, A. Chattaraj, N. Lester, W. Webber, H.E. Williams, J. Yiannis, J. Zobel.

Professional activities

Editorships

• Information Retrieval (member of editorial board, 2011--present)

Invited talks

• Principles for Robust Evaluation Infrastructure, NTCIR-9, December 2011 (with Mark Sanderson).

Conference chairs and committees

• EVIA Workshop 2010 (co-chair), 2011 (co-chair)
• ACM CIKM Conference, 2009 (Program Committee), 2010 (Program Committee), 2011 (Program Committee)
• ACM SIGIR Conference, 2009 (Program Committee), 2010 (Program Committee), 2011 (Program Committee)
• Australian Document Computing Symposium, 2007 (Program Committee), 2009 (Program Committee)
• String Processing and Information Retrieval Symposium, 2008 (Organizing Committee)
• Asian Information Retrieval Symposium, 2009 (Publicity Chair, Australia), 2010 (Area Co-Chair: Evaluation)
• International Joint Conference on Artificial Intelligence, 2011 (senior program committee)

Reviewing:

• ACM Transactions on Information Systems
• Journal of the American Society for Information Science and Technology
• Information Retrieval (journal)
• The Computer Journal
• Australian Document Computing Symposium, 2005, 2007, 2008 (shared Best Reviewer award), and 2009
• ACM Conference on Information and Knowledge Management, 2006, 2007, 2008, and 2009
• Symposium on Combinatorial Pattern Matching, 2008
• European Conference on Information Retrieval, 2008
• ACM SIGIR Conference, 2008 and 2009
• Conference on Very Large Databases, 2007 and 2008
• ACM International Conference on Web Search and Data Mining, 2008
• IEEE Transactions on Knowledge and Data Engineering, 2011

In Spring 2012, I am teaching LBSC 690, Information Technology at the University of Maryland, College Park.

In 2009, I co-taught 433467, Text and Document Management, in first semester, at the University of Melbourne.

• An implementation of the rank-biased overlap (RBO) indefinite rank similarity measure.
• I made a distributed fork of the Zettair search engine for my Masters research, back in 2004-2006. This fork is not being maintained, and is primarily of historical interest.

• An example of creating name tags in latex.
• I did some thread analysis for the TREC W3C collection.

William Webber
17/May/2012