This project plans to develop a new evaluation framework to understand and characterise web users and their situation within complex, multi-faceted search tasks, exemplified through job-search. While evaluation of web search engine effectiveness is relatively well understood, measuring information retrieval performance in the context of complex tasks with heterogeneous users is a largely neglected problem. This project plans to mine user-specific characteristics and situations from complex profiles and interaction logs for online information services run by the industry partner, SEEK. The new techniques are intended to redefine understanding of task-oriented search, and have the potential to reinvent the user experience for complex search tasks. This project will transform how practical search systems are measured within complex task scenarios. This will result in substantial economic impact by enabling businesses providing task-based search services to provide more customized offerings. Within the target domain (job search), this greatly enhances a service highly relevant to Australia’s productivity.
- E. Amigó, D. Spina, J. Carrillo-de-Albornoz. An Axiomatic Analysis of Diversity Evaluation Metrics: Introducing the Rank-Biased Utility Metric. In Proceedings of SIGIR’18, 2018.
- A. Moffat, A. Wicaksono. Users, Adaptivity, and Bad Abandonment. In Proceedings of SIGIR’18, 2018.
- B. Salehi, D. Spina, A. Moffat, S. Sadeghi, F. Scholer, T. Baldwin, L. Cavedon, M. Sanderson, W. Wong, J. Zobel. A Living Lab Study of Query Amendment in Job Search. In Proceedings of SIGIR’18, 2018.
- S. Shiga, H. Joho, R. Blanco González, J. R. Trippas, M. Sanderson. Modelling information needs in collaborative search conversations. In Proceedings of SIGIR’17, 2017.
- D. Spina, M. Maistro, Y. Ren, S. Sadeghi, W. Wong, T. Baldwin, L. Cavedon, A. Moffat, M. Sanderson, F. Scholer, J. Zobel. Understanding User Behavior in Job and Talent Search: An Initial Investigation. In Proceedings of SIGIR Workshop on eCommerce (eCom 2017), 2017.