EECS500 Spring 2018 Department Colloquium

Jiepu Jiang
Building Search Engines without Human Judgments: Distant Supervision and Behavioral Data
University of Massachusetts Amherst
White 411
11:30 AM - 12:30 PM
March 8, 2018

Search engines often require large-scale human annotation to improve and evaluate their services. In this talk, we introduce research towards reducing such human labor requirements, especially methods without using private user data. We first present a method called similarity-based distant supervision, which automatically generates text retrieval training labels in a new environment using only open data sources such as Wikipedia and existing test collections. The method can also be refined substantially with the help of only a few human judgments, which by themselves are insufficient to train search engines directly. Second, we also introduce methods for evaluating user experience with a new search engine service ---intelligent voice assistant ---without asking users' explicit feedback. The techniques predict user experience based on behavioral data and user modeling. The methods diagnose the goodness of a running system instantly without interrupting users and show excellent generalizability to new, unseen intelligent assistant tasks.


Jiepu Jiang is a Ph.D. candidate in Computer Science at the University of Massachusetts Amherst. His research aims to help people better access and use information, including both designing effective systems to support information access and studying human factors during this process. He has been regularly publishing and serving in top information retrieval and data mining conferences. He received the best student paper award from CHIIR 2017 for his work on understanding dynamics of search result judgments in information retrieval. He also received a Ph.D. in Library & Information Science from the University of Pittsburgh.