Understanding & Inferring User Tasks and Needs

Venues

WWW 2018, Lyon, France

CIKM 2017, Singapore

Introduction

Search behavior, and information behavior more generally, is often motivated by tasks that prompt search processes that are often lengthy, iterative, and intermittent, and are characterized by distinct stages, shifting goals and multitasking. Current search systems do not provide adequate support for users tackling complex tasks due to which the cognitive burden of keeping track of such tasks is placed on the searcher. Developing a comprehensive understanding of user’s tasks would help in providing better support and recommendations to users based on their contextual information and as a result, help users accomplish the task.

In this tutorial, we begin by discussing recent advancements towards building task based IR systems and present analytical results which highlight the importance of considering tasks as the focal unit of modelling search behavior. Additionally, we consider the challenge of extracting tasks from a given collection of search log data and present some recently proposed task extraction techniques which rely on recent advancements in bayesian non parametrics, word embeddings, structured predictions and deep learning. Finally, we present applications of task inference techniques alongside discussing the implications of task based systems & summarize few key open research questions.

Target Audience

Intermediate researchers, industry technologists and practitioners. We are expecting a general background in information retrieval and machine learning. Given the interest of the WWW & CIKM community in information access, retrieval and user modelling, this tutorial on task based systems will provides attendees with a novel perspective of analysing log data, enabling them to view user interactions in a new light. The algorithmic approaches presented would help not only academic researchers but also industrial practitioners in better developing systems which support users in accomplishing their task.

Outline of the tutorial

1. Introduction
  - Evolution of search
  - Functionality levels of search offerings
  - Conversational & Task completion engines
  
2. Characterizing Tasks
  - Understanding Intents & Tasks
    - Query Intents in IR
    - Session based modelling
    - From sessions to tasks
  - Characterizing Tasks across devices
    - Desktop based search
    - Digital Assistants
    - Voice-only assistants
  
3. Task extraction algorithms
  - Latent task extraction
    - Clustering based approaches
    - Entity based task extraction
    - Structured learning approach
    - LDA-Hawkes model
  - Subtask Extraction
    - Chinese Restaurant Processes
    - dd-CRPs with deep embeddings
  - Hierarchies of Tasks & Subtasks
    - Agglomerative Clustering
    - Bayesian Rose Trees
    - Bayesian Non-parametric approach
  - Task understanding in Digital Assistants
  - Evaluation Mechanisms
    - Gold standard dataset
    - Designing user studies for hierarchical evaluation
    - Alternate evaluation techniques
    - TREC Tasks Tracks

4. Task based Evaluation
  - User behavior signals
  - Predictive models of SAT
  - Explicit satisfaction signals
  
5. Applications
  - Task based Personalization
  - Task based recommendations
  - Task tours
  - Predicting Task continuation
  - Task completion dialogue systems
  
6. Conclusion
  - Open Research questions

Slides

Parts 1 & 2: link to slides

Part 3: link to slides

Part 4: link to slides

Parts 5 & 6: link to slides

Organizers

1. Rishabh Mehrotra, Research Scientist, Spotify Research, London

2. Emine Yilmaz, Associate Professor, University College London; Faculty Fellow, Alan Turing Institute, London

3. Ahmed Hassan Awadallah, Research Manager/Researcher, Microsoft Research