Shaping Answers with Rules through Conversation

Conversational Question Answering

Most work in machine reading focuses on question answering problems where the answer is directly expressed in the text to read. However, many real-world question answering problems require the reading of text not because it contains the literal answer, but because it contains a recipe to derive an answer together with the reader's background knowledge. We formalise this task and develop a crowd-sourcing strategy to collect 32k task instances.

Task Overview

In our task, the goal is to answer questions by possibly asking follow-up questions first.

We assume that the question is often underspecified, in the sense that the question does not provide enough information to be answered directly. However, an agent can use the supporting rule text to infer what needs to be asked in order to determine the final answer. In the example in the figure, a reasonable follow-up question is "Have you been working abroad 52 weeks or less?"

More explanation on the task and the dataset can be found in the paper.


marzieh.saeidi [at]

maxbartolo [at]

patrick.s.h.lewis [at]

s.riedel [at]