- Siri natural language understanding.
Building conversational machines, with a focus on meaningful statistical evaluations. Creative with machine learning and mathematics.
- Leading research
- Building a scalable machine learning platform for conversational agents, applied to customer service
As well as my PhD from Cambridge, I have an MSc in Speech and Language Processing from the University of Edinburgh, and an MA in Mathematics from the University of Cambridge.
For a full list, please see my Google Scholar profile.
Slot labeling framework that achieves a new leap in performance for few-shot slot labeling. Introduces a new pretraining task, pairwise cloze, that allows pre-training all sequence-level layers.
A highly-optimized transformer architecture, pre-trained on response selection. ConveRT achieves state-of-the-art performance across widely established response selection and intent classification tasks.
An evaluation of dual sentence encoders such as ConveRT on few-shot intent detection tasks.
A collection of large datasets for Conversational AI, with hundreds of millions of examples, and a standardised evaluation framework.
An evaluation of fine-tuning techniques for adapting large neural response selection models, trained on large datasets like Reddit, to task-oriented dialogue domains.
This paper presents a question answering system, that selects sentences from product descriptions using a neural-network ranking model. This is trained and evaluated on a dataset of 590K questions and answers from Carousell marketplace conversations.
The modelling approach behind the Smart Reply feature launched to Gmail, using hierarchical feedforward neural networks, and an efficient dot-product search. The system is 100x times faster than the original LSTM sequence-to-sequence approach and is higher quality.
A review paper on Machine Learning methods for Dialog State Tracking. An invited talk, presented at the first Machine Learning in Spoken Language Processing workshop in Aizuwakamatsu, Japan.
This thesis presents how discriminative machine learning methods can be used to develop high accuracy spoken language understanding and state tracking modules for spoken dialogue systems.
The Third Dialog State Tracking Challenge, with 7 research teams submitting 26 total entries, evaluated the state of the art in dialog state tracking. The focus was on adapting to expanding conversational domains. Presented as a poster at IEEE SLT 2014, in Lake Tahoe.
This paper presents a method to robustly transfer recurrent neural networks from one dialog domain to another, including a technique for adapting the parameters online with no supervision. This was the top performing tracker in the Third Dialog State Tracking Challenge. Presented as a poster at IEEE SLT 2014, in Lake Tahoe.
This paper presents the results of the Second Dialog State Tracking Challenge, a research challenge focused on improving the state of the art in tracking the state of spoken dialog systems. In total 9 research teams from across the world competed, with 31 total entries.
This paper presents a new method for dialog state tracking that relies directly on the words spoken, i.e. the output of a speech recogniser, rather than a semantic representation. The method is validated in the Second Dialog State Tracking Challenge, and found to be one of the top two tracking methods. Nominated for best paper.
Inspired by recent promising results using Deep Neural Networks in speech applications, this paper shows how to apply DNNs to Dialog State Tracking. This was presented as a poster at SIGdial and was entered in the Dialog State Tracking Challenge.
This paper presents a new form of semantic decoding which uses the entire distribution of speech recognition hypotheses to infer the meaning of an utterance. This was presented as a poster at SLT. The dataset used for training the decoder and for the offline evaluation is freely available: In Car SLU Corpus.
A paper stemming from work done in my MSc evaluating a set of strategies a conversational (rather than task-oriented) dialogue system can use to recover from situations where it is unable to understand what the user has asked. The dialogue system concerned is for a robot tourguide which shows visitors around the Edinburgh Informatics Forum. This was an oral presentation at Semdial.