Matthew Henderson Building conversational machines, with a focus on meaningful statistical evaluations. Creative with machine learning and mathematics.

Experience

Apple

Apple

ML Researcher
Edinburgh, March 2021 - present

  • Siri natural language understanding.
PolyAI

PolyAI

VP of Research
Singapore, Jan 2018 - Feb 2021

  • Leading research
  • Building a scalable machine learning platform for conversational agents, applied to customer service
Google

Google

Senior Software Engineer
Mountain View, California (on O1 visa), Mar 2015 - June 2017

  • Lead researcher behind Smart Reply in Gmail
  • Tech lead on Ray Kurzweil's natural language understanding & dialog research team in Mountain View California
  • Inventor on three patents
  • Worked for first 6 months in London on text-to-speech research team with Heiga Zen

As well as my PhD from Cambridge, I have an MSc in Speech and Language Processing from the University of Edinburgh, and an MA in Mathematics from the University of Cambridge.

Selected Publications

For a full list, please see my Google Scholar profile.

ConVEx: Data-Efficient and Few-Shot Slot Labeling

Matthew Henderson and Ivan Vulić
2021

Slot labeling framework that achieves a new leap in performance for few-shot slot labeling. Introduces a new pretraining task, pairwise cloze, that allows pre-training all sequence-level layers.

@inproceedings{henderson-vulic-2021-convex,
    title = "{ConVEx}: Data-Efficient and Few-Shot Slot Labeling",
    author = "Henderson, Matthew  and
      Vuli{'c}, Ivan",
    booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
    month = jun,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2021.naacl-main.264",
    pages = "3375--3389",
}

ConveRT: Efficient and Accurate Conversational Representations from Transformers

Matthew Henderson, Iñigo Casanueva, Nikola Mrkšić, Pei-Hao Su, Tsung-Hsien Wen, and Ivan Vulić
2020

A highly-optimized transformer architecture, pre-trained on response selection. ConveRT achieves state-of-the-art performance across widely established response selection and intent classification tasks.

@inproceedings{Henderson2020a,
    author      = {Matthew Henderson and I{\~{n}}igo Casanueva and  Nikola Mrk{{s}}i'c and Pei-Hao Su and Tsung-Hsien Wen and Ivan Vuli'c},
    title       = {A Repository of Conversational Datasets},
    year        = {2020},
    url         = {https://arxiv.org/abs/1911.03688},
    booktitle   = {Findings of {EMNLP}},
}

Efficient Intent Detection with Dual Sentence Encoders

Iñigo Casanueva, Tadas Temcinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić
2019

An evaluation of dual sentence encoders such as ConveRT on few-shot intent detection tasks.

@inproceedings{Casanueva:2020ws,
  author    = {I{\~{n}}igo Casanueva and
               Tadas Temcinas and
               Daniela Gerz and
               Matthew Henderson and
               Ivan Vuli'c},
  title     = {Efficient Intent Detection with Dual Sentence Encoders},
  booktitle   = "Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI",
  year      = {2020},
  url       = {https://arxiv.org/abs/2003.04807},
  pages = {38--45},
}

A Repository of Conversational Datasets

Matthew Henderson, Paweł Budzianowski, Iñigo Casanueva, Sam Coope, Daniela Gerz, Girish Kumar, Nikola Mrkšić, Georgios Spithourakis, Pei-Hao Su, Ivan Vulić, and Tsung-Hsien Wen
2019

A collection of large datasets for Conversational AI, with hundreds of millions of examples, and a standardised evaluation framework.

@inproceedings{Henderson2019,
    author      = {Matthew Henderson and Pawe{\l} Budzianowski and I{\~{n}}igo Casanueva and Sam Coope and Daniela Gerz and Girish Kumar and Nikola Mrk{{s}}i'c and Georgios Spithourakis and Pei-Hao Su and Ivan Vuli'c and Tsung-Hsien Wen},
    title       = {A Repository of Conversational Datasets},
    year        = {2019},
    month       = {jul},
    note        = {Data available at github.com/PolyAI-LDN/conversational-datasets},
    url         = {https://arxiv.org/abs/1904.06472},
    booktitle   = {Proceedings of the Workshop on {NLP} for Conversational {AI}},
}

Training Neural Response Selection for Task-Oriented Dialogue Systems

Matthew Henderson, Ivan Vulić, Daniela Gerz, Iñigo Casanueva, Paweł Budzianowski, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrkšić, and Pei-Hao Su
2019

An evaluation of fine-tuning techniques for adapting large neural response selection models, trained on large datasets like Reddit, to task-oriented dialogue domains.

@inproceedings{Henderson2019b,
    author      = {Matthew Henderson and Ivan Vuli'c and Daniela Gerz and I{\~{n}}igo Casanueva and Pawe{\l} Budzianowski and Sam Coope and Georgios Spithourakis and Tsung-Hsien Wen and Nikola Mrk{{s}}i'c and Pei-Hao Su},
    title       = {Training Neural Response Selection for Task-Oriented Dialogue Systems},
    year        = {2019},
    month       = {jul},
    url         = {https://arxiv.org/abs/1906.01543},
    booktitle   = {Proceedings of ACL},
}

Question-Answer Selection in User to User Marketplace Conversations

Girish Kumar, Matthew Henderson, Shannon Chan, Hoang Nguyen, and Lucas Ngoo
2018

This paper presents a question answering system, that selects sentences from product descriptions using a neural-network ranking model. This is trained and evaluated on a dataset of 590K questions and answers from Carousell marketplace conversations.

@inproceedings{Kumar2018,
    author = {Kumar, Girish and Henderson, Matthew and Chan, Shannon and Nguyen, Hoang and Ngoo, Lucas},
    title = "{Question-Answer Selection in User to User Marketplace Conversations}",
    booktitle = {International Workshop on Spoken Dialog Systems (IWSDS)},
    month = {May},
    year = {2018}
}

Efficient Natural Language Response Suggestion for Smart Reply

Matthew Henderson, Rami Al-Rfou, Brian Strope, Yun-hsuan Sung, Laszlo Lukacs, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, and Ray Kurzweil
2017

The modelling approach behind the Smart Reply feature launched to Gmail, using hierarchical feedforward neural networks, and an efficient dot-product search. The system is 100x times faster than the original LSTM sequence-to-sequence approach and is higher quality.

@article{Henderson2017,
   author = {Henderson, Matthew and {Al-Rfou}, Rami and Strope, Brian and Sung, Yun-hsuan and
	 Luk{'{a}}cs, L{'{a}}szl{'{o}} and Guo, Ruiqi and Kumar, Sanjiv and Miklos, Balint and
	Kurzweil, Ray},
    title = "{Efficient Natural Language Response Suggestion for Smart Reply}",
  journal = {ArXiv e-prints},
archivePrefix = "arXiv",
   eprint = {1705.00652},
 primaryClass = "cs.CL",
     year = 2017,
    month = may,
   adsurl = {http://adsabs.harvard.edu/abs/2017arXiv170500652H},
}

Machine Learning for Dialog State Tracking: A Review

Matthew Henderson
2015

A review paper on Machine Learning methods for Dialog State Tracking. An invited talk, presented at the first Machine Learning in Spoken Language Processing workshop in Aizuwakamatsu, Japan.

@inproceedings{Henderson2015,
    title = {Machine Learning for Dialog State Tracking: A Review},
    author  = {Matthew Henderson},
    year  = 2015,
    booktitle = {The First International Workshop on Machine Learning in Spoken Language Processing}
}

Discriminative Methods for Statistical Spoken Dialogue Systems

PhD Thesis
2015

This thesis presents how discriminative machine learning methods can be used to develop high accuracy spoken language understanding and state tracking modules for spoken dialogue systems.

@phdthesis{MattHendersonThesis,
    Title = {{Discriminative Methods for Statistical Spoken Dialogue Systems}},
    Author = {Henderson, Matthew},
    Year = {2015},
    School = {{University of Cambridge}},
}

The Third Dialog State Tracking Challenge

Matthew Henderson, Blaise Thomson, and Jason Williams
2014

The Third Dialog State Tracking Challenge, with 7 research teams submitting 26 total entries, evaluated the state of the art in dialog state tracking. The focus was on adapting to expanding conversational domains. Presented as a poster at IEEE SLT 2014, in Lake Tahoe.

@inproceedings{Henderson2014c,
    author = {Henderson, M. and  Thomson, B. and Williams, J.},
    booktitle = {Proceedings of IEEE Spoken Language Technology},
    title = {{The Third Dialog State Tracking Challenge}},
    year = {2014}
}

Robust Dialog State Tracking Using Delexicalised Recurrent Neural Networks and Unsupervised Adaptation

Matthew Henderson, Blaise Thomson, and Steve Young
2014

This paper presents a method to robustly transfer recurrent neural networks from one dialog domain to another, including a technique for adapting the parameters online with no supervision. This was the top performing tracker in the Third Dialog State Tracking Challenge. Presented as a poster at IEEE SLT 2014, in Lake Tahoe.

@inproceedings{Henderson2014d,
    author = {Henderson, M. and  Thomson, B. and Young, S. J.},
    booktitle = {Proceedings of IEEE Spoken Language Technology},
    title = {{Robust Dialog State Tracking Using Delexicalised Recurrent Neural Networks and Unsupervised Adaptation}},
    year = {2014}
}

The Second Dialog State Tracking Challenge

Matthew Henderson, Blaise Thomson, and Jason Williams
2014

This paper presents the results of the Second Dialog State Tracking Challenge, a research challenge focused on improving the state of the art in tracking the state of spoken dialog systems. In total 9 research teams from across the world competed, with 31 total entries.

@inproceedings{Henderson2014a,
    author = {Henderson, M. and  Thomson, B. and Williams, J.},
    booktitle = {Proceedings of SIGdial},
    title = {{The Second Dialog State Tracking Challenge}},
    year = {2014}
}

Word-based Dialog State Tracking with Recurrent Neural Networks

Matthew Henderson, Blaise Thomson, and Steve Young
2014

This paper presents a new method for dialog state tracking that relies directly on the words spoken, i.e. the output of a speech recogniser, rather than a semantic representation. The method is validated in the Second Dialog State Tracking Challenge, and found to be one of the top two tracking methods. Nominated for best paper.

@inproceedings{Henderson2014b,
    author = {Henderson, M. and  Thomson, B. and Young, S. J.},
    booktitle = {Proceedings of SIGdial},
    title = {{Word-based Dialog State Tracking with Recurrent Neural Networks}},
    year = {2014}
}

Deep Neural Network Approach for the Dialog State Tracking Challenge

Matthew Henderson, Blaise Thomson, and Steve Young
2013

Inspired by recent promising results using Deep Neural Networks in speech applications, this paper shows how to apply DNNs to Dialog State Tracking. This was presented as a poster at SIGdial and was entered in the Dialog State Tracking Challenge.

@inproceedings{Henderson2013a,
    author = { Henderson, M. and  Thomson, B. and Young, S. J.},
    booktitle = {Proceedings of SIGdial},
    title = {{Deep Neural Network Approach for the Dialog State Tracking Challenge}},
    year = {2013}
}

Discriminative Spoken Language Understanding Using Word Confusion Networks

Matthew Henderson, Milica Gašić, Blaise Thomson, Pirros Tsiakoulis, Kai Yu, and Steve Young
2012

This paper presents a new form of semantic decoding which uses the entire distribution of speech recognition hypotheses to infer the meaning of an utterance. This was presented as a poster at SLT. The dataset used for training the decoder and for the offline evaluation is freely available: In Car SLU Corpus.

@inproceedings{Henderson2012a,
    author = {Henderson, Matthew and Ga{s}i'{c}, Milica and Thomson, Blaise and Tsiakoulis, Pirros and Yu, Kai and Young, Steve},
    booktitle = {Spoken Language Technology Workshop, 2012. IEEE},
    title = {{Discriminative Spoken Language Understanding Using Word Confusion Networks}},
    year = {2012}
}

Recovering from Non-Understanding Errors in a Conversational Dialogue System

Matthew Henderson, Colin Matheson, and Jon Oberlander
2012

A paper stemming from work done in my MSc evaluating a set of strategies a conversational (rather than task-oriented) dialogue system can use to recover from situations where it is unable to understand what the user has asked. The dialogue system concerned is for a robot tourguide which shows visitors around the Edinburgh Informatics Forum. This was an oral presentation at Semdial.

@inproceedings{Henderson2012a,
    author = {Henderson, Matthew and Matheson, Colin and Oberlander, Jon},
    booktitle = {Workshop on the Semantics and Pragmatics of Dialogue},
    title = {{Recovering from Non-Understanding Errors in a Conversational Dialogue System}},
    year = {2012}
}