your-photo

John Hewitt

johnhew [at] stanford.edu

Hi! I’m a third-year PhD student in computer science, conducting research in natural language processing at Stanford University. I am grateful to be co-advised by Chris Manning and Percy Liang, and to be supported by a NSF Graduate Research Fellowship.

For Winter 2021, I’m Head TA of CS224N!

I aim to design systems that robustly and efficiently learn to understand human languages to the end of advancing human communication and education, and to teach others.

Feel free to look me up on Google Scholar or Twitter, or take my CV.

Current Research Interests

  • Understanding unsupervised learning of language
  • NLP theory
  • Out-of-domain extrapolation in NLP
  • Multilinguality and low-resource language processing
  • Publications

    Publications

    2020

    RNNs can generate bounded hierarchical languages with optimal memory.
    John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, Christopher D. Manning.
    EMNLP 2020 (long papers).
    (pdf) (blog) (code:analytic) (code:learning) (codalab)

    The EOS Decision and Length Extrapolation.
    Benjamin Newman, John Hewitt, Percy Liang, Christopher D. Manning
    BlackBoxNLP 2020. (Outstanding Paper).
    (pdf) (code)

    Emergent Linguistic Structure in Artificial Neural Networks Trained by Self-Supervision.
    Christopher D. Manning, Kevin Clark, John Hewitt, Urvashi Khandelwal, Omer Levy
    Proceedings of the National Academy of Sciences. 2020.
    (pdf)

    Finding Universal Grammatical Relations in Multilingual BERT.
    Ethan A. Chi, John Hewitt and Christopher D. Manning.
    ACL 2020 (long papers).
    (pdf) (bib) (code) (viz)

    2019

    Designing and Interpreting Probes with Control Tasks.
    John Hewitt and Percy Liang.
    EMNLP 2019 (long papers). (Runner Up Best Paper).
    (pdf) (bib) (blog) (code) (codalab) (slides) (talk).

    A Structual Probe for Finding Syntax in Word Representations.
    John Hewitt and Christopher D. Manning.
    NAACL 2019 (short papers).
    (pdf) (bib) (blog) (code) (nlp highlights podcast) (slides) (talk).

    Simple, Fast, Accurate Intent Classification and Slot Labeling for Goal-Oriented Dialogue Systems.
    Arshit Gupta*, John Hewitt* and Katrin Kirchhoff.
    SIGDIAL 2019.
    (pdf)
    *: Equal contribution; authors listed alphabetically

    2018

    A Distributional and Orthographic Aggregation Model for English Derivational Morphology.
    Daniel Deutsch*, John Hewitt* and Dan Roth.
    ACL 2018 (long papers).
    (pdf)
    *: Equal contribution; authors listed alphabetically

    Learning Translations via Images with a Massively Multilingual Image Dataset.
    John Hewitt*, Daphne Ippolito*, Brendan Callahan, Reno Kriz, Derry Tanti Wijaya and Chris Callison-Burch.
    ACL 2018 (long papers).
    (pdf)
    *: Equal contribution; authors listed alphabetically

    XNMT: The eXtensible Neural Machine Translation Toolkit.
    Graham Neubig, Matthias Sperber, Xinyi Wang, Matthieu Felix, Austin Matthews, Sarguna Padmanabhan, Ye Qi, Devendra Singh Sachan, Philip Arthur, Pierre Godard, John Hewitt, Rachid Riad, and Liming Wang.
    AMTA 2018.
    (pdf)

    2017

    Learning Translations via Matrix Completion.
    Derry Tanti Wijaya, Brendan Callahan, John Hewitt , Xiao Ling, Marianna Apidianaki, and Chris Callison-Burch.
    EMNLP 2017 (long papers).
    (pdf)

    2016

    Automatic Construction of Morphologically-Motivated Translation Models for Highly Inflected Low-Resource Languages.
    John Hewitt, Matt Post, David Yarowsky.
    AMTA 2016.
    (pdf)

    Invited Talks

    A Natural Language Processing perspective on supervised analysis of neural representations.
    EvLab, MIT. December 2, 2020.

    The Unreasonable Syntactic Expressivity of RNNs.
    USC ISI NLP Seminar. (video) November 5, 2020.

    Ongoing work on the probing methodology in NLP.
    NLP with Friends. September 9, 2020.

    Probing Neural NLP: Ideas and Problems.
    Berkeley NLP Seminar. November 18, 2019.

    Emergent Linguistic Structure in Neural NLP.
    Amazon AI. July 25, 2019.

    A Structural Probe for Finding Syntax in Word Representations.
    NLP Highlights Podcast. May, 2019.

    Abstracts

    RNNs can generate bounded hierarchical languages with optimal memory.
    John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, Christopher D. Manning
    2020 Conference on the Mathematical Theory of Deep Learning (abstracts).

    Semantic Bootstrapping in Frames: A Computational Model of Syntactic Category Acquisition.
    John Hewitt, Jordan Kodner, Mitch Marcus, and Charles Yang.
    Conference of the Cognitive Science Society (CogSci), (member posters) 2017. (pdf) (abstract)

    Patents

    Capturing Rich Response Relationships with Small-Data Neural Networks.
    John Hewitt.
    US Patent App 15/841,963. December 2017. (granted). (application)

  • Blog

    Newest Articles

    See all articles
  • Projects

    • Transformers lecture

      • I wrote a lecture on Transformers in my role as Head TA for Stanford’s CS 224N: Natural Language Processing with Deep Learning.
  • About

    Tidbits

    This talk by Rajiv Gandhi, to whom I am grateful. For you if you think, like I used to, that research–or any success in STEM–is out of your reach.

    Scott Aaronson’s old note on frameworks for reasoning about large numbers, for enjoyment

    Kevin Knight’s note on unix commands, to help you with your bash skills

    The Fundamental Whiteboard Difficulty (Scott Aaronson):

    I figured that chalk has its problems—it breaks, the dust gets all over—but I could live with them, much more than I could live with the Fundamental Whiteboard Difficulty, of all the available markers always being dry whenever you want to explain anything.

    I highly suggest Arch Linux for its configurability and the educational experience it provides…

    Contact

    Take my school email johnhew@stanford, and predict the TLD using your internal knowledge base.

Join My Newsletter

Sign up to receive weekly updates.

x