Yoav Goldberg
Krill Prize 2017
Bar-Ilan University
Dr. Yoav Goldberg
Description of Research
I work on developing automatic methods for understanding and generating human languages by computers (Natural Language Processing, NLP). Language is at the core of human intelligence, and is arguably what sets us apart from all other species. It is central to effective com mun icati on, as it facilitates the transfer of ideas across space and time. In the digital age, we see unprecedented growth in the scale at which individuals and institutions generate, communicate and access information in written or spoken form. Finding ways for effectively leveraging the vast amounts of available data to discover and address people’s needs {social, professional , scientific or otherwise) is a fundamental problem in modern societies. Over the past decade we’ve seen tremendous progress in NLP capabilities, with NLP commonly facilitating breaking language barriers (machine translation), finding required information (search engines), and digesting large amounts of unstructured text into more convenient forms (summarization, information extr action). Despite this progress, the NLP problem is far from being solved, especially when moving from types of text the systems are familiar with (mostly well edited English text ) to other genres or languages.
My research focuses on the building-blocks of NLP — the core algorithms that other language understanding systems rely on. These include computationally representing the meaning of words (lexical semantics); how words are combined to form phrases and sentences (synta x); and how this process can be reversed, inferring the structure of a sentence from its words (syntactic parsi ng).
Within lexical semantics, I worked on analyzing the foundation of “word embedding” algorithms. Word embeddings are the dominant and very popular framework for lexical semantics, and were at the time not well underst ood. In a series of works with PhD student Omer Levy we analyzed the mathematical and linguistic foundations of these algorithms and unified many algorithms under a common framework. This in turn allowed us to further generalize and improve the word embeddings technique. This line of work
was highly ·influentia I, accumulating over 720 citations since 2014. Products of the work are also featured in the upcoming 3rd edition of the definitive NLP textbook by Jurafsky and Martin .
Within syntactic parsing , my most recent contribution (with PhD student Eliyahu Kiperwasser) is a parser based on bidirectional recurrent neural networks, which as of April 2016 had the best accuracy in the world (the work has since been extended by research groups at Stanford, CMU and Harvard) . One of my earlier innovations in parsing, the dynamic -oracl e training techniques, is integrated in several
widely-used parsers, including the open-source s pa Cy. i o NLP packa ge.
The field of natural-language processing interacts closely with the field machine learning. My research is deeply rooted in both fields, borrowing state-of-the-art methods in machine learning and adapting them to fit the needs of linguistic tasks, leading to innovations in both machine learning and NLP. In the past two years, I became interested in the effective use of neural-n etwo rk s (a.k.a deep lea rn in g, DL) for NLP. A common trend in my work is to combine DL methods with linguistically grounded mechanisms, relying on in-depth understa nding of the DL algorithms as well as the underlying linguistic phenomena . Besides the accurate parser mentioned above, this line of work resulted in several best paper awards as detailed in my CV. I authored an extensive tutorial paper on DL techniques for NLP which is now being used in academic courses worldwide.
My focus is improving the coverage of these NLP building blocks. Current algorithms achieve impressive accuracies on well-formed English newswire text, but degrade considerably on other genres, severely limiting their applicability. I work on methods for extending their coverage to a diverse set of textual genres (scienti fic, le gal, literary, colloquial , etc) as well as to spoken language. Accuracy also drops when attempting to work with languages with richer morphological sys t em s and freer word order than English (most of the world ‘s languages). In the Universa I Dependencies mu!ti -nati onal collaboration, we collect data on such languages under a common scheme. I recen tly started using this data to devise
DL-based lexical semantics and syntactic parsing algorithms tailored for such languages. If success ful, this will transform NLP from being applicable to a relatively narrow range of commercial scenarios to being universally applicable in a wide set of scenarios for the benefit of science and society worldwide .