• Digital Humanities and Cultural Heritage
• HELDA - Digital Repository of the U... close

The article is devoted to a discussion of dominant approaches developed within the framework of Corpus Linguistics (CL) and their influence on the general theory of language. Based on the research co-authored with his colleagues, the author describes three approaches to linguistic research in CL. First, corpus-informed analysis assumes that the data collected in the corpus are used as a source of examples in a natural language. Second, corpus-based analysis presupposes that the data are examined not only qualitatively but also quantitatively. Third, corpus-driven analysis assumes that the research task is to create an algorithm for data processing, the results of which require theoretical interpretation or practical application. The article concludes with a discussion of those implications that CL brings into the general understanding of language. The most important of them are: reduction of the role of introspection, increase of attention to peripheral linguistic phenomena, and reliance on quantitative data. It is still too early to sum up the impact of corpus linguistics on the general theory of language, but it is already clear that syntagmatic connections, in particular idiomatization in a broad sense, have moved into the focus of linguistic attention and are recognized as one of the main phenomena of language and its evolution. Moreover, an adequate description of a language is not limited to the rules of interaction of units divided into levels, but the description of all - both individual and the most general - probabilistic parameters of use, representing a single continuum in which the division into language and speech is conventional. Peer reviewed

Word embeddings have advanced the state of the art in NLP across numerous tasks. Understanding the contents of dense neural representations is of utmost interest to the computational semantics community. We propose to focus on relating these opaque word vectors with human-readable definitions, as found in dictionaries This problem naturally divides into two subtasks: converting definitions into embeddings, and converting embeddings into definitions. This task was conducted in a multilingual setting, using comparable sets of embeddings trained homogeneously. Peer reviewed

Pretrained embeddings based on the Transformer architecture have taken the NLP community by storm. We show that they can mathematically be reframed as a sum of vector factors and showcase how to use this reframing to study the impact of each component. We provide evidence that multi-head attentions and feed-forwards are not equally useful in all downstream applications, as well as a quantitative overview of the effects of finetuning on the overall embedding space. This approach allows us to draw connections to a wide range of previous studies, from vector space anisotropy to attention weights. Accepted at TACL (pre-MIT Press publication version)