• Digital Humanities and Cultural Heritage
• Academy of Finland close
• English close

The digitised collections of Finnic oral poetry present exceptional material for studying oral tradition. According to the oral-formulaic theory, in oral tradition singers recreate the poems during performance relying on a store of composition units of various lengths. The formula was defined by Lord as "a group of words, employed repeatedly under the same metric conditions to express a certain idea". In the framework of oral-formulaic theory the concept of formula has got a wider meaning of recurrent units of various lengths. Recently, we have been applying a clustering based on cosine similarity of character bigrams to identify equivalent poetic lines across dialectal and orthographic variation. Building up on that, in this paper we explore quantitative criteria for identifying formulas, starting on the level of a single line. The first obvious criterion is frequency. Furthermore, we introduce a measure of “stereotypy” as entropy of the distribution of line over high-level context, which is approximated by automatic text clustering. Finally, we apply a statistical co-occurrence metric (log-likelihood ratio) to identify lines frequently occurring together. We examine the capability of the method to identify parallel lines and multi-line formulas.

Interview guide for service developers interviews regarding a research infrastructure for the digital humanities and computational social sciencies.

The actor network is a living document that includes relevant information regarding the DARIAH-FI research infrastructure, such as the activities conducted so far (internal meetings, roadshows), the future of the consortium and details of the different WPs.

An entry from the Cambridge Structural Database, the world’s repository for small molecule crystal structures. The entry contains experimental data from a crystal diffraction study. The deposited dataset for this entry is freely available from the CCDC and typically includes 3D coordinates, cell parameters, space group, experimental conditions and quality measures. Related Article: Pasi Salonen, Jörg A. Schachner, Anssi Peuronen, Manu Lahtinen, Ferdinand Belaj, Nadia Carmen Mösch-Zanetti, Ari Lehtonen|2023|Mol.Catal.|540|113034|doi:10.1016/j.mcat.2023.113034

An entry from the Cambridge Structural Database, the world’s repository for small molecule crystal structures. The entry contains experimental data from a crystal diffraction study. The deposited dataset for this entry is freely available from the CCDC and typically includes 3D coordinates, cell parameters, space group, experimental conditions and quality measures. Related Article: Pasi Salonen, Jörg A. Schachner, Anssi Peuronen, Manu Lahtinen, Ferdinand Belaj, Nadia Carmen Mösch-Zanetti, Ari Lehtonen|2023|Mol.Catal.|540|113034|doi:10.1016/j.mcat.2023.113034

Video is an important format of information. Humans use videos for a variety of purposes such as entertainment, education, communication, information sharing, and capturing memories. To this date, humankind accumulated a colossal amount of video material online which is freely available. Manual processing at this scale is simply impossible. To this end, many research efforts have been dedicated to the automatic processing of video content. At the same time, human perception of the world is multi-modal. A human uses multiple senses to understand the environment and objects, and their interactions. When watching a video, we perceive the content via both audio and visual modalities, and removing one of these modalities results in less immersive experience. Similarly, if information in both modalities does not correspond, it may create a sense of dissonance. Therefore, joint modelling of multiple modalities (such as audio, visual, and text) within one model is an active research area. In the last decade, the fields of automatic video understanding and multi-modal modelling have seen exceptional progress due to the ubiquitous success of deep learning models and, more recently, transformer-based architectures in particular. Our work draws on these advances and pushes the state-of-the-art of multi-modal video understanding forward. Applications of automatic multi-modal video processing are broad and exciting! For instance, the content-based textual description of a video (video captioning) may allow a visually- or auditory-impaired person to understand the content and, thus, engage in brighter social interactions. However, prior work in video content description relies on the visual input alone, missing vital information only available in the audio stream. To this end, we proposed two novel multi-modal transformer models that encode audio and visual interactions simultaneously. More specifically, first, we introduced a late-fusion multi-modal transformer that is highly modular and allows the processing of an arbitrary set of modalities. Second, an efficient bi-modal transformer was presented to encode audio-visual cues starting from the lower network layers allowing more rich audio-visual features and stronger performance as a result. Another application is the automatic visually-guided sound generation that might help professional sound (foley) designers who spend hours searching a database for relevant audio for a movie scene. Previous approaches for automatic conditional audio generation support only one class (e. g. “dog barking”), while real-life applications may require generation for hundreds of data classes and one would need to train one model for every data class which can be infeasible. To bridge this gap, we introduced a novel two-stage model that, first, efficiently encodes audio as a set of codebook vectors (i. e. trains to make “building blocks”) and, then, learns to sample these audio vectors given visual inputs to make a relevant audio track for this visual input. Moreover, we studied the automatic evaluation of the conditional audio generation model and proposed metrics that measure both quality and relevance of the generated samples. Finally, as video editing is becoming more common among non-professionals due to the increased popularity of such services as YouTube, automatic assistance during video editing grows in demand, e. g. off-sync detection between audio and visual tracks. Prior work in audio-visual synchronization was devoted to solving the task on lip-syncing datasets with “dense” signals, such as interviews and presentations. In such videos, synchronization cues occur “densely” across time, and it is enough to process just a few tens of a second to synchronize the tracks. In contrast, opendomain videos mostly have only “sparse” cues that occur just once in a seconds-long video clip (e. g. “chopping wood”). To address this, we: a) proposed a novel dataset with “sparse” sounds; b) designed a model which can efficiently encode seconds-long audio-visual tracks in a small set of “learnable selectors” that is, then, used for synchronization. In addition, we explored the temporal artefacts that common audio and video compression algorithms leave in data streams. To prevent a model from learning to rely on these artefacts, we introduced a list of recommendations on how to mitigate them. This thesis provides the details of the proposed methodologies as well as a comprehensive overview of advances in relevant fields of multi-modal video understanding. In addition, we provide a discussion of potential research directions that can bring significant contributions to the field.

An entry from the Cambridge Structural Database, the world’s repository for small molecule crystal structures. The entry contains experimental data from a crystal diffraction study. The deposited dataset for this entry is freely available from the CCDC and typically includes 3D coordinates, cell parameters, space group, experimental conditions and quality measures. Related Article: Pasi Salonen, Jörg A. Schachner, Anssi Peuronen, Manu Lahtinen, Ferdinand Belaj, Nadia Carmen Mösch-Zanetti, Ari Lehtonen|2023|Mol.Catal.|540|113034|doi:10.1016/j.mcat.2023.113034

An entry from the Cambridge Structural Database, the world’s repository for small molecule crystal structures. The entry contains experimental data from a crystal diffraction study. The deposited dataset for this entry is freely available from the CCDC and typically includes 3D coordinates, cell parameters, space group, experimental conditions and quality measures. Related Article: Pasi Salonen, Jörg A. Schachner, Anssi Peuronen, Manu Lahtinen, Ferdinand Belaj, Nadia Carmen Mösch-Zanetti, Ari Lehtonen|2023|Mol.Catal.|540|113034|doi:10.1016/j.mcat.2023.113034

An entry from the Cambridge Structural Database, the world’s repository for small molecule crystal structures. The entry contains experimental data from a crystal diffraction study. The deposited dataset for this entry is freely available from the CCDC and typically includes 3D coordinates, cell parameters, space group, experimental conditions and quality measures. Related Article: Pasi Salonen, Jörg A. Schachner, Anssi Peuronen, Manu Lahtinen, Ferdinand Belaj, Nadia Carmen Mösch-Zanetti, Ari Lehtonen|2023|Mol.Catal.|540|113034|doi:10.1016/j.mcat.2023.113034

An entry from the Cambridge Structural Database, the world’s repository for small molecule crystal structures. The entry contains experimental data from a crystal diffraction study. The deposited dataset for this entry is freely available from the CCDC and typically includes 3D coordinates, cell parameters, space group, experimental conditions and quality measures. Related Article: Pasi Salonen, Jörg A. Schachner, Anssi Peuronen, Manu Lahtinen, Ferdinand Belaj, Nadia Carmen Mösch-Zanetti, Ari Lehtonen|2023|Mol.Catal.|540|113034|doi:10.1016/j.mcat.2023.113034