Work

Processing and querying large web corpora with the COW14 architecture

Year: 2015

Type: article

Abstract: In this paper, I present the COW14 tool chain, which comprises a web corpus creation tool called texrex, wrappers for existing linguistic annotation tools as well as an online query software called Colibri2. By detailed descriptions of the implementation and systematic evaluations of the performance of the software on different types of systems, I show that the COW14 architecture is capable of handling the creation of corpora of up to at least 100 billion tokens. I also introduce our running demo system which currently serves corpora of up to roughly 20 billion tokens in Dutch, English, French, German, Spanish, and Swedish

Source: Publication Server of the Institute for German Language (Institute for German Language)

Author: Roland Schäfer

Institution: Freie Universität Berlin

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Deutsche Forschungsgemeinschaft, Freie Universität Berlin

Award: G7819390555

OpenAlex ID: W2222142803