| Sketch Engine | |
| Logo Caption: | Logo of Sketch Engine |
| Logo Size: | 250px |
| Author: | Adam Kilgarriff, Pavel Rychlý |
| Developer: | Lexical Computing CZ s.r.o. |
| Released: | [1] |
| Programming Language: | Go, JavaScript, jQuery, C++, Python |
| Operating System: | Linux, Mac OS X |
| Platform: | IA-32, x64 or IA-64 |
| Language: | Arabic, Crimean Tatar, Czech, English, French, German, Irish, Italian, Nko, Spanish, Ukrainian |
| Language Count: | 11 |
| Genre: | Corpus manager for 90+ languages, database management system |
| License: | Proprietary software
|
| Standard: | Unicode |
Sketch Engine is a corpus manager and text analysis software developed by Lexical Computing since 2003. Its purpose is to enable people studying language behaviour (lexicographers, researchers in corpus linguistics, translators or language learners) to search large text collections according to complex and linguistically motivated queries. Sketch Engine gained its name after one of the key features, word sketches: one-page, automatic, corpus-derived summaries of a word's grammatical and collocational behaviour.[2] Currently, it supports and provides corpora in over 90 languages.[3]
Sketch Engine is a product of Lexical Computing, a company founded in 2003 by the lexicographer and research scientist Adam Kilgarriff.[4] He started a collaboration with Pavel Rychlý, a computer scientist working at the Natural Language Processing Centre, Masaryk University,[5] and the developer of Manatee and Bonito (two major parts of the software suite). Kilgarriff also introduced the concept of word sketches.
Since then, Sketch Engine has been commercial software, however, all the core features of Manatee and Bonito that were developed by 2003 (and extended since then) are freely available under the GPL license within the NoSketch Engine suite.[6]
A list of tools available in Sketch Engine:
Sketch Engine can perform automatic term extraction by identifying words typical of a particular corpus, document, or text. Single words and multi-word units can be extracted from monolingual or bilingual texts. The terminology extraction feature provides a list of relevant terms based on comparison with a large corpus of general language. This functionality is also available as a separate service called OneClick Terms with a dedicated interface.[8]
A free web service based on Sketch Engine and aimed at language learners and teachers is SKELL (formerly SkELL). It exploits Sketch Engine's proprietary GDEX (Good Dictionary Examples) scoring function to provide authentic example sentences for specific target words. Results are drawn from a special corpus of high-quality texts covering everyday, standard, formal, and professional language and displayed as a concordance. SKELL also includes simplified versions of Sketch Engine's word sketch and thesaurus functions.[9]
It has been suggested that SKELL can be used, for instance, to help students understand the meaning and/or usage of a word or phrase; to help teachers wanting to use example sentences in a class; to discover and explore collocates; to create gap-fill exercises; to teach various kinds of homonyms and polysemous words.[10] [11] SKELL was first presented in 2014, when only English was supported. Later, support was added for Russian,[12] Czech,[13] German,[14] Italian[15] and Estonian.[16]
Sketch Engine provides access to more than 700 text corpora. There are monolingual as well as multilingual corpora of different sizes (from thousand of words up to 60 billions of words) and various sources (e.g. web, books, subtitles, legal documents). The list of corpora includes British National Corpus, Brown Corpus, Cambridge Academic English Corpus and Cambridge Learner Corpus, CHILDES corpora of child language, OpenSubtitles (a set of 60 parallel corpora), 24 multilingual corpora of EUR-Lex documents, the TenTen Corpus Family (multi-billion web corpora), and Trends corpora (monitor corpora with daily updates).
Sketch Engine consists of three main components: an underlying database management system called Manatee, a web interface search front-end called Bonito, and a web interface for corpus building and management called Corpus Architect.[17]
Manatee is a database management system specifically devised for effective indexing of large text corpora. It is based on the idea of inverted indexing (keeping an index of all positions of a given word in the text). It has been used to index text corpora comprising tens of billions of words.[18]
Searching corpora indexed by Manatee is performed by formulating queries in the Corpus Query Language (CQL).[19]
Manatee is written in C++ and offers an API for a number of other programming languages including Python, Java, Perl and Ruby. Recently, it was rewritten into Go for faster processing of corpus queries.[20]
Bonito is a web interface for Manatee providing access to corpus search. In the client–server model, Manatee is the server and Bonito plays the client part. It is written in Python.
Corpus Architect is a web interface providing corpus building and management features. It is also written in Python.
Sketch Engine has been used by major British and other publishing houses for producing dictionaries such as Macmillan English Dictionary, Dictionnaires Le Robert, Oxford University Press or Shogakukan. Four of United Kingdom's five biggest dictionary publishers use Sketch Engine.[21]