• ARF [statistics ]

    a modified frequency of a word form which prevents words with high frequency only in one part of a corpus (e.g. in one document) from high ranking in frequency lists (e.g. two words with the same frequency in the corpus, but first one is only in three documents in contrast to second one appearing in the whole corpus). For more details, see the ARF definition.
  • CAT tool

    A CAT tool is a computer assisted translation tool, a software that helps translators maintain consistency in terminology across their translation jobs and also aids the translation process by suggesting (or translating automatically) passages which the translator already translated in the past.
  • cluster

    a process of creating groups of words in the thesaurus or word sketch. Words are connected to their shared collocational behaviour. See more on the Clustering Neighbours documentation
  • collocate

    a part of a collocation that is not a node, e.g. the collocate strong and the node wind, make up the collocation strong wind
  • collocation

    a collocation is a sequence of words or terms that co-occur more often than would be expected by chance (from Wikipedia|Collocation) A collocation, e.g. fatal error, typically consists of a node (error) and a collocate (fatal). Collocations can have different strength, e.g. nice house is a week collocation because both nice and house can combine with lots of other words, on the other hand the Opera House is a strong collocation because it is very typical for opera to occur next to house
  • concordance [feature ]

    a list of all examples of the search word or phrase found in a corpus, usually in the format of a KWIC concordance with the search word highlighted in the centre of the screen and some context to the right and to the left read more»
  • concordancer

    a program which displays a concordance. It is a result of a search for corpora in Sketch Engine.
  • corpus

    a large collection of texts used for studying language. A corpus is usually annotated (=word are labelled with information about the part of speech and grammatical category). The terms corpus and text corpus and language corpus are interchangeable. Using a corpus for any type of linguistic or language oriented work ensures the outcomes reflect the real use of the language. more on copora»
  • corpus architect

    an intuitive tool inside Sketch Engine for creating corpora from documents or the Web which does not require any expert knowledge. See the create your own corpus    page.
  • corpus manager

    a program used to manage text corpora, i.e. to build, edit, annotate and search corpora. Sketch Engine is the user interface to the corpus manager Manatee.
  • CQL

    The Corpus Query Language is a code used to set criteria for complex searches which cannot be carried out using the standard user interface controls. The criteria may not only include words or lemmas but also tags, text types and other attributes. Logical operators (AND/OR/NOT) can be used. See the CQL manual.
  • CSV

    a type of plain text document used for saving tabular data. It is seamlessly accepted by a large variety of applications and is therefore ideal for exporting Sketch Engine results to be used in other software. CSV can be opened directly in Microsoft Excel, Open Office, Google Documents and many others.
  • disambiguation

    a process of identifying meanings of words (lemma, part of speech) when a word has multiple meanings. The result of this process is one word with one meaning.
  • distributional thesaurus

    an automatically produced thesaurus which finds words that tend to occur in similar contexts as the target word. It is not a man-made thesaurus of synonyms. See the User manual.
  • freq/mill

    a frequency of an item normalised per million. If a token has normalised frequency per million 10, it means that in a corpus with million tokens this token would have frequency 10.
  • GDEX

    Good Dictionary Examples are sentences which can be useful as dictionary example sentences, i.e. are illustrative and representative. more on GDEX
  • global subcorpus

    a subcorpus that is shared with all users. See instructions how to set the subcorpus shared all users»
  • header field

    various types of information associated with documents of a corpus, e.g. a corpus with documents from different domains can be structured according to these domains with a usage of header fields <doc domain> and their values "nameofdomain" = <doc domain="nameofdomain">
  • KWIC

    the most common format of a concordance with the search word or phrase displayed in the centre (and usually highlighted) and with some context to the right and some context to the left. KWIC is the acronym for Key Word in Context.
  • lc [attribute ]

    word form lowercase, i.e. case insensitive word form, done is the same as Done. see word form
  • learner corpus

    A collection of texts produced by learners of a language used to study the most frequent errors made by learners. Learner corpora in Sketch Engine can use both error and correction annotation. A special search interface is available to search by the former or the latter or both.
  • lemma [attribute ]

    Lemma is the basic form of a word, typically the form found in dictionaries. Lemmas are used for searching all forms of a word, e.g. searching for lemma go will find go, goes, went, going, gone. Lemma is case sensitive. go and Go are two different lemmas. see also lemma-lc or compare with word form
  • lemma_lc [attribute ]

    lemma-lc is a case insensitive lemma. All upper-case characters are converted to lowercase. apple and Apple is the same thing. see lemma
  • Lemmatization

    Lemmatization is a process of assigning a lemma to each word form in a corpus using an automatic tool called a lemmatizer. Lemmatization ensures that searching for a lemma will also return all related words forms, e.g. searching for go will also find goes, went, gone, going.
  • lempos [attribute ]

    lempos is a combinatin of lemma and part of speech (pos) consisting of the lemma, hyphen and a one-letter abbreviation of the part of speech, eg. go-vhouse-n. The part of speech abbreviations differ between corpora. Lempos is case sensitive, house-n is different from House-n.  see also lempos_lc
  • lempos_lc [attribute ]

    lempos_lc is a case insensitive counterpart of lempos. All uppercase letters are converted to lowercase, thus House-n becomes identical with house-n.
  • likelihood [statistics ]

    a function of parameters of a statistical model, it plays a key role in statistical inference and is the basis for the log-likelihood function. see Statistics in Sketch Engine
  • log-likelihood [statistics ]

    one of the functions used in computed statistics of Sketch Engine. It is the association measures based on the likelihood function, using in tests for significance (see the log-likelihood calculator and more details)
  • logDice [statistics ]

    a statistic measure based only on a frequency of words w_1 and w_2 and the bigram w_1w_2, it is not affected by a size of the corpus See logDice in Statistics used in Sketch Engine.
  • Longest-commonest match

    The longest-commonest match is a concept coined by Adam Kilgarriff to name the most common realisation of the collocational pair, i.e. the chunk of language in which the collocation appears most frequently. The longest-commonest match is part of the word sketch result screen to facilitate the understanding of how the collocation typically behaves.
  • metadata

    structured information containing added information about the text of a corpus. The metadata is e.g. lemma (basic form a word) or a name of each documentation in the corpus.  
  • multilevel list

    a list sorted at more than one level e.g. a frequency list sorted by word form followed by lemma and then tag, see this multilevel list in the BAWE corpus.
  • n-gram

    is a sequence of a number of structures (bigram = 2 structures, trigram = 3 structures...n-gram = n structures) typically letters or words but also phonemes or syllables. Generating a frequency list of such sequences can help us notice which structures tend to combine in a language. n-grams are generated using the word list feature.
  • node

    (collocation) central word in a collocation, e.g. strong wind consists of the collocate strong and the node wind (concordance) the search word or phrase, sometimes called a query, appears in the centre of a KWIC concordance or highlighted in other types of concordances
  • non-word

    generally speaking, non-words are tokens which do not start with a letter of the alphabet. Examples of non-words: !mportant, 2U (There might be rare cases when the corpus author uses a different definition in their corpus. Such a definition is part of the corpus configuration file.)
  • overall score [statistics ]

    score of the relation based on logDice in word sketches. The score is displayed in the header of each column of the relation.
  • parallel corpus

    A parallel corpus is a corpus consisting of the same text in two languages. The texts are aligned (matching segments, usually sentences are linked). The corpus allows searches in one or both languages to look up translations. parallel_key
  • PoS

    part of speech, some typical examples of parts of speech are: noun, adjective, verb, adverb etc.
  • POS tagger

    POS (part of speech) tagging is a process of annotating each token with a tag carrying information about the part of speech and often also morphological and grammatical information such as number, gender, case, tense etc. The automatic tagging tool is called a tagger or POS tagger.
  • positional attribute

    added information to each token in a corpus. It is a description of a token, an example of an attribute is a lemma (basic form of a word). Each corpus can have its own positional attribute. For example,
    word lemma tag lempos
    dogs dog n dog-n
  • preloaded corpus

    a ready-to-use corpus included in Sketch Engine subscription or Trial access, not created by a user, e.g. British National corpus
  • query

    a sequence of characters or words or their combinations input by a user into Sketch Engine in order to retrieve a concordance. Each search for corpora is based on a query = your request.
  • reference

    an attribute of the document describing this document, e.g. a URL of a document. These are information about each document in a corpus.
  • reference corpus

    a corpus chosen as a standard of comparison with your corpus.  The reference corpus is used for the search terms (keywords).
  • regular expressions

    a collection of special symbols that can be used to search for patterns rather than specific characters, e.g. to find all words starting, containing or ending in a specific sequence of characters, for example .*tion will find all words ending in tion and having an unlimited number of characters at the beginning read more»  
  • relative text type frequency

    compares the frequency in a specific text type (part of corpus) to the whole corpus or compares frequencies in different text types (parts of corpus) even if they are not the same size. Thus the user can see whether the search word(s) is typical only for a specific text type (e.g. in newspapers only) but not in the rest of the corpus. The number is relative frequency of the query result divided by relative size of the particular text type. It can be interpreted as “how much more/less often is the result of the query in this text type in comparison to the whole corpus”. Higher frequency means higher value, bigger text type size means lower value. E.g. The word 'test' has 2000 hits in the corpus. 400 of them are in the text type “Spoken” and this text type represents 10 % of the corpus. Then the Relative Text Type frequency will be (400 / 2000) / 0.1 = 200 % and it means 'test' is twice as common in “Spoken” than in the whole corpus.  
  • salience [statistics ]

    a statistical measure of the significance of a specific token in the given context. This is measured with logDice, for more information, see section 3 of Statistics used in Sketch Engine)
  • search attribute

    the attribute that is used for the search and creating a word list. You can have the word list of words, lemmas, tags, etc.
  • search span

    the number of tokens either side of the node that will be matched for filtering concordance. The set search span from -5 to 5 means filter all concordance lines which containing a requirement of the filter in the range of 5 tokens around the node.
  • simple math [statistics ]

    the number used for the calculations and finding terms (keywords). The low number of simple math searches terms with lower frequency whereas the higher number find terms with higher frequency. For further details, see Simple math.
  • structure

    a corpus structure refers to the segments into which a corpus can be divided. Typically a corpus can be divided into sentences, paragraphs and documents but corpora can use various other structures based on the type of corpus.
  • subcorpus

    a corpus can be subdivided into an unlimited number of parts called subcorpora. Subcorpora can be used to divide the corpus by the type (fiction, newspaper), media (spoken, written) or time (e.g. by years) or by any other criteria. A subcorpus can also be created from a concordance by including all concordance lines and the documents they come from into a subcorpus. How to create a subcorpus»
  • tag [attribute ]

    (also called morphological tag or POS tag) a label assigned to each token in an annotated corpus to indicate the part of speech and grammatical category . The tool used to annotate a corpus is called a tagger. A collection of tags used in a corpus is called a tagset.
  • TBL

    application in Sketch Engine for collecting usage-example sentences to build dictionaries. Find more on the Tick Box Lexicography page
  • term

    a keyword or multi-word term that is more frequent in one corpus compared to another one and at the same time it is not a common word(s) like "the, house, at the, ...". Hence, this is the term significant for the corpus. See more on term extraction»
  • term base

    In connection with CAT tools, a term base is a database of subject-specific terminology and other lexical items which need to be translated consistently. The CAT tool uses the term base to check the consistency of translation, to look for untranslated segments, and to suggest (or automatically supply) translations of the terms from the database.
  • term extraction

    the process of identifying subject specific vocabulary in a subject specific text usually using specialized software. The finding of one-word and multi-word terms in Sketch Engine is based on a comparison with the frequency of these words and phrases in a reference corpus.
  • text type

    a text type is a generic name for types assigned to documents inside a corpus. Text types can refer to source (newspaper, book etc.), medium (spoken, written), but also time (year, century). Not all corpora have documents annotated for text types. Corpora can be divided into subcorpora based on text types and searches or word lists can be generated based on this division.
  • token

    Token is the smallest unit that each corpus divides to. Typically each word form and punctuation (comma, dot, ...) is a separate token. Therefore, corpora contain more tokens than words. Spaces between words are not tokens. A text is divided into tokens by a tool called tokenizer which is often specific for each language, for example don't  in English consists of 2 tokens.
  • tokenization

    Tokenization is the automatic process of separating text into tokens.
  • Tokenizer

    A tokenizer is a tool (software) used for dividing text into tokens. A tokenizer is language specific and takes into account the peculiarities of the language, e.g. don't in English is tokenized as two tokens. Sketch Engine contains tokenisers for many languages and also a universal tokenizer used for languages not yet supported by Sketch Engine. The universal tokenizer only recognizes whitespace characters as token boundaries ignoring any language specific rules. This, however, is sufficient for the use of many Sketch Engine features.
  • translation memory

    A translation memory is a database inside a CAT tool which holds segments of text translated in the past. The CAT tool can suggest (or automatically supply) translations based on matching text from the translation memory.
  • trends

    Trends is a feature used for diachronic analysis, i.e. for identifying how the frequency of the word (or other attributes) changes over time. read more
  • UMS

    feature available to users with local installation for the administration of users and corpora.
  • user corpus

    a corpus created by a user. Users can create corpora by uploading their own data or using Sketch Engine to collect data from the Web. User corpora can be shared with other users.
  • word form [attribute ]

    A word form refers to one form that a word can take, e.g. the word go can take these word forms go, went, gone, goes, going. Searching for the word form going will not find any other forms of the word. It is case sensitiveapple and Apple are two different word forms.
  • word list

    A word list is a generic name for various types of lists such as list of words, lemmas, POS tags or other attributes with their frequency (hit counts, document counts or others).
  • word sketch

    A word sketch is a one-page, automatic, corpus-derived summary of a word’s grammatical and collocational behaviour. more»
  • Word Sketch grammar

    Word Sketch grammar (WSG) is a set of rules defining the grammatical relations (=columns/categories) in a Word Sketch. WSG is language dependent, the same WSG cannot be shared across languages. Different corpora in the same language can use the same or different WSG. Users can write their own WSG to match their specific need. Corpora in unsupported languages can make use of a universal WSG which provides only basic statistics of words surrounding the keywords ignoring the grammar of the language. The universal WSG can also be modified by the user. more»