Token
Introduction
The Token level is the third level of output and provides analysis of individual words within a sentence. The Token Object contains subtypes such as Meaning and Lemmatizer.
The Meaning subtype provides information about the definition and context of the word, while the Lemmatizer subtype identifies the base form of the word.
Overall, the Token returned by the API offers a detailed analysis of each word within a sentence, providing insight into the meaning and context of the word.
Format
Tokens are received inside an Array()
| Key | Type | Description |
|---|---|---|
source | String | Source of the document without normalisation |
tag | String | List of all tags |
source_pure | String | Source of the document without normalisation |
indexes | [number, number] | Indexes of the token in the source |
lemma | String | Lemma of the token |
lemmatizer | Lemmatizer | Lemmatizer of the token |
meaning | Meaning | List of all meanings |
transform | Transform | |
value | object | Value of the token |
len | Number | Length of the token |
dep | String | List of all dependencies |
ref | Number | Reference of the token |
index | Number | Index of the token |
coreference | number [] | Coreference of the token |