What is a token and how does it work

Tokens are what we use to measure your usage on Dr Tech AI, and you can easily track your usage on your account. Let’s dive into what a token is and how it works.

What is a token

A token is a unit of measurement used by AI models to understand and process text. Tokens can be words or chunks of characters, depending on the language. For instance, the word “Artificial” is broken up into the tokens “Art,” and “ificial,” while “doctor” is one token. 

How many token are used

The number of tokens used in a given request is the sum of the number of tokens in your inputs and outputs. For example, if you input 50 tokens and receive an output of 100 tokens, the total token usage for that request would be 150.

It’s important to note that the number of tokens per word varies depending on the language. In English, one token is approximately four characters or 0.75 words. In contrast, one token in French is around three characters or 0.5 words. 

Number of words included in our plans

The free plan on Dr Tech AI provides 25,000 tokens, which is roughly 18,000 words in English and 12,500 words in French. Using token measurements instead of words allows Dr Tech AI to support every language, even those with a higher number of tokens per word, such as Spanish (1 token is approximately 3 characters), Polish (1 token for 2 characters), and Japanese (1 token for 0,75 characters).

If you plan to use another language and want to know how many characters are in a token, contact Dr Tech AI’s support team for assistance. 

Token usage for the Chat Feature

It’s also essential to keep in mind that token usage on the chat feature can increase rapidly. When you send a new question, the input includes all previous messages from the chat (including yours and the AI’s messages), and the output includes all previous messages, your question, and the AI’s answer. Starting a new chat when you have a new topic to discuss can help keep token usage under control.

Conclusion

In conclusion, tokens are units of measurement used by AI models to process and understand text. Dr Tech AI uses token measurements to track usage, and the number of tokens per word varies depending on the language.You can track your token usage on your account.

Contact our support team for assistance with token measurements in other languages, and keep in mind that the chat feature can increase your token usage quickly.

Written by Dr Tech AI