You have the option of selecting a Generative AI model that suits your conversational needs. The bot (chatbot, mailbot, or callbot) can have a default AI model defined for all its generative features (use of generative text responses, generative audio responses, and AI agents).

To access the selection of models available for each bot, go to the toolbar on the left within the bot designer and select “Generative model.”

Once inside, you can choose from a range of models. Please note that the range of models is constantly being updated, so we will periodically update the models available to choose from.

What is the difference between one model and another?

AI models differ in cost, speed, and level of intelligence in their responses. This information is visible within each model card. You can also add information from the LLM page, such as OpenIA, Gemini, or Llama.

When choosing a model, we recommend taking these three variables into account, as well as the conversational purpose you are seeking for your bot.

If you do not select a particular model, GPT-4 Mini will be used by default.

It may be necessary to modify the prompts so that they work after changing a model. Not all models work the same way. Changes to the instructions may be necessary.

A smarter model does not guarantee better bot performance. You should test the bot when making these changes and modify the instructions as necessary.

What are tokens and how are they charged?

Tokens are the basic units that language models use to process text. Their size and composition vary depending on multiple factors:

• The language used
• The context of the surrounding words
• Special characters and punctuation marks
• The specific AI model being used

For example, in the sentence “Hello, how are you?!” which contains 19 characters, the model consumes 6 tokens. This occurs because the model can:

• Separate whole words such as “Hello” into one token
• Group punctuation marks such as “?!” into a single token
• Split longer words into multiple tokens based on common language patterns

It is important to understand that there is no fixed rule for calculating tokens, as each language model (LLM) has its own tokenization system based on how it learns to give meaning to words and their components (n-grams).

You can find details of tokens by model and where that consumption took place (Co-pilot, content base training, generative text response, or generative natural language) in the “Settings” > ‘Account’ > “Products and consumption” section.  

https://go.botmaker.com/#/account/products

How to select a generative AI model in an AI agent block?

From the “AI Agent” block, you can choose a generative AI model that best suits the specific needs of an agent.

Within the AI agent block, go to the last step, “Generative Model.” By default, the option “Use the model configured in bot identity” is selected. You can select “Use a different model” and choose a different one.

At Botmaker, we recommend using the AI Agent block model change sparingly to ensure the bot works as well as possible.