Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func EstimateTokens ¶
EstimateTokens estimates the number of LLM tokens in the given text. Uses a simple heuristic of approximately 4 characters per token, which is a common approximation for English text with GPT-style tokenizers.
Types ¶
This section is empty.
Source Files
Click to show internal directories.
Click to hide internal directories.