defaultTokenCounter

const defaultTokenCounter: TokenCounter

Default token counter: chars / 4 heuristic.

~85% accurate for English text. The 80% default context limit in memory.compaction() provides a safety margin for this inaccuracy.

Param

Text to estimate token count for

Returns

Estimated token count