What invisible characters cost you in LLM tokens

Short answer: about one token each. A 1,000-word text with a single zero-width space after every word goes from 1,103 tokens to 2,103 — a 90.7% increase — before you have added a single word of meaning. The characters are invisible on screen, but the tokenizer sees every one of them and you pay for every one of them.

Measured, not estimated

These are the numbers from a 1,000-word English sample, tokenized twice — once with o200k_base and once with cl100k_base, the encodings behind current OpenAI models. Only difference between rows: invisible characters inserted at a fixed interval.

The pattern is close to linear: one invisible character in, roughly one token out. It holds while each character stands on its own, which is what every row above does. Move them and the rate moves too — a character buried inside a word costs about two, a run of identical ones costs a quarter each. Where the character lands prices the same 1,000 characters four ways.

Why a character you cannot see still costs a token

Tokenizers are built from byte-pair merges learned on ordinary text. Common English words collapse into single tokens. Zero-width and invisible characters are rare in training data, so they never earned a merge of their own — the tokenizer falls back to splitting them into their raw UTF-8 bytes, and each of those bytes becomes a token. U+200B is three bytes in UTF-8 (E2 80 8B), which is why one invisible space lands you roughly one extra token rather than zero.

That is the whole mechanism. There is nothing exotic about it, which is exactly why it gets missed: everything about these characters says they are not there.

Where the character lands changes the price

Every row below carries the same 1,000 zero-width spaces through the same 1,000-word document — 1,140 tokens clean in o200k_base, 1,160 in cl100k_base. Nothing changes except where they sit, and the bill moves by a factor of eight.

Three mechanisms are at work and none of them belongs to the character. The first is that a word which fits in one token stops fitting once something invisible appears in the middle of it. authentication is a single token; with one zero-width space between authent and ication it comes out as four — auth, ent, the character, ication. One of those is the character you inserted. The other two are the word you cut. Measured one word at a time across the sample, 860 of the 1,000 words cost more with the character inside than with it trailing, and the surplus comes to 939 tokens before the characters themselves are counted.

The second is a detail of how English gets tokenized: the space in front of a word belongs to the word.  alpha is one token. Put an invisible character between that space and alpha and you have two. In this sample 905 of the 1,000 words are a single token in that leading-space form, and for 193 of them the character in front cost two tokens or more rather than one. That is the gap between 1.00 and 1.22.

The third runs the other way. A run of identical characters is cheap, because the tokenizer holds a token for the repetition: in o200k_base a single token decodes back to four zero-width spaces, so eight in a row cost 2 tokens, sixteen cost 4, and a thousand cost 250. In cl100k_base the same token holds two, which is why the identical block costs 500 there. This is the one arrangement where counting characters overcharges you, by four.

Do not carry that discount over to other characters. A block of 1,000 identical tag characters cost +4,001 tokens against +4,000 spread one per word — no collapse whatsoever. The discount exists only where that exact repetition earned a token, and nothing about the character tells you whether it did. It is also why the flat "one token each" rule survives unchallenged: it is tested on characters sitting alone after a word, the one arrangement where it happens to be correct.

There is an argument running underneath all of this. One side holds that a single added character cannot move the count, because it ought to merge into the text around it. The other answers that sometimes it merges and the count stays where it was, and sometimes it does not and the count goes up. Both are describing the same machinery from opposite ends, and the version with ordinary text in it is familiar enough: a number such as 123456789 comes out as three tokens or four depending on where the splits fall, so the count was never a function of how many characters were typed. It is the same reason character-level work is where these models are weakest — the digits of 101 arrive as 10 and 1, not as three characters, and a model fed raw bytes has no such trouble. A hidden character gets no exemption from any of it.

Which leaves an awkward shape for anyone trying to budget. The cheapest place to put a thousand hidden characters is the easiest place to find: a run of a thousand is caught by any scanner that flags three in a row. The most expensive place — inside words — is invisible to that scanner, and invisible to an estimate built by counting characters. Counting is still the right first move, but it counts what is present, not what will be billed. The multiplier is set by position, and it runs from 0.25 to about 2.1 per character over the same 1,000.

Measured 2026-10-12 with tiktoken 0.14.0 on the same 1,000-word sample, in both o200k_base and cl100k_base; the four rows differ only in placement.

Where this actually bites

• Pasting model output back into a model. The most common way to carry a watermark forward is to feed yesterday's output into today's prompt. You pay for the hidden characters on every round trip.
• RAG pipelines. Chunks scraped from pages or documents that carry invisible characters inflate every retrieval, and the inflation is invisible in your usage dashboard because the text looks short.
• Batch jobs. A 90% token increase on a nightly job over a large corpus is not a rounding error, and it does not show up as an error at all.
• Context limits. Tokens spent on characters that carry no meaning are tokens unavailable for the content you actually wanted in the window.

What it costs in dollars

Tokens are abstract until they are multiplied by a rate and a volume. The table below takes the measured token counts from this page and prices them at the input rates on OpenAI's API pricing page, checked 14 November 2026: gpt-6.1-terra $1.00 and gpt-6-astra $10.00 per 1M input tokens. Rates change and cached input is billed lower, so re-check before budgeting — the arithmetic is the point, not the rate.

Two things follow. First, the per-call amounts look like nothing — a tenth of a cent at the cheap end — which is why this survives review: nobody investigates a tenth of a cent. It only becomes money at volume, and the volume is exactly where pasted and scraped text lives: RAG pipelines and batch jobs that run the same dirty corpus thousands of times. Second, the cheapest arrangement is again the one that is easiest to catch: the 1,000-character block costs a quarter of the spread-out version and is caught by any scanner that flags three in a row, while the inside-words arrangement costs more than double and is invisible to that scanner. Stripping the characters before they reach the model costs nothing per call; paying for them costs a little per call, every call, for as long as the dirty text keeps circulating.

What it costs to hide an instruction

A watermark only has to be present. A hidden instruction has to be read. Both are carried the same way, so the same question applies: what does it cost to put a sentence into a document where nobody can see it? Measured, hiding one 72-character sentence costs about 24 times what typing it in the open would — roughly 280 tokens instead of 12. And the carriers that cost the least to hide are the ones that survive the most cleaning.

Put into a 1,000-word document that costs 1,140 tokens on its own, that payload adds 24.4% to 30.4% to the bill. The same sentence typed in the open adds 1.05%. Hiding text is not the cheap version of sending it. Per character it is between 23 and 29 times more expensive, and the cheapest of the three is not the one most people watch for. Measured one at a time inside a word: a variation selector costs five tokens in both encodings, a tag character five in o200k_base and four in cl100k_base, a zero-width space two.

The more awkward part is what survives. Seven ordinary cleaning steps were run against all three payloads, in JavaScript and in Python, and the two languages agreed on every row.

The name people reach for when they want to strip invisible characters is \p{C}, and it works on two of the three. It leaves the variation-selector payload completely readable: all 72 characters came back out of the cleaned string. The reason is a classification detail rather than a clever trick — U+E0100 to U+E01EF are Mn, combining marks, not C. A selector with nothing to select still takes up a code point, still carries its bytes, and still costs its tokens. In Chromium at 16 px it measured 0.00 px wide with zero painted pixels, against 503.06 px and 2,619 painted pixels for the same sentence typed in the open.

Which sounds like an argument for stripping all combining marks instead, and that is where the trade-off actually sits. Run \p{M} over a single line of ordinary text — three accented words, two emoji that carry a presentation selector, and two words in Devanagari — and it removed 9 of 41 code points in decomposed form, 5 of 37 in composed form. Every accent went, both emoji lost the selector that makes them render as emoji rather than as the plain black symbol, and the Devanagari words lost three of the signs they need, which leaves them misspelled rather than merely unaccented. The choice is not between cleaning and not cleaning. It is between which kind of real text you are willing to damage, and there is no setting that damages only the part you wanted gone.

Detection has the same shape. Blocked together at the end of a document, the three payloads produce runs of 576, 72 and 72 consecutive invisible code points and are caught immediately by any scanner that flags three or more in a row. Spread one code point per word, the longest run of invisible characters in the whole document is one, and such a scanner sees nothing at all. Spreading costs nothing: with the payload spread, the rendered box measured 1280 × 201.52 px, exactly what the clean version of the same paragraph measured, and the token count was unchanged. The one-code-point carriers need 7.2% of the words to carry one; the binary carrier needs 57.6%.

Nothing in the ordinary path removes it either. After selecting the whole rendered paragraph and pressing Ctrl+C, the full 72-character sentence was recovered intact from the clipboard, with the payload spread one code point per word. innerText and textContent both returned 1,585 characters — the page's own view of itself does not drop any of it. Whether a model then obeys a hidden instruction is a separate question from whether it receives one, and it is the question people argue about. The receiving end is not in dispute: the characters are in the token stream, they are billed, and no step in a normal copy-and-paste takes them out. Of the seven cleaning steps above, only the ASCII-only filter removed all three.

Measured 2026-10-02 with tiktoken on a 1,000-word English sample, in both o200k_base and cl100k_base; survival checked in Python against Unicode 15.1 and in Node against Unicode 17, which returned the same answer for every row; rendering and clipboard measured in Chromium.

Check your own text

Count the invisible characters first, then multiply. The zero-width space remover shows you where they sit and how many there are; the invisible characters list has the code points if you want to look them up by hand. If you are cleaning a payload rather than prose, the JSON cleaner and Excel cell cleaner do the same job on structured input.

Reproduce it

The measurement above is three lines of Python. Run it on your own text rather than trusting a table on someone else's site:

import tiktoken
enc = tiktoken.get_encoding("o200k_base")   # or "cl100k_base"
dirty = "Your text here"                    # paste as-is, do not clean it
clean  = dirty.replace("\u200b", "").replace("\u200c", "") \
                .replace("\u200d", "").replace("\ufeff", "")
print(len(enc.encode(dirty)), len(enc.encode(clean)),
      len(enc.encode(dirty)) - len(enc.encode(clean)))

The third number is what the hidden characters are costing you per call. Multiply it by your request volume and your provider's rate.

What this does not tell you

This page measures cost, not authorship. Extra tokens tell you characters are present; they do not tell you who put them there or why, and they are not evidence that a text was generated by a model. Token counts also vary by encoding, so the percentages above apply to the two encodings tested — run the snippet against your own model's tokenizer if you need a number you can bill against.

Measured 2026-09-21 with tiktoken on a 1,000-word English sample. Dollar figures added 2026-11-14 from OpenAI API pricing (gpt-6.1-terra $1.00, gpt-6-astra $10.00 per 1M input tokens). Tools: Claude watermark remover, zero-width space remover, Word cleaner, JSON cleaner, Excel cell cleaner, invisible characters list. The most expensive per code point are the variation selectors in the U+E0100 block, at five tokens each in both encodings, against five and four for the Unicode tag characters.