Compare Two Texts and Find Invisible Differences

How to read the result

For each differing position you get the codepoint of the character in A and the codepoint of the character in B, with the Unicode name if it has one. Position is counted in Unicode code points, not bytes — a position that lands on an emoji family with a zero width joiner inside it will still make sense.

If you press Clean both, then compare, the page strips invisible characters from both sides and compares again. If the cleaned versions are equal, the mismatch was hidden characters and nothing more. If they still differ, there is real wording or visible-punctuation difference to investigate.

Why this is useful

The classic case is a string that renders identically but will not match a key, will not compare equal in code, or will not find a record. Most of the time, invisible characters are the cause, and you want to know exactly where and which ones before you delete anything.

This page tells you exactly that, with codepoints, so you can decide whether to strip them and whether to trust the cleaned version.

Which engine sees it, and which one does not

Two strings that look the same and compare as different are not a contradiction: two engines, two rules. The part that catches people out is that the rule can change underneath you. The same pair comes out equal on one machine and different on another, and the same file gives two different counts to two tools in the same pipeline — so the number you are looking at may be correct for the tool that produced it and wrong for the question you asked.

Take one record, INV-1001 Paid, and the same record with a pair of directional marks wrapped around the number — U+202D before it, U+202C after it, two code points that print nothing. It goes from 13 bytes to 19. Handed both versions, this is what nine engines report:

The same split turns into a counting problem as soon as the strings are lines in a file. A five-line log was written with the same error message three times: once clean, once wrapped in U+202D and U+202C, once with a U+200B dropped inside the word payments, plus two identical warnings:

Three copies of one error, and two tools in the same pipeline disagree about how many distinct errors there are: sort -u says three, sort | uniq says four, on the same file under the same locale. Neither is broken. sort -u compares by the locale's collation rules, uniq compares bytes. Under en_US.UTF-8 the collation treats the directional marks as if they were not there and the wrapped line collapses into the clean one; the zero-width space inside the word survives that comparison, because these tables do not ignore U+200B while ICU's do. The two collation implementations disagree with each other about which invisible characters count.

How big that set is depends on which tables you are running. Of 26 invisible code points run through ICU's English collator, 17 compare as equal to nothing at all: the zero-width space, the joiner and non-joiner, the word joiner, the byte order mark, the soft hyphen, the combining grapheme joiner, the directional marks U+200E, U+200F, U+202A, U+202D, U+202C, U+2066, U+2069 and U+061C, the Mongolian vowel separator, and the tag characters. Nine are not ignorable: the spaces U+00A0, U+2007, U+2009, U+202F and U+3000, the line and paragraph separators, and the two Hangul fillers. A merge, a join, or a unique index on a collation-aware engine silently folds the first 17 into the record you already have; one built on byte equality rejects all of them; and the characters that behave like spaces are rejected by both.

Where they come from is less disputed than what to do about them. People who have lost hours to this describe the same handful of sources: a certificate fingerprint copied out of a Windows dialog, a name out of a word processor that carries a left-to-right mark on the end, a chat client that turns tabs into non-breaking spaces inside pasted configuration files and breaks the build, an input method, a paragraph out of a PDF. The disagreement is about blame. One camp says the application that inserted the character is at fault. The other, smaller and more convincing, says the field should never have accepted it — the character arrived, the record was stored, and every lookup afterwards failed for a reason nobody could see. A third report has the opposite shape: the merge failed while the spreadsheet's own comparison of the same two values returned true, which is the split in the table above showing up inside one product.

The checks people reach for first are the ones that miss. A widely-read answer to the question "how do I find the difference between two strings that look identical" suggests testing with Contains("\u8203"). The escape is read as hexadecimal, so that looks for U+8203, a CJK ideograph, not for U+200B — 8203 is the decimal value of the zero-width space. Measured: on a string with the character at position 4, the search with "\u8203" returns -1, while chr(8203) finds it at 4. Length is not a test either. The two records above are 15 characters against 13 and 19 bytes against 13, and in the other direction one person checked the length, saw it match, concluded there were no invisible characters, and spent the afternoon proving both halves of that wrong.

What does work, in order of cost. Compare bytes, not characters: cmp gives you the offset of the first difference, which is the position you need, and it costs nothing to run. Paste both sides into the box at the top if you want the code point named. Only then decide what to delete — normalising first is worth one row of the table, because NFKC folded the U+00A0 pair into a single record and left every other pair untouched. The blunt fix, keep only U+0020 to U+007E, does remove the marks: on a 37-code-point line carrying two directional marks, an em dash, two accented letters and a non-breaking space, it left 31 code points — and took the em dash, both accents and the spacing with it.

Limits

Up to fifty differing positions are listed. If there are more than fifty, the page tells you the total and shows the first fifty. The first-difference panel always shows position 1, 2 or 3 with surrounding context, regardless of how many differences there are.

Nothing is uploaded. The comparison runs in this browser tab.