Text Encodings, and Where the é Went

We ran the mismatch through FileFlip's own conversion code rather than just describing it. A CSV saved in the wrong encoding still runs through csv to json and reports success, with the wrong characters baked in permanently: nothing anywhere in that path stops to tell you. SRT to VTT behaves differently, because FFmpeg's decoder rejects invalid bytes outright and drops the affected cue instead of mangling it.

Why did my accented characters turn into weird symbols?

Something decoded your file's bytes with the wrong encoding, and every character outside plain ASCII broke.

• A diamond with a question mark (�) means a decoder tried UTF-8 on bytes that aren't valid UTF-8, most often a Windows-1252 file. The single byte E9 is é in Windows-1252, but alone it's not a legal UTF-8 sequence, so the decoder substitutes the Unicode replacement character, U+FFFD.
• é in place of é is the opposite mistake: a real UTF-8 file, where é is the two bytes C3 A9, decoded one byte at a time as Windows-1252 instead. C3 reads as Ã and A9 reads as ©, so one character splits into two wrong ones.
• Neither is a corrupted file. The bytes on disk never change; only the decoding step got the wrong instruction. Convert CSV to JSON and convert SRT to VTT run in your browser, and re-running the same file after fixing its encoding, covered further down, produces a clean result.

ASCII, Latin-1, and UTF-8 at a glance

Latin-1 and Windows-1252 get used interchangeably, but they're not the same table. True ISO 8859-1 leaves bytes 80 through 9F undefined; Windows-1252 fills that range with printable characters, 80 for €, 93 and 94 for curly quotes. The mixup is common enough that the WHATWG Encoding Standard, which every browser and Node follow, maps the labels "latin1" and "iso-8859-1" straight to the Windows-1252 decoder.

ASCII is a strict subset of both: plain English text is byte-for-byte identical under all three, which is why encoding bugs stay invisible until the first accented name shows up.

What actually happens when FileFlip guesses wrong?

For CSV and JSON, it produces a file that looks successful and has the wrong characters baked into it, permanently, with no error. For SRT, the cues with bad bytes vanish from the output instead.

convertData, the function behind csv>json and json>csv, reads your file with JavaScript's own file.text(), defined to always run UTF-8 decode regardless of what the bytes actually are, and it never throws. We fed it a two-row CSV saved as Windows-1252 and it came back readable but wrong:

name,city              name,city
José,Curitiba    →     Jos�,Curitiba
naïve,Zürich            na�ve,Z�rich

JSON.parse doesn't reject the result either, because U+FFFD is a legal character inside a JSON string. The conversion succeeds and downloads, and the diamonds are now the only version of that data that exists; the original bytes never touched the output.

SRT behaves differently, because srt>vtt runs through FFmpeg's own subtitle demuxer instead of file.text(), and FFmpeg checks its input for valid UTF-8 rather than substituting. We built a two-cue SRT, saved a UTF-8 copy and a Windows-1252 copy, and ran both through FFmpeg 8.1.2, the version behind FileFlip's WebAssembly build, with no encoding flag, because the srt-to-vtt converter's UI doesn't expose one.

The UTF-8 file converted cleanly, both cues intact. The Windows-1252 file with one accent-free cue lost only the cue containing é and ï, a decode error on that one block, and kept the plain cue next to it. The version where every cue carried an accent lost all of it: FFmpeg's error rate check gave up on the whole file and handed back a .vtt with a WEBVTT header and nothing else, a 7-byte file that opens fine and plays no subtitles.

Nothing in FileFlip's UI tells you which of these happened; every one of these runs exits successfully and produces a file to download.

What is the byte order mark, and who chokes on it?

A byte order mark, or BOM, is three bytes, EF BB BF, that some programs write at the start of a UTF-8 file to flag it as UTF-8 to anything reading it. It's optional. The Unicode Standard permits it and doesn't require it, and most tools built after 2010 don't bother writing one.

FileFlip's CSV and JSON converters are unaffected by a BOM either way, because file.text() runs the UTF-8 decode algorithm, and that algorithm strips a leading BOM as one of its first steps, before your CSV or JSON parser ever sees the text. We confirmed this directly: a CSV built with a BOM prepended, run through the same Papa.parse call convertData uses, produced a first column header of id, not id. A raw JSON.parse on BOM-prefixed text throws immediately, since a BOM character isn't valid at the start of a JSON document, but by the time our code calls JSON.parse, file.text() has already removed it.

Why does Excel guess encoding differently from everything else?

Because Excel decides a CSV's encoding by looking for a BOM, and falls back to your Windows install's system codepage, not UTF-8, when there isn't one.

Microsoft's own guidance on opening CSV UTF-8 files in Excel states that a CSV opens correctly "if it was saved with BOM," and recommends importing through Power Query instead of double-clicking for one that wasn't. That's the reverse of the browser and Node behaviour above: FileFlip's decoder ignores the BOM's presence and always assumes UTF-8, while Excel needs the BOM present to assume UTF-8 and otherwise falls back to a single-byte codepage. A CSV FileFlip writes is correct, BOM-less UTF-8, and double-clicking it in Excel can still mangle any accented name in it, for exactly the reason above.

RFC 4180, the closest thing CSV has to a specification, only says "common usage of CSV is US-ASCII" and leaves any other encoding to an external charset parameter the file itself never carries. That's the whole problem in one sentence: the format has nowhere inside it to say what it is.

Which of our formats declare their own encoding?

Two do, by specification. Two don't, and never will, because there's no field in either format to put it in.

A JSON or WebVTT file that isn't UTF-8 is, by its own format's rules, not valid JSON or WebVTT, whatever a lenient parser lets through. CSV and SRT carry no such rule, which is why they're the two formats in this comparison that actually break in practice: nothing about opening one is wrong until you look at the bytes.

Fixing a file that is already wrong

Fix the encoding before you convert, not after. Once FileFlip has written diamonds or dropped a cue, that output is the only copy of the data left; converting it again just moves the damage into a new format.

• Find out what encoding the file actually is. On a Mac or Linux machine, file -i suspect.csv prints its best guess. On Windows, opening the file in Notepad and checking File → Save As shows the current encoding in the dropdown at the bottom of that dialog.
• Re-save it as UTF-8 before uploading it again. In Excel, use "CSV UTF-8 (Comma delimited)" from the Save As file-type list rather than plain "CSV (Comma delimited)". From a terminal, iconv -f WINDOWS-1252 -t UTF-8 broken.csv > fixed.csv does the same for CSV, JSON, or SRT files, substituting the encoding you found in the step above.
• Reconvert the fixed file. Convert CSV to JSON or convert SRT to VTT again once the source is actually UTF-8, and the accented characters that were diamonds or missing entirely will be correct, because the original bytes are still on disk in the source file. Only the already-converted output is unrecoverable.

Common questions

Can I fix a file full of diamonds by re-encoding the output?

No, not the specific characters that turned into diamonds. U+FFFD is a real, distinct Unicode character standing in for whatever byte sequence the decoder couldn't read, and the information about what that byte sequence actually was is gone the moment the substitution happens.

Re-encoding a file full of diamonds only changes how the diamonds themselves are stored. Fixing the problem means going back to the original file and decoding it with the right encoding the first time.

Why does my file show ’ instead of an apostrophe?

That's the same double-decoding mistake as é, just with a different source character. A curly apostrophe, ', is E2 80 99 in UTF-8, three bytes. Decode those three bytes one at a time as Windows-1252 and you get â, €, and ™ stitched together, which is exactly what you're seeing.

It's UTF-8 read as a single-byte encoding, same cause as everywhere else on this page.

Does converting the same broken file twice make it worse?

No, not by itself. U+FFFD decodes back to the exact same three UTF-8 bytes every time it's re-encoded, so the diamond stays a stable, if wrong, character through any number of further conversions. It doesn't compound the way repeatedly re-saving a lossy image or audio file does; it just never gets better, because there's nothing left in the file to recover from.

Convert between them

FileFlip converts CSV to JSON and SRT to VTT entirely in your browser, on the same code paths tested above. Nothing is uploaded to a server, there's no account, and the conversion runs the moment you drop the file in. Fixing the source encoding first is still on you: FileFlip decodes CSV and JSON as UTF-8 unconditionally and has no control to set anything else.

How we measured this

• The CSV and JSON test used Node 25's Blob.text(), which implements the same WHATWG File API text() method convertData calls in src/utils/convert/data.ts, on a two-row CSV written once as UTF-8 and once as Windows-1252 with Python's codecs module.
• The BOM test ran the CSV row through the same Papa.parse call convertData uses, and ran JSON.parse directly on BOM-prefixed text with and without first passing it through Blob.text().
• The SRT test built a two-cue .srt file, wrote a UTF-8 and a Windows-1252 copy with Python, and ran both through ffmpeg -i in.srt out.vtt with FFmpeg 8.1.2, no -sub_charenc flag, matching what src/utils/convert/subtitle.ts passes today. A third run added -sub_charenc CP1252 to confirm the cues were fully recoverable once the encoding was told correctly.
• This is one subtitle file with two cues, not a corpus; a file with more varied byte sequences could lose a different fraction to the same decode error, though the pattern, whole cues dropping rather than degrading, held in every mis-encoded run we tried.
• Reproduce it yourself: encode any small CSV, JSON, or SRT file as Windows-1252 with iconv -f UTF-8 -t WINDOWS-1252 in.file > out.file, then drop it on the matching FileFlip converter.