Garbled character repair/analysis & Excel CSV character code mutual conversion tool | ZeroTools

This tool consists of two primary tabs: "Code Conversion & Hex" and "Mojibake Candidates". Use each tab according to your current data state and intended objective.

If you have the original text file or clean string and want to recreate it in a character encoding readable by other software or systems, use the "Code Conversion & Hex" tab.

On the other hand, if your data has already turned into an unreadable string of symbols or kanji like 縺薙ｓ縺ｫ縺｡縺ｯ and no original byte sequence remains, use the "Mojibake Candidates" tab to attempt deciphering the original data.

The target encodings selectable in the "Code Conversion & Hex" tab are limited to four types: UTF-8, Shift_JIS, EUC-JP, and ISO-2022-JP.

UTF-8 is the current dominant standard representing Unicode characters as variable-length byte sequences and can record characters from across the globe.

Shift_JIS and EUC-JP are Japanese encodings used historically in Japanese computing environments, but they cannot represent all characters existing in Unicode.

ISO-2022-JP is a 7-bit format used in email and similar media that switches character sets using escape sequences.

In this tool, UTF-16 and UTF-32 cannot be selected as file output targets. In internal processing, JavaScript strings are handled as code units equivalent to UTF-16 that can be retrieved via charCodeAt(), but only the four aforementioned types are offered when saving externally as files.

Although the tool name contains the word "CSV", this feature does not parse or edit comma delimiters, quotation marks, or column counts as CSV syntax.

Even for CSV files, it treats them simply as text byte sequences just like TXT or HTML, specializing solely in character encoding conversion.

If you want to rearrange columns or convert to an Excel workbook format (XLSX), please use dedicated tools on this site.

The "Code Conversion & Hex" tab provides two input methods: "Manual Input", where you directly type the text to convert, and "Select File" (or drag and drop) to load a local file.

When entering text in the manual input field, that string is received in the browser as a JavaScript Unicode string. The source encoding is displayed on screen as "UTF-8 (Manual Input)", and the entered string is encoded directly into the specified target encoding.

This is convenient when you want to quickly convert a short, cleanly readable text into a byte sequence of another encoding and save it.

When a file is selected, the manual input area is hidden; clearing the file restores manual input mode.

When a file is loaded, its data is retained in memory as a Uint8Array byte sequence. Immediately after loading, the auto-detection feature of the internal library attempts to determine the character encoding of the original text.

If it is detected as ASCII, UTF-8, Shift_JIS, EUC-JP, ISO-2022-JP, or UTF-16 variants, it decodes the byte sequence into a JavaScript Unicode string according to that encoding.

If auto-detection does not identify the data as one of these supported character encodings, the tool stops with an error. Because binary files may occasionally be misidentified as text encodings by heuristic detection, ensure you select valid text files.

If the loaded file size is under 50 KB (50,000 bytes), the text decoded using the detected encoding is displayed on screen as a preview.

This allows you to visually check whether auto-detection produced the intended result. Encoding conversion is still attempted for files of 50 KB or larger; however, to prevent browser lag caused by expanding lengthy text on screen, only the preview display is omitted, showing file size info instead.

Automatic character encoding detection is merely an inference based on byte sequence characteristics. It does not analyze the linguistic meaning of text written in the file or determine the operating system on which it was created.

Therefore, for short data consisting solely of ASCII characters or byte sequences that happen to match multiple encoding rules, it may detect an encoding different from the actual format.

Do not rely solely on the detected name shown on screen; inspect the preview for files under 50 KB, the converted Hex display, and the actual result opened in target software after saving.

Text loaded as a Unicode string from a file or manual input is encoded into one of the four formats selected by the user. An essential aspect of this conversion process is the behavior when the text contains characters that cannot be represented in the destination encoding.

This tool does not silently substitute unsupported environment-dependent characters or newer emoji with question marks (?) or replacement characters during conversion.

When an unmappable character is detected, it halts processing and displays an error message: "Contains characters that cannot be represented in the target encoding." This design prevents incompletely converted files with missing characters from being saved as valid data.

Only when "UTF-8" is selected as the destination encoding does the checkbox "Prepend UTF-8 BOM" appear. When enabled, the site program explicitly prepends the 3-byte marker EF BB BF (BOM: Byte Order Mark) before saving.

The BOM is not text body data, but rather an indicator to inform reading software that the file is written in UTF-8. Not all systems require a BOM, and some import destinations reject files with a BOM, so decide whether to add it based on the receiving software specifications.

When Shift_JIS, EUC-JP, or ISO-2022-JP is selected, this checkbox is hidden and no BOM is attached.

The download file name for the converted file is generated automatically by combining the original file name, target format, and BOM presence.

For example, if the original name is data.csv, converting to Shift_JIS yields data_sjis.csv, and converting to UTF-8 with a BOM yields data_utf8_bom.csv.

If the file has no extension, txt is appended, and manual input defaults to the base name converted. Proposing a new name without overwriting the original file allows you to compare before and after data side by side.

At the bottom of the screen, a "Hex Preview" displays up to the first 1 KB (maximum 1,024 bytes) of the converted data in hexadecimal notation.

Divided into 16 bytes per line, it lists the 0-based offset on the left, two-digit hex values in the middle, and printable ASCII characters on the right.

Non-printable bytes are represented by periods (.). When a UTF-8 BOM is added, you can visually verify that the leading byte sequence ef bb bf has been prepended.

This is not for checking the entire file, but an auxiliary feature to inspect leading byte structure and escape sequences prior to saving.

The "Mojibake Candidates" tab is designed to decipher mojibake caused by a mismatch between the original encoding and the encoding assumed by a text editor or application when opening the file.

Focusing on UTF-8, Shift_JIS, and EUC-JP—the three encodings most commonly involved in Japanese mojibake—it generates combinations of the assumed "original format" and "misread format".

Excluding pairs where both encodings are identical yields up to 6 directional conversion paths.

In this reverse-calculation process, the entered mojibake text is first re-encoded back into a byte sequence under the assumption that it was misread under a given encoding.

The resulting byte sequence is then decoded again assuming the original intended encoding. For example, to fix garbled text where an original UTF-8 file was misread as Shift_JIS, the input string is converted back to bytes according to Shift_JIS rules and then re-read as UTF-8 text to output the result.

The screen displays the results of these 6 paths as a list, but not all 6 candidates are necessarily shown. Paths where the current string cannot be re-encoded into bytes under the assumed encoding, paths producing empty output, paths yielding output identical to the input, and duplicate paths matching other candidates are excluded.

Thus, the number of displayed candidates varies between 0 and 6 depending on the input string.

Among the displayed candidates, those meeting specific criteria receive a green badge indicating "Contains Japanese Characters". This is a simple indicator confirming that the converted string contains hiragana, katakana, or kanji, and lacks unexpected control characters or replacement characters (U+FFFD) that indicate conversion failure.

However, this badge does not guarantee that the string is the correct restoration. It does not parse linguistic vocabulary, grammar, or character frequencies; alternative mojibake paths may satisfy this condition by coincidence.

Furthermore, this feature cannot restore all instances of mojibake. If unrepresentable characters were already converted to question marks (?) or replacement characters (U+FFFD) during copying of the garbled text, the original byte data is permanently lost and cannot be recovered by reverse calculation.

Data subjected to multiple rounds of mis-encoding and overwrite-saving, or mojibake involving other encodings such as Chinese, may also remain undecipherable across these 6 paths.

Users must evaluate context and source materials to verify which candidate is correct and whether numbers or critical symbols remain intact.

A common reason for converting character encodings is to display CSV files in Microsoft Excel without mojibake. While this tool can be used for that purpose, the way Excel reads files varies by version and environment settings, so several characteristics must be understood before importing.

According to official Microsoft guidance, CSV files saved in UTF-8 format can generally be opened normally in Excel if a BOM is present at the beginning of the file.

Therefore, when opening directly in Excel, one approach is to select "UTF-8" as the target encoding, enable "Prepend UTF-8 BOM", and test with a duplicate file.

However, this does not guarantee success across all CSV structures, Excel versions, or regional settings.

If you prefer to keep UTF-8 files without a BOM, you can use the "From Text/CSV" import feature located in Excel's "Data" tab. This feature provides a preview dialog where you can inspect the file origin and delimiter before loading.

Since UI labels and steps may differ depending on the Excel edition and OS, review the actual preview before completing the import.

Another approach is converting to Shift_JIS for Excel, but Shift_JIS does not cover all Unicode characters. If emoji or unmapped characters are present, this tool stops with a conversion error.

Additionally, while the underlying library's SJIS implementation handles Windows-31J and CP932 extension areas, Shift_JIS and CP932 are not strictly identical standards.

Compatibility with private-use characters or vendor-specific mappings in certain applications cannot be guaranteed.

Even when character encoding matches and mojibake is resolved, secondary issues can arise from Excel's automatic data type interpretation.

Excel may automatically convert date-like strings, long numbers with leading zeros, or strings resembling scientific notation into numeric or date types.

This tool only converts character encodings; it does not assign CSV column types or validate and repair delimiters, quotes, or multiline cell structures.

Before converting production data, test with a small copy containing representative Japanese text, symbols, and identifiers with leading zeros.

Checking whether mojibake is resolved separately from whether column data types loaded as intended makes it easier to spot discrepancies caused by OS locale settings or Excel versions.

File reading, auto-detection, encoding and decoding, candidate calculation, and export file generation are executed entirely as client-side JavaScript within your browsing tab.

The conversion logic contains no routine that transmits selected file contents or typed text to external encoding APIs or saves them on a server.

However, this does not mean the entire website operates completely offline without network activity. Standard communications that constitute web pages—such as page delivery, analytics, and advertising scripts—can occur independently.

It also does not block browser extensions from inspecting page contents or prevent data transfer to the OS clipboard when copying candidate text.

Complete site-wide security cannot be assured when handling highly confidential data.

The conversion logic depends on the standard interfaces of encoding-japanese. It does not implement custom per-character parsers for each encoding byte array.

It does not feature lookup tables to reconcile OS differences for wave dashes, nor does it automatically normalize decomposed voiced consonant marks into Unicode NFC form.

For mojibake candidates, the tool does not analyze text for natural Japanese grammar to automatically pick the single best result.

Batch processing across multiple files and conversion history persistence are also unsupported. Whether a generated string forms coherent text must be verified by a human reviewer.

The 50 KB threshold serves only to omit screen text previews and is not a limit on conversion processing itself. However, because files are loaded into browser RAM, processing duration and memory usage increase with file size and device capabilities, which may cause unstable behavior.

Processing speed cannot be guaranteed for large files.

Encoding conversion is not a mechanism for recovering lost data. Inspect the converted file and Hex preview, comparing them against the original whenever in doubt.

Retaining the original file and testing on copies remains the fundamental safeguard against accidental data loss.