@spec decode_escapes(binary()) :: binary()

Decode backslashed escape sequences (\x, \u, \U, octal, and the single-character escapes), even in the presence of other Unicode.

Unlike the rest of ftfy, this must be called explicitly; escaped text is not necessarily a mistake.

@spec decode_inconsistent_utf8(binary()) :: binary()

Fix UTF-8 mojibake that is embedded within otherwise-fine text, by fixing the detected sequences in isolation.

@spec fix_c1_controls(binary()) :: binary()

Treat any remaining C1 control characters as their Windows-1252 equivalents, the way web browsers do.

@spec fix_character_width(binary()) :: binary()

Replace fullwidth and halfwidth characters with their standard forms.

@spec fix_latin_ligatures(binary()) :: binary()

Replace single-character Latin ligatures with their component letters.

@spec fix_line_breaks(binary()) :: binary()

Convert all line breaks to the Unix \n style.

@spec fix_surrogates(binary()) :: binary()

Replace properly-paired UTF-16 surrogate codepoints with the character they represent, or with U+FFFD otherwise.

Note: the BEAM cannot represent lone surrogate codepoints in a UTF-8 binary, so for any valid string input this is a no-op. It exists for API parity and to handle the (rare) case of a charlist that carries surrogate codepoints.

@spec remove_bom(binary()) :: binary()

Remove a byte-order mark decoded as if it were part of the text.

@spec remove_control_chars(binary()) :: binary()

Remove control characters that have no displayed effect on text.

@spec remove_terminal_escapes(binary()) :: binary()

Strip out ANSI terminal escape sequences, such as color codes.

@spec replace_lossy_sequences(binary()) :: binary()

Replace lossy UTF-8 sequences (where a continuation byte became 0x1A or '?') with the UTF-8 encoding of U+FFFD. Operates on raw bytes.

@spec restore_byte_a0(binary()) :: binary()

Put back byte A0 (non-breaking space) where a Windows-1252 program replaced it with an ASCII space, when doing so makes a fixable UTF-8 sequence. Operates on raw bytes.

@spec uncurl_quotes(binary()) :: binary()

Replace curly quotation marks with straight equivalents.

@spec unescape_html(binary()) :: binary()

Decode HTML entities and character references, including some nonstandard all-caps ones, but only the unambiguous ones that end in semicolons.