Perl: Convert File Encoding

By Xah Lee. Date: . Last updated: .

How to convert a file encoding in Perl?

Use the shell util /usr/bin/piconv.

# convert utf16 encoding to utf8
piconv -f UTF-16 -t utf8 file.txt > filenew.txt

Type piconv for help.

piconv is installed by Perl and written in Perl. You can also look at the code to see how it's done.

◆ piconv
piconv [-f from_encoding] [-t to_encoding] [-s string] [files...]
piconv -l
piconv -r encoding_alias
  -l,--list
     lists all available encodings
  -r,--resolve encoding_alias
    resolve encoding to its (Encode) canonical name
  -f,--from from_encoding
     when omitted, the current locale will be used
  -t,--to to_encoding
     when omitted, the current locale will be used
  -s,--string string
     "string" will be the input instead of STDIN or files
The following are mainly of interest to Encode hackers:
  -D,--debug          show debug information
  -C N | -c           check the validity of the input
  -S,--scheme scheme  use the scheme for conversion
Those are handy when you can only see ascii characters:
  -p,--perlqq
  --htmlcref
  --xmlcref

File Encoding

  1. Unicode Basics: Character Set, Encoding, UTF-8, Codepoint
  2. HTML: Character Sets and Encoding
  3. Unicode in Ruby, Perl, Python, JavaScript, Java, Emacs Lisp, Mathematica
  4. Python: Unicode Tutorial 🐍
  5. Python: Convert File Encoding
  6. Python: Convert File Encoding for All Files in a Dir
  7. Perl: Unicode Tutorial 🐪
  8. Perl: Convert File Encoding
  9. Ruby: Unicode Tutorial 💎
  10. Java: Convert File Encoding
  11. Linux: Convert File Encoding with iconv

If you have a question, put $5 at patreon and message me.

Perl

  1. Perl Overview
  2. Version String
  3. Help System

Detail

  1. Quoting String
  2. Format String
  3. String Operations
  4. True, False
  5. if then else
  6. Loop
  7. List / Array
  8. Loop Thru List
  9. Map f to List
  10. List Comprehension
  11. Hash Table
  12. Function Optional Param
  13. regex

Text Processing

  1. Unicode 🐪
  2. Convert File Encoding
  3. Read Write File
  4. Traverse Dir
  5. Find Replace
  6. Validate Local Links
  7. Split Line by Regex

Advanced

  1. Sort List, Matrix, Object
  2. Sort Matrix
  3. Sort Unstable
  4. Sort Misc
  5. List Modules, Search Paths
  6. Write a Module
  7. Complex Numbers
  8. System Call
  9. gzip
  10. Get Env Var
  11. GET Web Content
  12. Email