Unicodeencodeerror When Using Os.listdir

OS: Windows 7, 64-bit Python 3.1.3 When I try to do this os.listdir('F:\\music') I get this UnicodeEncodeError: 'gbk' codec can't encode character '\xe3' in position 643: illega

Solution 1:

It's a windows console unicode problem, you can fix it by install the win-unicode-console library

$ pip install win-unicode-console
$ edit a.py

  import win_unicode_console

  win_unicode_console.enable()
  print('non-gbk-character Résumé or 欧•亨利 works')

I've tested in python 3.4 in Chinese Windows 8

Solution 2:

UnicodeEncodeError indicates that you are trying to print the filenames. If it was os.lisdir() that had a problem you should see a UnicodeDecodeError (Decode, not Encode).

Because you use a Unicode pathname, os.listdir() returns readily decoded filenames; on Windows the filesystem uses UTF-16 to encode filenames and those are easily decoded in Python (sys.getfilesystemencoding() tells Python what codec to use).

However, the Windows console uses a different encoding; in your case gbk, and that codec cannot display all the different characters that UTF-16 can encode.

You are looking for a print() statement here. You perhaps could use print(filename.encode('gbk', errors='replace')) to try and print the filenames instead; unprintable characters will be replaced by a question mark.

Alternatively, you could use a b'F:\\music' as the path and work with raw bytestrings instead of Unicode.