In this example, unicode characters will be dropped from varB.

If you encode with ascii an decide to throw out the unicode characters ,use the below option.

This basically tells the text editor what codec to use.

When you use IDLE (Python 2) and the file contains non-ASCII characters, then it will prompt you to add an encoding declaration, using the Emacs -*- style.

ThisLocale=tlocale(category=locale.LC_ALL, locale='en_GB.UTF-8')

You can Set the encoding in the code also.ī = str1.encode('utf-8', 'ignore').decode('utf-8').

Let’s see the the options to set the UTF-8 Encoding (If you are using Python 3, UTF-8 is the default source encoding)

Else there would be invisible characters which are not interpreted as UTF-8. Verify if the text editor encodes properly your code in UTF-8.A Standard option is to use “UTF-8” as a encode option which more or less works fine.

For example – If the default is UTF-8, these would be LANG=”UTF-8″, LC_ALL=”UTF-8″, LC_CTYPE=”UTF-8″.Check the values set against these variables.The default is defined by the variables LANG, LC_ALL, LC_CTYPE.The encoding default can be located in – /etc/default/locale.Check value – sometimes it is set to “None”.Python string function uses the default character encoding.When the encoding is not correctly set-up, it is commonly seen to throw an “”UnicodeDecodeError: ‘ascii’ codec can’t encode” error.In Python 3 UTF-8 is the default source encoding.In this post, we will see – How to Enable UTF-8 in Python.

Python 3 write to file utf 8