• 9
name

A PHP Error was encountered

Severity: Notice

Message: Undefined index: userid

Filename: views/question.php

Line Number: 191

Backtrace:

File: /home/prodcxja/public_html/questions/application/views/question.php
Line: 191
Function: _error_handler

File: /home/prodcxja/public_html/questions/application/controllers/Questions.php
Line: 433
Function: view

File: /home/prodcxja/public_html/questions/index.php
Line: 315
Function: require_once

name Punditsdkoslkdosdkoskdo

python locale strange error. what's going on here exactly?

So today I upgraded to bazaar 2.0.2, and I started receiving this message (I'm on snow leopard, btw):

bzr: warning: unknown locale: UTF-8
  Could not determine what text encoding to use.
  This error usually means your Python interpreter
  doesn't support the locale set by $LANG (en_US.UTF-8)
  Continuing with ascii encoding.

very strange, since my LANG is actually empty. Similar thing happen when I try to tinker with the locale module

Python 2.5.4 (r254:67916, Nov 30 2009, 14:09:22) 
[GCC 4.3.4] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getdefaultlocale()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/sbo/runtimes/lib/python2.5/locale.py", line 443, in getdefaultlocale
    return _parse_localename(localename)
  File "/Users/sbo/runtimes/lib/python2.5/locale.py", line 375, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

exporting LANG does not help

sbo@dhcp-045:~ $ export LANG=en_US.UTF-8
sbo@dhcp-045:~ $ bzr
bzr: warning: unknown locale: UTF-8
  Could not determine what text encoding to use.
  This error usually means your Python interpreter
  doesn't support the locale set by $LANG (en_US.UTF-8)
  Continuing with ascii encoding.

However, this solved the problem

sbo@dhcp-045:~ $ export LANG=en_US.UTF-8
sbo@dhcp-045:~ $ export LC_ALL=en_US.UTF-8

Python 2.5.4 (r254:67916, Nov 30 2009, 14:09:22) 
[GCC 4.3.4] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getdefaultlocale()
('en_US', 'UTF8')

Could you please explain what's going on here, for better googlability ?

2016 UPDATE: Turns out that this is a Python bug since at least 2013, very probably earlier too, consisting in Python not reacting well to non-GNU locales - like those found in Mac OS X and the BSDs. The bug is still open as of September 2016, and affects every Python version.


If there was no LANG environment variable set, chances are you had either an LC_CTYPE (the key variable) or LC_ALL (which overrides if set) environment variable set to UTF-8, which is not a valid OS X locale. It's easy enough to reproduce with the Apple-supplied /usr/bin/python or with a custom python, as in your case, that was built with the 10.6 SDK (probably also the 10.5 SDK). You won't be able to reproduce it that way with a python.org python; they are currently built with the 10.4 SDK where the locale APIs behave differently.

$ unset LANG
$ env | grep LC_
$ export LC_CTYPE="UTF-8"
$ /usr/bin/python  # Apple-supplied python
Python 2.6.1 (r261:67515, Jul  7 2009, 23:51:51) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale ; locale.getdefaultlocale()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/locale.py", line 459, in getdefaultlocale
    return _parse_localename(localename)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/locale.py", line 391, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8
^D
$ /usr/local/bin/python2.6   # python.org python
Python 2.6.4 (r264:75821M, Oct 27 2009, 19:48:32) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale ; locale.getdefaultlocale()
(None, 'mac-roman')
>>> 

EDIT:

There may be another piece to the puzzle. A quick look at the bzr 2.0.1 I have installed indicates that the message you cite should only show up if locale.getpreferredencoding() raises a locale.Error. One way that can happen is if the python _locale.so C extension can't be loaded and that can happen if there are permission problems on it. For example, MacPorts currently is known to have problems setting permissions if you have a customized umask; I've been burned by that issue myself. Check the permissions of _locale.so in the python lib/python2.5/lib-dynload directory and ensure it is 755. The full path for MacPorts should be:

/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload/
  • 17
Reply Report
    • The python I am using is installed by hand, it's not the standard python installation bundled with OSX. Also, it appears I have no _locale.so or _locale.dylib.... uh ?
      • 2
    • There should be a _locale.so in lib-dynlib. If not, your python was not built correctly on OS X and locale.py will fall back to some default behaviors.
    • @hmijail, Thanks for jogging the memory. I just took a quick look at it but I'm not sure that the OS X 10.6 Terminal app behaves the same way that the 10.9+ one does; certainly the System Preferences options are different. If anyone wants to take the time to investigate further, feel free to edit the answer; that's a feature of SO.

I faced the same problem. When I ran locale, I noticed that the LANG and LC_ALL were unset. So I fixed this by adding the following lines in the .bash_profile file:

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

Then I simply ran:

source ~/.bash_profile 

And this issue was fixed thereafter on my Mac.

  • 9
Reply Report

It's a Mac OS X problem. To see your locale settings, run locale in terminal. locale -a should list all locales that you have defined (that you may use as argument to LC_ALL).

Notice that LC_ALL and other LC_* variables take precedence over LANG when defined.

  • 4
Reply Report
      • 1
    • It certainly can be confusing and there is the issue of the python.org pythons using the 10.4 SDK/libs where locale was kind of broken (something I forgot about when writing my original response). Things are cleaner starting with 10.5. Also, python 3.1 has some cleanup in the locale area, e.g. you won't see mac-roman anymore. The OP, though, is clearly not using a python.org python.
      • 1
    • More specifically this is a problem with the environment not just with Mac OS X. Linux and other UNIX clones are prone to the same problems if you customize your environment and inadvertently leave things out. Sometimes the problems appear immediately, and other times not until you really need them to work. Yet another symptom of having too many ways to do the same thing...
      • 1
    • @synack: This is something that has come up before with OS X, which is why I don't think it's because Stefano changed his environment.
      • 1
    • @kaiser.se: Why do you think this is a problem and why with OS X? Having an LC_CTYPE or LC_ALL set explains the behavior seen by the OP and is working as documented. The example I gave fails exactly the same way on a current Debian Linux system with the exception that the newer bash on that system actually warns you when exporting LC_CTYPE to the invalid value "UTF-8".

Trending Tags