Airtime

media monitor encoding problem

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 2.1.0
  • Fix Version/s: 2.1.0
  • Component/s: Media Monitor
  • Labels:
    None
  • Originating Party:
    Developer

Description

sync_database_to_filesystem()] : LINE 153 - /srv/airtime/stor/imported/Dana Hilliot and his friends/I WAS A RABBIT AND I WON/06-I'm your alcool-499kbps.ogg
2012-04-18 20:47:24,776 ERROR - [MainThread] [MediaMonitor.py : <module>()] : LINE 95 - Exception: 'ascii' codec can't encode character u'\u2019' in position 85: ordinal not in range(128)
2012-04-18 20:47:24,776 ERROR - [MainThread] [MediaMonitor.py : <module>()] : LINE 96 - traceback: Traceback (most recent call last):
  File "/usr/lib/airtime/media-monitor/MediaMonitor.py", line 66, in <module>
    bootstrap.scan()
  File "/usr/lib/airtime/media-monitor/airtimefilemonitor/airtimemediamonitorbootstrap.py", line 45, in scan
    self.sync_database_to_filesystem(id, dir)
  File "/usr/lib/airtime/media-monitor/airtimefilemonitor/airtimemediamonitorbootstrap.py", line 154, in sync_database_to_filesystem
    if os.path.exists(full_file_path):
  File "/usr/lib/airtime/airtime_virtualenv/lib/python2.6/genericpath.py", line 18, in exists
    st = os.stat(path)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 85: ordinal not in range(128)

2012-04-18 20:47:28,732 INFO - [MainThread] [MediaMonitor.py : <module>()] : LINE 32 -

Issue Links

is duplicated by

Bug - A problem which impairs or prevents the functions of the product. CC-3629 Media Monitor DB sync

  • Major - Major loss of function.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Martin Konecny added a comment -
This looks like its using a smart quote in the filename, and os.path.exists expects a non-unicode string. This is a case that would be fixed in Python 3....
Show
Martin Konecny added a comment - This looks like its using a smart quote in the filename, and os.path.exists expects a non-unicode string. This is a case that would be fixed in Python 3....
Hide
Permalink
Martin Konecny added a comment -
Yup, unicode code point u+2019 is an apostrophe.
Show
Martin Konecny added a comment - Yup, unicode code point u+2019 is an apostrophe.
Hide
Permalink
Martin Konecny added a comment -
Naomi, this looks like a system configuration option. Python respects the system config of how it should read the interpret filepath strings. If your filesystem supports UTF-8 encoded strings, then this error shouldn't happen because Python would encode the String into UTF-8 instead of ascii: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 85: ordinal not in range(128)
Show
Martin Konecny added a comment - Naomi, this looks like a system configuration option. Python respects the system config of how it should read the interpret filepath strings. If your filesystem supports UTF-8 encoded strings, then this error shouldn't happen because Python would encode the String into UTF-8 instead of ascii: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 85: ordinal not in range(128)
Hide
Permalink
Naomi added a comment -
hey I did this like you said

naomiaro@naomiaro-HP-Elite-7200-MT-Business-PC:~$ env | grep LANG
LANG=en_US.UTF-8
LANGUAGE=en_US:en

it looks like I do have UTF8
Show
Naomi added a comment - hey I did this like you said naomiaro@naomiaro-HP-Elite-7200-MT-Business-PC :~$ env | grep LANG LANG=en_US.UTF-8 LANGUAGE=en_US:en it looks like I do have UTF8
Hide
Permalink
Martin Konecny added a comment -
These guys have the same problem: http://stackoverflow.com/questions/2076708/python-os-stat-and-unicode-file-names
Show
Martin Konecny added a comment - These guys have the same problem: http://stackoverflow.com/questions/2076708/python-os-stat-and-unicode-file-names
Hide
Permalink
Martin Konecny added a comment -
https://code.djangoproject.com/ticket/9696
Show
Martin Konecny added a comment - https://code.djangoproject.com/ticket/9696
Hide
Permalink
Martin Konecny added a comment -
Will fix this so that media-monitor doesn't start if we have a non UTF-8 locale.
Show
Martin Konecny added a comment - Will fix this so that media-monitor doesn't start if we have a non UTF-8 locale.
Hide
Permalink
Martin Konecny added a comment -
http://publib.boulder.ibm.com/infocenter/wmbhelp/v7r0m0/index.jsp?topic=%2Fcom.ibm.etools.mft.doc%2Fae19494_.htm
Show
Martin Konecny added a comment - http://publib.boulder.ibm.com/infocenter/wmbhelp/v7r0m0/index.jsp?topic=%2Fcom.ibm.etools.mft.doc%2Fae19494_.htm
Hide
Permalink
Martin Konecny added a comment -
When file names aren't being saved as UTF-8 in the filesystem, media-monitor should refuse to start.
Show
Martin Konecny added a comment - When file names aren't being saved as UTF-8 in the filesystem, media-monitor should refuse to start.
Hide
Permalink
Cliff Wang added a comment -
Fixed, verified at Airtime 2.1.0+08d939d
Show
Cliff Wang added a comment - Fixed, verified at Airtime 2.1.0+08d939d

People

  • Assignee:
    Cliff Wang
    Reporter:
    Naomi
    Implemented by:
    Martin Konecny
    Tested by:
    Cliff Wang
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: