Json utf8 response object

In Horde 3.3.5 the response object from Kronolith is using w

Fri, 06 Nov 2009 17:35:05 +0000

In Horde 3.3.5 the response object from Kronolith is using wrong charset. If Horde API is used to Power External Sites like this one: http://theupstairsroom.com/67 Problem: Strings get cut off Reason: kronolith/lib/Driver/sql.php (line 453) Kronolith_Event_sql turns all utf8 strings into $GLOBALS['_HORDE_STRING_CHARSET'] = 'iso-8859-1'; Solution: /lib/Horde/RPC/jsonrpc.php class Horde_RPC_jsonrpc(); function getResponse($request); Line 91 -- return Horde_Serialize::serialize($response, SERIALIZE_JSON); ++ return Horde_Serialize::serialize($response, SERIALIZE_JSON, $GLOBALS['_HORDE_STRING_CHARSET']);

> In Horde 3.3.5 the response object from Kronolith is using

Fri, 06 Nov 2009 19:09:02 +0000

> In Horde 3.3.5 the response object from Kronolith is using wrong charset. > > If Horde API is used to Power External Sites like this one: > http://theupstairsroom.com/67 > > Problem: > Strings get cut off > > Reason: > kronolith/lib/Driver/sql.php (line 453) > Kronolith_Event_sql turns all utf8 strings into > $GLOBALS['_HORDE_STRING_CHARSET'] = 'iso-8859-1'; What version of Kronolith is this? That line number has nothing to do with charset conversion, and furthermore, nowhere do we use $GLOBALS['_HORDE_STRING_CHARSET']. Are you sure this isn't a local modification?

>> In Horde 3.3.5 the response object from Kronolith is usin

Sat, 07 Nov 2009 12:46:50 +0000

>> In Horde 3.3.5 the response object from Kronolith is using wrong charset. >> >> If Horde API is used to Power External Sites like this one: >> http://theupstairsroom.com/67 >> >> Problem: >> Strings get cut off >> >> Reason: >> kronolith/lib/Driver/sql.php (line 453) >> Kronolith_Event_sql turns all utf8 strings into >> $GLOBALS['_HORDE_STRING_CHARSET'] = 'iso-8859-1'; > > What version of Kronolith is this? Horde Groupware 1.2.4 Kronolith H3 (2.3.2) kronolith/lib/api.php,v 1.126.2.64 2009/09/04 10:38:36 jan Exp $ >That line number has nothing to do with charset conversion, > and furthermore, nowhere do we use > $GLOBALS['_HORDE_STRING_CHARSET']. Are you sure this isn't a local > modification? Procedure during the a json call server side: 1. json is calling function &_kronolith_eventFromUID($uid) --> kronolith/lib/api.php line 1133 2. the event object get fetched: $event = $GLOBALS['kronolith_driver']->getByUID($uid) --> kronolith/lib/api.php line 1137 ------------- Get an event or events with the given UID value in kronolith/lib/Driver/sql.php,v 1.213 2008/10/18 04:20:42 3. call function: &getByUID($uid, $calendars = null, $getAll = false) --> kronolith/lib/Driver/sql.php line 416 4. the event get cached : $this->_cache[$this->_calendar][$event['event_id']] = &new Kronolith_Event_sql($this, $event) --> kronolith/lib/Driver/sql.php line 453 5. call fromDriver(SQLEvent) in the class Kronolith_Event_sql method fromDriver($SQLEvent) kronolith/lib/Driver/sql.php line 871 6. call $driver->convertFromDriver($SQLEvent['event_description']) --> kronolith/lib/Driver/sql.php line 948 7. Converts a value from the default charset to the driver's charset String::convertCharset($value, $this->_params['charset']) kronolith/lib/Driver/sql.php line 803 ------------ The String:: class provides static methods for charset and locale safe framework/Util/String.php,v 1.43.6.37 2009/03/30 15:31:38 8.call function convertCharset($input, $from, $to = null) framework/Util/String.php line 76 on top of this file the $GLOBALS['_HORDE_STRING_CHARSET'] = 'iso-8859-1'; is set. ------------ This is way we get an iso-8859-1 formated event back from kronolith. json does not like iso-8859-1.This leads us back on track: ------------ framework/RPC/RPC/jsonrpc.php,v 1.9 2009/02/21 01:58:14 chuck Exp call function getResponse($request) --> framework/RPC/RPC/jsonrpc.php line 47 return Horde_Serialize::serialize($response, Horde_Serialize::JSON) --> framework/RPC/RPC/jsonrpc.php line 88 We have to make sure, that all strings in this responseheader are utf8. for this we should add the propper charset in Horde_Serialize::serialize($response, Horde_Serialize::JSON, "CHARSET") <--- or make somehow sure, that we get all respondehaeder for json utf8. ------------ $Horde: framework/Serialize/Serialize.php,v 1.25.10.16 2009/01/06 15:23:34 jan Exp $ // $params = Source character set case SERIALIZE_JSON in function _serialize($data, $mode, $params = null) --> framework/Serialize/Serialize.php line 264 regards roman

Changes have been made in CVS for this ticket: JSON is alwa

Sat, 07 Nov 2009 12:54:38 +0000

Changes have been made in CVS for this ticket: JSON is always UTF-8 (Bug #8685). http://cvs.horde.org/diff.php/framework/RPC/RPC/Attic/jsonrpc.php?rt=horde&r1=1.4.2.4&r2=1.4.2.5&ty=u

Please try what I committed.

Sat, 07 Nov 2009 12:55:23 +0000

Please try what I committed.

Changes have been made in Git for this ticket: JSON is alwa

Sat, 07 Nov 2009 17:47:32 +0000

Changes have been made in Git for this ticket: JSON is always UTF-8 (Bug #8685). http://git.horde.org/diff.php/framework/Rpc/lib/Horde/Rpc/Jsonrpc.php?rt=horde-git&r1=bfbf37ce80786f3a0d6c904509cc27d53698dd52&r2=bff9b6b204d6dac17d167c8aae9a64791c5535fd

> Changes have been made in CVS for this ticket: > > JSON

Sat, 07 Nov 2009 19:53:37 +0000

> Changes have been made in CVS for this ticket: > > JSON is always UTF-8 (Bug #8685). > http://cvs.horde.org/diff.php/framework/RPC/RPC/Attic/jsonrpc.php?rt=horde&r1=1.4.2.4&r2=1.4.2.5&ty=u this breaks the json request. --> Horde_NLS::setCharsetEnvironment('UTF-8');

> this breaks the json request. Care to tell us how?

Sun, 08 Nov 2009 10:38:13 +0000

> this breaks the json request. Care to tell us how?

>> this breaks the json request. > > Care to tell us how?

Sun, 08 Nov 2009 10:44:25 +0000

>> this breaks the json request. > > Care to tell us how? under investigation.... right now... I let you know as soon as i know more...

>>> this breaks the json request. >> >> Care to tell us ho

Mon, 09 Nov 2009 10:14:32 +0000

>>> this breaks the json request. >> >> Care to tell us how? [09-Nov-2009 11:06:04] PHP Fatal error: Class 'Horde_NLS' not found in /horde3/lib/Horde/RPC/jsonrpc.php on line 44

framework/RPC/RPC/jsonrpc.php,v 1.4.2.4 2009/01/06 15:23:32

Mon, 09 Nov 2009 10:43:19 +0000

framework/RPC/RPC/jsonrpc.php,v 1.4.2.4 2009/01/06 15:23:32 Line 91 + return Horde_Serialize::serialize($response, SERIALIZE_JSON, NLS::getCharset()); - return Horde_Serialize::serialize($response, SERIALIZE_JSON); Works for me.

Changes have been made in CVS for this ticket: Fix class na

Mon, 09 Nov 2009 13:20:21 +0000

Changes have been made in CVS for this ticket: Fix class name (Bug #8685). http://cvs.horde.org/diff.php/framework/RPC/RPC/Attic/jsonrpc.php?rt=horde&r1=1.4.2.5&r2=1.4.2.6&ty=u

How about now? > Line 91 > + return Horde_Serialize::ser

Mon, 09 Nov 2009 13:21:13 +0000

How about now? > Line 91 > + return Horde_Serialize::serialize($response, SERIALIZE_JSON, > NLS::getCharset()); > - return Horde_Serialize::serialize($response, SERIALIZE_JSON); There is no such line.

> How about now? > >> Line 91 >> + return Horde_Serialize

Mon, 09 Nov 2009 13:40:01 +0000

> How about now? > >> Line 91 >> + return Horde_Serialize::serialize($response, SERIALIZE_JSON, >> NLS::getCharset()); >> - return Horde_Serialize::serialize($response, SERIALIZE_JSON); > > There is no such line. Jsonrpc.php, which is delivered 2009-09-14: Horde Groupware 1.2.4: Might be this one: http://cvs.horde.org/h/chora/annotate.php/framework/RPC/RPC/Attic/jsonrpc.php?sa=1&rev=1.5

> Changes have been made in CVS for this ticket: > > Fix c

Mon, 09 Nov 2009 13:59:17 +0000

> Changes have been made in CVS for this ticket: > > Fix class name (Bug #8685). > http://cvs.horde.org/diff.php/framework/RPC/RPC/Attic/jsonrpc.php?rt=horde&r1=1.4.2.5&r2=1.4.2.6&ty=u This does not change the charsetcoding of strings in $response. @return string The JSON encoded response from the server. Still the same behavior for me.

Please download a fresh copy of this file, revision 1.4.2.6.

Mon, 09 Nov 2009 14:14:07 +0000

Please download a fresh copy of this file, revision 1.4.2.6. You might still have local modifications that interfere.

> Please download a fresh copy of this file, revision 1.4.2.

Mon, 09 Nov 2009 14:33:54 +0000

> Please download a fresh copy of this file, revision 1.4.2.6. You > might still have local modifications that interfere. Still no luck. Strings get cut off. $response is still in ISO-8859-1 Will do a fresh install. Hope that helps.

> In Horde 3.3.5 the response object from Kronolith is using

Mon, 09 Nov 2009 16:13:31 +0000

> In Horde 3.3.5 the response object from Kronolith is using wrong charset. > > If Horde API is used to Power External Sites like this one: > http://theupstairsroom.com/67 > > Problem: > Strings get cut off > > Reason: > kronolith/lib/Driver/sql.php (line 453) > Kronolith_Event_sql turns all utf8 strings into > $GLOBALS['_HORDE_STRING_CHARSET'] = 'iso-8859-1'; I can confirm this solution (thanks to Roman too). It solves the described problem for me (had that problems with strings containing german 'umlauts') - i use JSON with python-simplejson (in ZOPE) externally. With XMLRPC i had not such charset problems. good work ;) hth cheers, Niels --- Niels Dettenbach http://www.syndicat.com http://www.stadtradio-goettingen.de

I can see clearly now what the patch of Jan is trying to do.

Mon, 09 Nov 2009 17:12:11 +0000

I can see clearly now what the patch of Jan is trying to do. :-) The muche smarter way as i tried. Nomore back and forth conversions. In theory this should fix the issue. Unfortunately in the setCharsetEnvironment(){} function is another "bug". (in NLS.php) the NLS::setLang(); turns the charset back to iso, before the utf-8 can take action. Regards roman

> the NLS::setLang(); turns the charset back to iso, > befo

Sat, 10 Apr 2010 12:30:11 +0000

> the NLS::setLang(); turns the charset back to iso, > before the utf-8 can take action. Only if your system locales don't support UTF-8. Install UTF-8 locales.

>> the NLS::setLang(); turns the charset back to iso, >> be

Sat, 10 Apr 2010 12:51:19 +0000

>> the NLS::setLang(); turns the charset back to iso, >> before the utf-8 can take action. > > Only if your system locales don't support UTF-8. Install UTF-8 locales. there was a problem with the test environment client server on the same machine, virtual hosting etc... I tested this on a productive machines and as fare is i can see the json is working as expected. thanks.