6.0.0-beta1
7/5/25

[#8685] Json utf8 response object
Summary Json utf8 response object
Queue Horde Framework Packages
Queue Version FRAMEWORK_3
Type Bug
State Resolved
Priority 2. Medium
Owners
Requester roman (at) stachura (dot) ch
Created 11/06/2009 (5720 days ago)
Due
Updated 04/10/2010 (5565 days ago)
Assigned 11/06/2009 (5720 days ago)
Resolved 04/10/2010 (5565 days ago)
Github Issue Link
Github Pull Request
Milestone 3.3.7
Patch No

History
04/10/2010 12:51:19 PM stachrom (at) gmail (dot) com Comment #21 Reply to this comment
the NLS::setLang(); turns the charset back to iso,
before the utf-8 can take action.
Only if your system locales don't support UTF-8. Install UTF-8 locales.
there was a problem with the test environment client server on the 
same machine,  virtual hosting etc...

I tested this on a productive machines and as fare is i can see the   
json is working as expected.

thanks.
04/10/2010 12:30:11 PM Jan Schneider Comment #20
State ⇒ Resolved
Reply to this comment
the NLS::setLang(); turns the charset back to iso,
before the utf-8 can take action.
Only if your system locales don't support UTF-8. Install UTF-8 locales.
12/17/2009 12:12:11 PM Jan Schneider Milestone ⇒ 3.3.7
 
11/09/2009 05:12:11 PM roman (at) stachura (dot) ch Comment #19 Reply to this comment
I can see clearly now what the patch of Jan is trying to do. :-)

The muche smarter way as i tried. Nomore back and forth conversions.
In theory this should fix the issue.

Unfortunately in the setCharsetEnvironment(){} function is another "bug".
(in NLS.php)

the NLS::setLang(); turns the charset back to iso,
before the utf-8 can take action.

Regards roman
11/09/2009 04:13:31 PM nd (at) syndicat (dot) com Comment #18 Reply to this comment

[Show Quoted Text - 12 lines]
I can confirm this solution (thanks to Roman too). It solves the 
described problem for me (had that problems with strings containing 
german 'umlauts') - i use JSON with python-simplejson (in ZOPE) 
externally. With XMLRPC i had not such charset problems.

good work ;)

hth
cheers,


Niels
---
Niels Dettenbach
http://www.syndicat.com
http://www.stadtradio-goettingen.de
11/09/2009 02:33:54 PM roman (at) stachura (dot) ch Comment #17 Reply to this comment
Please download a fresh copy of this file, revision 1.4.2.6. You 
might still have local modifications that interfere.
Still no luck. Strings get cut off.  $response is still in ISO-8859-1
Will do a fresh install.  Hope that helps.





11/09/2009 02:14:07 PM Jan Schneider Comment #16 Reply to this comment
Please download a fresh copy of this file, revision 1.4.2.6. You might 
still have local modifications that interfere.
11/09/2009 01:59:17 PM roman (at) stachura (dot) ch Comment #15 Reply to this comment
This does not change the charsetcoding of strings in $response.
@return string  The JSON encoded response from the server.

Still the same behavior for me.
11/09/2009 01:40:01 PM roman (at) stachura (dot) ch Comment #14 Reply to this comment
How about now?
Line 91
+ return Horde_Serialize::serialize($response, SERIALIZE_JSON,
NLS::getCharset());
- return Horde_Serialize::serialize($response, SERIALIZE_JSON);
There is no such line.
Jsonrpc.php, which is delivered 2009-09-14: Horde Groupware 1.2.4:

Might be this one:
http://cvs.horde.org/h/chora/annotate.php/framework/RPC/RPC/Attic/jsonrpc.php?sa=1&rev=1.5




11/09/2009 01:21:13 PM Jan Schneider Comment #13 Reply to this comment
How about now?
Line 91
+ return Horde_Serialize::serialize($response, SERIALIZE_JSON, 
NLS::getCharset());
- return Horde_Serialize::serialize($response, SERIALIZE_JSON);
There is no such line.
11/09/2009 01:20:21 PM CVS Commit Comment #12 Reply to this comment
11/09/2009 10:43:19 AM roman (at) stachura (dot) ch Comment #11 Reply to this comment
framework/RPC/RPC/jsonrpc.php,v 1.4.2.4 2009/01/06 15:23:32

Line 91
+ return Horde_Serialize::serialize($response, SERIALIZE_JSON, 
NLS::getCharset());
- return Horde_Serialize::serialize($response, SERIALIZE_JSON);

Works for me.
11/09/2009 10:14:32 AM roman (at) stachura (dot) ch Comment #10 Reply to this comment
this breaks the json request.
Care to tell us how?
[09-Nov-2009 11:06:04] PHP Fatal error:  Class 'Horde_NLS' not found 
in /horde3/lib/Horde/RPC/jsonrpc.php on line 44

11/08/2009 10:44:25 AM roman (at) stachura (dot) ch Comment #9 Reply to this comment
this breaks the json request.
Care to tell us how?
under investigation.... right now...

I let you know as soon as i know more...

11/08/2009 10:38:13 AM Jan Schneider Comment #8 Reply to this comment
this breaks the json request.
Care to tell us how?
11/07/2009 07:53:37 PM roman (at) stachura (dot) ch Comment #7 Reply to this comment
this breaks the json request.
--> Horde_NLS::setCharsetEnvironment('UTF-8');
11/07/2009 12:55:23 PM Jan Schneider Comment #5
Summary ⇒ Json utf8 response object
Milestone ⇒ 3.3.6
Reply to this comment
Please try what I committed.
11/07/2009 12:54:38 PM CVS Commit Comment #4 Reply to this comment
Changes have been made in CVS for this ticket:

JSON is always UTF-8 (Bug #8685).
http://cvs.horde.org/diff.php/framework/RPC/RPC/Attic/jsonrpc.php?rt=horde&r1=1.4.2.4&r2=1.4.2.5&ty=u
11/07/2009 12:46:50 PM roman (at) stachura (dot) ch Comment #3 Reply to this comment

[Show Quoted Text - 14 lines]
Horde Groupware 1.2.4
Kronolith H3 (2.3.2)
kronolith/lib/api.php,v 1.126.2.64 2009/09/04 10:38:36 jan Exp $
That line number has nothing to do with charset conversion,
and furthermore, nowhere do we use 
$GLOBALS['_HORDE_STRING_CHARSET']. Are you sure this isn't a local 
modification?
Procedure during the a json call server side:

1. json is calling function &_kronolith_eventFromUID($uid) --> 
kronolith/lib/api.php line 1133
2. the event object get fetched:  $event = 
$GLOBALS['kronolith_driver']->getByUID($uid) --> kronolith/lib/api.php 
line 1137
-------------
    Get an event or events with the given UID value in   
kronolith/lib/Driver/sql.php,v 1.213 2008/10/18 04:20:42
    3. call function: &getByUID($uid, $calendars = null, $getAll = 
false) --> kronolith/lib/Driver/sql.php line 416
    4. the event get cached : 
$this->_cache[$this->_calendar][$event['event_id']] = &new 
Kronolith_Event_sql($this, $event) --> kronolith/lib/Driver/sql.php 
line 453
    5. call fromDriver(SQLEvent) in the class Kronolith_Event_sql 
method fromDriver($SQLEvent) kronolith/lib/Driver/sql.php line 871
    6. call $driver->convertFromDriver($SQLEvent['event_description']) 
--> kronolith/lib/Driver/sql.php line 948
    7. Converts a value from the default charset to the driver's 
charset String::convertCharset($value, $this->_params['charset']) 
kronolith/lib/Driver/sql.php line 803
------------
       The String:: class provides static methods for charset and 
locale safe framework/Util/String.php,v 1.43.6.37 2009/03/30 15:31:38
       8.call function convertCharset($input, $from, $to = null) 
framework/Util/String.php line 76
       on top of this file the $GLOBALS['_HORDE_STRING_CHARSET'] = 
'iso-8859-1'; is set.
------------
This is way we get an iso-8859-1 formated event back from kronolith.

json does not like iso-8859-1.This leads us back on track:
------------
framework/RPC/RPC/jsonrpc.php,v 1.9 2009/02/21 01:58:14 chuck Exp

call function getResponse($request) --> framework/RPC/RPC/jsonrpc.php line 47
return Horde_Serialize::serialize($response, Horde_Serialize::JSON) 
--> framework/RPC/RPC/jsonrpc.php line 88

       We have to make sure, that all strings in this responseheader are utf8.
       for this we should add the propper charset in   
Horde_Serialize::serialize($response, Horde_Serialize::JSON, 
"CHARSET") <---
       or make somehow sure, that we get all respondehaeder for json utf8.

------------
    $Horde: framework/Serialize/Serialize.php,v 1.25.10.16 2009/01/06 
15:23:34 jan Exp $

    // $params = Source character set
    case SERIALIZE_JSON in function _serialize($data, $mode, $params = 
null) --> framework/Serialize/Serialize.php line 264


regards roman
11/06/2009 07:09:02 PM Michael Rubinsky Comment #2
State ⇒ Feedback
Reply to this comment

[Show Quoted Text - 12 lines]
What version of Kronolith is this? That line number has nothing to do 
with charset conversion, and furthermore, nowhere do we use 
$GLOBALS['_HORDE_STRING_CHARSET']. Are you sure this isn't a local 
modification?

11/06/2009 05:35:05 PM roman (at) stachura (dot) ch Comment #1
Priority ⇒ 2. Medium
State ⇒ Unconfirmed
Patch ⇒ No
Milestone ⇒ Horde 3.3.5
Summary ⇒ Json utf8 respons object
Type ⇒ Bug
Queue ⇒ Horde Framework Packages
Reply to this comment
In Horde 3.3.5 the response object from Kronolith is using wrong charset.

If Horde API is used to Power External Sites like this one:
http://theupstairsroom.com/67

Problem:
Strings get cut off

Reason:
kronolith/lib/Driver/sql.php (line 453)
Kronolith_Event_sql turns all utf8 strings into 
$GLOBALS['_HORDE_STRING_CHARSET'] = 'iso-8859-1';


Solution:

/lib/Horde/RPC/jsonrpc.php
class Horde_RPC_jsonrpc();
function getResponse($request);

Line 91
--   return Horde_Serialize::serialize($response, SERIALIZE_JSON);
++ return Horde_Serialize::serialize($response, SERIALIZE_JSON, 
$GLOBALS['_HORDE_STRING_CHARSET']);

Saved Queries