6.0.0-beta1
7/7/25

[#11899] Horde_String::validUtf8() doesn't work for servers with small stacks
Summary Horde_String::validUtf8() doesn't work for servers with small stacks
Queue Horde Framework Packages
Queue Version Git master
Type Bug
State Resolved
Priority 1. Low
Owners slusarz (at) horde (dot) org
Requester mm (at) freebsd (dot) org
Created 12/17/2012 (4585 days ago)
Due
Updated 12/20/2012 (4582 days ago)
Assigned 12/20/2012 (4582 days ago)
Resolved 12/20/2012 (4582 days ago)
Github Issue Link
Github Pull Request
Milestone
Patch No

History
12/20/2012 09:06:13 PM Michael Slusarz Comment #13
State ⇒ Resolved
Reply to this comment
Fixed in Horde_Util 2.0.2.
12/20/2012 08:58:25 PM Git Commit Comment #12 Reply to this comment
Changes have been made in Git (master):

commit 4f849a390cd6be890d43793110bc531d50f9ba91
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Thu Dec 20 13:58:14 2012 -0700

     Bug #11899: Cast to string or else objects will continually call 
the __toString() magic method

  framework/Util/lib/Horde/String.php |    2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)

http://git.horde.org/horde-git/-/commit/4f849a390cd6be890d43793110bc531d50f9ba91
12/20/2012 08:10:57 PM Git Commit Comment #11 Reply to this comment
Changes have been made in Git (master):

commit 91c3666f133ad96cd15cf24ee3e881130458e1e6
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Thu Dec 20 13:10:08 2012 -0700

     [mms] Convert Horde_String#validUtf8() from a preg-regex to a 
string iterator to eliminate PCRE issues relating to stack sizes (Bug 
#11899).

  framework/Util/lib/Horde/String.php           |   61 
++++++++++++-------------
  framework/Util/package.xml                    |    4 +-
  framework/Util/test/Horde/Util/StringTest.php |   37 +++++++++++++++
  3 files changed, 68 insertions(+), 34 deletions(-)

http://git.horde.org/horde-git/-/commit/91c3666f133ad96cd15cf24ee3e881130458e1e6
12/20/2012 07:35:54 PM Michael Slusarz Assigned to Michael Slusarz
Summary ⇒ Horde_String::validUtf8() doesn't work for servers with small stacks
 
12/20/2012 07:34:27 PM Michael Slusarz Comment #10
State ⇒ Assigned
Queue ⇒ Horde Framework Packages
Version ⇒ Git master
Reply to this comment
For the record - we already have code in place to prevent stack overflows:

commit 7053f7ae5f7f735d288618d7a54e15c658d6e34c
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Wed May 9 14:01:11 2012 -0600

     [mms] Fix Horde_String::validUtf8() to workaround PHP/PCRE bug 
where segfault will occur on input strings > ~5000 characters

See: https://bugs.php.net/bug.php?id=37793

(We are even mentioned by name in that bug report as a proper way to 
workaround this issue!)

This is not an IMP issue - it is an issue with Horde_String::validUtf8().
12/20/2012 08:14:40 AM oliver (at) freebsd (dot) org Comment #9 Reply to this comment
http://stackoverflow.com/questions/7620910/regexp-in-preg-match-function-returning-browser-error

Has a wonderfull explanation.
The default pcre.recursion_limit on PHP is quite to high.

Setting pcre.recursion_limit in php.ini to a reasonable value like 
1048 makes the bug go away.

5554 is the highest value I can set. with 5555 the test case segfaults 
again. No idea what the cause is, my stacksize is 512MB regarding to 
ulimit -s
12/20/2012 07:34:37 AM oliver (at) freebsd (dot) org Comment #8 Reply to this comment
I executed the PHPUnit test cases.

testLargeEnvelopeData()
and
testBug11899()

causing PHP to segfault. Unfortunally no debugging symbols, but it 
seems to be a Regexp-Problem:

#0  0x0000000800d0fc5d in match () from /usr/local/lib/libpcre.so.3
[...]
#5554 0x0000000800d0fe25 in match () from /usr/local/lib/libpcre.so.3
#5555 0x0000000800d12040 in match () from /usr/local/lib/libpcre.so.3
#5556 0x0000000800d21733 in pcre_exec () from /usr/local/lib/libpcre.so.3
#5557 0x00000000004539f2 in php_pcre_match_impl ()
#5558 0x0000000000454428 in php_do_pcre_match ()
#5559 0x00000000005a47b6 in zend_do_fcall_common_helper_SPEC ()
#5560 0x00000000005aa75d in execute ()

match() is being called 5555 times according to the backtrace of gdb
12/20/2012 07:07:24 AM oliver (at) freebsd (dot) org Comment #7
New Attachment: debug.txt Download
Reply to this comment
I have the same error while Running PHP 5.4.9, FreeBSD 9, Qmail, 
Vpopmail + Dovecot.

Please find attached the IMAP debug output from generating the 
Maildir-Index in IMP.

The error happens in Horde/Imap/Client/Socket.php Method _parseEnvelop 
when doing the final assignment:

    $ret->$env_data[$key] = $tmp;

The attached httpd process dies while $tmp being kinda high (in my 
case 4826 chars while my $key index was 35)

   4561 httpd    NAMI  "/usr/share/locale/de_DE.UTF-8/LC_CTYPE"
   4561 httpd    RET   open 28/0x1c
....
   4561 httpd    CALL  read(0x1c,0x80f457000,0x8000)
   4561 httpd    GIO   fd 28 read 4096 bytes
....
   4561 httpd    RET   read 10596/0x2964
   4561 httpd    CALL  close(0x1c)
   4561 httpd    RET   close 0
   4561 httpd    PSIG  SIGSEGV caught handler=0x801298180 mask=0x0 code=0x2
12/20/2012 05:46:20 AM Michael Slusarz Comment #6 Reply to this comment
I've added a unit test.  It can be run (via a git checkout) by going 
to the Imap_Client/test/Horde/Imap/Client directory and running:

phpunit SocketTest.php

Obviously, PHPUnit must be installed on your system.
12/20/2012 05:44:13 AM Git Commit Comment #5 Reply to this comment
Changes have been made in Git (master):

commit 13e6032279cb629d54db3675c523452bd0c16474
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Wed Dec 19 22:44:03 2012 -0700

     Bug #11899: Add unit test (which passes)

  .../test/Horde/Imap/Client/SocketTest.php          |   14 ++++++++++++++
  1 files changed, 14 insertions(+), 0 deletions(-)

http://git.horde.org/horde-git/-/commit/13e6032279cb629d54db3675c523452bd0c16474
12/20/2012 04:47:48 AM Michael Slusarz Comment #4 Reply to this comment
Update: cannot reproduce with original message.  Thinking this is 
either a PHP or charset related problem with the host system (or 
both).  Problem appears to be with processing ENVELOPE data with 
(multiple?) MIME-Encoded addresses.  Waiting for further information 
from OP.
12/17/2012 08:38:09 PM Michael Slusarz Comment #3
Priority ⇒ 1. Low
State ⇒ Feedback
Reply to this comment
Works fine here.  At a minimum will need an IMAP debug log.



To further debug this issue, we need details of the IMP -> IMAP/POP 
communication.

To enable debugging, see instructions contained in 
imp/config/backends.php (the 'debug' config parameter).

Debugging should not be enabled on a production server,   Attach/post 
only the portion of the log that directly deals with the problem 
reported (it may be simplest to clear the log file and then perform 
the event that causes the error).
12/17/2012 07:40:44 PM mm (at) freebsd (dot) org Comment #2
New Attachment: mail.txt Download
Reply to this comment
Messages with many recipients or message indexes with such messages 
bail out with "Error communicating with server". Even highest debug 
level doesn't provide any hints. IMAP server used is Cyrus IMAPD. 
Other mailclients work well.

Error did not happen with horde4 (imp 5.x), sample stripped IMAP 
message causing such an error is attached.
12/17/2012 07:39:38 PM mm (at) freebsd (dot) org Comment #1
Priority ⇒ 2. Medium
State ⇒ Unconfirmed
Patch ⇒ No
Milestone ⇒
Summary ⇒ IMP cannot message or generate message index and/or read IMAP message if many recipients
Type ⇒ Bug
Queue ⇒ IMP
Reply to this comment
Messages with many recipients or message indexes with such messages 
bail out with "Error communicating with server". Even highest debug 
level doesn't provide any hints. IMAP server used is Cyrus IMAPD. 
Other mailclients work well.

Error did not happen with horde4 (imp 5.x), sample stripped IMAP 
message causing such an error is attached.

Saved Queries