6.0.0-beta1
9/24/25

[#5874] Mnemo2 locks up when encountering large notes
Summary Mnemo2 locks up when encountering large notes
Queue Mnemo
Queue Version 2.1.1
Type Bug
State Resolved
Priority 2. Medium
Owners
Requester jwinius (at) umrk (dot) to
Created 11/08/2007 (6530 days ago)
Due
Updated 11/17/2007 (6521 days ago)
Assigned 11/10/2007 (6528 days ago)
Resolved 11/17/2007 (6521 days ago)
Github Issue Link
Github Pull Request
Milestone
Patch No

History
11/17/2007 08:06:50 PM jwinius (at) umrk (dot) to Comment #8 Reply to this comment
Which one, 2, or 2.2-alpha?
2.2-alpha. I could never load the entire list with 2.
Hardware probably isn't the issue here,  it's download time. Make sure
you've enabled page compression, etc.
I've installed the deflate module in Apache2 and turned off the debug 
mode for both Horde3 and Apache2, but that didn't seem to make any 
difference. I'll keep an eye on this issue, though.
11/17/2007 06:08:59 PM Chuck Hagenbuch Comment #7
State ⇒ Resolved
Reply to this comment
Which one, 2, or 2.2-alpha? Hardware probably isn't the issue here, 
it's download time. Make sure you've enabled page compression, etc.
11/17/2007 06:03:44 PM jwinius (at) umrk (dot) to Comment #6 Reply to this comment
Yes, swapping in Mnemo 2.2-alpha fixes the problem for me. Thanks!



It would be nice, though, if there was simply a way to disable the 
previews -- something which I'm sure would also help the performance 
when many notes are involved. I've got over 300, and while my old 
system takes only one second to show the entire list, Mnemo2 on Horde3 
takes almost four seconds despite newer, faster hardware.
11/17/2007 06:02:46 AM Chuck Hagenbuch Comment #5 Reply to this comment
Mnemo 2.2-alpha cuts previews to not be the whole note body. If that 
is indeed your problem, the alpha (which is compatible with your 
current Horde version) will fix it. Please test that and let us know.
11/16/2007 08:19:20 PM jwinius (at) umrk (dot) to Comment #4 Reply to this comment
After some more experimentation I've made an important discovery. 
Although the problem does have to do with the size of the notes, it 
now appears not to be a problem with the size of the individual notes. 
Rather, it seems to be caused by the total size of all of the notes.



Not wanting to wait for a solution, I wrote a perl script to process 
my horde2 database dump and print out the sizes of the individual 
mnemo notes. My intention was to identify the notes that were too 
large and chop them up into smaller ones so that I could get mnemo2 to 
work again. However, the printout showed that a number of other notes, 
that were being successfully read by mnemo2, were much larger than the 
34.6K size limit I mentioned earlier -- they were up to 77K in size.



Another test showed that if I started with a known non-working 
situation -- leaving that larger 34.6K note intact on the end -- and 
then deleted a few smaller notes (instead of making the 34.6K note 
smaller), mnemo2 would suddenly start to work again.



I calculated the total number of bytes and found that if all of the 
notes amounted to 1,252,799 bytes or less, mnemo2 would work. However, 
if the total was 1,266,774 bytes or more, mnemo2 would not work. The 
actual limit is somewhere in between.



I'm guessing, but could it be that this bug has something to do with 
the mnemo2 preview function?
11/10/2007 12:28:27 PM jwinius (at) umrk (dot) to Comment #3 Reply to this comment
Can you please be a little more specific than "lock up"? Do you get 
PHP errors (either in
the generated page, or in your server logs? Do you get apache 
segfaulting? Do you get
javascript errors? Is the server unresponsive, or the browser? Etc...
Only mnemo (including the icon bar above) becomes unresponsive. This 
is what the Horde3 log says when I start up mnemo (debug mode):



=== Begin Horde3 log ===



Nov 10 13:11:03 HORDE [debug] [] SQL Query by Prefs_sql::retrieve(): 
SELECT pref_scope, pref_name, pref_value FROM horde_prefs WHERE 
pref_uid = ? AND (pref_scope = ? OR pref_scope = 'horde') ORDER BY 
pref_scope [on line 137 of "/usr/share/horde3/lib/Horde/Prefs/sql.php"]

Nov 10 13:11:03 HORDE [debug] [mnemo] Hook _horde_hook_share_init in 
application horde not called. [on line 1462 of 
"/usr/share/horde3/lib/Horde.php"]

Nov 10 13:11:03 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name 
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON 
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND 
(a1.attribute_name = ? AND a1.attribute_key = ?)   GROUP BY 
c.datatree_id, c.datatree_name, c.datatree_order ORDER BY 
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:03 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name 
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON 
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND 
((a1.attribute_name = ? AND a1.attribute_value = ?) OR 
(a1.attribute_name = ? AND a1.attribute_key = ? AND CASE WHEN 
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN 
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END) OR 
(a1.attribute_name = ? AND CASE WHEN CAST(a1.attribute_value AS 
VARCHAR) ~ '^-\?[0-9]+$' THEN (CAST(a1.attribute_value AS INTEGER) \& 
?) <> 0 ELSE FALSE END) OR (a1.attribute_name = ? AND CASE WHEN 
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN 
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END))   
GROUP BY c.datatree_id, c.datatree_name, c.datatree_order ORDER BY 
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:03 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::_loadById(): SELECT datatree_id, datatree_parents FROM 
horde_datatree WHERE datatree_id IN (5) AND group_uid = 
'horde.shares.mnemo' ORDER BY datatree_id [on line 342 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:03 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::_loadById(): SELECT datatree_id, datatree_name, 
datatree_parents, datatree_order FROM horde_datatree WHERE datatree_id 
IN (?, ?) AND group_uid = ? ORDER BY datatree_id [on line 375 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:03 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::getAttributes(): SELECT datatree_id, attribute_name AS 
name, attribute_key AS "key", attribute_value AS value FROM 
horde_datatree_attributes WHERE datatree_id IN (5) [on line 992 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:03 HORDE [debug] [mnemo] Mnemo_Driver_sql::retrieve(): 
SELECT * FROM mnemo_memos WHERE memo_owner = ? [on line 344 of 
"/usr/share/horde3/mnemo/lib/Driver/sql.php"]

Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name 
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON 
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND 
(a1.attribute_name = ? AND a1.attribute_key = ?)   GROUP BY 
c.datatree_id, c.datatree_name, c.datatree_order ORDER BY 
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name 
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON 
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND 
((a1.attribute_name = ? AND a1.attribute_value = ?) OR 
(a1.attribute_name = ? AND a1.attribute_key = ? AND CASE WHEN 
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN 
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END) OR 
(a1.attribute_name = ? AND CASE WHEN CAST(a1.attribute_value AS 
VARCHAR) ~ '^-\?[0-9]+$' THEN (CAST(a1.attribute_value AS INTEGER) \& 
?) <> 0 ELSE FALSE END) OR (a1.attribute_name = ? AND CASE WHEN 
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN 
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END))   
GROUP BY c.datatree_id, c.datatree_name, c.datatree_order ORDER BY 
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::getAttributes(): SELECT datatree_id, attribute_name AS 
name, attribute_key AS "key", attribute_value AS value FROM 
horde_datatree_attributes WHERE datatree_id IN (5) [on line 992 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::_buildParentIds(): SELECT datatree_id, datatree_parents 
FROM horde_datatree WHERE datatree_name = ? AND group_uid = ? ORDER BY 
datatree_id [on line 261 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::_load(): SELECT c.datatree_id, c.datatree_name, 
c.datatree_parents, c.datatree_order FROM horde_datatree c WHERE 
c.group_uid = 'horde.perms'  [on line 130 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name 
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON 
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND 
(a1.attribute_name = ? AND a1.attribute_key = ?)   GROUP BY 
c.datatree_id, c.datatree_name, c.datatree_order ORDER BY 
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name 
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON 
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND 
((a1.attribute_name = ? AND a1.attribute_value = ?) OR 
(a1.attribute_name = ? AND a1.attribute_key = ? AND CASE WHEN 
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN 
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END) OR 
(a1.attribute_name = ? AND CASE WHEN CAST(a1.attribute_value AS 
VARCHAR) ~ '^-\?[0-9]+$' THEN (CAST(a1.attribute_value AS INTEGER) \& 
?) <> 0 ELSE FALSE END) OR (a1.attribute_name = ? AND CASE WHEN 
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN 
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END))   
GROUP BY c.datatree_id, c.datatree_name, c.datatree_order ORDER BY 
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]

Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by 
DataTree_sql::getAttributes(): SELECT datatree_id, attribute_name AS 
name, attribute_key AS "key", attribute_value AS value FROM 
horde_datatree_attributes WHERE datatree_id IN (5) [on line 992 of 
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]



=== End Horde3 log ===



The apache2 log say a lot less (debug mode):



=== Begin Apache2 log ===



atheris.umrk.to - - [10/Nov/2007:13:17:53 +0100] "GET 
/horde3/mnemo/?Horde=06118e37f6e6c0f6e81582eefafb19b7 HTTP/1.1" 200 - 
"https://webmail.umrk.to/horde3/services/portal/sidebar.php?Horde=06118e37f6e6c0f6e81582eefafb19b7" "Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.8 (like Gecko) 
(Debian)"



=== End Apache2 log ===



Otherwise, I have not found any particular errors associated with this 
problem. Although the Horde3 iconbar above becomes unresponsive, the 
sidebar continues to work, so it is not a problem to switch to another 
Horde3 application. The rest of the browser and all other open pages 
are also totally unaffected.
11/10/2007 03:59:07 AM Chuck Hagenbuch Comment #2
State ⇒ Feedback
Priority ⇒ 2. Medium
Reply to this comment
Can you please be a little more specific than "lock up"? Do you get 
PHP errors (either in the generated page, or in your server logs? Do 
you get apache segfaulting? Do you get javascript errors? Is the 
server unresponsive, or the browser? Etc...
11/08/2007 02:14:22 PM jwinius (at) umrk (dot) to Comment #1
Priority ⇒ 3. High
State ⇒ Unconfirmed
Queue ⇒ Mnemo
Summary ⇒ Mnemo2 locks up when encountering large notes
Type ⇒ Bug
Reply to this comment
While migrating a horde2 database (including mnemo v1.1) to horde3, I 
notice that mnemo2 locks up if any of the notes contain more than 
about 33.6 KB of text. If there is even one note in several hundred 
that is slightly too large, the list of notes will not be generated 
and the icons across the top of the page will either stop working 
(Konqueror), or the entire icon bar will disappear (Firefox).



In such cases I have found that if I use phppgadmin to edit the 
offending text field and reduce it slightly in size, mnemo2 will 
suddenly start to work again. It will even allow me to save extra text 
in the same note again, after which it will once again cease to 
function as a result.



Curiously, if I use Konqueror the limit for mnemo2 is a text size of 
34606 bytes, with 34607 causing the application to lock up. However, 
when using Firefox (Iceweasel) the maximum supported size was 34485 
bytes, while for both Mozilla (Iceape) and Epiphany it was 34489 bytes.



My horde2 system is still running on a Debian sarge machine with 
postgresql v7.4.7 and apache-ssl v1.3.33.  So far, I am not aware that 
mnemo v1.1 has any such text size limitations. My new horde3 system is 
on Debian etch with postgresql v7.4.17 and apache2 v2.2.3. The 
browsers are on a workstation running Debian sid.

Saved Queries