Summary | Mnemo2 locks up when encountering large notes |
Queue | Mnemo |
Queue Version | 2.1.1 |
Type | Bug |
State | Resolved |
Priority | 2. Medium |
Owners | |
Requester | jwinius (at) umrk (dot) to |
Created | 11/08/2007 (6530 days ago) |
Due | |
Updated | 11/17/2007 (6521 days ago) |
Assigned | 11/10/2007 (6528 days ago) |
Resolved | 11/17/2007 (6521 days ago) |
Github Issue Link | |
Github Pull Request | |
Milestone | |
Patch | No |
you've enabled page compression, etc.
mode for both Horde3 and Apache2, but that didn't seem to make any
difference. I'll keep an eye on this issue, though.
State ⇒ Resolved
it's download time. Make sure you've enabled page compression, etc.
It would be nice, though, if there was simply a way to disable the
previews -- something which I'm sure would also help the performance
when many notes are involved. I've got over 300, and while my old
system takes only one second to show the entire list, Mnemo2 on Horde3
takes almost four seconds despite newer, faster hardware.
is indeed your problem, the alpha (which is compatible with your
current Horde version) will fix it. Please test that and let us know.
Although the problem does have to do with the size of the notes, it
now appears not to be a problem with the size of the individual notes.
Rather, it seems to be caused by the total size of all of the notes.
Not wanting to wait for a solution, I wrote a perl script to process
my horde2 database dump and print out the sizes of the individual
mnemo notes. My intention was to identify the notes that were too
large and chop them up into smaller ones so that I could get mnemo2 to
work again. However, the printout showed that a number of other notes,
that were being successfully read by mnemo2, were much larger than the
34.6K size limit I mentioned earlier -- they were up to 77K in size.
Another test showed that if I started with a known non-working
situation -- leaving that larger 34.6K note intact on the end -- and
then deleted a few smaller notes (instead of making the 34.6K note
smaller), mnemo2 would suddenly start to work again.
I calculated the total number of bytes and found that if all of the
notes amounted to 1,252,799 bytes or less, mnemo2 would work. However,
if the total was 1,266,774 bytes or more, mnemo2 would not work. The
actual limit is somewhere in between.
I'm guessing, but could it be that this bug has something to do with
the mnemo2 preview function?
PHP errors (either in
the generated page, or in your server logs? Do you get apache
segfaulting? Do you get
javascript errors? Is the server unresponsive, or the browser? Etc...
is what the Horde3 log says when I start up mnemo (debug mode):
=== Begin Horde3 log ===
Nov 10 13:11:03 HORDE [debug] [] SQL Query by Prefs_sql::retrieve():
SELECT pref_scope, pref_name, pref_value FROM horde_prefs WHERE
pref_uid = ? AND (pref_scope = ? OR pref_scope = 'horde') ORDER BY
pref_scope [on line 137 of "/usr/share/horde3/lib/Horde/Prefs/sql.php"]
Nov 10 13:11:03 HORDE [debug] [mnemo] Hook _horde_hook_share_init in
application horde not called. [on line 1462 of
"/usr/share/horde3/lib/Horde.php"]
Nov 10 13:11:03 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND
(a1.attribute_name = ? AND a1.attribute_key = ?) GROUP BY
c.datatree_id, c.datatree_name, c.datatree_order ORDER BY
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:03 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND
((a1.attribute_name = ? AND a1.attribute_value = ?) OR
(a1.attribute_name = ? AND a1.attribute_key = ? AND CASE WHEN
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END) OR
(a1.attribute_name = ? AND CASE WHEN CAST(a1.attribute_value AS
VARCHAR) ~ '^-\?[0-9]+$' THEN (CAST(a1.attribute_value AS INTEGER) \&
?) <> 0 ELSE FALSE END) OR (a1.attribute_name = ? AND CASE WHEN
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END))
GROUP BY c.datatree_id, c.datatree_name, c.datatree_order ORDER BY
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:03 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::_loadById(): SELECT datatree_id, datatree_parents FROM
horde_datatree WHERE datatree_id IN (5) AND group_uid =
'horde.shares.mnemo' ORDER BY datatree_id [on line 342 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:03 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::_loadById(): SELECT datatree_id, datatree_name,
datatree_parents, datatree_order FROM horde_datatree WHERE datatree_id
IN (?, ?) AND group_uid = ? ORDER BY datatree_id [on line 375 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:03 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::getAttributes(): SELECT datatree_id, attribute_name AS
name, attribute_key AS "key", attribute_value AS value FROM
horde_datatree_attributes WHERE datatree_id IN (5) [on line 992 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:03 HORDE [debug] [mnemo] Mnemo_Driver_sql::retrieve():
SELECT * FROM mnemo_memos WHERE memo_owner = ? [on line 344 of
"/usr/share/horde3/mnemo/lib/Driver/sql.php"]
Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND
(a1.attribute_name = ? AND a1.attribute_key = ?) GROUP BY
c.datatree_id, c.datatree_name, c.datatree_order ORDER BY
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND
((a1.attribute_name = ? AND a1.attribute_value = ?) OR
(a1.attribute_name = ? AND a1.attribute_key = ? AND CASE WHEN
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END) OR
(a1.attribute_name = ? AND CASE WHEN CAST(a1.attribute_value AS
VARCHAR) ~ '^-\?[0-9]+$' THEN (CAST(a1.attribute_value AS INTEGER) \&
?) <> 0 ELSE FALSE END) OR (a1.attribute_name = ? AND CASE WHEN
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END))
GROUP BY c.datatree_id, c.datatree_name, c.datatree_order ORDER BY
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::getAttributes(): SELECT datatree_id, attribute_name AS
name, attribute_key AS "key", attribute_value AS value FROM
horde_datatree_attributes WHERE datatree_id IN (5) [on line 992 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::_buildParentIds(): SELECT datatree_id, datatree_parents
FROM horde_datatree WHERE datatree_name = ? AND group_uid = ? ORDER BY
datatree_id [on line 261 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::_load(): SELECT c.datatree_id, c.datatree_name,
c.datatree_parents, c.datatree_order FROM horde_datatree c WHERE
c.group_uid = 'horde.perms' [on line 130 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND
(a1.attribute_name = ? AND a1.attribute_key = ?) GROUP BY
c.datatree_id, c.datatree_name, c.datatree_order ORDER BY
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::getByAttributes(): SELECT c.datatree_id, c.datatree_name
FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON
a1.datatree_id = c.datatree_id WHERE c.group_uid = ? AND
((a1.attribute_name = ? AND a1.attribute_value = ?) OR
(a1.attribute_name = ? AND a1.attribute_key = ? AND CASE WHEN
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END) OR
(a1.attribute_name = ? AND CASE WHEN CAST(a1.attribute_value AS
VARCHAR) ~ '^-\?[0-9]+$' THEN (CAST(a1.attribute_value AS INTEGER) \&
?) <> 0 ELSE FALSE END) OR (a1.attribute_name = ? AND CASE WHEN
CAST(a1.attribute_value AS VARCHAR) ~ '^-\?[0-9]+$' THEN
(CAST(a1.attribute_value AS INTEGER) \& ?) <> 0 ELSE FALSE END))
GROUP BY c.datatree_id, c.datatree_name, c.datatree_order ORDER BY
c.datatree_order, c.datatree_name, c.datatree_id [on line 1111 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
Nov 10 13:11:04 HORDE [debug] [mnemo] SQL Query by
DataTree_sql::getAttributes(): SELECT datatree_id, attribute_name AS
name, attribute_key AS "key", attribute_value AS value FROM
horde_datatree_attributes WHERE datatree_id IN (5) [on line 992 of
"/usr/share/horde3/lib/Horde/DataTree/sql.php"]
=== End Horde3 log ===
The apache2 log say a lot less (debug mode):
=== Begin Apache2 log ===
atheris.umrk.to - - [10/Nov/2007:13:17:53 +0100] "GET
/horde3/mnemo/?Horde=06118e37f6e6c0f6e81582eefafb19b7 HTTP/1.1" 200 -
"https://webmail.umrk.to/horde3/services/portal/sidebar.php?Horde=06118e37f6e6c0f6e81582eefafb19b7" "Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.8 (like Gecko)
(Debian)"
=== End Apache2 log ===
Otherwise, I have not found any particular errors associated with this
problem. Although the Horde3 iconbar above becomes unresponsive, the
sidebar continues to work, so it is not a problem to switch to another
Horde3 application. The rest of the browser and all other open pages
are also totally unaffected.
State ⇒ Feedback
Priority ⇒ 2. Medium
PHP errors (either in the generated page, or in your server logs? Do
you get apache segfaulting? Do you get javascript errors? Is the
server unresponsive, or the browser? Etc...
Priority ⇒ 3. High
State ⇒ Unconfirmed
Queue ⇒ Mnemo
Summary ⇒ Mnemo2 locks up when encountering large notes
Type ⇒ Bug
notice that mnemo2 locks up if any of the notes contain more than
about 33.6 KB of text. If there is even one note in several hundred
that is slightly too large, the list of notes will not be generated
and the icons across the top of the page will either stop working
(Konqueror), or the entire icon bar will disappear (Firefox).
In such cases I have found that if I use phppgadmin to edit the
offending text field and reduce it slightly in size, mnemo2 will
suddenly start to work again. It will even allow me to save extra text
in the same note again, after which it will once again cease to
function as a result.
Curiously, if I use Konqueror the limit for mnemo2 is a text size of
34606 bytes, with 34607 causing the application to lock up. However,
when using Firefox (Iceweasel) the maximum supported size was 34485
bytes, while for both Mozilla (Iceape) and Epiphany it was 34489 bytes.
My horde2 system is still running on a Debian sarge machine with
postgresql v7.4.7 and apache-ssl v1.3.33. So far, I am not aware that
mnemo v1.1 has any such text size limitations. My new horde3 system is
on Debian etch with postgresql v7.4.17 and apache2 v2.2.3. The
browsers are on a workstation running Debian sid.