6.0.0-git
2019-03-19

[#13149] Filter rules fail depending on mumber of rules and all/any setting
Summary Filter rules fail depending on mumber of rules and all/any setting
Queue Ingo
Queue Version 3.1.4
Type Bug
State Not A Bug
Priority 1. Low
Owners slusarz (at) horde (dot) org
Requester aria (at) bsc (dot) gwu (dot) edu
Created 2014-04-25 (1789 days ago)
Due
Updated 2014-05-08 (1776 days ago)
Assigned 2014-04-28 (1786 days ago)
Resolved 2014-05-08 (1776 days ago)
Milestone
Patch No

History
2014-05-08 18:55:05 Michael Slusarz State ⇒ Not A Bug
 
2014-05-08 18:54:58 Michael Slusarz Comment #8 Reply to this comment
The IMAP server does support US-ASCII so this must be a bug in the 
server code.
For the record, this is what the IMAP server is reporting also:

  S: 3 NO [BADCHARSET (US-ASCII)] unsupported CHARSET specified

The optional parenthesized list to BADCHARSET response code lists the 
search charsets available on the server.  US-ASCII is correctly listed 
(notable, your server doesn't support UTF-8).

I would guess the server is either misparsing the syntax of the search 
query -- using one of the parsed atoms, like UNDELETED, as the CHARSET 
incorrectly -- or else it's US-ASCII parsing code of the search values 
is broken -- e.g., it is misinterpreting "Webmail" as something other 
than US-ASCII.
2014-05-08 13:13:38 aria (at) bsc (dot) gwu (dot) edu Comment #7 Reply to this comment
Thanks Michael for your analysis.  I will report this as a bug to the 
IMAP vendor.  The IMAP server does support US-ASCII so this must be a 
bug in the server code.

Thanks and please close this ticket.
2014-05-08 05:31:49 Michael Slusarz Comment #6 Reply to this comment
All strings in this query are in US-ASCII, so if the IMAP server is 
misparsing this SEARCH string, there's not much we can do about it.
testing on Dovecot 2.2:

3 UID SEARCH UNDELETED OR (SUBJECT Hello) (OR (SUBJECT BSC) (SUBJECT Webmail))
* SEARCH 59466 62431 62432 62452 62530 62531 62568 62573 62590 62596 
62606 62607 62625 62639
3 OK Search completed (0.000 secs).

Those are actually correct results in my mailbox...  I have a bunch of 
messages from a client's ticket queue that contain "Webmail" in the 
subject line, so it is correctly returning the set of messages that 
are undeleted and contain subject "Webmail" (Hello and BSC aren't 
matching).
2014-05-08 05:25:29 Michael Slusarz Comment #5
Assigned to Michael Slusarz
Reply to this comment
Your IMAP server is broken.

Your AND search:

C: 3 UID SEARCH UNDELETED SUBJECT Webmail SUBJECT BSC SUBJECT "BSC Webmail"
S: * SEARCH 620
S: 3 OK UID SEARCH completed

This is OK.

Results from OR search:

C: 3 UID SEARCH UNDELETED OR (SUBJECT Hello) (OR (SUBJECT BSC) 
(SUBJECT Webmail))
S: 3 NO [BADCHARSET (US-ASCII)] unsupported CHARSET specified
Command 3 took 0.0041 seconds.
Specifying a charset in a SEARCH/UID SEARCH command is optional.  In 
lieu of specifying charset, the server *MUST* support US-ASCII.  See 
RFC 3501[6.4.4]: "US-ASCII MUST be supported; other [CHARSET]s MAY be 
supported.".

All strings in this query are in US-ASCII, so if the IMAP server is 
misparsing this SEARCH string, there's not much we can do about it.

(Note that the query generated by Ingo is actually: "All UNDELETED 
messages with a subject of either Hello, BSC, or Webmail".
But this is expected, as the UNDELETED query is added to the 
user-defined query - we don't filter messages already marked as 
deleted.)

In short, I guess I don't see this as an issue to be fixed in Horde 
but rather something that needs to be fixed in the IMAP server instead.
2014-05-05 17:01:05 aria (at) bsc (dot) gwu (dot) edu Comment #4
New Attachment: Trace.zip Download
Reply to this comment
My attachment to the previous message didn't make it but I just 
noticed why.  I completed the comment section, then attached the file 
then hit submit but had forgotten to fill in my e-mail and the spam 
protection filed.  As a result, the attachment was removed when I 
received the error about the missing fields.  Not sure if this a a bug!
Thanks.
2014-05-05 16:56:55 aria (at) bsc (dot) gwu (dot) edu Comment #3 Reply to this comment
To further debug this issue, we need details of the IMP/Ingo -> 
IMAP/POP communication.
Sorry for the delay.  I have attached one zip file containing two 
traces.  One is for a successful AND filter with three anded 
requirements and a second for a failed OR filter with three 
requirements.

The trace is started at user login and stopped after the filter is 
applied.  The OR filter generates no message at the user end but the 
following log is written:

ERR: HORDE [imp] Charset used in search query is not supported on the 
mail  server. unsupported CHARSET specified [pid 6797 on line 570 of 
"/data/www/horde/imp/lib/Imap.php"]


Please let me know if I can provide anything else.
Thanks,
Aria
2014-04-28 05:36:39 Michael Slusarz Comment #2
State ⇒ Feedback
Reply to this comment
To further debug this issue, we need details of the IMP/Ingo -> 
IMAP/POP communication.

To enable debugging, see instructions contained in 
imp/config/backends.php (the 'debug' config parameter).

Debugging should not be enabled on a production server,   Attach/post 
only the portion of the log that directly deals with the problem 
reported (it may be simplest to clear the log file and then perform 
the event that causes the error).
2014-04-25 15:02:16 aria (at) bsc (dot) gwu (dot) edu Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 1. Low
Summary ⇒ Filter rules fail depending on mumber of rules and all/any setting
Queue ⇒ Ingo
Milestone ⇒
Patch ⇒ No
Reply to this comment
Hi,
Let me preface by saying that my IMAP server does not support UTF8 
characters and this is how I noticed this bug.

Create a filter rule as follows:

1-Select 'ANY of the following'
2-For field, pick anything (I used Subject or Sender).  Then select 
'Contains', then type in any string to check.
3-Now add two more like the one in the above step for a total of 3 
checks to be 'ORed' together in this single filter rule.
4-Save the rule.

When a rule contains 3 or more checks as in the above and they are 
ORed, you will get this error on the server and no filter is executed 
and no response or error is given to the user:

ERR: HORDE [imp] Charset used in search query is not supported on the 
mail server. unsupported CHARSET specified [pid 18030 on line 570 of 
"/data/www/horde/imp/lib/Imap.php"]

Here is the interesting part:

a) If you edit the rule and remove just one of the criterion, leaving 
just 2 possible checks.  Then the rule executes without error.

OR

b) if you edit the rule with the 3 or more criteria and change the 
selection from 'Any' to 'All the following' causing them to be 'ANDed' 
then all is well and the rule executes with 3 or more criteria!

Thanks,

Aria

Saved Queries