[#14710] Headers encoded wrongly when replying to certain senders
Summary Headers encoded wrongly when replying to certain senders
Queue Synchronization
Queue Version FRAMEWORK_5_2
Type Bug
State No Feedback
Priority 1. Low
Owners mrubinsk@horde.org
Requester alexh@boxed.no
Created 2017-10-16 (761 days ago)
Due
Updated 2018-11-21 (360 days ago)
Assigned 2017-10-28 (749 days ago)
Resolved 2018-11-21 (360 days ago)
Milestone
Patch No

Comments
alexh@boxed.no 2017-10-16 02:27:35
This is a followup on #14456. Here is what postfix spits out when an 
iPhone tires to reply to a problematic email:
Oct 16 04:15:08 popper postfix/submission/smtpd[29358]: NOQUEUE: 
reject: RCPT from localhost[127.0.0.1]: 550 5.1.1 
<=?utf-8?Q?"Lanesskog>: Recipient address rejected: User unknown in 
local recipient table; from=<alexh@boxed.no> to=<=?utf-8?Q?"Lanesskog> 
proto=ESMTP helo=<mail.boxed.no>

The original From-header from the mail storage is this:
From: =?utf-8?B?TGFuZXNza29nLCBKw7hyZ2Vu?= <Jorgen.Lanesskog@marinit.no>



alexh@boxed.no 2017-10-16 02:43:02
> This is a followup on #14456. Here is what postfix spits out when an 
> iPhone tires to reply to a problematic email:
> Oct 16 04:15:08 popper postfix/submission/smtpd[29358]: NOQUEUE: 
> reject: RCPT from localhost[127.0.0.1]: 550 5.1.1 
> <=?utf-8?Q?"Lanesskog>: Recipient address rejected: User unknown in 
> local recipient table; from=<alexh@boxed.no
> to=<=?utf-8?Q?"Lanesskog> proto=ESMTP helo=<mail.boxed.no>
>
> The original From-header from the mail storage is this:
> From: =?utf-8?B?TGFuZXNza29nLCBKw7hyZ2Vu?= <Jorgen.Lanesskog@marinit.no>
>
>

Also,doing tcpdump on lo0 with TLS disabled, shows this interaction 
between Horde and postfix:
[root@popper ~]# tshark -r utf8-fun.dump
Running as user "root" and group "root". This could be dangerous.
   1          0    127.0.0.1 -> 127.0.0.1    TCP 74 50784 > submission 
[SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=1439637087 
TSecr=0 WS=128
   2  380105375    127.0.0.1 -> 127.0.0.1    TCP 74 submission > 50784 
[SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 
TSval=1439637087 TSecr=1439637087 WS=128
   3          0    127.0.0.1 -> 127.0.0.1    TCP 66 50784 > submission 
[ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=1439637087 TSecr=1439637087
   4          0    127.0.0.1 -> 127.0.0.1    SMTP 99 S: 220 
mail.boxed.no ESMTP Postfix
   5          0    127.0.0.1 -> 127.0.0.1    TCP 66 50784 > submission 
[ACK] Seq=1 Ack=34 Win=43776 Len=0 TSval=1439637088 TSecr=1439637088
   6          0    127.0.0.1 -> 127.0.0.1    SMTP 86 C: EHLO mail.boxed.no
   7          0    127.0.0.1 -> 127.0.0.1    TCP 66 submission > 50784 
[ACK] Seq=34 Ack=21 Win=43776 Len=0 TSval=1439637088 TSecr=1439637088
   8          0    127.0.0.1 -> 127.0.0.1    SMTP 178 S: 250 
mail.boxed.no | 250 PIPELINING | 250 SIZE 20480000 | 250 ETRN | 250 
ENHANCEDSTATUSCODES | 250 8BITMIME | 250 DSN
   9          0    127.0.0.1 -> 127.0.0.1    SMTP 178 C: MAIL 
FROM:<alexh@boxed.no> SIZE=9623 | RCPT TO:<"=?utf-8?Q?\"Lanesskog"> | 
RCPT TO:<Jorgen.Lanesskog@marinit.no>
  10          0    127.0.0.1 -> 127.0.0.1    SMTP 195 S: 250 2.1.0 Ok 
| 550 5.1.1 <=?utf-8?Q?"Lanesskog>: Recipient address rejected: User 
unknown in local recipient table | 250 2.1.5 Ok
  11          0    127.0.0.1 -> 127.0.0.1    SMTP 72 C: RSET
  12          0    127.0.0.1 -> 127.0.0.1    SMTP 80 S: 250 2.0.0 Ok
  13          0    127.0.0.1 -> 127.0.0.1    SMTP 72 C: QUIT
  14          0    127.0.0.1 -> 127.0.0.1    SMTP 81 S: 221 2.0.0 Bye
  15          0    127.0.0.1 -> 127.0.0.1    TCP 66 submission > 50784 
[FIN, ACK] Seq=304 Ack=145 Win=43776 Len=0 TSval=1439637109 
TSecr=1439637108
  16          0    127.0.0.1 -> 127.0.0.1    TCP 66 50784 > submission 
[FIN, ACK] Seq=145 Ack=305 Win=44800 Len=0 TSval=1439637109 
TSecr=1439637109
  17          0    127.0.0.1 -> 127.0.0.1    TCP 66 submission > 50784 
[ACK] Seq=305 Ack=146 Win=43776 Len=0 TSval=1439637109 TSecr=1439637109

-A

Git Commit <commits@lists.horde.org> 2017-10-28 20:40:18
Changes have been made in Git (master):

commit a4cf059021fe2ccfdd6c520604230ab67db6634f
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Sat, 28 Oct 2017 16:40:00 -0400

Add (passing) test re: bug: 14710

  M test/Horde/Mime/HeadersTest.php

https://github.com/horde/Mime/commit/a4cf059021fe2ccfdd6c520604230ab67db6634f

Michael Rubinsky <mrubinsk@horde.org> 2017-10-28 22:12:45
I cannot reproduce this locally.

Please provide the full source of an email that exhibits this behavior.

alexh@boxed.no 2017-10-30 00:25:03
> I cannot reproduce this locally.
>
> Please provide the full source of an email that exhibits this behavior.

Do have a non-public place I can email the source of this and other 
emails that exhibit this behaviour?

They have in common that the person I'm replying to have [åøæÅØÆ] in 
their names.

-A

Michael Rubinsky <mrubinsk@horde.org> 2017-10-30 00:32:16

> Do have a non-public place I can email the source of this and other 
> emails that exhibit this behaviour?

mrubinsk@horde.org


Git Commit <commits@lists.horde.org> 2017-10-30 20:06:02
Changes have been made in Git (FRAMEWORK_5_2):

commit 4d9db500eeb53f6939ccf736f5e485d5a1df800f
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Mon, 30 Oct 2017 20:55:29 +0100

Add (passing) test re: bug: 14710

  M test/Horde/Mime/HeadersTest.php

https://github.com/horde/Mime/commit/4d9db500eeb53f6939ccf736f5e485d5a1df800f

alexh@boxed.no 2017-11-13 00:12:24
>> Do have a non-public place I can email the source of this and other
>> emails that exhibit this behaviour?
>
> mrubinsk@horde.org
>

Did the raw mail make sense of how the emails gets rejected?

-A

Michael Rubinsky <mrubinsk@horde.org> 2017-11-13 02:53:46
>>> Do have a non-public place I can email the source of this and other
>>> emails that exhibit this behaviour?
>>
>> mrubinsk@horde.org
>>
>
> Did the raw mail make sense of how the emails gets rejected?
>
> -A

Unfortunately, no, and testing a local reply to it from my EAS test 
devices works as expected.


alexh@boxed.no 2017-11-16 08:26:21
>>>> Do have a non-public place I can email the source of this and other
>>>> emails that exhibit this behaviour?
>>>
>>> mrubinsk@horde.org
>>>
>>
>> Did the raw mail make sense of how the emails gets rejected?
>>
>> -A
>
> Unfortunately, no, and testing a local reply to it from my EAS test 
> devices works as expected.
>

If you look at line 9 and 10 from the tcpdump provided a few entries 
down, clearly Horde generates something that the MTA can't parse 
properly, though?

-A

Michael Rubinsky <mrubinsk@horde.org> 2017-11-17 17:43:48

> If you look at line 9 and 10 from the tcpdump provided a few entries 
> down, clearly Horde generates something that the MTA can't parse 
> properly, though?

Yes, and that is what I cannot reproduce.

alexh@boxed.no 2017-12-11 09:27:27
>
>> If you look at line 9 and 10 from the tcpdump provided a few entries
>> down, clearly Horde generates something that the MTA can't parse
>> properly, though?
>
> Yes, and that is what I cannot reproduce.

As a last ditch, can I make one of the recepients I have issues with 
send you a test email directly and see if that makes your server 
behave the same?

-A

alexh@boxed.no 2017-12-11 09:34:46
>>
>>> If you look at line 9 and 10 from the tcpdump provided a few entries
>>> down, clearly Horde generates something that the MTA can't parse
>>> properly, though?
>>
>> Yes, and that is what I cannot reproduce.
>
> As a last ditch, can I make one of the recepients I have issues with 
> send you a test email directly and see if that makes your server 
> behave the same?
>
> -A

Also, I found this snipped in the postix documentation, that seems to 
explain it, and explains why you cannot reproduce it (you seem to run 
exim from some bounce messages I've got):

"When the Postfix SMTP server receives a message WITHOUT the SMTPUTF8 
request, Postfix handles the message as it has always done (at least 
that is the default, see autodetection below). Specifically, the 
Postfix SMTP server does not accept UTF-8 in the envelope sender 
domain name or envelope recipient domain name, and the Postfix SMTP 
client does not issue the SMTPUTF8 request when delivering that 
message to an SMTP or LMTP server that announces SMTPUTF8 support 
(again, that is the default). Postfix will accept UTF-8 in message 
header values and in the localpart of envelope sender and recipient 
addresses, because it has always done that."

The two first sentences seem to be the problem.

-A

Michael Rubinsky <mrubinsk@horde.org> 2017-12-25 04:25:24

> Also, I found this snipped in the postix documentation, that seems 
> to explain it, and explains why you cannot reproduce it (you seem to 
> run exim from some bounce messages I've got):

Nope, I run Postfix.


alexh@boxed.no 2018-11-21 15:18:40
>
>> Also, I found this snipped in the postix documentation, that seems to
>> explain it, and explains why you cannot reproduce it (you seem to run
>> exim from some bounce messages I've got):
>
> Nope, I run Postfix.
>

What further input can I give you to convince you this is still a 
problem and fully reproducible?