Correct.
No.

i see.
but 6.0 is horde5, correct?
so there won't be a "bugfix" [*] for horde4?

fmasdr
IOhannes

[*] my users are also affected by receiving broken AppleMail 
attachments. unfortunately more and more people out there in the wild 
are franzied by iStuff, so my users need a possibility to handle such 
broken attachments.

There won't be a 5.1 release, it's part of the 6.0 code base though.

[...]
i see that this affects imp5.1.
however current stable imp is 5.0.23

will this changeset propagate into a released version of imp (either 
bugfix:5.0.24, or release:5.1) and if so, could you estimate when?

btw, horde/imp are great

Changes have been made in Git (master):

commit dbc6169ae27f7c02a50c3abec17b77f9498f8586
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Sat Nov 19 01:41:34 2011 -0700

     [mms] Add ability to view all message parts in dynamic view 
(Request #9827).

  imp/docs/CHANGES                             |    1 +
  imp/js/dimpbase.js                           |   25 +++++++++++++++-
  imp/js/message-dimp.js                       |   15 ++++++++++
  imp/lib/Ajax/Application.php                 |   38 
++++++++++++++++++++++++++
  imp/lib/Contents.php                         |    4 ++-
  imp/lib/Views/ShowMessage.php                |    4 +++
  imp/message-dimp.php                         |    6 ++++
  imp/package.xml                              |    1 +
  imp/templates/dimp/index.inc                 |    5 +--
  imp/templates/dimp/javascript_defs.php       |    1 +
  imp/templates/dimp/message/message.html      |   16 ++++++-----
  imp/themes/default/dimp/screen.css           |    3 ++
  imp/themes/default/graphics/email_attach.png |  Bin 0 -> 744 bytes
  imp/themes/silver/dimp/screen.css            |    3 ++
  imp/themes/silver/graphics/email_attach.png  |  Bin 0 -> 744 bytes
  15 files changed, 109 insertions(+), 13 deletions(-)

http://git.horde.org/horde-git/-/commit/dbc6169ae27f7c02a50c3abec17b77f9498f8586

Allowing viewing of all message parts has been added to the dynamic 
view for IMP 5.1:

http://git.horde.org/horde-git/-/commit/dbc6169ae27f7c02a50c3abec17b77f9498f8586

I can see and download the pdf attachment now.
Thanks.

Changes have been made in Git for this ticket:

Bug #9827: Better determination of whether related part was referenced 
in the root part

  2 files changed, 12 insertions(+), 8 deletions(-)
http://git.horde.org/horde-git/-/commit/52ed28d4b5c2030725a40a500cfcf7fad4115815

Actually, it isn't related to the primary focus of this ticket 
(displaying attachments contained in the non-viewed alternative part). 
  This instead relates to the fix mentioned below for broken 
multipart/related messages.  The fix on place theoretically *should* 
show the attachment... except this message example is twice as broken 
as normal because the PDF part does not even contain a content-ID.

This should be fixed now.

using git and courier-imap 4.9.3

I think the issue we're having is related to this ticket.

the attached message contains 2 attachments (png, pdf). IMP doesn't 
provide a link to download the pdf file.

imaplog :
S: 4 OK [READ-WRITE] Ok
C: 5 UID FETCH 1723 (BODY[HEADER])
S: * 1 FETCH (UID 1723 BODY[HEADER] {1814}
S: [LITERAL DATA - 1814 bytes]
S: )
S: 5 OK FETCH completed.
C: 6 UID FETCH 1723 (BODYSTRUCTURE)
S: * 1 FETCH (UID 1723 BODYSTRUCTURE (("text" "html" ("charset" 
"utf-8") NIL NIL "8bit" 156 7 NIL NIL NIL)("image" "png" ("charset" 
"utf-8" "name" "logo001.png") 
"<logo001.png@e3132cc6aa18c08f87b3375c69db99e5>" NIL "base64" 92 NIL 
("attachment" ("filename" "logo001.png")) NIL)("application" "pdf" 
("charset" "utf-8" "name" "2111100221.pdf") NIL NIL "base64" 112 NIL 
("attachment" ("filename" "2111100221.pdf")) NIL) "related" ("charset" 
"utf-8" "boundary" "ba8632ed0d4705f58db3b6b58bc6daba8") NIL NIL))
S: 6 OK FETCH completed.
C: 7 UID FETCH 1723 (BODY.PEEK[1])
S: * 1 FETCH (UID 1723 BODY[1] {156}
S: [LITERAL DATA - 156 bytes]
S: )
S: 7 OK FETCH completed.
C: 8 UID FETCH 1723 (BODY.PEEK[HEADER])
S: * 1 FETCH (UID 1723 BODY[HEADER] {1814}
S: [LITERAL DATA - 1814 bytes]
S: )
S: 8 OK FETCH completed.

I am also looking for a solution to same problem. Our salary 
processing partner sends inline HTML using multipart/related messages. 
Due to confidentiality issues, I could not attach the message.

It wasn't intended to fix this issue.  It was intended (and does) fix 
this issue: A multipart/related message such as the following:

multipart/related:
   text/html
   index/png (not referenced in the text/html part).

Will allow index/png to be viewed.

This patch has nothing to do with the problem discussed elsewhere in 
the ticket.  As mentioned multiple times previously, you simply cannot 
treat non-viewable parts inside of a multipart/alternative as an 
attachment.  So that is what is needed to be worked around.

This patch doesn't fix our problem with multipart/related messages. An 
example message is attached.

Changes have been made in Git for this ticket:

Workaround broken messages by allow viewing multipart/related parts 
that are not referenced in the base part (Request #9827).
The formatting of the status message could use some tweaking, but it
works.

  4 files changed, 39 insertions(+), 1 deletions(-)
http://git.horde.org/horde-git/-/commit/33aeef48376c186697ca02d8362219cdd7881b37

This solution doesn't really deal with this problem.  It is 
multipart/related specific.  It would not fix, for example, this issue:

multipart/alternative
   text/plain
   multipart/mixed
     text/html (not viewable inline)
     image/png

The image/png would not be available with your solution, and would not 
be shown in the multipart/alternative display.

An adaption of this solution may be useful, however, within 
multipart/related to catch those attachments that are not properly 
referred to in the base part.

Now with attachment...

I know that this is not the desired approach but the attached patch is 
working pretty well for us.

We are showing all downloadable parts inside a multipart/related that 
are not referenced in another part as attachments.

The code is not perfect but this is only a temporary fix.

PS: Please ignore all bugs and Portugalmail references in this patch :P

Uhm, I just realized that "1 attachment" string won't be shown for 
these messages at all.

Something like this?

Any suggestions on how to do the UI in the dynamic view?  This can't 
be a simple [+] expand graphic, since that is confusing with the 
current list of *actual* attachments and defeats the whole purpose of 
trying to detect what the actual attachments are.

Then this pretty much means that MUA sniffing is out of the question.

You are definitely not living in Portugal :P Just two examples:

- Train tickets sent by www.cp.pt (Trains of Portugal) match this kind 
of messages. How you can imagine its a big frustration paying a train 
ticket and not being able to print it (pdf)...
- Social Services from Portugal send this badly formatted messages too.

This refers to the MUA-sniffing workaround.  Adding the show all 
attachments ability to the dynamic view is higher on my priority list, 
since it is more generally useful.

Not a high priority for me at the moment, since I have *never* 
received a message like this in 16 years.

Any chance this gets solved soon? I've switched some users to dynamic 
interface and they cannot access attachments for these kind of 
messages at all.

That sounds very right for me.

Because I end up viewing a message that has 10 formatting images in 
the unviewable part and that message now shows 10 attachments.  So as 
a user, if I see a message with 10 attachments, there is going to be 
some sort of (substantial) time penalty to go through and determine 
whether those attachments are legitimate or not.  This is unacceptable 
especially since, as mentioned previously, this example represents 99% 
of the messages received vs. the broken message that is the subject of 
this ticket which 1 broken e-mail client is sending.

The default behavior must remain the way it is.  I am all for finding 
a way to workaround this behavior without destroying the user 
experience in the vast majority of messages received.

Displaying all message parts needs to be implemented for dimp, 
notwithstanding anything in this ticket.  So that solves the problem 
partially since the attachment will be accessible.  However, it does 
not indicate that there may be an attachment.

As much as it pains me to say it... I am thinking that we ARE going to 
need to do MUA-sniffing to fix this.  I am thinking that this sniffing 
can be accomplished in a status box rather than automatically adding 
the attachments to the list.  E.g. for messages received from Apple 
Mail, if an attachment is found within an unviewable alternative part, 
a yellow box will be displayed indicating that an attachment might be 
in another part and that the user should view all message parts to 
verify.  This prevents the situation above from happening (unwanted 
attachments polluting the attachment list) while providing some 
indication to the user that an attachment *may* be available, and the 
user can factor the context of the displayed part into their decision 
as to whether to view all message parts.

In that example, I agree that it's not great to show the user those 
images as attachments, but what's the actual harm or user frustration 
caused by it, as opposed to not being able to find a file that an 
email claims is attached?

Here's just one example I can come up with of why we can't treat 
anything inside of a non-viewable alternative part as an attachment.

multipart/alternative
   text/plain
   multipart/related
     text/html
     image/png (content-disposition: attachment; image is used for 
HTML formatting)
     image/png (same)
     image/png (same)
     image/png (same)
     image/png (same)

This is a not uncommon message structure.  If text/html is disabled, 
the image/png's should definitely NOT be shown as attachments.  (This 
examples show that content-disposition is worthless for determining 
what parts are attachments).

To me, "view all message parts" is a good workaround for power users 
or really broken messages. But I'm guessing that most end users won't 
find it (or know when to use it) without specific help, so I'd prefer 
another solution. I'm okay if we want to treat messages from a 
specific MUA differently, though it doesn't feel *quite* right to me.

I'm still wondering how many real-world use cases there are for 
multipart/alternative parts that aren't text/plain or text/html? The 
one that's been mentioned is calendar invites and that seems like one 
we should special-case even if the invite *isn't* an alternative part 
(i.e., text message with calendar attachment - we should show the 
calendar response UI - along with any text - anyway). Beyond that I 
don't know of any - just cases where a buggy MUA sends an excel 
spreadsheet as one of the multipart/alternative parts, despite clear 
user intent for the spreadsheet to be an attachment.

As told earlier, interpreting particular MUA messages differently 
would be better, but at least for my users "View All Message Parts" 
method is enough too.

This was not right in IMP4 also. In this case most messages has been 
marked as with attachments, and when user opened those attachments, 
they saw alternative or empty parts.

I'm more in favor of making an exception for *major* buggy clients 
only. Like Apple's.
True.

A valid workaround already exists to see these parts in standard view 
- by clicking on view all message parts.  If this feature was added to 
the dynamic view also, would this be sufficient?

As I think I mentioned in this ticket, another option would be to 
inform the user that alternate views exist and give them the 
opportunity to view them.  But we used to have this in IMP 4 (or 3?) 
and all it did was confuse users so it was removed.  So that does not 
seem like a viable option.

Here's another way of putting it: I think the vast majority of 
multipart/alternative usage that isn't text/plain vs. text/html is 
broken. So yes, with multipart/alternative messages, I think we should 
show the richer of html or text, depending on whether both are 
available in the message and how IMP is configured, and then make 
everything else available as an attachment.

Otherwise, IMP users are going to continue to get messages from broken 
MUAs and consider IMP to be buggy because they can't get to the 
attachment. It won't matter if we're right.

I totally disagree with this statement.  A sender is given a set of 
rules on how to send a message.  We use those rules to interpret their 
intent.  What you are saying is that somehow we need to determine that 
their intent is completely opposite of what they actually sent.  Or, 
more succinctly, we simply need to ignore *all* message structure and 
display however we feel like.

But we simply CAN'T be doing this with these messages.  This defeats 
the ENTIRE purpose of multipart/alternative.  We might as well show 
all the alternative parts to the user every time by this reasoning.   
Which is essentially what you are asking us to do.
That may be true, but multipart/alternative is not one of these types 
that doesn't have a point.  Instead, it has a clear purpose that is 
very useful: to provide a single representation of a message part 
without the user even being aware that other representations exist.

I disagree - they're going against the expressed intent of the 
sender's client, which may or may not be what the person sending the 
mail intended.
I feel like we're not serving our user well in that case though, since 
a) that view of how the message should be viewed could have been 
constructed by a buggy MUA, and b) even if it was the express intent 
of the sender, it could be really frustrating to the receiver, and the 
receiver is our user and who we should be serving imo.

I feel like this whole area is a case of "be strict in what you omit 
and liberal in what you accept".
I imagine an email spec written today would be greatly simplified. So 
... possibly in real world usage, some of them don't have a point.

Thank you.

This has been fixed.

Changes have been made in Git for this ticket:

Bug #9827: Fix displaying all message parts in standard view

  3 files changed, 3 insertions(+), 1 deletions(-)
http://git.horde.org/horde-git/-/commit/a246c24cd1d242d0a078b78e8646d9d8b41d327a

This feature has been properly added to IMP 5.0.1.

Changes have been made in Git for this ticket:

Request #9827: Allow .eml files to be imported into a mailbox

  6 files changed, 23 insertions(+), 8 deletions(-)
http://git.horde.org/horde-git/-/commit/2d50ea21baeedf10844df68825676081a170c54d

Then at least IMP's Save As functionality need to be adjusted to send 
.mbox extension instead of .eml. Or better yet, allow importing .eml 
also.

Anyway I've found a way to get attachments from these messages. There 
is a link "Attached Files" in the action bar. I can get all needed 
attachments (and more) with "Download All Attached Files", but "Show 
All Message Parts" still doesn't who anything new. Is this right?

For reference:
http://tools.ietf.org/html/rfc4155

But actually they're not.  mbox files require a 'From [...]' header 
before each message.  And mbox files require special quoting for lines 
beginning with 'From' in the body of the message.  Neither of these 
occurs in message source data.

So if it was working previously, it was a bug in IMP.

They are, and I import them all the time using IMP since years :)

I really understand what you say and I support every point made. I 
always teach people and encaurage to NOT use Apple products for the 
same reason. They are indeed very buggy under the hood. But also I 
always say that standards and technology should serve people, not the 
other way around. So just hoped that it is possible to make an 
exception for Apple messages and to display them differently.

But I don't understand how they could display these as attachments?   
If they do they are going entirely against the wishes of the original 
sender.

What I really think is happening here (and not to be braggy), but IMP 
is just much more intelligent about determining what to display as 
attachments.  I can almost guarantee you that these other MUA's are 
simply obtaining a list of MIME part types of the message, looping 
through the list, and marking as attachments anything that "looks" 
like an attachment.  But this approach completely ignores the 
container those parts are located in, which is simply broken behavior 
(or at the least, lazy coding).

If I send a message to you that says that you shouldn't see any 
attachments if you can only view the text/plain part, then that's the 
way I want the message to be viewed by you.  Or else, what's the point 
of any of these MIME types?

I've always thought that .eml files ARE mboxes just with one message in it?

Yes. I'm using the default.

Understood. However it is really difficult to explain the users, why 
Gmail, Outlook and even oldy The Bat! display it right. But with IMP 
they don't.

Do you have HTML viewing off?  If so, there aren't any attachments in 
this message.

This is a multipart/alternative message that looks like the following:

multipart/alternative
   text/plain
   multipart/mixed
     text/html
     application/msword
     text/html
     application/msword
     text/html

Unless at least 1 of the parts in a multipart/alternative part is 
viewable, we can't display that part.  Thus, if text/html is off, we 
can only display text/plain.  There are no attachments in this 
message.  Absent bringing back the confusing alternative parts status 
box, not sure what we can do here.

Related to Ticket #9365 and #7981.  As previously mentioned, and of 
which I am constantly reminded, Apple Mail continues to make TERRIBLE 
assumptions about how multipart/alternative works.

Because folder import only supports importing mbox files.

Another sample message. Looks like problematic messages are sent from 
Apple OSes.

A client received a message for which IMP doesn't show attachments at 
all. Example attached.

Strangely this message cannot be imported through Folder Navigator -> 
Import messages also. It just shows "Error importing Forward 
Message.eml".