6.0.0-git
2019-03-18

[#7680] gzip/bzip2 upload doubly compressed
Summary gzip/bzip2 upload doubly compressed
Queue IMP
Queue Version Git master
Type Bug
State Not A Bug
Priority 2. Medium
Owners
Requester marc (at) r4l (dot) com
Created 2008-11-13 (3777 days ago)
Due
Updated 2011-01-28 (2971 days ago)
Assigned 2008-12-08 (3752 days ago)
Resolved 2008-12-09 (3751 days ago)
Milestone
Patch No

History
2011-01-28 09:46:38 maciej (dot) uhlig (at) us (dot) edu (dot) pl Comment #15 Reply to this comment
I've investigated the issue in detail and here are my findings:

The problem model (summary): you are using Imp/Horde and a browser. 
Horde is configured with $conf[compress_pages] = true. You compose 
mail message to yourself. The mail message contains an attachment 
which is tar.gz file. You send the message. You download the 
attachment from the received message. You find out it doesn't open. 
Then you find out it is (gzip) compressed twice (name not changed). 
You need to investigate what goes on.

1. I used Firefox, MSIE, Chrome and Opera. The attached file was 
downloaded incorrectly by Firefox and Chrome. It was downloaded 
correctly by MSIE and Opera. So it's really not a Horde bug but rather 
a bug (or unpleasant feature) of Firefox or Chrome.

2. While downloading, you distinguish MSIE from the other browsers. 
You set Content-Type as application/x-msdownload for MSIE while for 
other browsers you use Content-Type sent in fact by browser while 
uploading file (from mail) (in case content type is not known, you use 
application/octet-stream, which is correct).

3. Firefox and Chrome send Content-Type: application/gzip while 
uploading an attachment. This is incorrect (application/gzip is not 
registered by IANA).

4. Firefox and Chrome don't want to uncompress server data in order to 
get the correct file. They simply write the stream to disk which is 
incorrect. In contrary, Opera does it well.

5. Firefox and Chrome correctly save the file when HTTP header shows 
Content-Type: application/octet-stream (forced by Horde temporary 
modification in function downloadHeaders in Browser.php)

Looks like Firefox and Chrome get *gz file and save it to disk without 
further thinking. Opera thinks more and therefore is able to 
uncompress data before writing file to disk.

So, now we know we are doing our job well but some browsers don't 
behave well. The question arises, are we doing our best to circumvent 
bad browser behavior? Maybe we should always send Content-Type: 
application/octet-stream to the browser to make it think and doing 
something more (i.e. uncompress compressed chunks)?
2011-01-26 18:02:16 Michael Slusarz Comment #14
Version ⇒ Git master
Reply to this comment
- enable "$conf[compress_pages]" in horde config
I've always had this setting enabled.  And I've never seen this problem.
While page compressing is enabled, Imp _shouldn't_ compress 
downloaded .gz files. But it does it.
Then it is not strictly related to this setting.  Because it is 
completely irrelevant if we compress output to a browser that is 
already compressed.  It may not be efficient, but there's absolutely 
nothing wrong with doing this.

So there must be something different on your server that is causing 
this issue.  And unless you can track it down and tell us exactly what 
is causing this, there is nothing us developers can do to help with 
since it is working fine for us.
2011-01-26 12:30:26 maciej (dot) uhlig (at) us (dot) edu (dot) pl Comment #13 Reply to this comment
This IS a bug. The clue is in the first post:

- enable "$conf[compress_pages]" in horde config

While page compressing is enabled, Imp _shouldn't_ compress downloaded 
.gz files. But it does it. Sincerely speaking, I don't know why it 
doesn't compress the other filetypes, I didn't trace this. But my user 
complained about 'damaged' *.tar.gz files, so I'm returning to this 
ticket...
2008-12-09 05:03:09 Michael Slusarz Comment #12
State ⇒ Not A Bug
Reply to this comment
# file archive.tgz
archive.tgz: gzip compressed data, from Unix, last modified: Mon Dec
8 11:28:54 2008
And this is correct.  This is a gzip'd file.  How can 'file' know what 
is inside the file without uncompressing it?  'file' simply guess what 
kind of data is in the file.
Not much of a difference. So where is the MIME type sent to HORDE/IMP
coming from?
The browser/OS.
For a ".tgz", IMP receives "application/x-gtar" and for a ".tar.gz",
it receives "application/x-gzip"
Technically, there is nothing wrong with application/x-gzip.  Because 
that is exactly what the file is.  On OS's like windows, there can 
only be a single extension.  So the MIME type 'application/x-gzip' is 
correct.  I realize Thunderbird is sending something different, but I 
don't necessarily agree with what they are doing (x-compressed-tar 
does not seem to be used by many people.).



So I think the previous analysis in this ticket remains.
2008-12-08 16:54:21 Chuck Hagenbuch State ⇒ Feedback
 
2008-12-08 16:52:19 marc (at) r4l (dot) com Comment #11
New Attachment: archive.tar.gz Download
Reply to this comment
Should have done the same test on the ".tgz" file in the previous 
post, but here it is:



# file archive.tgz

archive.tgz: gzip compressed data, from Unix, last modified: Mon Dec   
8 11:28:54 2008



Not much of a difference. So where is the MIME type sent to HORDE/IMP 
coming from?



For a ".tgz", IMP receives "application/x-gtar" and for a ".tar.gz", 
it receives "application/x-gzip"



Can someone shed some light on this?
2008-12-08 16:46:59 marc (at) r4l (dot) com Comment #10 Reply to this comment
Silly me, I tested with a ".tgz" instead of a ".tar.gz". So the 
problem is still there, and the "file" command on Linux reports this 
for a ".tar.gz" file:



# file archive.tar.gz

archive.tar.gz: gzip compressed data, from Unix, last modified: Mon 
Dec  8 11:28:54 2008



So it's probably the right behavior. Can someone confirm?

[Show Quoted Text - 13 lines]
2008-12-08 16:42:37 Chuck Hagenbuch State ⇒ Not A Bug
 
2008-12-08 16:37:01 marc (at) r4l (dot) com Comment #9 Reply to this comment
Since I reported this, I re-installed my OS from scratch, and now I 
don't experience the MIME type discrepancy anymore. So it seems there 
was something mis-configured with the MIME types in my previous 
environment.



Thanks for this precision, Michael!
- a .tar.gz attachment sent using Horde/IMP has a MIME type of
"application/x-gzip"
Is this what the browser is sending to Horde/IMP?  We use the MIME
type information provided to us by the user's browser/OS.
2008-12-07 20:41:12 Michael Slusarz Comment #8 Reply to this comment
- a .tar.gz attachment sent using Horde/IMP has a MIME type of
"application/x-gzip"
Is this what the browser is sending to Horde/IMP?  We use the MIME 
type information provided to us by the user's browser/OS.
2008-11-17 13:29:50 marc (at) r4l (dot) com Comment #7 Reply to this comment
More details for the problem when trying to view the content listing 
of a tar.gz file within IMP:



- a .tar.gz attachment sent using Horde/IMP has a MIME type of 
"application/x-gzip"

- a .tar.gz attachment sent using Thunderbird has a MIME type of 
"application/x-compressed-tar"



Which probably explain the problem (and is probably caused by a 
limitation in the MIME library, as Chuck already mentioned)
2008-11-17 00:03:40 marc (at) r4l (dot) com Comment #6 Reply to this comment
- the file will be compressed TWICE
I can't reproduce this.
I tested again, and it really needs to be a compressed TAR  FILE, not 
just a compressed FILE.




2008-11-16 23:54:12 marc (at) r4l (dot) com Comment #5 Reply to this comment
No. Individual gzipped files are different from tar files.
OK, but I really meant "tar.gz" files, not just gzipped files.
2008-11-16 22:49:08 Chuck Hagenbuch Comment #4 Reply to this comment
No. Individual gzipped files are different from tar files.
2008-11-16 22:44:49 marc (at) r4l (dot) com Comment #3 Reply to this comment
So, are you saying the problem happens with Horde's MIME library WHEN 
UPLOADING the gzipped attachment? Otherwise, If you're saying it's 
only when OPENING the attachment, then it would not explain why this 
does not happen with an attachment in an email created OUTSIDE of Horde.
2008-11-16 22:20:59 Chuck Hagenbuch State ⇒ Feedback
 
2008-11-16 22:20:51 Chuck Hagenbuch Comment #2 Reply to this comment
- the file will be compressed TWICE
I can't reproduce this.
Also, independantly of the setting above, any uploaded .tgz (or 
.tbz) file cannot be
opened in IMP (gives error: "Content type of compressed file: Unknown")
This is a limitation of the current MIME lib and can't be fixed until IMP 5.
2008-11-13 16:49:39 marc (at) r4l (dot) com Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 2. Medium
Summary ⇒ gzip/bzip2 upload doubly compressed
Queue ⇒ IMP
Milestone ⇒
Patch ⇒ No
Reply to this comment
Reproduce method:



- enable "$conf[compress_pages]" in horde config

- upload a gzip/bzip2 file in IMP compose window

- send that email to yourself

- open the email you just sent

- download the gzip/bzip2 file to local disk

- the file will be compressed TWICE



Also, independantly of the setting above, any uploaded .tgz (or .tbz) 
file cannot be

opened in IMP (gives error: "Content type of compressed file: Unknown")



If I receive an email that contains a .tgz/tbz file (and that email 
has not been created with IMP), the attached compressed tar file 
content can be read within IMP, which leads me to believe there is a 
link with the upload bug above.


Saved Queries