6.0.0-git
2019-03-21

[#4225] Some calendar events not imported
Summary Some calendar events not imported
Queue Kronolith
Queue Version 2.1.2
Type Bug
State Not A Bug
Priority 2. Medium
Owners
Requester stpierre (at) nebrwesleyan (dot) edu
Created 2006-08-01 (4615 days ago)
Due
Updated 2006-08-02 (4614 days ago)
Assigned 2006-08-02 (4614 days ago)
Resolved 2006-08-02 (4614 days ago)
Milestone
Patch No

History
2006-08-02 21:11:31 Jan Schneider Comment #15
State ⇒ Not A Bug
Reply to this comment
With regards to fixing this: will it import okay if I just remove the
UID: attribute from each event?  Will Kronolith be smart enough to
generate its own UIDs,
Yes.
2006-08-02 21:08:51 stpierre (at) nebrwesleyan (dot) edu Comment #14 Reply to this comment
Jan--



As can I.  As I mentioned in the initial bug ticket, the event imports 
fine by itself; its only in the context of a larger calendar that it 
fails.



Luckily, I had a sudden insight as I was poring over the events, 
trying to find similarities and differences.  For some reason, a 
number of the events have the same UID attribute; the reason Kronolith 
never barfs is that it's able to add each one, overwriting the old 
event with the same UID.  (How Sun Calendar Express can, in good 
faith, export a calendar with multiple events with the same primary 
key is another question, one that I suspect is far beyond the realm of 
mortals.)



With regards to fixing this: will it import okay if I just remove the 
UID: attribute from each event?  Will Kronolith be smart enough to 
generate its own UIDs, or do I need to come up with an algorithm to 
munge those myself?
2006-08-02 21:01:25 Jan Schneider Comment #13 Reply to this comment
I can import these fine.
2006-08-02 20:52:19 stpierre (at) nebrwesleyan (dot) edu Comment #12
New Attachment: failed-events.ics Download
Reply to this comment
I'm attaching a file, 'failed-events.ics', which contains two events, 
both of which fail.  I reduced the size of the calendar until I had a 
more manageable number of events, and then removed them one by one 
until the number of events in the .ics file matched number of events 
stored in the database for that file.



Out of 6171 events originally in the calendar, only 4039 imported 
initially, so there are plenty more examples out there if you need them.
2006-08-02 20:09:37 Chuck Hagenbuch Comment #11 Reply to this comment
By the way, let me just say that I know I'm asking you to do things 
that myself or another Horde developer could do, since you've provided 
the failing file. But it will really help us out, and speed up 
tracking this down, the more debugging that you contribute. Going 
through the troubleshooting steps on problems like this is as much a 
contribution to the community as a patch; sometimes even more so. So 
thanks in advance.
2006-08-02 20:07:42 Chuck Hagenbuch Comment #10 Reply to this comment
Like I said, the only thing I have to go on for now is that it's 
large. So anything that narrows down the problem would be useful.



For instance, if both halves load okay, but the whole thing doesn't, 
then we know it's some kind of memory/execution time problem. Speaking 
of memory, have you looked at your httpd logs for PHP/apache segfaults?



Or, if you can produce a smaller slice that fails, we can start 
investigating that more closely. So binary search sounds like a good 
place to start.



Another thing, esp. if you can test against a blank (non-production) 
database, might be to run the import (using the command line for now) 
and see where it stops. You can count the # of events put into the db 
(thus the blank database). Then maybe clear the db and run it again - 
see if it's consistent. If possible, find out if it's skipping 
entries, or if it just stops at some point. Etc... at this point 
simply more information is needed.
2006-08-02 19:16:27 stpierre (at) nebrwesleyan (dot) edu Comment #9 Reply to this comment
I bumped max_execution_time and max_input_time up to 360 seconds and 
the event still isn't imported.  Running the import from the command 
line with import_cals.php gives "6171 events successfully imported', 
but it's apparently lying. :)  It takes about 3 and a half minutes to 
import from the command line, so I don't think it's running into 
max_*_time.



Any idea what sort of things I should be looking for while paring down 
the calendar?  Or should I just do the old binary search method and 
hack in half and see which half fails?
2006-08-02 18:35:42 Chuck Hagenbuch Comment #8 Reply to this comment
So that is indeed a huge calendar. 
framework/iCalendar/docs/examples/parser.php tells me it finds 6171 
components in it; do you know if that's correct? It also does find the 
event you called out as not being imported. Is it possible that PHP 
simply hits max_execution_time and stops adding events? Have you found 
any pattern to the events that aren't imported? Can you possibly trim 
this down to a smaller example that still breaks?



Otherwise I'm left with it simply being large; we can try to speed up 
how we handle iCalendar imports but that's about it.
2006-08-02 16:21:32 stpierre (at) nebrwesleyan (dot) edu Comment #7 Reply to this comment
2006-08-02 16:18:17 Chuck Hagenbuch Deleted Original Message
 
2006-08-02 16:18:01 Chuck Hagenbuch Comment #6 Reply to this comment
Can you put it online somewhere?
2006-08-02 16:16:31 stpierre (at) nebrwesleyan (dot) edu Comment #5 Reply to this comment
Yep, it seems to have silently dropped my attachment.  Is there 
another way I can get the calendar to you?
2006-08-02 16:15:05 stpierre (at) nebrwesleyan (dot) edu Comment #4 Reply to this comment
Sorry, I must have flubbed the attachment.  If Whups drops attachments 
over a certain size, that could also be the issue; the full calendar 
(which fails) is about 5 Mb.
2006-08-02 16:08:48 Chuck Hagenbuch Comment #3 Reply to this comment
Okay, you said that on its own it works, my bad. So I need an example 
of data that _doesn't_ work. Nothing was attached to the ticket 
originally.
2006-08-02 16:07:31 Chuck Hagenbuch Comment #2
State ⇒ Feedback
New Attachment: bug_4225.ics
Reply to this comment
If I import the file that I'm attaching, which I got from your comment 
body, it works fine with HEAD. It's possible something has been fixed 
that isn't in the FRAMEWORK_3 branch, but before anyone spends too 
much time on it, can you confirm that what I'm attaching matches the 
data you have trouble with?
2006-08-01 12:54:43 stpierre (at) nebrwesleyan (dot) edu Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 2. Medium
Summary ⇒ Some calendar events not imported
Queue ⇒ Kronolith
Reply to this comment
I've attached a rather large calendar which, when imported into 
Kronolith, gives no errors but silently drops some events.  An example 
of one event which is dropped:



BEGIN:VEVENT

UID:42769abc0000048a0000207f00001178

RECURRENCE-ID:20060802T140000Z

DTSTAMP:20060727T201537Z

SUMMARY:All Staff Mtg

DTSTART:20060802T140000Z

DTEND:20060802T150000Z

CREATED:20050502T212516Z

LAST-MODIFIED:20051219T152244Z

PRIORITY:0

SEQUENCE:1

CLASS:PUBLIC

LOCATION:Callen

ORGANIZER;CN="John Q. Public"

  ;SENT-BY="jpublic@NebrWesleyan.edu"

  ;X-NSCP-ORGANIZER-UID=jpublic

  ;X-NSCP-ORGANIZER-SENT-BY-UID=jpublic

  ;X-S1CS-EMAIL=jpublic@NebrWesleyan.edu

  :jpublic

STATUS:CONFIRMED

TRANSP:OPAQUE

ATTENDEE;ROLE=REQ-PARTICIPANT;CUTYPE=INDIVIDUAL

  ;PARTSTAT=ACCEPTED;CN="John Q. Public"

  ;RSVP=TRUE

  ;X-NSCP-ATTENDEE-GSE-STATUS=2

  ;X-S1CS-EMAIL=jpublic

  :jpublic

X-NSCP-ORIGINAL-DTSTART:20060802T140000Z

X-NSCP-LANGUAGE:en

X-NSCP-DTSTART-TZID:America/Chicago

X-NSCP-TOMBSTONE:0

X-NSCP-ONGOING:0

X-NSCP-ORGANIZER-EMAIL:jpublic@NebrWesleyan.edu

X-NSCP-GSE-COMPONENT-STATE;X-NSCP-GSE-COMMENT="REQUEST-COMPLETED":131074

END:VEVENT



If that event is put into its own .ics file and imported alone, the 
import succeeds.



The attached .ics file was exported from Sun Calendar Express.

Saved Queries