6.0.0-git
2019-08-22

[#11972] importing ldif files from Thunderbird fails
Summary importing ldif files from Thunderbird fails
Queue Turba
Queue Version Git master
Type Bug
State Resolved
Priority 2. Medium
Owners jan (at) horde (dot) org, slusarz (at) horde (dot) org
Requester jhaar-horde (at) whanau (dot) org
Created 2013-01-17 (2408 days ago)
Due
Updated 2013-01-24 (2401 days ago)
Assigned
Resolved 2013-01-17 (2408 days ago)
Milestone
Patch No

History
2013-01-24 10:43:29 Git Commit Comment #18 Reply to this comment
Changes have been made in Git (master):

commit dbd6eee04d8a7e7d7e5e489a1c84a67c17d80596
Author: Jan Schneider <jan@horde.org>
Date:   Thu Jan 24 10:44:21 2013 +0100

     We still need to test if the method exists at all though. (Bug #11972).

  framework/Support/lib/Horde/Support/Randomid.php |    3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)

http://git.horde.org/horde-git/-/commit/dbd6eee04d8a7e7d7e5e489a1c84a67c17d80596
2013-01-23 20:52:22 jhaar-horde (at) whanau (dot) org Comment #17 Reply to this comment
Please use var_dump() instead of print_r() to output the function result.
All right - you win; I'm the fool :-)

I put your code back and it worked. I guess I edited the file in the 
/usr/share/pear... tree instead of the /var/www/usr/share... jailed tree

mea culpa :-)

Jason
2013-01-23 10:01:08 Jan Schneider Comment #16 Reply to this comment
Please use var_dump() instead of print_r() to output the function result.
2013-01-23 06:02:53 jhaar-horde (at) whanau (dot) org Comment #15 Reply to this comment
Not according to your test script.  You said that this:
...
output: '::'.  If sys_getloadavg() was true, it would have output: 
':1:'.  Thus sys_getloadavg() must be returning false (or null).
your code says

if ($var = sys_getloadavf()) {
   ...
}

$var = sys_getloadavg is true irrespective of whether or not Apache is 
running in a jail. My code

$var= sys_getloadavf();
if (is_array($var)){...

is better because you definitely want to ignore the data if it is NOT 
an array.

All I know if that when I used your patch, it didn't work. Replacing 
it with mine fixed it

Jason
2013-01-23 03:49:19 Michael Slusarz Comment #14 Reply to this comment
I found the solution. It's caused by sys_getloadavg
Should be fixed in Horde_Support 2.0.3.
Your patch doesn't work as "$var = function" always return true.
Not according to your test script.  You said that this:

echo ":";
print_r(sys_getloadavg());
echo ":";

output: '::'.  If sys_getloadavg() was true, it would have output: 
':1:'.  Thus sys_getloadavg() must be returning false (or null).
2013-01-23 03:14:32 jhaar-horde (at) whanau (dot) org Comment #13 Reply to this comment
I found the solution. It's caused by sys_getloadavg
Should be fixed in Horde_Support 2.0.3.
Your patch doesn't work as "$var = function" always return true. How 
about this instead - it worked for me

         $loadavg = sys_getloadavg();
         if (is_array($loadavg)) {
             $elts = array_merge($elts, $loadavg);
         }

2013-01-23 02:35:46 Michael Slusarz Comment #12
Assigned to Michael Slusarz
Reply to this comment
I found the solution. It's caused by sys_getloadavg
Should be fixed in Horde_Support 2.0.3.
2013-01-23 02:35:06 Git Commit Comment #11 Reply to this comment
Changes have been made in Git (master):

commit 437f3e9f983b0930f7c2323eba4b4e573e26d935
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Tue Jan 22 19:33:11 2013 -0700

     [mms] Fix detection of sys_getloadavg() on Windows (Bug #11972).

     Apparently when PHP tells you that a "function is not implemented"
     on a certain platform, they *really* mean that the "function is
     implemented" and good luck figuring out what it will return. Wonderful.

  framework/Support/lib/Horde/Support/Randomid.php |    4 ++--
  framework/Support/package.xml                    |    4 ++--
  2 files changed, 4 insertions(+), 4 deletions(-)

http://git.horde.org/horde-git/-/commit/437f3e9f983b0930f7c2323eba4b4e573e26d935
2013-01-23 02:04:40 jhaar-horde (at) whanau (dot) org Comment #10 Reply to this comment
What does sys_getloadavg() return on your system?
It returns an array with /proc mounted and an empty string when it's 
not (ie no error)

Basically I wrote a webpage that did

<?php
echo ":";
print_r(sys_getloadavg());
echo ":";
exit;
?>

And it returned "::" when there was no /proc and ":Array....:" when it was

Jason


2013-01-23 01:49:04 Michael Rubinsky Comment #9 Reply to this comment
What does sys_getloadavg() return on your system?
2013-01-23 01:31:24 jhaar-horde (at) whanau (dot) org Comment #8 Reply to this comment
I found the solution. It's caused by sys_getloadavg, so I just 
disabled that function via

disable_function = sys_getloadavg

... in /etc/php.ini and now Horde thinks it's dealing with a Windows 
system and does the randomization differently

Please let me know if I screwed up the logic completely, but for now 
it's all working :-)

Jason
2013-01-23 01:24:03 jhaar-horde (at) whanau (dot) org Comment #7 Reply to this comment
Cannot reproduce and doesn't make much sense since the contact's ID 
is a random value.
  I may have found the problem. I run Apache in a chroot-ed jail, and 
when I strace'ed httpd and looked at what happened just before the 
duplicate INSERT occurs, I see a read call to /proc/loadavg. I guess 
that is used to seed some random generator? As it doesn't exist, it 
probably reuses the last value - causing the problem?

I mounted /proc inside the jail and the problem is solved - but that 
isn't how I want to run it. I can't find the code that would be doing 
this, can you point me in the right direction so I can hack my way 
past this? Perhaps all that is needed is for whatever call is failing 
to fall back on something else; timestamp, pid, whatever?


Thanks

Jason
2013-01-21 17:19:00 Jan Schneider Comment #6 Reply to this comment
Cannot reproduce and doesn't make much sense since the contact's ID is 
a random value.
2013-01-17 22:21:32 jhaar-horde (at) whanau (dot) org Comment #5 Reply to this comment
That made a difference - now it gets as far as importing, but then 
fails due to re-use of object_id. I trashed all my addressbook and 
imported the LDIF file - but "1B2M2Y8AsgTpgAmY7PhCfg1" is getting reused

There was an error importing the data: QUERY FAILED: Duplicate entry 
'1B2M2Y8AsgTpgAmY7PhCfg1' for key 'PRIMARY' INSERT INTO turba_objects 
(object_firstname, object_lastname, object_email, owner_id, 
object_type, object_uid, object_id) VALUES ('Jason', 'Weinberg', 
'jweinberg@guardianedge.com', 'jhaar', 'Object', 
'20130117221807.1B2M2Y8AsgTpgAmY7PhCfg1@whanau.org', 
'1B2M2Y8AsgTpgAmY7PhCfg1')


After the import (100s of entries in the LDIF file), only the 
following 4 entries are in the turba_objects table

INSERT INTO `turba_objects` VALUES 
('1B2M2Y8AsgTpgAmY7PhCfg1','jhaar','Object','20130117220505.1B2M2Y8AsgTpgAmY7PhCfg1@whanau.org',NULL,'Andrew','ds',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,'ds@ds.com',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL);
INSERT INTO `turba_objects` VALUES 
('1B2M2Y8AsgTpgAmY7PhCfg2','jhaar','Object','20130117220617.1B2M2Y8AsgTpgAmY7PhCfg1@whanau.org',NULL,'Ferry','ds',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,'ferry.ds@e.ds.nz',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL);
INSERT INTO `turba_objects` VALUES 
('1B2M2Y8AsgTpgAmY7PhCfg3','jhaar','Object','20130117220617.1B2M2Y8AsgTpgAmY7PhCfg1@whanau.org',NULL,'ds','Landin',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,'sdin@dsd.se',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL);
INSERT INTO `turba_objects` VALUES 
('1B2M2Y8AsgTpgAmY7PhCfg4','1B2M2Y8AsgTpgAmY7PhCfg4','Object','20130111051719.1B2M2Y8AsgTpgAmY7PhCfg1@whanau.org',NULL,'Roy','ds','','','','xxx',NULL,NULL,NULL,'2a dfdfd 
Place','','Raglan','','','NZ','','','','','','','',NULL,'dsd@ds.org','','','','','','','',NULL,NULL,'','','','','','','',NULL,'','','',NULL,'','skype:xxx','','');
INSERT INTO `turba_objects` VALUES 
('1B2M2Y8AsgTpgAmY7PhCfg8','jhaar','Object','20130117220712.1B2M2Y8AsgTpgAmY7PhCfg2@whanau.org',NULL,'Support',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,'support@guardianedge.com',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL);

2013-01-17 21:59:39 Git Commit Comment #4 Reply to this comment
Changes have been made in Git (master):

commit 46022e68b0bec5be01c7e84c67feb512fe1ea01c
Author: Jan Schneider <jan@horde.org>
Date:   Thu Jan 17 22:58:18 2013 +0100

     Fix unsetting storage key (Bug #11972).

  turba/lib/Data/Ldif.php |    2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)

http://git.horde.org/horde-git/-/commit/46022e68b0bec5be01c7e84c67feb512fe1ea01c
2013-01-17 20:30:59 jhaar-horde (at) whanau (dot) org Comment #3 Reply to this comment
Hi there

I have cut-n-pasted the diff into my turba/data.php, ran "php 
turba/data.php" to ensure there were no obvious errors and tried 
again. Now I don't even get the crash page - I get a blank page! I 
have DEBUG level logging and no error is reported, but my php logging 
reports

[17-Jan-2013 20:28:07 UTC] PHP Fatal error:  Call to undefined method 
Horde_Core_Data_Storage::remove() in 
/html/horde/turba/lib/Data/Ldif.php on line 289

2013-01-17 14:44:16 Jan Schneider Assigned to Jan Schneider
State ⇒ Resolved
 
2013-01-17 14:43:47 Git Commit Comment #2 Reply to this comment
Changes have been made in Git (master):

commit 93d6f6b98034a29a18c38875ed926b3fc59c1adc
Author: Jan Schneider <jan@horde.org>
Date:   Thu Jan 17 14:18:51 2013 +0100

     [jan] Fix importing of LDIF data (Bug #11972).

  turba/data.php     |   13 ++++++++-----
  turba/docs/CHANGES |    1 +
  turba/package.xml  |    4 ++--
  3 files changed, 11 insertions(+), 7 deletions(-)

http://git.horde.org/horde-git/-/commit/93d6f6b98034a29a18c38875ed926b3fc59c1adc
2013-01-17 07:47:56 jhaar-horde (at) whanau (dot) org Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 2. Medium
Summary ⇒ importing ldif files from Thunderbird fails
Queue ⇒ Turba
Milestone ⇒
Patch ⇒ No
Reply to this comment
I'm trying to import my personal contacts into Horde and even if I 
edit the ldif file down to one entry, it generates the error at the 
bottom.

This is a CentOS6 server...

[root@hostname horde]# rpm -q php php-pear httpd
php-5.3.20-13.el6.art.x86_64
php-pear-1.9.4-4.el6.noarch
httpd-2.2.15-15.el6.centos.1.x86_64


...with horde-5.0.3 installed as as pear install (only a week old). I 
have just done a "pear upgrade -a -B -c horde" and that didn't fix the 
error - only some ActiveSync changes came down

I have horde logging to a file, and at DEBUG level this event ends with

2013-01-17T07:46:04+00:00 DEBUG: HORDE [turba] PHP ERROR: Argument 1 
passed to Horde_Data_Base::__construct() must implement interface 
Horde_Data_Storage, array given, called in /html/horde/turba/data.php 
on line 191 and defined [pid 6803 on line 75 of 
"/usr/share/pear/Horde/Data/Base.php"]
2013-01-17T07:46:04+00:00 DEBUG: HORDE 1. 
Horde_Data_Base->__construct() /html/horde/turba/data.php:191
2. Horde_ErrorHandler::errorHandler() /usr/share/pear/Horde/Data/Base.php:75

2013-01-17T07:46:04+00:00 EMERG: HORDE [turba] Missing browser 
parameter. [pid 6803 on line 273 of 
"/usr/share/pear/Horde/Data/Base.php"]
2013-01-17T07:46:04+00:00 DEBUG: HORDE 1. Turba_Data_Ldif->nextStep() 
/html/horde/turba/data.php:213
2. Horde_Data_Base->nextStep() /html/horde/turba/lib/Data/Ldif.php:227

2013-01-17T07:46:04+00:00 DEBUG: HORDE [turba] Max memory usage: 
4718592 bytes [pid 6803 on line 543 of 
"/usr/share/pear/Horde/Registry.php"]
2013-01-17T07:46:04+00:00 WARN: HORDE [turba] PHP ERROR: Cannot modify 
header information - headers already sent [pid 6803 on line 0 of 
"Unknown"]
2013-01-17T07:46:04+00:00 DEBUG: HORDE 1. ob_gzhandler()
2. Horde_ErrorHandler::errorHandler()


Thanks, Jason

------------

A fatal error has occurred
Missing browser parameter.

1. Turba_Data_Ldif->nextStep() /html/horde/turba/data.php:213
2. Horde_Data_Base->nextStep() /html/horde/turba/lib/Data/Ldif.php:227

Details
The full error message is logged in Horde's log file, and is shown 
below only to administrators. Non-administrative users will not see 
error details.

LogicException Object
(
     [message:protected] => Missinga browser parameter.
     [string:Exception:private] =>
     [code:protected] => 0
     [file:protected] => /usr/share/pear/Horde/Data/Base.php
     [line:protected] => 273
     [trace:Exception:private] => Array
         (
             [0] => Array
                 (
                     [file] => /html/horde/turba/lib/Data/Ldif.php
                     [line] => 227
                     [function] => nextStep
                     [class] => Horde_Data_Base
                     [type] => ->
                     [args] => Array
                         (
                             [0] => 11
                             [1] => Array
                                 (
                                     [time_fields] => Array
                                         (
                                         )

                                     [file_types] => Array
                                         (
                                             [csv] => CSV
                                             [tsv] => TSV
                                             [vcard] => vCard
                                             [mulberry] => Mulberry 
Address Book
                                             [pine] => Pine Address Book
                                             [ldif] => LDIF Address Book
                                         )

                                     [import_mapping] => Array
                                         (
                                             [Title] => namePrefix
                                             [First Name] => firstname
                                             [Middle Name] => middlenames
                                             [Last Name] => lastname
                                             [Nickname] => nickname
                                             [Suffix] => nameSuffix
                                             [Company] => company
                                             [Department] => department
                                             [Job Title] => title
                                             [Business Street] => workStreet
                                             [Business City] => workCity
                                             [Business State] => workProvince
                                             [Business Postal Code] => 
workPostalCode
                                             [Business Country] => workCountry
                                             [Home Street] => homeStreet
                                             [Home City] => homeCity
                                             [Home State] => homeProvince
                                             [Home Postal Code] => 
homePostalCode
                                             [Home Country] => homeCountry
                                             [Business Fax] => workFax
                                             [Business Phone] => workPhone
                                             [Home Phone] => homePhone
                                             [Mobile Phone] => cellPhone
                                             [Pager] => pager
                                             [Anniversary] => anniversary
                                             [Assistant's Name] => assistant
                                             [Birthday] => birthday
                                             [Business Address PO Box] 
=> workPOBox
                                             [Categories] => category
                                             [Children] => children
                                             [E-mail Address] => email
                                             [Home Address PO Box] => homePOBox
                                             [Initials] => initials
                                             [Internet Free Busy] => 
freebusyUrl
                                             [Language] => language
                                             [Notes] => notes
                                             [Profession] => role
                                             [Office Location] => office
                                             [Spouse] => spouse
                                             [Web Page] => website
                                             [e-mail] => email
                                             [homeaddress] => homeAddress
                                             [businessaddress] => workAddress
                                             [homephone] => homePhone
                                             [businessphone] => workPhone
                                             [mobilephone] => cellPhone
                                             [businessfax] => fax
                                             [jobtitle] => title
                                             [internetfreebusy] => freebusyUrl
                                             [Dept] => department
                                             [Work Street Address] => 
workStreet
                                             [Work City] => workCity
                                             [Work State] => workProvince
                                             [Work Zip] => workPostalCode
                                             [Work Country/Region] => 
workCountry
                                             [Home Street Address] => 
homeStreet
                                             [Home Zip] => homePostalCode
                                             [Home Country/Region] => 
homeCountry
                                             [Work Fax] => workFax
                                             [Work Phone 1] => workPhone
                                             [Home Phone 1] => homePhone
                                             [Instant Messaging 1] => 
instantMessenger
                                             [Primary Email] => email
                                             [Fax Number] => fax
                                             [Pager Number] => pager
                                             [Mobile Number] => Mobile Phone
                                             [Home Address] => homeStreet
                                             [Home ZipCode] => homePostalCode
                                             [Work Address] => workStreet
                                             [Work ZipCode] => workPostalCode
                                             [Work Country] => workCountry
                                             [Work Phone] => workPhone
                                             [Organization] => company
                                             [Web Page 1] => website
                                         )

                                 )

                         )

                 )

             [1] => Array
                 (
                     [file] => /html/horde/turba/data.php
                     [line] => 213
                     [function] => nextStep
                     [class] => Turba_Data_Ldif
                     [type] => ->
                     [args] => Array
                         (
                             [0] => 11
                             [1] => Array
                                 (
                                     [time_fields] => Array
                                         (
                                         )

                                     [file_types] => Array
                                         (
                                             [csv] => CSV
                                             [tsv] => TSV
                                             [vcard] => vCard
                                             [mulberry] => Mulberry 
Address Book
                                             [pine] => Pine Address Book
                                             [ldif] => LDIF Address Book
                                         )

                                     [import_mapping] => Array
                                         (
                                             [Title] => namePrefix
                                             [First Name] => firstname
                                             [Middle Name] => middlenames
                                             [Last Name] => lastname
                                             [Nickname] => nickname
                                             [Suffix] => nameSuffix
                                             [Company] => company
                                             [Department] => department
                                             [Job Title] => title
                                             [Business Street] => workStreet
                                             [Business City] => workCity
                                             [Business State] => workProvince
                                             [Business Postal Code] => 
workPostalCode
                                             [Business Country] => workCountry
                                             [Home Street] => homeStreet
                                             [Home City] => homeCity
                                             [Home State] => homeProvince
                                             [Home Postal Code] => 
homePostalCode
                                             [Home Country] => homeCountry
                                             [Business Fax] => workFax
                                             [Business Phone] => workPhone
                                             [Home Phone] => homePhone
                                             [Mobile Phone] => cellPhone
                                             [Pager] => pager
                                             [Anniversary] => anniversary
                                             [Assistant's Name] => assistant
                                             [Birthday] => birthday
                                             [Business Address PO Box] 
=> workPOBox
                                             [Categories] => category
                                             [Children] => children
                                             [E-mail Address] => email
                                             [Home Address PO Box] => homePOBox
                                             [Initials] => initials
                                             [Internet Free Busy] => 
freebusyUrl
                                             [Language] => language
                                             [Notes] => notes
                                             [Profession] => role
                                             [Office Location] => office
                                             [Spouse] => spouse
                                             [Web Page] => website
                                             [e-mail] => email
                                             [homeaddress] => homeAddress
                                             [businessaddress] => workAddress
                                             [homephone] => homePhone
                                             [businessphone] => workPhone
                                             [mobilephone] => cellPhone
                                             [businessfax] => fax
                                             [jobtitle] => title
                                             [internetfreebusy] => freebusyUrl
                                             [Dept] => department
                                             [Work Street Address] => 
workStreet
                                             [Work City] => workCity
                                             [Work State] => workProvince
                                             [Work Zip] => workPostalCode
                                             [Work Country/Region] => 
workCountry
                                             [Home Street Address] => 
homeStreet
                                             [Home Zip] => homePostalCode
                                             [Home Country/Region] => 
homeCountry
                                             [Work Fax] => workFax
                                             [Work Phone 1] => workPhone
                                             [Home Phone 1] => homePhone
                                             [Instant Messaging 1] => 
instantMessenger
                                             [Primary Email] => email
                                             [Fax Number] => fax
                                             [Pager Number] => pager
                                             [Mobile Number] => Mobile Phone
                                             [Home Address] => homeStreet
                                             [Home ZipCode] => homePostalCode
                                             [Work Address] => workStreet
                                             [Work ZipCode] => workPostalCode
                                             [Work Country] => workCountry
                                             [Work Phone] => workPhone
                                             [Organization] => company
                                             [Web Page 1] => website
                                         )

                                 )

                         )

                 )

         )

     [previous:Exception:private] =>
)

Saved Queries