But regardless, the current method is the only way that makes sense as 
default.  It makes no sense to use Horde SMTP parameters as a default. 
  (Not to mention your setup is extremely rare in practice).

Just imagine the case where an admin allows login to two different 
mail backends - a local backend and Gmail.  For both of these, the 
obvious default is the same credentials for logging into the IMAP/POP 
backend, NOT the default SMTP configuration for the Horde 
installation.  (Almost every mail system setup I know shares 
authentication credentials for both the sending and receiving parts.   
In fact, in 10+ years of dealing with e-mail systems, I can't think of 
a setup otherwise.  I'm not saying this isn't true somewhere - but I 
would have seen this at least once by now if it was a common 
occurrence).

This is especially evident with the Gmail setup example from above.   
You absolutely cannot use the Horde default SMTP server to send 
messages via a Gmail address.  The odds are about 0% a receiving mail 
agent is going to accept a message like this - it is going to be 
almost immediately marked as spam.  Thus the default needs to be the 
only authentication credentials that will most likely work - the ones 
used to login to the Gmail IMAP backend.

Do understand that your use case is in the extreme minority.  So it 
needs to be documented b your use case must not be prominently 
documented - since that's going to confuse the vast majority of other 
users that almost certainly want to use the IMP authentication 
credentials.

I will use your input and try to make this clearer in the documentation.

You are assuming people will read the smtp parameters especifications 
after they already discarded the need of configuring smtp in IMP at all.

Also, the horde_auth explanation is a little ambiguous. It references 
horde's authentication, when actually it should reference horde 
mailer's authentication (in my case they differ). And when it says 
"those fields", it's not clear if those fields are the ones defined in 
horde mailer tab or in imp's backends.

Either way, it's confusing because you are led to think you do not 
need to set imp's smtp options when you are not overwriting horde 
mailer settings.

My suggestion would be something like this:

  * smtp: (array) These parameters will be used by IMP when
  *    sending email. If you want to use the default SMTP mailer
  *    already configured in Horde, you must set horde_auth
  *    parameter below. If you define any other parameters in this
  *    array, they will overwrite the default Horde SMTP parameters.


For the horde_auth explanation, I would suggest something like this:

  *     - horde_auth: (boolean) If true, populates the 'password' and
  *                   'username' parameters with Horde mailer's authentication
  *                   credentials (only if those fields are not 
defined in here).
  *                   Otherwise, IMP's authentication credentials are 
used instead
  *                   (this option only makes a difference if IMP's 
authentication
  *                   credenitals differ from Horde's).


It's just a suggestion and I'm sure it can be improved.

All of this must be very clear in your head because you wrote the 
program, but for those who just use it, I can assure you it's not that 
obvious and some clarification would certainly help. It's up to you. 
I'm just trying to contribute. Please receive my comments with an open 
spirit. Thanks.

From backends.php:

  *     - horde_auth: (boolean) If true, populates the 'password' and
  *                   'username' parameters with Horde's authentication
  *                   credentials if those fields are not defined. Otherwise,
  *                   IMP's authentication credentials are used instead (this
  *                   option only makes a difference if IMP's authentication
  *                   credenitals differ from Horde's).

I'm really not sure how much clearer we can make this.

Then I think the documentation needs clarification.

In backends.php we see:

smtp: (array) If Horde is configured to use SMTP as the mailer, 
entries defined in this array will overwrite the default Horde SMTP 
parameters.

So, it's pretty reasonable to imply if you don't overwrite the default 
Horde SMTP parameters, then they will be used by IMP. Specially 
because this was the previous behavior.

The way it is now, unless you ask for support, you won't just guess 
you need to set horde_auth to true in order to use parameters you 
didn't overwrite...


PS: In the $servers['advanced'] example, there's a parameter named 
'imp_auth' under smtp array, which I think is supposed to be 
'horde_auth'.

You are incorrect.  This changed for IMP 6.  You need the horde_auth 
parameter in IMP's backends.php, as has been mentioned several times.   
Your setup is working perfectly given your current configuration.

I my case, IMP credentials are equal to Horde credentials, since I'm 
using IMP as authentication system.

I have no SMTP credentials defined in imp's backends.php or 
backends.local.php. I was expecting it would use mailer credentials, 
as it always did in previous horde versions.

The problem is IMP is ignoring the username and password set in 
Administration -> Configuration -> Horde -> Mailer fields 
($conf[mailer][params][username] and $conf[mailer][params][password]).

I just installed horde in a fresh virtual machine and I could 
reproduce the problem again.

Here are the steps to reproduce it:

1 - Install horde and IMP via PEAR;

2 - Define IMP as auth driver for horde (Let a Horde application 
handle authentication);

3 - Let backends files untouched;

4 - Choose "Use a SMTP server (HIGHLY RECOMMENDED)" in Mailer config 
tab and set the proper fields, in my case:

$conf['mailer']['params']['host'] = 'smtp.googlemail.com';
$conf['mailer']['params']['port'] = 587;
$conf['mailer']['params']['auth'] = true;
$conf['mailer']['params']['username'] = 'john_doe@example.com.br';
$conf['mailer']['params']['password'] = 'password';
$conf['mailer']['type'] = 'smtp';

5 - Logout and login;

6 - Add Horde::debug($params); to lines 43 and 49 of 
imp/lib/Factory/Mail.php to see what username and password IMP is 
using before and after the array_merge function when you click "Send".

7 -Try to send an email.

8 - Look at horde_debug.txt

You will see the first Horde::debug() will return the correct username 
and password set in $params array, I mean, the ones defined in 
Configurarion -> Horde -> Mailer;

The second Horde::debug(), the one inserted after the array_merge 
function will contain IMP credentials, instead of the ones defined in 
Mailer tab, and, thus, the SMTP server authentication will fail.

Here's the output of horde_debug.txt:

2013-07-16T00:19:17+00:00 DEBUG: Variable information:
array(5) {
   ["host"]=>
   string(19) "smtp.googlemail.com"
   ["port"]=>
   int(587)
   ["auth"]=>
   bool(true)
   ["username"]=>
   string(22) "john_doe@example.com.br"                     <---- SMTP 
credential defined in Mailer tab (Ok!)
   ["password"]=>
   string(17) "password"                                               
        <---- SMTP credential defined in Mailer tab (Ok!)
}

Backtrace:
  1. Horde_Core_Ajax_Application->doAction() 
/var/www/horde/services/ajax.php:58
  2. call_user_func() /usr/share/php/Horde/Core/Ajax/Application.php:162
  3. IMP_Ajax_Application_Handler_Common->sendMessage()
  4. IMP_Compose->buildAndSendMessage() 
/var/www/horde/imp/lib/Ajax/Application/Handler/Common.php:528
  5. IMP_Compose->sendMessage() /var/www/horde/imp/lib/Compose.php:842
  6. Horde_Injector->getInstance() /var/www/horde/imp/lib/Compose.php:1066
  7. Horde_Injector->createInstance() /usr/share/php/Horde/Injector.php:247
  8. Horde_Injector_Binder_Factory->create() 
/usr/share/php/Horde/Injector.php:213
  9. IMP_Factory_Mail->create() 
/usr/share/php/Horde/Injector/Binder/Factory.php:111
10. Horde::debug() /var/www/horde/imp/lib/Factory/Mail.php:43

2013-07-16T00:19:17+00:00 DEBUG: Variable information:
array(5) {
   ["host"]=>
   string(19) "smtp.googlemail.com"
   ["port"]=>
   int(587)
   ["auth"]=>
   bool(true)
   ["username"]=>
   string(4) "test"                                          <------ 
IMP credential (Not Ok!)
   ["password"]=>
   string(3) "test"                                          <------ 
IMP credential (Not Ok!)
}

Backtrace:
  1. Horde_Core_Ajax_Application->doAction() 
/var/www/horde/services/ajax.php:58
  2. call_user_func() /usr/share/php/Horde/Core/Ajax/Application.php:162
  3. IMP_Ajax_Application_Handler_Common->sendMessage()
  4. IMP_Compose->buildAndSendMessage() 
/var/www/horde/imp/lib/Ajax/Application/Handler/Common.php:528
  5. IMP_Compose->sendMessage() /var/www/horde/imp/lib/Compose.php:842
  6. Horde_Injector->getInstance() /var/www/horde/imp/lib/Compose.php:1066
  7. Horde_Injector->createInstance() /usr/share/php/Horde/Injector.php:247
  8. Horde_Injector_Binder_Factory->create() 
/usr/share/php/Horde/Injector.php:213
  9. IMP_Factory_Mail->create() 
/usr/share/php/Horde/Injector/Binder/Factory.php:111
10. Horde::debug() /var/www/horde/imp/lib/Factory/Mail.php:49

If I comment out the array_merge function from lines 45 to 48 of 
imp/lib/Factory/Mail.php then username and password params defined in 
Configurarion -> Horde -> Mailer are used correctly and I can send 
emails. I did the test in the fresh install again and all went well.

So something during the process of this array_merge, which is executed 
by $injector->getInstance('IMP_Imap')->config->smtp is pulling the 
credentials from IMP instead of SMTP mailer params defined in Mailer 
tab of horde configuration.

can't reproduce.  I can verify that if you pass horde_auth as a smtp 
param in imp's backends.php, you will get Horde authentication 
credentials.  Without it, you will get IMP credentials.  There is no 
other way that you could possibly get Horde credentials **unless** you 
either define horde_auth, username, or password.

Yes, as I posted, one trace is from when I click "new message" and the 
other - the one I think matters - is when I click "send". I just 
showed them both because I don't really know what's going wrong.

What I realized is this. The username and password parameters are 
passed correctly - when I click send - until the array_merge performed 
on lines 45-48 of imp/lib/Factory/Mail.php. After that they get 
replaced by horde login credentials.

So I commented out this merge and things started working again. Mailer 
credentials defined in horde settings are now being used to send 
emails from IMP and I stopped getting the gmail authentication erros:

if ($transport == 'smtp') {
     /*Commented out as temporary FIX
     $params = array_merge(
         $params,
         $injector->getInstance('IMP_Imap')->config->smtp
     );
     */
     ...

https://github.com/horde/horde/blob/master/imp/lib/Factory/Mail.php#L45-L48

In fact, I have been running the system like this for days now, 
because for this set up I cannot use individual usernames/passwords 
for sending emails.

If I can perform any other tests, please instruct me.

Hope it helps.

My backends.php is untouched. I don't mess with it, as recommended in 
the docs. As for the backends.local.php, the only things I've added 
are imap options, because I'm using imapproxy (on the same server):

imp/config/backends.local.php
$servers['imap']['cache'] = true;
$servers['imap']['port'] = 1143;
$servers['imap']['secure'] = false;

What Jan said.  You have provided traces of 2 completely different 
SMTP object instantiations.  The first is caused by the Alarm code (it 
has nothing to do with IMP).

So there is no expectation that these two should be identical.

Do you have 'smtp' settings in IMP's backends.(local.)php?

My horde/config/conf.php:

$conf['mailer']['params']['host'] = 'smtp.gmail.com';
$conf['mailer']['params']['port'] = 587;
$conf['mailer']['params']['auth'] = true;
$conf['mailer']['params']['username'] = 'mailer@tesdomain.com.br';
$conf['mailer']['params']['password'] = 'gmail-dummy-password';
$conf['mailer']['type'] = 'smtp';

But IMP seems to be ignoring the above settings and is using horde 
credentials anyway to send the message.


Jul  5 22:49:47 mserver HORDE: [imp] Failed to set sender: 
xxxxxx@xxxxxx.xxx.xx [SMTP: Invalid response code received from server 
(code: 530, response: 5.5.1 Authentication Required. Learn more 
at#0125.5.1 http://support.google.com/mail/bin/answer.py?answer=14257 
h26sm17104593yhb.21 - gsmtp)] [pid 6800 on line 1068 of 
"/var/www/horde/imp/lib/Compose.php"]


I've added Horde::debug($this->_params) at line 147 of 
/usr/share/php/Horde/Mail/Transport/Smtp.php


When I click the *New Message* button I get this, which seems just fine:


2013-07-05T22:48:00-03:00 DEBUG: Variable information:
array(10) {
   ["auth"]=>
   bool(true)
   ["debug"]=>
   bool(false)
   ["host"]=>
   string(14) "smtp.gmail.com"
   ["localhost"]=>
   string(9) "localhost"
   ["password"]=>
   string(17) "gmail-dummy-password"                         <-- Ok!
   ["persist"]=>
   bool(false)
   ["pipelining"]=>
   bool(false)
   ["port"]=>
   int(587)
   ["timeout"]=>
   NULL
   ["username"]=>
   string(22) "mailer@tesdomain.com.br"                    <-- Ok!
}

Backtrace:
  1. IMP_Dynamic_Base->__construct() /var/www/horde/imp/dynamic.php:31
  2. IMP_Dynamic_Compose->_init() /var/www/horde/imp/lib/Dynamic/Base.php:90
  3. Horde_Notification_Handler->notify() 
/var/www/horde/imp/lib/Dynamic/Compose.php:260
  4. Horde_Notification_Handler_Decorator_Alarm->notify() 
/usr/share/php/Horde/Notification/Handler.php:317
  5. Horde_Core_Factory_Alarm->create() 
/usr/share/php/Horde/Notification/Handler/Decorator/Alarm.php:61
  6. Horde_Injector->getInstance() 
/usr/share/php/Horde/Core/Factory/Alarm.php:93
  7. Horde_Injector->createInstance() /usr/share/php/Horde/Injector.php:247
  8. Horde_Injector_Binder_Factory->create() 
/usr/share/php/Horde/Injector.php:213
  9. Horde_Core_Factory_MailBase->create() 
/usr/share/php/Horde/Injector/Binder/Factory.php:111
10. Horde_Core_Factory_Mail->create() 
/usr/share/php/Horde/Core/Factory/MailBase.php:37
11. Horde_Mail_Transport_Smtp->__construct() 
/usr/share/php/Horde/Core/Factory/Mail.php:55
12. Horde::debug() /usr/share/php/Horde/Mail/Transport/Smtp.php:147



But when I click the *Send* button on the compose window I get 
differente credentials. IMP pulls the credentials used to login to 
horde, and not the ones defined in $conf['mailer']['params'] array:



2013-07-06T01:49:44+00:00 DEBUG: Variable information:
array(10) {
   ["auth"]=>
   bool(true)
   ["debug"]=>
   bool(false)
   ["host"]=>
   string(14) "smtp.gmail.com"
   ["localhost"]=>
   string(9) "localhost"
   ["password"]=>
   string(6) "horde-dummy-password"                   <-- Not Ok!
   ["persist"]=>
   bool(false)
   ["pipelining"]=>
   bool(false)
   ["port"]=>
   int(587)
   ["timeout"]=>
   NULL
   ["username"]=>
   string(4) "horde-dummy-username"                   <-- Not Ok!
}

Backtrace:
  1. Horde_Core_Ajax_Application->doAction() 
/var/www/horde/services/ajax.php:58
  2. call_user_func() /usr/share/php/Horde/Core/Ajax/Application.php:162
  3. IMP_Ajax_Application_Handler_Common->sendMessage()
  4. IMP_Compose->buildAndSendMessage() 
/var/www/horde/imp/lib/Ajax/Application/Handler/Common.php:528
  5. IMP_Compose->sendMessage() /var/www/horde/imp/lib/Compose.php:842
  6. Horde_Injector->getInstance() /var/www/horde/imp/lib/Compose.php:1066
  7. Horde_Injector->createInstance() /usr/share/php/Horde/Injector.php:247
  8. Horde_Injector_Binder_Factory->create() 
/usr/share/php/Horde/Injector.php:213
  9. IMP_Factory_Mail->create() 
/usr/share/php/Horde/Injector/Binder/Factory.php:111
10. Horde_Core_Factory_Mail->create() 
/var/www/horde/imp/lib/Factory/Mail.php:64
11. Horde_Mail_Transport_Smtp->__construct() 
/usr/share/php/Horde/Core/Factory/Mail.php:55
12. Horde::debug() /usr/share/php/Horde/Mail/Transport/Smtp.php:147