Changes have been made in Git (master):

commit 11e1e868c6d1494323fe42f97f5b5a09eae71dce
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Mon Feb 20 00:00:59 2012 -0700

     Bug #10680: Use correct secret key to encrypt IMAP password

  imp/lib/Auth.php |   11 +++++++++++
  imp/package.xml  |    2 +-
  2 files changed, 12 insertions(+), 1 deletions(-)

http://git.horde.org/horde-git/-/commit/11e1e868c6d1494323fe42f97f5b5a09eae71dce

The authentication process has been rewritten in Horde 5/IMP 6.  I am 
not going to spend any more time personally trying to track down this 
issue in Horde 4/IMP 5.

Hi again,

applied patch from #9733 - but problem is still there. No success yet.

I'll try and will report asap if it is fixed.

Was this fixed by Bug #9733?

Some first report:

Patch does not seem to work. Both commits applied.
However - now thunderbird is asking again after time for username + 
password and i've got again those - authentication denied log messages 
(which was gone using session_control = 'none').

Ah nothing released yet - sorry should have looked before asking.
I am applying:

76faef61dc154be79b95946f6cd3c16c6c6f29a1
and
11e1e868c6d1494323fe42f97f5b5a09eae71dce

I'll report back if it works.

Yeah i would like to test it.
To confirm - i have to remove the patch with session_control = 'none' 
and upgrade to latest client? enough? Or have i have to apply the git 
patches manually yet to test?

If someone could report back on whether this fixes things, that would 
be great (since I was never able to reproduce in the first place).

Changes have been made in Git (develop):

commit 11e1e868c6d1494323fe42f97f5b5a09eae71dce
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Mon Feb 20 00:00:59 2012 -0700

     Bug #10680: Use correct secret key to encrypt IMAP password

  imp/lib/Auth.php |   11 +++++++++++
  imp/package.xml  |    2 +-
  2 files changed, 12 insertions(+), 1 deletions(-)

http://git.horde.org/horde-git/-/commit/11e1e868c6d1494323fe42f97f5b5a09eae71dce

Changes have been made in Git (develop):

commit 76faef61dc154be79b95946f6cd3c16c6c6f29a1
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Sun Feb 19 23:44:40 2012 -0700

     [mms] Add Horde_Imap_Client_Base#setParam() (Bug #10680).

  .../Imap_Client/doc/Horde/Imap/Client/UPGRADING    |    4 ++
  .../Imap_Client/lib/Horde/Imap/Client/Base.php     |   42 ++++++++++++++-----
  framework/Imap_Client/package.xml                  |    2 +
  3 files changed, 37 insertions(+), 11 deletions(-)

http://git.horde.org/horde-git/-/commit/76faef61dc154be79b95946f6cd3c16c6c6f29a1

Requires Horde_Imap_Client 1.5.0.  Currently only implemented hotfix 
in IMP 5.1.

Changes have been made in Git (master):

commit 76faef61dc154be79b95946f6cd3c16c6c6f29a1
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Sun Feb 19 23:44:40 2012 -0700

     [mms] Add Horde_Imap_Client_Base#setParam() (Bug #10680).

  .../Imap_Client/doc/Horde/Imap/Client/UPGRADING    |    4 ++
  .../Imap_Client/lib/Horde/Imap/Client/Base.php     |   42 ++++++++++++++-----
  framework/Imap_Client/package.xml                  |    2 +
  3 files changed, 37 insertions(+), 11 deletions(-)

http://git.horde.org/horde-git/-/commit/76faef61dc154be79b95946f6cd3c16c6c6f29a1

This is currently impossible to fix in IMP alone since there is no 
mechanism to change the stored password once the object is created.

I agree - this is what I figured out last week also.

Although I don't know if this is a limitation in Horde_Secret or an 
issue in IMP.  Because Horde_Secret doesn't clearly indicate in its 
API that this can occur.

I *think* this is what's happening (at least in my case):
- The user is logging without cookies
- Horde_Secret falls back to session_id()
- During the login process, the password is stored encrypted with session_id
- After logging in, the session id is generated to protect against 
session fixation
- The new session_id is no longer the valid key for the encrypted 
password, so decrypting fails

This could be related: I was trying cookie-less authentication for 
some other reason, which made IMAP authentication in follow-up 
requests fail. Most probably because the original secret was not used 
when decrypting the password.

I am seeing this one too - sorry:

array(2) {
    [0]=>
    string(3) "imp"
    [1]=>
    string(0) ""
}

But Bug #9733 does remove session_control = 'none' -> Michael S. told 
me to add this line (looks promising yet - enabled encryption and it 
seems to work - will tell you if this helps next week - need some time 
to verify things).
So the solution to remove the session_control = 'none' line would 
result in "original" code which does match the original report.

This is likely due to the same session issue as in Bug: 9733

This is not a good sign.  You should be seeing this also:

array(2) {
   [0]=>
   string(3) "imp"
   [1]=>
   string(0) ""
}

Thx for the input. Made the patch on line 211 + one at 154 where 
setCookie is called - i can confirm that only one call to setKey is 
done per request.
output is:

array(2) {
   [0]=>
   string(4) "auth"
   [1]=>
   string(0) ""
}

Using session_control = 'none' the key is always empty - guess thats correct?
Should i test it with session_control patch active or not?

Another debug option is to put the following in Horde_Secret on line 211:

     protected function _setCookie($keyname, $key)
     {
+     Horde::debug(array($keyname, $key));
         @setcookie(
             $keyname . '_key',

You should only see one call per RPC call to set the 'imp' key.

I can't reproduce doing the following:

1. Setup a shared calendar on the server that requires authentication 
(I'm using imp auth authentication).
2. Creating a new calendar in Thunderbird/Lightning.
3. Enter username/password
4. ICS file is downloaded.

I can verify in #3 that if I put in the wrong password the 
authentication fails.

Just for the heck of it... could you apply this diff:

diff --git a/horde/rpc.php b/horde/rpc.php
index a420f27..df8c789 100644
--- a/horde/rpc.php
+++ b/horde/rpc.php
@@ -42,6 +42,7 @@ if ((!empty($_SERVER['CONTENT_TYPE']) &&
  } elseif (!empty($_SERVER['PATH_INFO']) ||
            in_array($_SERVER['REQUEST_METHOD'], array('DELETE', 
'PROPFIND', 'PUT', 'OPTIONS'))) {
      $serverType = 'Webdav';
+    $session_control = 'none';
  } elseif (!empty($_SERVER['CONTENT_TYPE'])) {
      if (strpos($_SERVER['CONTENT_TYPE'], 
'application/vnd.syncml+xml') !== false) {
          $serverType = 'Syncml';

FYI: Here's what the Horde log looks like when authenticating via RPC:

2012-02-15T22:17:21-07:00 DEBUG: HORDE Load config file (hooks.php; 
app: horde) [pid 13427 on line 865 of 
"/disk2/src/horde/framework/Core/lib/Horde.php"]
2012-02-15T22:17:21-07:00 DEBUG: HORDE Hook browser_modify in 
application horde called. [pid 13427 on line 1826 of 
"/disk2/src/horde/framework/Core/lib/Horde.php"]
2012-02-15T22:17:21-07:00 DEBUG: HORDE Horde_Registry: retrieved app 
with cache ID 
horde_registry|app|1329184088|8bbed9e4eac921e350d64930c2cdc722 [pid 
13427 on line 1730 of 
"/disk2/src/horde/framework/Core/lib/Horde/Registry.php"]
2012-02-15T22:17:21-07:00 DEBUG: HORDE Load config file (hooks.php; 
app: imp) [pid 13427 on line 865 of 
"/disk2/src/horde/framework/Core/lib/Horde.php"]
2012-02-15T22:17:21-07:00 DEBUG: HORDE Hook preauthenticate in 
application imp called. [pid 13427 on line 1826 of 
"/disk2/src/horde/framework/Core/lib/Horde.php"]
2012-02-15T22:17:21-07:00 DEBUG: HORDE [imp] Load config file 
(conf.php; app: imp) [pid 13427 on line 865 of 
"/disk2/src/horde/framework/Core/lib/Horde.php"]
2012-02-15T22:17:21-07:00 DEBUG: HORDE [imp] Load config file 
(backends.php; app: imp) [pid 13427 on line 865 of 
"/disk2/src/horde/framework/Core/lib/Horde.php"]
2012-02-15T22:17:22-07:00 DEBUG: HORDE Hook preauthenticate in 
application imp called. [pid 13427 on line 1826 of 
"/disk2/src/horde/framework/Core/lib/Horde.php"]
2012-02-15T22:17:22-07:00 DEBUG: HORDE [gollem] Load config file 
(conf.php; app: gollem) [pid 13427 on line 865 of 
"/disk2/src/horde/framework/Core/lib/Horde.php"]
2012-02-15T22:17:22-07:00 DEBUG: HORDE [horde] Horde_Rpc::__construct 
complete [pid 13427 on line 96 of 
"/disk2/src/horde/framework/Rpc/lib/Horde/Rpc.php"]
2012-02-15T22:17:22-07:00 DEBUG: HORDE [horde] Hook preauthenticate in 
application imp called. [pid 13427 on line 1826 of 
"/disk2/src/horde/framework/Core/lib/Horde.php"]
2012-02-15T22:17:22-07:00 DEBUG: HORDE [horde] Hook postauthenticate 
in application imp called. [pid 13427 on line 1826 of 
"/disk2/src/horde/framework/Core/lib/Horde.php"]

Wondering if this might have something to do with this:

http://lists.horde.org/archives/dev/Week-of-Mon-20120213/026993.html

which was (possibly) fixed by this:

http://lists.horde.org/archives/commits/2012-February/013996.html

Most likely there are more than 1 secret key being used to 
encrypt/decrypt the password.  You will have to trace the code to 
discover why this is happening (adding Horde::debug() calls to 
IMP_Imap::getEncryptKey() to track the key value would be useful).

Horde::debug() documentation: http://wiki.horde.org/Doc/Dev/DebugH4

Some Update/Workaround i've found so far:

In /usr/share/php/Horde/Imap/Client/Base.php it does help to comment 
out the encryption stuff:

  175         // Encrypt password.
  176         /*
  177         try {
  178             $encrypt_key = $this->_getEncryptKey();
  179             if (strlen($encrypt_key)) {
  180                 $secret = new Horde_Secret();
  181                 $this->_params['password'] = 
$secret->write($encrypt_key, $this->_params['password']);
  182                 $this->_params['_passencrypt'] = true;
  183             }
  184         } catch (Exception $e) {}
  185         */

After this things work - althoug the password won't be encrypted 
anymore in the session. But if this does help, Horde_Secret stuff must 
be the cause. Anyone an idea what may be happen here?

This is precisely the issue - and what I have no knowledge of.  So you 
have to track this down yourself.

I'll would try to  make a patch, if somebody can help me to understand 
how the token process - encryption of the password in the session - 
should work do check where it fails; as it seems its the culprit here, 
because the provided user password from the request is correct.

Somebody who can reproduce this on their system is going to have to 
provide a patch.

Seems I have the same problem. Any progress?

I did some more testing and can tell this:

1. The password sent from the client (tbird+lightning) is correct, 
looking at the PHP_AUTH_PW line.
2. The stacktrace of the rpc call looks like this:

  1. Horde_Rpc_Webdav->getResponse() /var/www/horde/rpc.php:146
  2. Horde_Rpc_Webdav->ServeRequest() /usr/share/php/Horde/Rpc/Webdav.php:250
  3. Horde_Rpc_Webdav->_check_auth() /usr/share/php/Horde/Rpc/Webdav.php:953
  4. Horde_Rpc_Webdav->check_auth() /usr/share/php/Horde/Rpc/Webdav.php:2428
  5. Horde_Core_Auth_Application->authenticate() 
/usr/share/php/Horde/Rpc/Webdav.php:832
  6. Horde_Core_Auth_Application->authenticate() 
/usr/share/php/Horde/Core/Auth/Application.php:129
  7. Horde_Auth_Base->authenticate() 
/usr/share/php/Horde/Core/Auth/Application.php:132
  8. Horde_Core_Auth_Application->_authenticate() 
/usr/share/php/Horde/Auth/Base.php:146
  9. Horde_Registry->callAppMethod() 
/usr/share/php/Horde/Core/Auth/Application.php:161
10. call_user_func_array() /usr/share/php/Horde/Registry.php:1083
11. IMP_Application->authAuthenticate()
12. IMP_Auth::authenticate() /var/www/horde/imp/lib/Application.php:409
13. Horde::debug() /var/www/horde/imp/lib/Auth.php:80

3. The encoded session password taken from the imap->ob is garbage 
like reported.
4. The password send to the imap socket (verified via debug statement 
at lib level) is garbage too - but it differs from the one from 3.

I am stuck here. The client did sent the correct credentials - but 
credentials sent to the socket are wrong and does differ from the 
encoded one, to decoding seems to take place but there something is 
going wrong.

Adding Jan.  I am not familiar enough with kronolith to know where 
these RPC requests are generated/handled, so he is going to be more 
helpful regarding this.

Although it is very troubling that this works "some of the time".   
There's nothing worse than trying to track down an intermittent event.

Yes auth via imp is configured.
URLs are kronolith calendar ones like, e.g.:

https://<HOST>/horde/rpc.php/kronolith/horde-test@software-friends.de/pyBNNE7pT-BN06bxQTExq_A.ics

Errors are occuring if Thunderbirds lightning calendar extension wants 
to download the ics file; this fails with the mentioned auth denied 
errors (details like imap server output with auth details and debug 
stuff from imap session already mentioned in previous comments).
Sometimes it does succeed to fetch the ics - but in the long run, 
thunderbird is asking for credentials over and over again as the 
session becomes broken, although credentials submitted are valid.

Some note - maybe its relevant - i'll to not yet see this failure 
using firefox plain https request, evolution or kontact.
It does fail @thunderbird only it seems - but because lightning does 
submit correct credentials, it should not be the fault of the bird, 
shouldn't it?

Going to start back at the beginning and try to reproduce this locally.

IIRC, your Horde is set up to use imp authentication?
How/when are you seeing these errors.  You mentioned RPC calls - 
provide full details as to these calls (e.g. URLs).

Thats the password output:

     ["password"]=>
     string(16) "?k      ^Z)<B6>b<B0>.p<BE>^W<D8>\<9B>"

thats the _passencrypt output:

     ["_passencrypt"]=>
     bool(true)

Going back to Comment #11 below, is there any chance of checking to 
see if the Horde_Imap object params array contains the '_passencrypt' 
key?

It *looks* like the IMAP server is being sent the encrypted version of 
the password rather than the plaintext version.  The question becomes 
why this is happening?

I can tell you what my imap server does receive from horde, like 
mentioned in comment #9, thats the dovecot auth output:

MD5-CRYPT(;3<9A><CA><F1><B2>g:<FC><F1><F8>g<E3><FD><96>*) !=
'$1$a0dd526a$POYliZyTPtYiz.FGbo.0H0'

So it did receive this:

;3<9A><CA><F1><B2>g:<FC><F1><F8>g<E3><FD><96>*


The input from the user is correct, verified via debug statement in 
the horde code you told me.

What to do now? Should i still enter debug statements to the horde 
imap client code?

No, this should be correct.  The password is encrypted within the session.

The only way to debug this will be to find out exactly what Horde is 
sending to the IMAP server as a password.  And this will either 
require you to set up debugging on your IMAP server or by manually 
entering debug statements in the IMAP client code 
(Horde_Imap_Client_Socket class - _tryLogin() function).

Looking at the output it may contain sensitive data, right? At least 
there a password string in the imap object output - which does not 
macht the password from the credentials debug output:

["password"]=>
     string(16) "?k        )?b?.p??\?"



Is this what you are looking for? If you still need te dump please let 
me know, i'll sent you a private mail than.

It almost looks as if the cached imap object is being incorrectly 
generated.  Right before the "if (!$imp_imap->ob)" line, insert a 
debug statement:

Horde::debug($imp_imap->ob)

And report the debug output back here.

Debugged things but inserted the line 102, because the check:

if (!$imp_imap->ob)

is false.

The password is what i expect, but the raw imaplog does not show those 
password or the login command. The imap server does still deny 
authentication (and raw logs from dovecot) are showing "garbage" as 
password, not what the user entered above.

imap session output from horde:

S: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID 
ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
C: [SASL-IR AUTHENTICATE Command - username: horde-test@software-friends.de]
S: 1 NO [AUTHENTICATIONFAILED] Authentication failed.
C: [LOGIN Command - username: horde-test@software-friends.de]
S: 2 NO [AUTHENTICATIONFAILED] Authentication failed.

dovecot auth output:

MD5-CRYPT(;3<9A><CA><F1><B2>g:<FC><F1><F8>g<E3><FD><96>*) != 
'$1$a0dd526a$POYliZyTPtYiz.FGbo.0H0'


So what dovecot does receive, is not what the user did entered (and is 
show from the horde:debug line). Any ideas where things may break?

Re-looking at the imap client code, the authentication password is 
never output to the debug stream.  So this won't work.

Instead, we will need to manually debug the code.  In 
imp/lib/Auth.php, on line 88 (just before the line 
$imp_imap->createImapObject($credentials['userId'], 
$credentials['password'], $credentials['server']);), insert the 
following:

Horde::debug($credentials);

It should look like:

             try {
                 Horde::debug($credentials);
                 $imp_imap->createImapObject($credentials['userId'], 
$credentials['password'], $credentials['server']);
             } catch (IMP_Imap_Exception $e) {
                 self::_logMessage(false, $imp_imap);
                 throw $e->authException();
             }

Then check the debug output file (see below for further information).   
You are looking for the 'password' entry - see if the password is what 
you expect for that user.

----

Horde::debug() documentation: http://wiki.horde.org/Doc/Dev/DebugH4

So got some sample data - don't know if it does help you:

1. calendar via kornolith does fail - email does work, login to horde 
webfrontend
2. output from imap log does read:

S: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID 
ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
C: [SASL-IR AUTHENTICATE Command - username: horde-test@software-friends.de]
S: 1 NO [AUTHENTICATIONFAILED] Authentication failed.
C: [LOGIN Command - username: horde-test@software-friends.de]
S: 2 NO [AUTHENTICATIONFAILED] Authentication failed.

For now this is irrelevant.  You need to look at the IMAP logs 
generated by IMP and see why it is not authenticating correctly.   
unfortunately, you will have to do this on your own since this deals 
with sensitive data (passwords) that you probably don't want to share 
with the world.

Moving to kronolith queue for now.

What do you mean here with form? The failure case is only for RPC 
calls - it does work on the "http form" everytime. But e.g. kronolith 
calendars getch via http + basic auth does fail.

I do not mangle passwords or something else. Did only describe what 
users are reporting and what i've got from logs.
Will debug_raw = true also work for the successful "direct" imap 
authentication via horde or what is needed there for raw logs?

This doesn't make any sense, unless you are doing something to 
munge/mangle the password/username combination in Horde/IMP.  The 
login form is identical whether you are using 'IMP' authentication or 
'IMAP' authentication. (Similarly, this has nothing to do with special 
characters since the login form is the same).

You will need to provide full debug logs (debug_raw = true in 
imp/config/backends.php) of a successful vs. unsuccessful login for a 
given username for us to begin to debug this.

Authentication is done via IMAP. So i have 2 options - let imp handle 
things or do imap directly.
IMAP does work fine (with patch from #10679) for all my user accounts.
Using IMP (which does use imap too in the backend) should work too - 
but it does not for all accounts. Some users are unable to login and 
dovecot logs reports invalid passwords provided.
Looking at the debug output from dovecot it seems, that if imp is used 
to provide authentication, some passwords (guess with special chars in 
it - only a guess) are not passed correct to the imap server - they 
are different from what the user does provide.
Switching to imap auth using the same password does work. Looks like a 
bug to me in imp authentication - can't tell where exactly but thats 
the behaviour seen.