Any news with respect this issue?

Assigning back to horde folks to look at the new patch.

Hi,



I am providing a new patch for Signing messages with Firefox.

This version lets compose.php do all the MIME wrapping and then 
resubmit the signed message. I have tried to change as little code as 
possible.



This feature works for Horde 3.1.3 and IMP 4.1.3 releases, and Firefox 
only for the time being. I will port this code to HEAD version as soon 
as I have some more functionality.



To try this patch:

1) Download Horde 3.1.3 and IMP 4.1.3 releases.

2) Untar them and install as documented

3) Go to imp/config/prefs.php and set a 1 to use_smime value.

4) Patch the installation. Under linux I issue "patch -p0 < 
ffox-resubmit-noattachs.patch.2" outside horde directory.



You can sign plain text messages with no attachments.

This version is a kind of "demo" to show how easy it gets for 
unexperienced users to sign messages. It requires no certificate ( 
with private key) uploading, wich is the main reason I am working on 
this.



All comments are welcome.



Thanks,



Mariano.

I have detected another bug. Don't type special characters either for 
the time being, as the email comes with escapes them, thus making the 
content differ from the original data that has been signed.



In later versions I will move away the code in Javascript to the 
existing MIME functionality in IMP's phps.

Hi,



I have corrected a few things in the patch, so I attach a second version.

I hope this one is easier to use with the patch command. All files are 
relative to the directory containing the horde module, so it should be 
clear where a file is.



I had found a restriction to the mails you can sign right now. The 
lines in the message you write in the TEXTAREA should not wrap. I 
suggest you do not exceed 60 chars. Line breaks are handled alright 
for now. As I mentioned earlier, this implementation is just to show 
it can be done.



Bye,



Mariano.

I have produced a patch for Horde 3.1.3 IMP 4.1.3 releases, since I 
could not run the HEAD versions of these due to multiple errors. 
Nevermind, I intend to apply this feature to the final releases of 
Horde/IMP so this is fine for now. Later, I will produce a patch for 
the HEAD since I wish to integrate it in future Horde releases.

The patch file includes (hopefully) all necessary changes. I generated 
it using regular "diff -u". I could not do it with "cvs diff" due to 
bug #5017. Note that the file "SMIME_Delegate_Encryption.php" is a new 
file. To include it in the patch I created an empty 
"SMIME_Delegate_Encryption.php.newfile" and made a diff between them.



About the changes.

I have gone back to the minimal implementation of this feature, 
meaning I am doind it "hackishlky", as Jan Schneider commented. But 
the goal of the patch I am uploading right now is POC (Proof of 
concept). We will take the code to a better place later.



I really wish you give it a try. The install is straightforward. Use 
Horde 3.1.3 and IMP 4.1.3 final releases. No special Apache SSL nor 
OpenSSL required.

Apply this patch and try composing a text message with no attachments. 
Select "Sign message with SMIME" from the drop-down list and send it.

You will need to have a personal certificate in your FireFox capable 
of signing, of course. And I say Firefox. IE will come later.



Please tell me if it worked all right for you.



Thanks a lot.

If you can just upload an updated patch here, that would be great. Thanks!

Hi, I am back on this feature again.

I had a working version of IMP + client signing feature some months ago.

My intention is to add this feature to IMP. However, the patch is 
outdated right now. How can I keep this patch updated?



Thanks.

Mariano.

I will try to take a look at this as soon as possible.  I just don't 
have the time right now.

I finally know why new files are not included in the patch with cvs 
diff command.

It's becouse these files are not added to CVS, and I can't add them.

So, I will attach the new class here in case you are interested.

I am refactoring the code to minimize changes.

My goal now is to keep original openssl signature implementation 
untainted, as well as providing the new browser signing feature.

The patch has no major changes, only refactoring.

I said before my idea is to extend IMP, not to change code. So I 
created a new class IMP_SMIME_Delegate_Encryption extending IMP_SMIME 
that makes no use of openssl. Instead, it just uses the signature 
passed from the web form and set into this new object.

I had to extract some functionality from superclass 
Horde_Crypt_smime::signMIMEPart() into a new method 
Horde_Crypt_smime::wrapMessageInSMIME() so it could be called from the 
new class.

I guess the new class's name is awful. I would appreciate any 
suggestion of a better name for it.

I attach horde-imp-signing-2.patch

There is a lot more to do regarding functionality, but I want to clean 
up this code first while it's still small.

As usual, any comments are welcome!



Mariano.

Very well, I am submitting a patch for signing a simple text message 
with Mozilla Firefox.

Horde IMP needs to be configured for signing as if server side signing 
would be performed.

The code is far from its final version. I trust we can refactor it 
little by little.

I need help particularly for the IMP_Compose::_createMimeMessage() 
method. This is where I wrap a multipart/signed MIME around the message.

But I know that I am doing work here that is surely being done 
somewhere else. Only I don't know where.



Here is the list of files returned by "cvs -n -q update":



? imp/js/src/base64.js

M imp/compose.php

M imp/lib/Compose.php

M imp/lib/base.php

M imp/templates/compose/compose.html

M imp/templates/javascript/src/compose.js

C imp/templates/smime/import_key.html



I am not keen on cvs command line, so any help is appreciated.

The patch was generated by issuing "cvs -q diff  > patch" in horde directory.

Theres is a flie missing in this patch, base64.js wich is marked with 
"?" in the list, but I don't know what it means and why cvs diff does 
not include its source code in the patch. You don't need it unless you 
really want to try this patch.



All comments are welcome.



Mariano.

You're right, I am sorry for this text/plain confusion. I know the 
S/MIME wraps around whatever type of MIME is created. I just 
referenced text/plain for it is the simplest example, and the one I am 
testing this changes with.

The AJAX thing was about keeping the message sending flow control, 
particularly the switch case 'send_message'. My intention is to modify 
as few lines as little. In fact, I was expecting not to modify a 
single line of PHP code, but to extend it by subclassing and composing 
new objects.

Nevermind, I carry on the form resubmit strategy. As I get deeper into 
this I see some of the things I wrote on the Brief list are not 
nessesary.

I will submit a patch for signing a simple text/plain email the moment 
it works.

Thanks again Jan.

I think you misunderstand how signing works. If signing messages with 
s/mime, the *complete* message is being signed, not only the 
text/plain part containing the message body. It could be a 
multipart/alternative message if sending html messages or a 
multipart/mixed message if sending messages with attachements or a 
combination of both.

In any case the complete message is being signed, which means that you 
can simply send the body field with ajax to the server to be signed, 
but you need to send the complete form, build the complete message, 
send it back, sign it, put it into a hidden form field, submit the 
form again, construct a new multipart/signed message now with the 
signature in a separate mime part. Sounds familar? It's exactly the 
same amount of work necessary to do it with the form or with ajax, but 
the ajax version is a better user experience because it happens in the 
background.

I think Michael can comment more on the message building process.

I gave it another look at this and I believe some of the 
un/marshalling (i mean serializing) is unnecesary, just by changing 
the way I suggested to chop the switch case. The variables for 
buildAndSendMessage() are taken from the form submition. If I move up 
case 'submit_mime' to the top, all the variables will be re taken.



I am working to confirm this. Maybe the re-submition strategy isn't 
that painful, although I think we could do it better with Ajax later.

I don't know if you are interested in code details.

So I prepared a Brief and an Extended version,

both for AJAX and Without AJAX.

The extended version helps me to organize myself

(and also document these changes).

Here is the flow control of composing, singning and sending a message

the way I know today.



Without AJAX:

~~~~~~~~~~~~~

Brief:

------

1) Submit the form as usual.

2) Build a text/plain MIME for each recipient.

3) Marshall return values as form hidden inputs.

4) Render the compose page again.

5) Hook on onload event to sign text/plain MIMES.

6) ReSubmit this form.

7) Unmarshall submitted form hidden inputs to carry on.

8) Build a multipart/signed for each pair of (text/plain, signature)

9) Now send all as usual.



Extended:

---------

1) Send the form.



The page is loaded, the compose form is fulfilled, and submitted

with actionID="send-message'.



2) Build a text/plain MIME for each recipient.



/imp/compose.php is called, the following code is executed



case 'send_message':

...

$imp_compose->buildAndSendMessage(...);



My goal is to split this function in two, because I need to sing

in the browser between "build" and "send".

Right now, the patch I sent "composed" the text/plain part of the MIME

in a Javascript function.

I should leave that to the "build" part of this function.

So, I need 2 new functions:

/imp/lib/Compose.php::buildMessage()

/imp/lib/Compose.php::sendMessage()

The original buildAndSendMessage() is kept for backwards compatibility

and calls these two new ones.

I have done this refactoring and the coupling between these is strong.

Many parameters and array of return values.



     function buildAndSendMessage(<original parameters>)

     {

         list($messagesToSend, $recipients, $messageToSave) =

                         $this->buildMessage(<original parameters>);

         /**

          * Vars in the list() is the coupling between buildMessage()

          * and sendMessage()

          */

         $sent_saved = $this->sendMessages($messagesToSend, $recipients,

                                  $messageToSave, <original parameters>);

         return $sent_saved;

     }



Besides, buildMessage() does some work if there are multiple recipients

when signing, so <signing flag> so say "true":



     if (<signing flag>) {

         /* Must encrypt & send the message one recipient at a time. */

         foreach ($recipientArray as $val) {

             $res = $this->_createMimeMessage(array($val), $body, $encrypt);

             ...

         }

         ...

     }



Here, buildMessage() calls _createMimeMessage(), one for each recipient.

_createMimeMessage() is exactly the piece of code I want to replace the

Javascript with. However, this function also encrypts, wich I don't need right

now since encryption is carried out later on the browser.

I just want it to compose the text/plain part.



     function _createMimeMessage($to, $body, $encrypt)

     {

         ...

         <mostly encryption code>

         ...

         /* Add data to MIME_Message object. */

         $mime_message->addPart($body);\

         ...

         return array('recipients' => $to,

                      'to' => implode(', ', $to),

                      'msg' => &$mime_message);

     }



3) Marshall return values as form hidden inputs.



When all this calls return, I have to break the switch statement at

/imp/compose.php.



Right here I have an array of 3 values returned from buildMessage();





     function buildMessage(...)

     {

         ...

         return array($messagesToSend, $recipients, $messageToSave);

     }



     list($messagesToSend, $recipients, $messageToSave) = buildMessage();



In the original normal flow of actionID='send_message' these 3 values

are needed by the following lines of code of this switch case.

But now, I must break the flow and show the compose form again

with these 3 values rendered in the form as hidden inputs.

This task is not pretty. For example, $messagesToSend is an array of 3 
objects,

one of them is an IMP_MIME wich I have to transform toCanonicalString().

I have not completed this marshalling yet, $recipients and $messageToSave

are missing. Here is a bit of code from /imp/compose.php:



     /* Prepare the messages in the form to be signed in browser */

     if (isset($messagesToSend)) {

         $canonical_mimes = array();

         foreach ($messagesToSend as $mime_message )

             $canonical_mimes[] = $mime_message['msg']->toCanonicalString();

         $t->set('messagesToSend', $canonical_mimes);

     }



4) Render the compose page again.



This is done in /imp/templates/compose/compose.html:



<!-- These are the MIMEs to be signed -->

<loop:messagesToSend>

<input type="hidden" name="messagesToSend" value="<tag:messagesToSend />" />

</loop:messagesToSend>



Wich should write some hmtl code like this:



<input type="hidden" name="messagesToSend" value="Content-type: 
text/plain ..." />

<input type="hidden" name="messagesToSend" value="Content-type: 
text/plain ..." />

<input type="hidden" name="recipients" value="..." />

<input type="hidden" name="messageToSave" value="..." />



I am missing all code from here. It is not in the patch I attached just now.

5) Hook on onload event to sign text/plain MIMES.



With all of this values on the form, I have to hook page onload event

to fire the javascript that takes the text/plain messagesToSend and 
signs them.



6) ReSubmit this form.



7) Unmarshall submitted form hidden inputs to carry on.



The switch structure on /imp/compose.php now looks like this:



case 'send_message':

case 'compose_mime':

     ...

     if ($actionID == 'compose_mime') {

         $marshall_all_this = $imp_compose->buildMessage(...);

         break;

     }



case 'send_mime':



     if ($actionID == 'send_mime') {

         /* We are coming from a form submit */

         $unmarshalled_values = not_implemented_yet(<Util::getFormData()>);

         $sent = $imp_compose->sendMessages($unmarshalled_values);

     } else {

         /* The switch case did not break, so al usual variables are 
available */

         $sent = $imp_compose->buildAndSendMessage(<usual parameters>);

     }



8) Build a multipart/signed for each pair of (text/plain, signature)



Here's another problem, since now I would need to call buildMessage() again,

so a multipart/signed is composed with every signed text/plain.

Remember, we already called it to compose the text/plain.



9) Now send all as usual.



With the multipart/signed already composed, now I can send the messages

calling the refactored sendMessages().

But before I have to re-construct the parameters we stored as hiddens

in the form to pass them to this functions. Another not pretty task.



If all this succeeds, the messages are sent as multipart/signed.



The AJAX way:

~~~~~~~~~~~~~

Introduction:

-------------

I have written no code at all at this time. I have little experience

with AJAX, so maybe some of this can't be done the way I wished.

This approach should mean very little PHP code changes, plus a better

user experience. I am going to need much advise on how to embed Ajax

in the PHP templates, if this strategy is finally accepted.



Brief:

------

1) Fire the javascript signing function on form submit.

2) Call imp/lib/Compose.php::IMP_Compose::_createMimeMessage($to, 
$body, $encrypt)

    via Ajax to compose text/plain MIME(s if multiple recipients).

3) Sign the text/plain part returned by the Ajax call.

4) Set the signature(s if multiple recipients) as hidden inputs.

5) Submit this form with actionID='send_message'.



Extended:

---------



1) Fire the javascript signing function on form submit.



This change is already in the first patch. uniqSubmit() function

calls signMessage() if singning has been requested by selecting it

in the drop-down list.



2) Call imp/lib/Compose.php::IMP_Compose::_createMimeMessage($to, 
$body, $encrypt)

    via Ajax to compose text/plain MIME(s if multiple recipients).



To sign the message we need the message itself as a text/plain MIME.

This is provided by the PHP server side by wrapping the call to

class IMP_Compose. We need to expose _createMimeMessage() method to return

one or multiple text/plain MIMEs depending on how many recipients there are.



3) Sign the text/plain part returned by the Ajax call.

& 4) Set the signature(s if multiple recipients) as hidden inputs.



I sign each text/plain part we retrieved by AJAX, storing each

signature as a form hidden input.



5) Submit this form with actionID='send_message'.



Just go on and submit the form as usual.

Do not encrypt when buildAndSendMessage() calls _createMimeMessage(),

just compose the multipart/signed with the signature provided.

This change is already on the first patch (how do you reference different

patches so not to bring confusion, do you number/timestamp them?).



Feel free to correct my English. In fact, I would appreciate it.



Thanks,



Mariano.

Of course, but why would the help with the splitting of message 
building and sending?
Sure, talking about some real code is always easier.
Here is fine.

Hello,

I am making the changes to compose MIMEs in server side and re-submit 
them once the browser has signed each.

I am trying hard not to change much code, but honestly, I would need 
to do plenty function refactoring. For example, I have to split some 
functions like imp/lib/Compose.php::buildAndSendMessage() into 
buildMessage() and sendMessage(). The making and ecryption of the 
message flow is pretty hard to change.

Wouldn't you consider the AJAX alternative?

This way I could jump/skip to just the piece of code I need to execute 
on the server side and return without storing MIMEs in hidden inputs 
in the form, and leaving the flow of actionID='send_message' alone.

I could send you the changes so far as patched.



One last thing, Is this the right place to discuss this? Or would you 
prefer to make this comments on dev lists and reference this ticket?



Thanks.



Mariano.

That assumption is fine for now, we can sort it out later.
Does S/MIME not support multiple recipients in one message like PGP 
does? Anyway, it shouldn't make much difference whether you keep one 
mime type or several in the form. It's a matter of size though.
Not sure what you're asking here.

Given that this will require javascript anyway, why not use ajax? 
Should be a much better user experience.

First I'll try to configure the HEAD version I checked out and patched 
and see that signing emails is working all right before I make any 
more changes.



Then I'll tackle the serve side MIME composing, spliting actionID 
'send_message' in two parts for separate reuse as we agreed, taking 
care that current 'send_message' functionality is untainted.



Regarding wether detecting browser singing capability is possible, it 
is. However, that's not the problem. All signing behaviour is enabled 
by the IMP_SMIME_* constants, as expected. This constants are true 
only if you configured IMP (openssl, https, etc) wich is linked to the 
current server side signing strategy. So, though I can detect browser 
capability to sign, if I did not configure the server side signing 
strategy (openssl, etc) these IMP_SMIME_* constants will be false, 
excluding, for example, the lines of code in 'send_message' that 
creates multiple MIMEs (one for each recipient). The check box in the 
page will not show either. What is worse, I do not know how many other 
lines of code will not execute if we do not configure IMP for current 
signing strategy.

To sum it up, I can't make these two signing strategies independent. For now.

So, just for now, I will assume that, although I can detect browser 
signing, IMP is configured for server side signing anyway, so 
IMP_SMIME_* constants are true.



One shorter question. When signing an mail for multiple recipients, 
multiple MIMEs are created. How will we store these multiple MIMEs in 
the HTML form? We need all of them in the page for finally re-submit 
the form and send the messages.



In the 'send_message' actionID these are stored in the array 
$messagesToSend[], but if we split this code block in two, this array 
is comgin from the re-submition of the form, after the browser has 
signed each of them. How do we connect these two parts of the code 
block now?



I am so sorry this post is this lengthy. I won't post for 2 weeks so I 
want to ask all questions that pop up.



Thanks for your patience.



Mariano.

Yeah, I thought about using AJAX too initially, but I guess the form 
solution is easier for a start.
Correct.
Correct.
That's not too hard, the first case block would be called for 
"compose_mime" and "send_message", the second for "send_mime" and 
"send_message". You only need to take care that the first block 
doesn't break out at the end if the action is "send_message".
Is it possible to detect through the crypto api if client side 
crypting/signing is available?
Read docs/INSTALL and ask on the mailing list.

Completely agree with you, thanks for the comment.

When I was coding this, I thought about exposing the MIME creation 
function _createMimeMessage($to, $body) in a way I can call it AJAX 
style, so the double submit would not be needed. I would just replace 
my javascript code that composes the MIME hackishly and make an AJAX 
call the server side to compose it, then continue signing and finally 
submit it. However, I preferred to leave this strategy for later 
discussion with you. Besides, I don't think we could call the MIME 
composing part alone, could we? What do you think about it?



Getting back to your idea, I also thought about it, though I am not 
sure how to double submit the form.

The user first click the submit, we set actionID to *COMPOSE_MIME* to 
compose the MIME and not send it. The page would render again but here 
we need a javascript function to hook in body onload event and 
re-submit the form, this time setting actionID to *SEND_MESSAGE_AS_IS* 
so no other action is taken, only send the messages.



This is much like splitting current actionID 'send_message' in two. 
One that composes, the other just sends messages away.



I think we need to find a way o reusing the code block of actionID 
'send_message' but not sure how. I tried to change as little as I could.



We also would need to add another config preference other than 
IMP_SMIME_ENCRYPT so not to override current signing functionality.



Finally, I am now in a bit of a mess with versions. First I downloaded 
Horde 3.1.1 and IMP 4.1.1 to get it up and running fast and changed 
code there.

Then I checked out HEAD to produce this patch we are discussing, but I 
can not run this checked out source as it is, since I do not know how 
to configure it with apache. All modules are at the same level wich is 
different from release code where IMP directory goes inside Horde 
directory and so forth. In short, can you help me get up and running 
in Apache this checked out version of IMP so I can test the changes?



I'll be on hollidays for 2 weeks, so see you then.

And sorry If it takes me too many words to explain things.



Thanks,



Mariano.

This is indeed pretty simple, but the implementation is too hackish. 
There is much more happening inside IMP to create a valid mime part 
than what you have implemented in javascript.



If we really need to sign the complete mime part, we need to implement 
this differently. We could submit the form using a different actionID, 
one that completes all steps of the send_message action but doesn't 
send the message finally. Instead we would return to the compose view 
with the complete mime part, trigger the javascript to sign the part, 
and submit it again, this time with yet another actionID the runs the 
second half of the send_message action, but with the signed mime part.

Feedback in this case means that _we_ owe feedback on the patch.

This patch allows IMP configured for S/MIME signing to sign emails in 
FireFox, thus avoiding uploading certificates/private keys to the 
server and signing with PKCS#11 devices.



The patch is only to get advice from experienced Horde developers on 
how to put these changes in the right way. I know beforehand this is 
not the right way to extend IMP functionality, but as you'll see I 
profited from S/MIME code already there to tackle this proof of concept.



The goal is to provide signing/encryption capabilities for FireFox and IE.

At this point, you might just want to attache whatever code you have 
that currently works with the browser to accomplish this.  I (or 
someone else) can either look at this code and tell you where we need 
to add this to Horde/IMP; at the least, the code could be used as a 
reference if a developer wanted to help out on this task.

Take a look at http://www.horde.org/source/contribute.php

I accomplished signing an email with FireFox and verifying the 
signature with Outlook Express and Thunderbird.

I am now trying to look for some support to developers close to Horde 
to guide me through the patching.

Patch? :) You're probably going to need to supply at least equipment, 
and possibly either hire someone or sponsor a bounty to get this done, 
unless one of the core devs encounters a need for it.

I like the signing capabilities in IMP 4, but I wished it was done 
another way.

Rith now, the user has to upload the PKCS12 and trusted certificates.

This makes it impossible for Smart Cards to sign mails.



So, how about signing with the browser's certificates?

Yes, I am talking about CAPICOM, SignText etc.



This way the Keyring and certificate uploading would be avoided.