Tickets :: Comment on [#8535] Net_IMSP Sends too many strings as literals

* Your Email Address
* Spam protection	Enter the letters below: . ,. ..__..__..__ \./ \ /\| \|[__][__) \| \/ \|__\|\| \|\|
Comment	> Some users on our system have very large and complicated sets of > address books stored on a cyrus IMSP server. Performance of the IMSP > backend is a serious issue for us. One of the things I've noticed in > comparing IMSP conversations between horde's IMSP client and say, > Mulberry's, is that horde is sending many more strings as literals > instead of double quoted strings. From RFC, "In the case of literals > transmitted from client to server, the client must wait to receive a > command continuation request (described later in this document) > before sending the octet data (and the remainder of the command)." > Every time a string is sent as a literal there's an extra To/From > with the server, so per user, per request, it can add up. > > > > I didn't do a lot of time benchmarking to back this up, but this > seems like one of the easiest and best ways to improve performance. > In a particular address book on our system, the number of separate > IMSP sends To: the server on each request for Turba's browse > interface would normally be around 1900. More conservative use of > literals drops that to around 550 for the same request. We have a > custom mass batching fetchaddress function that also helps, so > mileage will vary. > > > > Literal usage in the IMSP backend is governed by this definition: > > define('IMSP_MUST_USE_LITERAL', "/[\W]/i"); > > > > The main reason /[\W]/i is overbroad is that it catches a vast > majority of contact names (generally "First Last" etc). Technically, > transferring all strings as literals would be valid behavior, but > there is a cost associated with waiting for all those command > continuation responses. It seems like there is some performance to > be gained by tightening the use of literals to (at least) not include > space characters (but still include CR & LF) and maybe commas. > > > > From reading the IMSP RFC, looking at other client conversations, and > testing with imspd, it seems like a better value for > IMSP_MUST_USE_LITERAL might be: > > define('IMSP_MUST_USE_LITERAL', "/[\x80-\xFF\\r\\n\"\\\\]/"); > > > > Any 8-bit char, CR, LF, double-quote <">, or backslash \. The Regex > is based on what a double-quoted string is NOT, and what a literal IS > (RFC bit follows below). It works well with cyrus imspd. > > > > The new regex also handles the cases that were being fixed last time > IMSP_MUST_USE_QUOTE was changed. Fields with multi-line input > continue to work fine, thanks to CR & LF in the regex. It's also been > tested with UTF-8, latin1, and ascii chars. > > > > In the provided patch, I've retained the /[\W]/i as > IMSP_MUST_USE_QUOTE, but I've only used it once in the > quoteSpacedString function, so it might not be necessary. I had been > thinking there might be additional places in the IMSP code where > quoteSpacedString would have to be called, but if there are, I have > not found them. > > > > From the RFC, regarding quoted strings and literals: > > "A quoted string is a sequence of zero or more 7-bit characters, > excluding CR and LF, with double quote (<">) characters at each end." > > TEXT_CHAR ::= <any CHAR except CR and LF> > > QUOTED_CHAR ::= <any TEXT_CHAR except quoted_specials> / > > "\" quoted_specials > > > > quoted_specials ::= <"> / "\" > > > > "A literal is a sequence of zero or more octets (including CR and LF)" > > literal ::= "{" number "}" CRLF *CHAR8 > > ;; The number represents the number of CHAR8 octets > > > > CHAR ::= <any 7-bit US-ASCII character except NUL, 0x01 - 0x7f> > > > > CHAR8 ::= <any 8-bit octet except NUL, 0x01 - 0xff>
Attachment
Watch this ticket

Saved Queries