Fri, 10 Apr 2026 18:27:13 +0000 https://bugs.horde.org/ticket/11983 limit synced mailboxes by user preferences I have a fairly large mailbox structure, plus access to various shared folders. When I synchronize with a remote device, I therefore generate tons of traffic and load, especially on the "small" remote device. On the other hand, my remote device won't let me access folders not under my INBOX at all, and I definitely don't need all the sub-folders as available through the web interface. As I have been unsuccessful to create a second IMAP user with access to the same INBOX, but only few of the folders below, I have created a patch to ActiveSync that allows a user to (currently) select - only to sync folders below and including the INBOX - limit the nesting level of the folders synced The according preferences are settable via the standard preferences dialog, per user. A future enhancement may be to explicitly select folders to include/exclude during email sync. I've attached two files: - the definitions for the user preferences - currently added as $webdir/config/prefs.d/30-activesync.php - the new method Horde_ActiveSync_Imap_Adapter::getMailboxes() from /usr/share/php5/PEAR/Horde/ActiveSync/Imap/Adapter.php I hope this change finds your approval. Regards, Jens Sun, 20 Jan 2013 16:09:02 +0000 https://bugs.horde.org/ticket/11983#t76313 > I have a fairly large mailbox structure, plus access to various > shared folders. When I synchronize with a remote device, I therefore > generate tons of traffic and load, especially on the "small" remote > device. Currently, it sends all "subscribed" folders in the FOLDERSYNC request. The rationale is that if you have subscribed to the folder, you are interested enough in it to want to see it on the device. Note that sending the folder via FOLDERSYNC does not necessarily mean it pushes that folder, just that it is available on the device for viewing. On most of the clients I test, the email is lazy loaded into folders other than the INBOX - it only polls the server when you view the folder. Some devices will THEN push to that folder for some time after it is loaded. Other clients have settings that allow you to pick and choose the folders you want pushed vs just available. > On the other hand, my remote device won't let me access > folders not under my INBOX at all, and I definitely don't need all > the sub-folders as available through the web interface. Curious as to what client this is. I've never seen a client limit folders to those only under INBOX. > As I have been unsuccessful to create a second IMAP user with access > to the same INBOX, but only few of the folders below, I have created > a patch to ActiveSync that allows a user to (currently) select > - only to sync folders below and including the INBOX > - limit the nesting level of the folders synced This of course will only work for IMAP servers that are structured/configured in this way. My server does not place folders under INBOX at all. I have a few hundred mailboxes (of course not all subscribed) and most are at the same level as the INBOX. > The according preferences are settable via the standard preferences > dialog, per user. > > A future enhancement may be to explicitly select folders to > include/exclude during email sync. I'd rather not go that route. I feel that this is a client responsibility. Also, filtering the folders that are returned via FOLDERSYNC responses could screw up search results Sun, 20 Jan 2013 19:35:16 +0000 https://bugs.horde.org/ticket/11983#t76314 >> I have a fairly large mailbox structure, plus access to various >> shared folders. When I synchronize with a remote device, I therefore >> generate tons of traffic and load, especially on the "small" remote >> device. > > Currently, it sends all "subscribed" folders in the FOLDERSYNC > request. The rationale is that if you have subscribed to the folder, > you are interested enough in it to want to see it on the device. When on the road, I usually only need to see if I have new mail and optionally reply to it. Some other folders will be required, too, like "Sent" and "Trash". > Note that sending the folder via FOLDERSYNC does not necessarily mean > it pushes that folder, just that it is available on the device for > viewing. On most of the clients I test, the email is lazy loaded into > folders other than the INBOX - it only polls the server when you view > the folder. Some devices will THEN push to that folder for some time > after it is loaded. Other clients have settings that allow you to > pick and choose the folders you want pushed vs just available. It's my observation too that only messages in the INBOX are actually pushed to the remote device. But with about 1083 folders (number increasing), already processing the list of available folders causes a significant slow-down on my remote... >> On the other hand, my remote device won't let me access >> folders not under my INBOX at all, and I definitely don't need all >> the sub-folders as available through the web interface. > > Curious as to what client this is. I've never seen a client limit > folders to those only under INBOX. It's the default client on WebOS 2.x (HP Pre3). >> As I have been unsuccessful to create a second IMAP user with access >> to the same INBOX, but only few of the folders below, I have created >> a patch to ActiveSync that allows a user to (currently) select >> - only to sync folders below and including the INBOX >> - limit the nesting level of the folders synced > > This of course will only work for IMAP servers that are > structured/configured in this way. My server does not place folders > under INBOX at all. I have a few hundred mailboxes (of course not all > subscribed) and most are at the same level as the INBOX. Yes, I had anticipated that situation - being able to select which folders to sync, similar to selecting which calendars and address books to sync, would cover that. >> The according preferences are settable via the standard preferences >> dialog, per user. >> >> A future enhancement may be to explicitly select folders to >> include/exclude during email sync. > > I'd rather not go that route. I feel that this is a client > responsibility. Also, filtering the folders that are returned via > FOLDERSYNC responses could screw up search results I fully agree to that position, and as I mentioned above, even I would have preferred to go a different route... a sufficient solution would have been if I could have used IMAP subscriptions and a second account - that would have been a favorable solution. But unfortunately I couldn't get that to work, probably to my insufficient knowledge of the details of the Cyrus IMAP server. That's when I thought that a client-side subscription model, implemented on the ActiveSync part (rather than the final mobile client) ought to help... and the long time it seems to take to process the results of a folder sync, I decided to code a filter option for that call. But of course, there may be (and your reply says there *are*) side effects I couldn't foresee. Do you see a different, better place to implement such a filter? I'd be willing to give this another try. BTW: After running this filter on my server/account, processing time and data transfer volume have already decreased. But I see many instances of supposedly "filtered" folders in the ActiveSync debug log, there still is much work left to actually reduce the handling to only those folders I actually want to see on the remote :/ Sun, 20 Jan 2013 21:12:46 +0000 https://bugs.horde.org/ticket/11983#t76315 Ha - seems you're right again: >> On the other hand, my remote device won't let me access >> folders not under my INBOX at all, and I definitely don't need all >> the sub-folders as available through the web interface. > > Curious as to what client this is. I've never seen a client limit > folders to those only under INBOX. I just noticed that during a development error, I actually had some "not under INBOX" folders in the list - and my WebOS client would show them. It must have been more of a problem of handling the large number of folders, the displayed list is rather bad to handle and either the "shared folders" where at the cut off at end of the list or lost somewhere in the too long (scrollable) list of folders. That list handling (within the WebOS mail client) is a PITA when having many folders - reason enough to limit the list of displayed folders. I couldn't find an option for this within that client, so I'm still after the ActiveSync filtering solution ;) Sun, 20 Jan 2013 21:28:45 +0000 https://bugs.horde.org/ticket/11983#t76316 >> Note that sending the folder via FOLDERSYNC does not necessarily mean >> it pushes that folder, just that it is available on the device for >> viewing. On most of the clients I test, the email is lazy loaded into >> folders other than the INBOX - it only polls the server when you view >> the folder. Some devices will THEN push to that folder for some time >> after it is loaded. Other clients have settings that allow you to >> pick and choose the folders you want pushed vs just available. > > It's my observation too that only messages in the INBOX are actually > pushed to the remote device. But with about 1083 folders (number > increasing), already processing the list of available folders causes > a significant slow-down on my remote... > >>> On the other hand, my remote device won't let me access >>> folders not under my INBOX at all, and I definitely don't need all >>> the sub-folders as available through the web interface. >> >> Curious as to what client this is. I've never seen a client limit >> folders to those only under INBOX. > > It's the default client on WebOS 2.x (HP Pre3). > >>> As I have been unsuccessful to create a second IMAP user with access >>> to the same INBOX, but only few of the folders below, I have created >>> a patch to ActiveSync that allows a user to (currently) select >>> - only to sync folders below and including the INBOX >>> - limit the nesting level of the folders synced >> >> This of course will only work for IMAP servers that are >> structured/configured in this way. My server does not place folders >> under INBOX at all. I have a few hundred mailboxes (of course not all >> subscribed) and most are at the same level as the INBOX. > > Yes, I had anticipated that situation - being able to select which > folders to sync, similar to selecting which calendars and address > books to sync, would cover that. That would introduce a whole other level of messiness since folders are more dynamic than address books/calendars. Not that it couldn't be done. The reason we do this for calendars/address books is because EAS only supports a single one of each of those. The pref allows the user to decide which sources to combine into the single data source that is sent to the client. We don't have to worry about subscriptions/namespaces etc... with those sources. If we do this, I think I'd do it as a INBOX and Special Folders only or all subscribed folders. > That's when I thought that a client-side subscription model, > implemented on the ActiveSync part (rather than the final mobile > client) ought to help... and the long time it seems to take to > process the results of a folder sync, I decided to code a filter > option for that call. But of course, there may be (and your reply > says there *are*) side effects I couldn't foresee. I'd need more testing to see exactly how this would affect search and such. > Do you see a different, better place to implement such a filter? I'd > be willing to give this another try. You can't use the prefs object inside the library code. This would have to be implemented inside Horde_Core_ActiveSync_Driver:: probably in _getMailFolders() or it's own filtering method. > BTW: After running this filter on my server/account, processing time > and data transfer volume have already decreased. But I see many > instances of supposedly "filtered" folders in the ActiveSync debug > log, there still is much work left to actually reduce the handling to > only those folders I actually want to see on the remote :/ This might be left over from data not updated in the syncCache of folder state yet. Once the device requests a new FOLDERSYNC it should only request data on those folders. In other words, when it receives a FOLDERSYNC response that indicates that folder FOO is no longer available, it thinks that folder has been deleted on the server, and should not issue any more requests for it. Update the patch with the above changes and I'll look at adding it in Horde 5.1. Sun, 20 Jan 2013 21:33:27 +0000 https://bugs.horde.org/ticket/11983#t76317 > If we do this, I think I'd do it as a INBOX and Special Folders only > or all subscribed folders. > [...] > You can't use the prefs object inside the library code. This would > have to be implemented inside Horde_Core_ActiveSync_Driver:: probably > in _getMailFolders() or it's own filtering method. > [...] > Update the patch with the above changes and I'll look at adding it in > Horde 5.1. I've move the code to Horde_Core_ActiveSync_Driver::_getMailFolders(), which works as expected (at the current stage of development). But two questions remain: Isn't "Horde_Core_ActiveSync_Driver::_getMailFolders()" part of "the library code" as well? For both places I've implemented this so far, accessing the preferences worked as expected, probably because the routines were called in the context of a logged-in user. If this may cause problems for other types of invocation (without user context), I'd need some architectural guidance. Secondly, is there a proper way to detect which folders returned by "_imap->getMailboxes()" are "special" mailboxes? I believe that even if I set preference to only see "INBOX", I'd need at least "Trash" and "Sent" as well, and probably "Drafts" (and "Templates"?) too. I've seen no specific attributes for these folders and remember that some clients I used were able to select which folder to use for these functions. I've seen code to somewhat handle this in IMP - would the "proper" way be to use the same code as IMP does, to determine which folder is a special folder? I'll then have to dig through IMP to find out if this is available to me within the ActiveSync context or if I'll have to somehow duplicate it (yuck ;) ). Mon, 21 Jan 2013 11:41:00 +0000 https://bugs.horde.org/ticket/11983#t76322 > I've move the code to > Horde_Core_ActiveSync_Driver::_getMailFolders(), which works as > expected (at the current stage of development). But two questions > remain: > > Isn't "Horde_Core_ActiveSync_Driver::_getMailFolders()" part of "the > library code" as well? For both places I've implemented this so far, > accessing the preferences worked as expected, probably because the > routines were called in the context of a logged-in user. If this may > cause problems for other types of invocation (without user context), > I'd need some architectural guidance. The framework libraries are designed to be used as stand alone libraries - outside of a Horde environment. Core is a special library that contains the core libraries needed for a Horde environment. > Secondly, is there a proper way to detect which folders returned by > "_imap->getMailboxes()" are "special" mailboxes? I believe that even > if I set preference to only see "INBOX", I'd need at least "Trash" > and "Sent" as well, and probably "Drafts" (and "Templates"?) too. > I've seen no specific attributes for these folders and remember that > some clients I used were able to select which folder to use for these > functions. This is already done in Horde_Core_ActiveSync_Driver::_getMailFolder(). You just need to check the type property of the returned folder object. Mon, 21 Jan 2013 14:44:49 +0000 https://bugs.horde.org/ticket/11983#t76324 >> Isn't "Horde_Core_ActiveSync_Driver::_getMailFolders()" part of "the >> library code" as well? [...] > The framework libraries are designed to be used as stand alone > libraries - outside of a Horde environment. Core is a special library > that contains the core libraries needed for a Horde environment. Thanks, that explains it! >> Secondly, is there a proper way to detect which folders returned by >> "_imap->getMailboxes()" are "special" mailboxes? [...] > This is already done in > Horde_Core_ActiveSync_Driver::_getMailFolder(). You just need to > check the type property of the returned folder object. If I had to call _getMailFolder() for every folder returned by _imap->getMailboxes(), wouldn't this get too expensive? My list may well be longer than 1k entries... each tme creating an instance of Horde_ActiveSync_Message_Folder, calling getSpecialMailboxes() etc, and then dumping the result in all but a handful of cases. Purely functionally speaking, the quickest approach would be to call _imap->getSpecialMailboxes() only once, re-map to an "folder name"-based array, then check every "_imap->getMailboxes()" result against that new array. Does that sound acceptable to you? Mon, 21 Jan 2013 16:33:04 +0000 https://bugs.horde.org/ticket/11983#t76327 > If I had to call _getMailFolder() for every folder returned by > _imap->getMailboxes(), wouldn't this get too expensive? My list may > well be longer than 1k entries... each tme creating an instance of > Horde_ActiveSync_Message_Folder, calling getSpecialMailboxes() etc, > and then dumping the result in all but a handful of cases. Well, getSpecialMailboxes() already caches the list so we only hit the imap server once. > Purely functionally speaking, the quickest approach would be to call > _imap->getSpecialMailboxes() only once, re-map to an "folder > name"-based array, then check every "_imap->getMailboxes()" result > against that new array. Does that sound acceptable to you? Feel free to submit a patch and I'll review. Mon, 21 Jan 2013 16:51:58 +0000 https://bugs.horde.org/ticket/11983#t76329 > Feel free to submit a patch and I'll review. So here's my attempt to a patch to /usr/share/php5/PEAR/Horde/Core/ActiveSync/Driver.php and a corresponding prefs.d file (who's contents ought to be merged into prefs.php in case your accepting the patch?) Tests here work fine so far. Mon, 21 Jan 2013 17:18:07 +0000 https://bugs.horde.org/ticket/11983#t76330 > Tests here work fine so far. Today I noticed that my events etc. don't get synced to my remote device. I had thought that this was handled by _getFolders(), which constructs a combined array of collections and mailnfolders, but obviously that is insufficient. Wed, 23 Jan 2013 00:57:34 +0000 https://bugs.horde.org/ticket/11983#t76360 >> Tests here work fine so far. > > Today I noticed that my events etc. don't get synced to my remote > device. I had thought that this was handled by _getFolders(), which > constructs a combined array of collections and mailnfolders, but > obviously that is insufficient. No, that's correct. Changes to the folders returned from _getMailFolders() would have no effect on synching non-email folders. Wed, 23 Jan 2013 01:24:38 +0000 https://bugs.horde.org/ticket/11983#t76362 >> Feel free to submit a patch and I'll review. > > So here's my attempt to a patch to > /usr/share/php5/PEAR/Horde/Core/ActiveSync/Driver.php 1) You need to review http://www.horde.org/apps/horde/docs/CODING_STANDARDS 2) _getMailFolder() still calls getSpecialMailboxes() and compares the names anyway. You should build the map of the special mailboxes regardless if it is $inbox_only or not, store it locally in the object and use it in getMailFolder() as well. Wed, 23 Jan 2013 01:42:31 +0000 https://bugs.horde.org/ticket/11983#t76364 >> Today I noticed that my events etc. don't get synced to my remote >> device.[...]. > > No, that's correct. Changes to the folders returned from > _getMailFolders() would have no effect on synching non-email folders. Yes, my fault: although I don't recall having changed it, the configuration for an add'l calendar to sync was gone - resulting in an almost empty calendar on the remote device. Once I re-added that calendar in my kronolith sync preferences and fully re-established the association (I really had to delete and re-add the account on the remote), both sets of events were synced. Wed, 23 Jan 2013 21:12:14 +0000 https://bugs.horde.org/ticket/11983#t76379 > 1) You need to review http://www.horde.org/apps/horde/docs/CODING_STANDARDS > 2) _getMailFolder() still calls getSpecialMailboxes() and compares > the names anyway. You should build the map of the special mailboxes > regardless if it is $inbox_only or not, store it locally in the > object and use it in getMailFolder() as well. I'll take your comments and accordingly will re-work the code - but this may take a day or two (not that I believe 5.1 is already lurking behind the next corner ;) ) That functional change really helped to reduce the load on my device to a reasonable level. Wed, 23 Jan 2013 21:15:09 +0000 https://bugs.horde.org/ticket/11983#t76380 >>> Feel free to submit a patch and I'll review. >> >> So here's my attempt to a patch to >> /usr/share/php5/PEAR/Horde/Core/ActiveSync/Driver.php > > 1) You need to review http://www.horde.org/apps/horde/docs/CODING_STANDARDS > 2) _getMailFolder() still calls getSpecialMailboxes() and compares > the names anyway. You should build the map of the special mailboxes > regardless if it is $inbox_only or not, store it locally in the > object and use it in getMailFolder() as well. I've tried to incorporate your comments - is this new patch sufficient for approval? Concerning the array of special mailboxes - is my approach (initializing the member to "null" in the c'tor and verifying against that value for lazy loading in "_getMailFolders()") correct? Let me know if I should further improve this patch. Mon, 04 Feb 2013 17:47:56 +0000 https://bugs.horde.org/ticket/11983#t76564 On reviewing this some more, I'm going to reject this. Exchange has a limit of 1000 folders by default, if a client cannot handle up to this many folders without falling apart, it's a client issue. This approach is just too fraught with the possibility of errors in other parts of the code (like searching, message moving etc...). Mon, 07 Apr 2014 16:32:33 +0000 https://bugs.horde.org/ticket/11983#t83224