6.0.0-alpha14
7/2/25

[#10775] VFS Memory Usage
Summary VFS Memory Usage
Queue Horde Framework Packages
Queue Version Git master
Type Bug
State Not A Bug
Priority 2. Medium
Owners
Requester slusarz (at) horde (dot) org
Created 11/17/2011 (4976 days ago)
Due
Updated 02/02/2017 (3072 days ago)
Assigned
Resolved 11/17/2011 (4976 days ago)
Github Issue Link
Github Pull Request
Milestone
Patch No

History
02/02/2017 05:15:26 PM Git Commit Comment #5 Reply to this comment
Changes have been made in Git (master):

commit 9198c3c8fac5e8b4213a3e5853281ce2439a5c08
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Thu Feb 2 12:11:17 2017 -0500

     Allow Horde_Vfs::writeData to accept a stream as input.

     For every driver except Mongo, the stream is maintained. Mongo
     must read it into a string since there is no way to stream the
     bytes to the backend.

     Related to Bug: 10775.

  framework/Vfs/lib/Horde/Vfs/Base.php    | 31 ++++++++++++++++++++++++++-----
  framework/Vfs/lib/Horde/Vfs/File.php    | 15 ++++++++++-----
  framework/Vfs/lib/Horde/Vfs/Ftp.php     | 12 ++++++++----
  framework/Vfs/lib/Horde/Vfs/Mongo.php   |  7 +++++++
  framework/Vfs/lib/Horde/Vfs/Musql.php   |  9 +++++----
  framework/Vfs/lib/Horde/Vfs/Smb.php     | 12 ++++++++----
  framework/Vfs/lib/Horde/Vfs/Sql.php     |  9 +++++----
  framework/Vfs/lib/Horde/Vfs/SqlFile.php |  3 +++
  framework/Vfs/lib/Horde/Vfs/Ssh2.php    | 12 ++++++++----
  framework/Vfs/package.xml               |  2 +-
  10 files changed, 81 insertions(+), 31 deletions(-)

http://github.com/horde/horde/commit/9198c3c8fac5e8b4213a3e5853281ce2439a5c08
11/27/2011 03:23:24 PM Chuck Hagenbuch Comment #4 Reply to this comment
Why don't we add stream support for blobs to Horde_Db? Is there an 
open enhancement for that?
11/17/2011 08:08:12 AM Michael Slusarz Comment #3
State ⇒ Not A Bug
Reply to this comment
Sure enough... moving to a file based VFS backend, memory usage was 
4-5 times less.  Simply put, Horde_Db is terribly inefficient.

Since that has nothing to do with VFS itself, closing this ticket.   
Horde_Db would benefit from log fixes in Ticket #10773, so that's a 
start.  Better would probably be informing users that they should NOT 
be using SQL VFS in production environments.
11/17/2011 07:44:19 AM Michael Slusarz Comment #2 Reply to this comment
So... for all the advances we have made in memory reduction in Horde 
4, VFS kills much of the joy.  Uploading a 1.2 MB attachment to IMP, 
a call to VFS::write(), using a Postgresql DB backend, uses 35.5 MB 
of memory.  That is unacceptable.
A quick pass at this is not promising, at least for the SQL driver: 
the critical issue seems to instead be that Horde_Db does not accept 
stream resources as input - it expects everything in strings.  Which 
is depressing - this is going to make tuning very difficult, and 
pretty much makes Horde_Db the bottleneck when it comes to memory 
usage improvements.
11/17/2011 06:50:48 AM Michael Slusarz Comment #1
Priority ⇒ 2. Medium
Patch ⇒ No
Milestone ⇒
Queue ⇒ Horde Framework Packages
Summary ⇒ VFS Memory Usage
Type ⇒ Bug
State ⇒ Unconfirmed
Reply to this comment
So... for all the advances we have made in memory reduction in Horde 
4, VFS kills much of the joy.  Uploading a 1.2 MB attachment to IMP, a 
call to VFS::write(), using a Postgresql DB backend, uses 35.5 MB of 
memory.  That is unacceptable.

This ticket is partially related to Ticket #10773 - eliminating 
logging within the DB object reduces memory usage to 15 MB.  Still too 
high, but not as bad.

Saved Queries