6.0.0-beta1
7/20/25

[#10286] Speed up browsing folders with a lot of files
Summary Speed up browsing folders with a lot of files
Queue Chora
Queue Version Git master
Type Enhancement
State No Feedback
Priority 1. Low
Owners
Requester vilius (at) lnk (dot) lt
Created 06/28/2011 (5136 days ago)
Due
Updated 01/30/2013 (4554 days ago)
Assigned
Resolved 01/30/2013 (4554 days ago)
Milestone
Patch No

History
01/30/2013 05:13:54 PM Jan Schneider State ⇒ No Feedback
 
08/06/2011 01:19:05 PM vilius (at) lnk (dot) lt Comment #8 Reply to this comment
Not really. I gave a quick look couple of times at Horde_Vcs code and 
it is still a mistery to me :) I will give it another try some day 
rather later than sooner. In the meantime maybe someone with more 
experience can patch it more easily.
08/06/2011 05:22:38 AM Chuck Hagenbuch Comment #7 Reply to this comment
Sounds promising - do you happen to have a patch for this, or can you 
make one? (definitely including the last commit per directory, imo)
08/05/2011 12:53:48 PM vilius (at) lnk (dot) lt Comment #6 Reply to this comment
Spoke a little bit too soon about the magic performance of a command 
in my comment. My previous tests with git log were made on a 
repository without a full log (git clone --depth 1 with 2 months worth 
of data). Just checked with full log. As an example root IMP folder 
listing on previous repository took 0.2 seconds. On full log it took 
2.2 seconds :(. Nevertheless it is ~3 times faster than current Chora 
code. On my server Chora lists files in IMP folder in ~6 seconds.

By the way it can be run on the bare repository too with this modification:
for a in $(git ls-tree --name-only fa5c5155d); do git log 
--pretty=format:"%h%x09%an%x09%ad%x09$a" -1 -- "imp/$a"; done

Where fa5c5155d is a tree object for imp folder.

This also includes last commits for folders which are missing 
currently from GUI and could be interesting for the user:

dc5d0f6 Ralf Lang       Wed Jun 15 08:44:43 2011 +0000  COPYING
97c55b2 Jan Schneider   Mon May 23 14:34:45 2011 +0200  README
a874ce8 Michael M Slusarz       Mon Apr 25 13:21:27 2011 -0600  attachment.php
b02d946 Jan Schneider   Tue Mar 29 16:08:21 2011 +0200  bin
6f99348 Michael M Slusarz       Mon Aug 1 16:40:02 2011 -0600   
compose-dimp.php
6f99348 Michael M Slusarz       Mon Aug 1 16:40:02 2011 -0600   
compose-mimp.php
6f99348 Michael M Slusarz       Mon Aug 1 16:40:02 2011 -0600   compose.php
6f99348 Michael M Slusarz       Mon Aug 1 16:40:02 2011 -0600   config
5b4709a Michael M Slusarz       Fri Mar 25 11:43:40 2011 -0600  contacts.php
12cdfbb Michael M Slusarz       Wed Aug 3 17:29:02 2011 -0600   docs
70161b3 Michael M Slusarz       Mon Apr 25 23:01:49 2011 -0600   
folders-mimp.php
1af27ae Michael M Slusarz       Sat Jun 4 13:59:50 2011 -0600   folders.php
3a34c30 Michael M Slusarz       Mon Jul 25 23:22:34 2011 -0600  index-dimp.php
12d62db Michael M Slusarz       Tue Mar 22 17:15:24 2011 -0600  index.php
12cdfbb Michael M Slusarz       Wed Aug 3 17:29:02 2011 -0600   js
2d380a5 Michael M Slusarz       Wed Aug 3 17:26:26 2011 -0600   lib
6930c5b Jan Schneider   Tue Aug 2 15:53:32 2011 +0200   locale
3a43505 Michael M Slusarz       Tue May 17 00:12:31 2011 -0600   
mailbox-mimp.php
6e93a8c Michael M Slusarz       Tue Jul 12 12:41:47 2011 -0600  mailbox.php
6e93a8c Michael M Slusarz       Tue Jul 12 12:41:47 2011 -0600   
message-dimp.php
6f99348 Michael M Slusarz       Mon Aug 1 16:40:02 2011 -0600   
message-mimp.php
6f99348 Michael M Slusarz       Mon Aug 1 16:40:02 2011 -0600   message.php
ae35555 Jan Schneider   Sat Mar 26 22:55:54 2011 +0100  migration
6f1f0e0 Michael M Slusarz       Thu May 5 09:57:44 2011 -0600   mobile.php
12cdfbb Michael M Slusarz       Wed Aug 3 17:29:02 2011 -0600   package.xml
3b6c3f2 Jan Schneider   Fri Jan 28 22:31:54 2011 +0100  pgp.php
16436a1 Michael M Slusarz       Sat May 14 00:06:36 2011 -0600  rss.php
ca28ddc Michael J Rubinsky      Thu Jun 16 16:02:21 2011 -0400  saveimage.php
32c3fac Michael M Slusarz       Tue May 17 00:13:49 2011 -0600   
search-basic.php
54dac27 Michael M Slusarz       Mon Aug 1 16:37:27 2011 -0600   search.php
e3b18b4 Jan Schneider   Wed Mar 30 14:39:55 2011 +0200  smime.php
6f99348 Michael M Slusarz       Mon Aug 1 16:40:02 2011 -0600   templates
f5d1d97 Gunnar Wrobel   Mon Jun 20 23:19:10 2011 +0200  test
6f99348 Michael M Slusarz       Mon Aug 1 16:40:02 2011 -0600   themes
9a46eae Michael M Slusarz       Tue May 17 17:32:16 2011 -0600  thread.php
2aa8698 Michael M Slusarz       Tue Jun 21 23:18:04 2011 -0600  view.php

08/04/2011 01:14:39 PM vilius (at) lnk (dot) lt Comment #5 Reply to this comment
Good question. As it uses git log on a file instead of the index I 
suspect yes, the repository should be full.
08/04/2011 12:33:19 PM Chuck Hagenbuch Comment #4
State ⇒ Feedback
Reply to this comment
That requires running on a full (non-bare) checkout though, right?
08/04/2011 12:15:40 PM vilius (at) lnk (dot) lt Comment #3 Reply to this comment
OK, I finally got some time to inspect the issue. Seems like 
queryLastLog() in Git driver is very expensive. And it is performed on 
every file in a directory.

As far as I know currently git doesn't support retrieving last log 
entries for all folder files. git ls-tree is very similar, but it 
doesn't include last commit author and date. However these limitations 
can be avoided by using something like:

for a in $(ls); do git log --pretty=format:"%h%x09%an%x09%ad%x09$a" -1 
-- "$a"; done
06/28/2011 12:37:41 PM Jan Schneider Comment #2
State ⇒ Rejected
Reply to this comment
This is not a very helpful request. *Everything* can *always* been 
improved somehow. Unless you can pinpoint a specific reason why those 
listings are slow, this request doesn't make sense.
06/28/2011 12:16:36 PM vilius (at) lnk (dot) lt Comment #1
Priority ⇒ 1. Low
Type ⇒ Enhancement
Summary ⇒ Speed up browsing folders with a lot of files
Queue ⇒ Chora
Milestone ⇒
Patch ⇒ No
State ⇒ New
Reply to this comment
Currently Chora is very slow when browsing folders with a lot of files 
(at least for Git). There should be a way to speed things up.

Saved Queries