Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

[Derry Shribman]

Hi,

> "lsta1" is Cygwin's lstat with full stat
> info, including correct POSIX permissions.  "lsta2" is with /bin
> mounted "noacl,notexec,ihash".
> Average ratio QDF/lsta1 is 1/3.2, QDF/lsta2 is 1/2.1

So in the lstat2 case, using QDF could bring a 200% performance improvement.
When doing the lstat2 mount, the user gains speed (over lstat1) but does not 
have correct st_ino and st_nlink. Using QDF in this case will give the user a 
210% performance improvement over lstat2 PLUS give him correct st_ino (on 
filesystems that support it).

FYI: A quick method to see EXACTLY what system-calls in cygwin1.dll take up time 
in the lstat1 and lstat2 case is to do while (1) on the test code, run it under 
gdb, and press CTRL-C every few seconds and look at the backtraces. That will 
give a statistically accurate pin-point on the existing bottlenecks in the code.

Regarding opening files and reading their content for "!<symlink>" magic checks 
and "#!/bin/xxx" headers: once a file is read - it can be stored in cache. Next 
access to that file will do QDF to get its size+mtime(or possible also st_ino), 
and if un-modified, it will use the cached information regarding this file.

Derry

On 10/5/2010 3:29 PM, Corinna Vinschen wrote:
> On Oct  4 12:23, Corinna Vinschen wrote:
>> On Oct  3 23:33, Derry Shribman wrote:
>>> We have NOD32 anti-virus installed on all XP PCs at our work place.
>>> So that may probably be one of the reasons for the XP slowdown. Can
>>> you run the test program on an XP at your place and see how results
>>> differ?
>>
>> Well, I can try, but it's a VM, too.
>
> The results on a VM are apparently very unreliable.  Here are the
> numbers on a XP VM using the latest Cygwin from CVS.  I ran the
> tests 10 times in a row.  "lsta1" is Cygwin's lstat with full stat
> info, including correct POSIX permissions.  "lsta2" is with /bin
> mounted "noacl,notexec,ihash".
>
> Worst results:
>
>    testing /bin lsta1 2015 files stat()  366.2ms,  per file: 0.1817ms
>    testing /bin lsta2 2015 files stat()  215.8ms,  per file: 0.1071ms
>    testing /bin GFA   2015 files stat()   86.91ms, per file: 0.04313ms
>    testing /bin QIF   2015 files stat()  144.5ms,  per file: 0.07173ms
>    testing /bin QDF   2015 files stat()  103.5ms,  per file: 0.05137ms
>
> Best results:
>
>    testing /bin lsta1 2015 files stat()  232.4ms,  per file: 0.1153ms
>    testing /bin lsta2 2015 files stat()  175.8ms,  per file: 0.08724ms
>    testing /bin GFA   2015 files stat()   76.17ms, per file: 0.0378ms
>    testing /bin QIF   2015 files stat()  117.2ms,  per file: 0.05816ms
>    testing /bin QDF   2015 files stat()   85.94ms, per file: 0.04265ms
>
> Average ratio QDF/lsta1 is 1/3.2, QDF/lsta2 is 1/2.1
>
>
> Corinna
>