IOImprovementsDesc
I/O Improvements
Contents |
Lightweight FileObject operations
- Delete of files was incredibly slow in 6.0 (IZ 106128, IZ 122594) especially in case of deep file hierarchies. The overhead caused by creation of lock file on disk, will be minimized by identifying cases where the lock file is really needed and creating it only in such situations.
- when there is just one locked file, it does not need lock on disk
- given the locking system on masterfs, it is only necessary to create file locks if we are locking file named as already locked file, then one needs to create lock files for all files with the same name and verify there are not identical
- The speedup is about 50%+-20% compared to the poor 6.0 case. In almost all cases delete just deletes and no writes are done anymore.
Number of file object instances
There is already stuff prototyped in the excessive disk usage area and it can also significantly decrease network access. Actually I work on evaluation. See Radek's issue, #123542 for details.
todo:
- Patched security manager for testing,
- native libraries evaluation,
- ...
Additional Measurements
I tried to recursively traverse NetBeans sources on different partitions (clone of NetBeans repository, built up).
HW: Tecra M3.1st measuring is immediately after system boot and login.
| OS | 1st (s) | 2nd (s) | 3rd (s) | 4th (s) | 5th |
|---|---|---|---|---|---|
| WinXP (ntfs) | 279 | 19 | 27 | 19 | 20 |
| Ubuntu 7.10 (ext3) | 166 | 4 | 4 | 4 | 4 |
| WinXP (samba) | 714 | 682 | N/A | N/A | N/A |
| Ubuntu 7.10 (nfs) | 155 | 150 | 163 | 91 | 177 |
ntfs - almost defragmented, partition was almost new.
ext3 - extensively used before
nfs - hs-eprg06-01.czech:/export/home1/pf124312 on /home/pf124312 type nfs (rw,addr=129.157.16.144)
I tried also to measure how single read byte works on both platforms. Reading limewire project byte by byte:
| WinXP (ntfs) | 76.079 | 78.875 | 65.042 | 61.016 | 69.562 |
|---|---|---|---|---|---|
| Ubuntu 7.10 | 44.719 | 42.768 | 42.745 | 42.867 | 42.814 |
The measurement was done immediately when system fully started.
What I found interesting, when using system for a while (making different task), I got a better results:
| WinXP | 53.422 | 53.046 | 53.047 | 53.172 | 54.438 |
|---|---|---|---|---|---|
| Ubuntu 7.10 | 39.617 | 33.914 | 33.286 | 33.480 | 33.572 |
Conclusion:
The results vary, only the thing I can confirm is slower ntfs filesystem.
Eliminate single byte reads/writes in file input/output streams
We are trying to reduce the number of I/O hits. This effort is tracked in Umbrella issue.
Single byte reads/writes (number of single byte reads/writes):
| Start-up | 6.0.1 | 6.1 |
|---|---|---|
| w/o userdir | 1098863 | 1699 |
| existing userdir | 7747 | 1265 |
Update tracking files during start-up
We eliminated 300 update_tracking files read during start-up in autoupdate (they are no longer read and parsed.)
| Action | Ubuntu 6.0.1 | Ubuntu 6.1 (0317) | WinXP 6.0.1 | WinXP 6.1 (0317) |
|---|---|---|---|---|
| Out Of The Box Startup -- Classes-Loaded | 6986 | 4755 | N/A | 6691 |
| Scanning Java Project Classpath | 34.0s | 19.6s | 35.4s | 20.6s |
| Expand Source Packages Node 1 | 603ms | 459ms | 1440ms | 3996ms (?) |
| Expand Source Packages Node 2 | 37ms | 35ms | 40ms | 34ms |
| Invoke Code Completion dialog in Editor 1 | 1081ms | 339ms | 1142ms | 311ms |
| Invoke Code Completion dialog in Editor 2 | 632ms | 29ms | 749ms | 36ms |
| Expand folder with 100 java files 1 | 1949ms | 620ms | 2128ms | 1202ms |
| Expand folder with 100 java files 2 | 554ms | 423ms | 285ms | 259ms |
Network Measurements
Client: Ubuntu Linux Sources: NFS, Solaris 10 Project: limewire, nbextra Actions: src node -- expand project src node, wait until all packages are computed (progress finished) dir node -- expand com.limegroup.gnutella package (>100files, several dirs)
First number in each measurement indicates time immediately after system boot and NetBeans start. Other numbers are measured after NetBeans restart.
| Action | NetBeans 6.0.1 | NetBeans 6.1 trunk (b1471) |
|---|---|---|
| src node 1 | 17.4s 17.2s 16.0s | 9.1s 9.3s 8.6s |
| src node 2 | 18.3s 16.7s 16.0s | 10.1s 9.4s 8.4s |
| dir node 1 | 2.7s 2.5s 2.6s | 1.4s 1.5s 1.4s |
| dir node 2 | 2.7s 2.7s 2.5s | 1.6s 1.3s 1.3s |
(Stopwatch measurements)
Scan scheduled
| Projects opening | 1st | 2nd | 3rd | 4th | 5th |
|---|---|---|---|---|---|
| postponed scanning | 4697 ms | 4661 ms | 4760 ms | 4983 ms | 4716 ms |
| immediate scanning | 7428 ms | 6754 ms | 7412 ms | 6785 ms | 7135 ms |
