Model expansion Model Memory JTree expansion JTree Memory nodes2exp/time[ms] [objects:KiB] nodes/time[ms] [objects:KiB] 1) no opt 240.002/4.392 2.838.095:220.609 434.175/251.410.764¹ 6.580.396: 398.545+ 25.145.931:1.244.200¹ ² 2) no opt + inc GC 240.001/3.955 2.852.286:221.770 434.175/ 12.465.036 na (still running) 2.353.458/493.204.009 8.252.781: 483.346 3) expandState opt 240.005/3.323 3.910.251:228.954 2.366.989/ 9.011 6.207.505: 338.923 4) +LayoutCache opt 240.002/5.919 3.226.081:191.729 2.356.394/ 1.246 ~ < 100: 1.024 thousand separator = . 1) default: no optimization, the only JVM parameter: -d64 - time for JTree node expansion increases "exponential": - 1st block 0K-8K nodes: 1.411 ms - 2nd block 8k-16K: 5.314 ms - 3rd block 16K-24K: 10.003 ms ... ... - 53th block 416K-424K: 2.676.313 ms (44min 36,313sec) ¹- stopped after ~ 3 days - unexpanded nodes ~ 2.5 M - 99.x% in Survivor Space - memory consumption ~ 2 GiB -² ~ 8M awt InvocationEvents + 8M awt EventQueueItems + 8M awt related TimerItems using ~ 1.2 GiB !!! Probably GC stops AWT-thread for a very long time, so that its "queue" gets filled up quickly ... - DefaultTreeModel (the "raw" data) uses ~ 220 MB (MutableTreeNodes) - JTree uses ~ 396 MB just for keeping/caching state - most of them huge Hashtables and underutilized Vectors (only 30-40% capacity used) - see mat1.png 2) as 1) but with GC tuning to keep stuff in Eden Space/Old Gen (skip Survivor Space step): -JVM params: -d64 -Xmx4096m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC \ -XX:+CMSIncrementalMode -XX:+UseAdaptiveGCBoundary \ -XX:MaxTenuringThreshold=0 -XX:CMSInitiatingOccupancyFraction=60 - time for JTree node expansion still increases over time, but not as dramatically as with non-inc GC -> time for first 53x8K nodes reduced by ~ factor 20, i.e. to 5% wrt. (1)): - 1st block 0K-8K nodes: 1.403 ms - 2nd block 8k-16K: 5.334 ms - 3rd block 16K-24K: 8.132 ms ... ... - 53th block 416K-424K: 472.021 ms (7min 52,021sec) ... - 287th block 2286K-2292K: 3.555.147 ms (59min 15,147sec) - 3.5GB -> 750 MB after explicit GC; ~ 730 MB in CMS Old Gen - DefaultTreeModel (the "raw" data) uses ~ 220 MB (MutableTreeNodes) - JTree uses ~ 480 MB just for keeping/caching state - most of them huge Hashtables and underutilized Vectors (only 30-40% capacity used) - see mat2.png 3) Same as (2), but with the following modifications: - JTree expand state hijacked and autoexpansion optimization activated, i.e. a) when autoexpand is "ON", modify JTrees expandedState Hashtable directly, and swallow didChange() propagation b) fire didChange() when autoexpand state gets back to "OFF" - DefaultTreeModel for "raw" data, but SyntaxTreeNodes do not extend DefaultMutableTreeNode any more - implement TreeNode and use a TreeNodeBase[] array to store children - avoid under-utilized vectors - Results: - overall mem consumption after GC: ~ 590 MiB - treeModel data uses slightly more mem: 228 MiB - Jtree uses still a lot of memory: ~ 340 MiB just for state stuff - expansion of JTree nodes is for this example ~ 55.000x faster!!! - time for JTree node expansion is more or less constant (hotspot in action): - 1st block 0K-8K nodes: 188 ms - 2nd block 8k-16K: 95 ms - 3rd block 16K-24K: 55 ms ... ... - 53th block 416K-424K: 22 ms ... - 286th block 2280K-2288K: 22 ms - see mat3.png 4) like (3), but with a flat direct indexing treeModel and a flat LayoutCache: - stores expansion state in a BitSetX, - row mapping in a simple int[] array - i.e. nowhere Hashmaps any more - for convinience and central storage allocation states are kept in the treeModel - LayoutCache takes advantage of "autoexpand" state and treeModel: faster row re-enumeration Results: - ~ 250 MB -> 205 MB after explicit GC - JTree memory usage is negligible (even if one takes the memory used in in the treeModel for state caching into account - ~ 10..12 MB) - size of "raw" data for TreeNodes and TreeModel even reduced to ~ 190 MB - further speedup in autoexpanding nodes - see mat4.png Finally: - autoexpansion is ~400.000x faster than with DefaultTreeModel/DefaultMutable- TreeNodes/JTree/LayoutCache4JTree for this example, i.e. a branch with ~2.4M nodes, whereby ~ 90% of those nodes are leafs - memory used by JTree & friends reduced by ~500 MB to almost zero - tree node/model memory usage reduced by ~ 30 MB to ~ 190 MB Test environment: X4440 with 4x Opteron 8439 (six-core) 2.8 GHz, 64 GB via SunRay 170 Solaris 5.11 (SXCE Nevada b129) java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode)