Open file handles is in proportion to number of collections

[kjdon]

On start up, each collection is loaded up, and file handles are opened to the GDBM databases. There is a limit to the number of file handles allowed (linux only, or all systmes??), so if there are too many collections, then potentially there could be too many file handles, and tomcat would crash.

We only need to keep open the GDBM database to improve efficiency - in theory could just open tup the database for each query, then close it again.

Two possible solutions:

1. have a fail safe mode where we don't care so much about efficiency, and close each file handle after using it.
2. Keep a limited size buffer of loaded collections. All the collection info could be put into collection_list, but only 50/100?? actually loaded into memory and referenced in module_map. When a new collection is accessed, it will be loaded up and added into module_map if space available, otherwise one of the collections will be unloaded (least recent access?), and the new one will replace it. This will mean that currently accessed collections will remain fast, and if you access a colleciton that hasn't been used for a while, then the initial request will be slower, but then will be fast.

[dmn]

we could allow collections to have a flag to request 'fast access' and then report when the number was > some_threshold to warn maintainers of possible instability.

Maybe the admin tool has a role here?