Ticket #250 (new defect)

Opened 11 years ago

Last modified 11 years ago

Open file handles is in proportion to number of collections

Reported by: kjdon Owned by: nobody
Priority: low Milestone:
Component: Greenstone3 Runtime Severity:
Keywords: Cc:

Description

On start up, each collection is loaded up, and file handles are opened to the GDBM databases. There is a limit to the number of file handles allowed (linux only, or all systmes??), so if there are too many collections, then potentially there could be too many file handles, and tomcat would crash.

We only need to keep open the GDBM database to improve efficiency - in theory could just open tup the database for each query, then close it again.

Two possible solutions: 1. have a fail safe mode where we don't care so much about efficiency, and close each file handle after using it. 2. Keep a limited size buffer of loaded collections. All the collection info could be put into collection_list, but only 50/100?? actually loaded into memory and referenced in module_map. When a new collection is accessed, it will be loaded up and added into module_map if space available, otherwise one of the collections will be unloaded (least recent access?), and the new one will replace it. This will mean that currently accessed collections will remain fast, and if you access a colleciton that hasn't been used for a while, then the initial request will be slower, but then will be fast.

Change History

Changed 11 years ago by dmn

we could allow collections to have a flag to request 'fast access' and then report when the number was > some_threshold to warn maintainers of possible instability.

Maybe the admin tool has a role here?

Note: See TracTickets for help on using tickets.