manifest handling

[kjdon]

When John T was using manifests for parallel processing, he found that the behaviour was not as expected. When proceesing a manifest specifying one file out of a million, the million were still scanned in global file_block_read. Apparently this may add metadata?? John invented version 2 manifests which explicitly lists all files and metadata files (? implemented yet?) and then stops any global pass through the import folder.

Should there be a global pass when using a manifest (version 1 or 2)? I guess we might need to discover metadata?? Think about whether we need to change the code or not.