1 | From: "Adam Murdoch" <[email protected]>
|
---|
2 | Subject: RE: Virtual FileSystem Layer
|
---|
3 | Date: Sat, 22 Dec 2001 12:06:37 +1000
|
---|
4 |
|
---|
5 | Hi,
|
---|
6 |
|
---|
7 | I've also been doing a bit of work on the VFS. No code yet - instead, I've
|
---|
8 | done a survey of the Ant 1 code, to help get a better idea of what we need
|
---|
9 | the VFS to actually do.
|
---|
10 |
|
---|
11 | I've put together a rough list of the sort of features the current tasks
|
---|
12 | require from the file system. This list is entirely from the task writer's
|
---|
13 | POV. I've ignored the build file writer completely - though, the action
|
---|
14 | list is a good summary of the build file writer's concerns. I've tried to
|
---|
15 | steer clear of assumptions about what is actually going to provide each
|
---|
16 | feature to the tasks, or what the API will look like to the tasks.
|
---|
17 |
|
---|
18 | The goal for doing up this list, was to help identify the features we want
|
---|
19 | to support, and the API that the tasks will use to get at them. This should
|
---|
20 | be largely independent of how we decide to represent the file system in the
|
---|
21 | build files. In addition, it doesn't matter too much whether the list below
|
---|
22 | is complete (I'm sure it isn't), or that the VFS provide every single one of
|
---|
23 | the features. Whatever it doesn't provide, can stay up in the tasks, and be
|
---|
24 | refactored down later.
|
---|
25 |
|
---|
26 | The assumption here is that we do actually want to put together a file
|
---|
27 | system API. I think it's a good idea to at least put together some
|
---|
28 | interfaces, even if the implementation is stolen from somewhere else.
|
---|
29 | Without a doubt, the file system is the most widely used "service" in the
|
---|
30 | current crop of tasks. The API that we choose has to have a good semantic
|
---|
31 | match with what the tasks need to do, so that writing the tasks is easy.
|
---|
32 | The API also has to be general enough to deal with stuff we haven't thought
|
---|
33 | of yet. On that note, I personally think that JNDI might be a touch too
|
---|
34 | general for what we need.
|
---|
35 |
|
---|
36 | So, the features. Note that many of these will be optional - not every
|
---|
37 | feature will be available for every node in the file system. I've used the
|
---|
38 | term "node" to mean both directories and files. I'm not suggesting we
|
---|
39 | actually call them "nodes" in the API. I've used the term "root node" to
|
---|
40 | mean the root of a file system.
|
---|
41 |
|
---|
42 | * Naming
|
---|
43 |
|
---|
44 | - Locate a node by absolute name.
|
---|
45 | - Get the absolute name for a node.
|
---|
46 | - Resolve a name to a node, relative to some base node - like
|
---|
47 | FileUtils.resolveFile().
|
---|
48 | - Get the relative name of a node, relative to some base node.
|
---|
49 | - Determine the base name (with and without the extension), and extension of
|
---|
50 | the node.
|
---|
51 | - Deal with file systems that are case sensitive, and case insentitive.
|
---|
52 |
|
---|
53 | * Properties
|
---|
54 |
|
---|
55 | - Determine what properties are available on the node.
|
---|
56 | - Determine if the node exists.
|
---|
57 | - Determine the type of node (file vs. directory, could be "has-content" vs
|
---|
58 | "has-children").
|
---|
59 | - Determine if the node is readable.
|
---|
60 | - Determine if the node is writeable.
|
---|
61 | - Get/set the permissions on the node. This covers things like chmod &
|
---|
62 | chown, making read-only, making executable, etc.
|
---|
63 |
|
---|
64 | * Content
|
---|
65 |
|
---|
66 | - Determine if the node can/does have content.
|
---|
67 | - Get the size of the node.
|
---|
68 | - Get/set the last-modified time of the node.
|
---|
69 | - Get/set the mime-type of the node.
|
---|
70 | - Get/set the encoding of the node.
|
---|
71 | - Get a checksum of the node.
|
---|
72 | - Get content as InputStream.
|
---|
73 | - Get content as Reader.
|
---|
74 | - Set content as an OutputStream.
|
---|
75 | - Set content as a Writer.
|
---|
76 | - Implicit creation of node and its ancestors when content is written.
|
---|
77 | - Compare nodes for equality (last modified timestamp, checksum, bytewise
|
---|
78 | compare).
|
---|
79 |
|
---|
80 | * Hierarchy
|
---|
81 |
|
---|
82 | - Get the parent node of a node.
|
---|
83 | - Get the child nodes of a node.
|
---|
84 | - Iterate over (or visit) the descendants of a node.
|
---|
85 | - With or without a selector.
|
---|
86 | - In various orders - depthwise, etc.
|
---|
87 | - Be able to modify the nodes during traversal.
|
---|
88 |
|
---|
89 | * Modification
|
---|
90 |
|
---|
91 | - Create a new node of a particular type. Create all missing ancestors.
|
---|
92 | - Move, copy, delete a node.
|
---|
93 | - All descendants.
|
---|
94 | - Optional selector. E.g. ignore empty dirs, ignore default excludes, etc.
|
---|
95 | - Optional filter.
|
---|
96 |
|
---|
97 | * Conversion
|
---|
98 |
|
---|
99 | - Convert the node to a java.net.URL.
|
---|
100 | - Make the node content available as a local file (to hand off to external
|
---|
101 | commands).
|
---|
102 | - Get the OS specific *filename* for a node.
|
---|
103 | - Resolve an OS specific *filename* to a node.
|
---|
104 |
|
---|
105 | * File System Types
|
---|
106 |
|
---|
107 | - Local file.
|
---|
108 | - HTTP.
|
---|
109 | - FTP.
|
---|
110 | - Classloader, uses Classloader.getResource().
|
---|
111 | - Temporary files.
|
---|
112 | - etc ...
|
---|
113 |
|
---|
114 | - Compound file system. Made up of a bunch of mount points. The VFS
|
---|
115 | itself.
|
---|
116 |
|
---|
117 | - Layered file systems (that sit on top of another file system or node):
|
---|
118 | - zip, bzip, jar, tar
|
---|
119 | - filtering - token replacement, etc
|
---|
120 |
|
---|
121 | - Factories for creating and configuring file system root nodes.
|
---|
122 | - Ability to easily add new file system implementations.
|
---|
123 |
|
---|
124 | * Task Container
|
---|
125 |
|
---|
126 | - A mechanism for a task to get hold of the project's root node.
|
---|
127 |
|
---|
128 | - A mechanism that allows a task to create its own private root nodes,
|
---|
129 | without letting it mess with the project's file system, or the file systems
|
---|
130 | of other tasks.
|
---|
131 |
|
---|
132 | - A mechanism for cleaning up all the node InputStream, OutputStream, Reader
|
---|
133 | and Writers opened by a task during its execute() method. Cleaning up files
|
---|
134 | is one thing the current tasks don't do very well at all. Something like
|
---|
135 | this would take care of anything the task did not explictly close. Would
|
---|
136 | include root nodes it created.
|
---|
137 |
|
---|
138 | * Other things
|
---|
139 |
|
---|
140 | - Maybe some way to explicitly close a node and release all resources used
|
---|
141 | by it.
|
---|
142 |
|
---|
143 | - Maybe detection of concurrent updates, in the case of parallel tasks.
|
---|
144 |
|
---|
145 | - Netbeans has an event model in its VFS. Something like this might be
|
---|
146 | useful in dependency management.
|
---|
147 |
|
---|
148 | - nodesets. The replacement for, or generalisation of, FileSet, Path,
|
---|
149 | FileList, etc
|
---|
150 | - A nodeset that contains the descendents of a node that match a selector
|
---|
151 | (like the current FileSet implementation).
|
---|
152 | - A nodeset that contains arbitrary nodes.
|
---|
153 | - An aggregating nodeset.
|
---|
154 | - Custom nodeset implementations.
|
---|
155 |
|
---|
156 | - Reimplement the Ant 1 Fileset, Path and Filelist as adaptors sitting on
|
---|
157 | top of the VFS.
|
---|
158 |
|
---|
159 | - A classloader that can load classes from a node.
|
---|
160 |
|
---|
161 | - etc ..
|
---|
162 |
|
---|
163 | What's missing? What shouldn't be on the list?
|
---|
164 |
|
---|
165 |
|
---|
166 | Adam
|
---|
167 |
|
---|
168 | > -----Original Message-----
|
---|
169 | > From: Magesh Umasankar
|
---|
170 | > Sent: Saturday, 22 December 2001 10:44 AM
|
---|
171 | > To: [email protected]
|
---|
172 | > Subject: Virtual FileSystem Layer
|
---|
173 | >
|
---|
174 | >
|
---|
175 | > I have been spending some time now on the VFS
|
---|
176 | > layer... Nothing major to report yet, but I just wanted
|
---|
177 | > to sound off so that if I am going down the wrong
|
---|
178 | > route, I correct it right away.
|
---|
179 | >
|
---|
180 | > I evaluated at WebNFS, NetBeansFS (NBFS) and
|
---|
181 | > JNDI.
|
---|
182 | >
|
---|
183 | > 1. WebNFS seems to be going nowhere. It has
|
---|
184 | > been dormant for quite sometime now. Licensing
|
---|
185 | > is rigid. Technically, it doesn't look so bad as it
|
---|
186 | > closely replicates java.io.File's API. But then,
|
---|
187 | > that really gives us very little.
|
---|
188 | >
|
---|
189 | > 2. NBFS looks OK. It has got a few filesystems
|
---|
190 | > already built. There may be some licensing issues,
|
---|
191 | > I don't know, but that shouldn't concern us too
|
---|
192 | > much as, according to Peter, it is Mozilla (I haven't
|
---|
193 | > really check the license out, sorry). But, as far as I
|
---|
194 | > can see, it seems to lack in sophisticated API features
|
---|
195 | > like searching based on attributes, etc., which
|
---|
196 | > we will definitely be needing for the Selector APIs.
|
---|
197 | >
|
---|
198 | > 3. JNDI, by far, beats the above to, in my
|
---|
199 | > evaluation. It is generic enough. We don't have
|
---|
200 | > any licensing issues. It has also become part of
|
---|
201 | > the core JRE (1.4 onwards). Technically, it fits to a T
|
---|
202 | > what we are looking for - virtual file system that
|
---|
203 | > provides search controls, access attributes,
|
---|
204 | > url mounting, etc. Furthermore, there's been
|
---|
205 | > some ground work already done for us at Jakarta/Apache
|
---|
206 | > (Catalina). I have written a SPI for a FTPFileSystem
|
---|
207 | > - though it is in a real crude stage right now. I believe
|
---|
208 | > this is the way to go because Ant's code would be
|
---|
209 | > operating at the (Dir)Context level and we can keep
|
---|
210 | > adding SPIs as we need them. Furthermore,
|
---|
211 | > JNDI has been stable for quite sometime now and
|
---|
212 | > we can depend on a widely used API.
|
---|
213 | >
|
---|
214 | > I don't think JNDI is a heavyweight API for our needs.
|
---|
215 | > It seems to be the only one, so far, which encompasses
|
---|
216 | > at the APIP level, all the new functionalities that we
|
---|
217 | > desire to introduce.
|
---|
218 | >
|
---|
219 | > Let me know if my approach, so far, to go the JNDI
|
---|
220 | > route seems reasonable.
|
---|
221 | >
|
---|
222 | > Cheers,
|
---|
223 | > Magesh
|
---|
224 | >
|
---|
225 | >
|
---|
226 | >
|
---|
227 | >
|
---|
228 | > --
|
---|
229 | > To unsubscribe, e-mail: <mailto:[email protected]>
|
---|
230 | > For additional commands, e-mail: <mailto:[email protected]>
|
---|
231 |
|
---|
232 |
|
---|
233 | --
|
---|
234 | To unsubscribe, e-mail: <mailto:[email protected]>
|
---|
235 | For additional commands, e-mail: <mailto:[email protected]>
|
---|
236 |
|
---|
237 |
|
---|