1 | <?xml version="1.0" encoding="UTF-8"?>
|
---|
2 | <!DOCTYPE videocollection [
|
---|
3 | <!ENTITY ndash "–">
|
---|
4 | <!ENTITY mdash "—">
|
---|
5 | ]>
|
---|
6 | <xsl:stylesheet version="1.0"
|
---|
7 | xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
|
---|
8 | xmlns:java="http://xml.apache.org/xslt/java"
|
---|
9 | xmlns:util="xalan://org.greenstone.gsdl3.util.XSLTUtil"
|
---|
10 | xmlns:gslib="http://www.greenstone.org/skinning"
|
---|
11 | extension-element-prefixes="java util"
|
---|
12 | exclude-result-prefixes="java util">
|
---|
13 |
|
---|
14 | <!-- use the 'main' layout -->
|
---|
15 | <xsl:include href="layouts/main.xsl"/>
|
---|
16 |
|
---|
17 | <!-- set page title -->
|
---|
18 | <xsl:template name="pageTitle"><gslib:collectionName/></xsl:template>
|
---|
19 |
|
---|
20 | <!-- set page breadcrumbs -->
|
---|
21 | <xsl:template name="breadcrumbs"><gslib:siteLink/><gslib:rightArrow/></xsl:template>
|
---|
22 |
|
---|
23 | <!-- the page content -->
|
---|
24 | <xsl:template match="/page">
|
---|
25 |
|
---|
26 | <!--Display the description text of the current collection,
|
---|
27 | and if some services are available then create a list
|
---|
28 | of links for each service within a <ul id="servicelist"> element.-->
|
---|
29 |
|
---|
30 | <xsl:variable name="httpPath">
|
---|
31 | <xsl:value-of select="/page/pageResponse/collection/metadataList/metadata[@name='httpPath']" />
|
---|
32 | </xsl:variable>
|
---|
33 |
|
---|
34 |
|
---|
35 | <style>
|
---|
36 | p.about { text-align: justify; }
|
---|
37 | h2 { /* line-height: 135%; */ }
|
---|
38 | h3 { margin-top: 4pt; margin-bottom: 7pt; line-height: 135%; }
|
---|
39 |
|
---|
40 | a.mashup {text-decoration: none; }
|
---|
41 | a.mashup:link {color:#000080;} /* unvisited link */
|
---|
42 | a.mashup:visited {color:#303080;} /* visited link */
|
---|
43 | a.mashup:hover {color:#000040;} /* mouse over link */
|
---|
44 | a.mashup:active {color:#0000a0;}
|
---|
45 |
|
---|
46 | </style>
|
---|
47 |
|
---|
48 |
|
---|
49 | <script src="{$httpPath}/script2/browser-detect.js" type="text/javascript">
|
---|
50 | /* space filler needed */
|
---|
51 | </script>
|
---|
52 |
|
---|
53 |
|
---|
54 | <div style="width: 800px; margin-left: auto; margin-right: auto;">
|
---|
55 |
|
---|
56 |
|
---|
57 | <h2>A Musical Web Observatory?</h2>
|
---|
58 |
|
---|
59 | <h3><i><font size="+2">Connecting People with Data, Documents, and Algorithms</font></i></h3>
|
---|
60 |
|
---|
61 | <!--
|
---|
62 | <div style="background-color: #DD3611; color: white; padding: 4pt; margin-bottom: 6pt;">
|
---|
63 | <table>
|
---|
64 |
|
---|
65 | <tr>
|
---|
66 | <td>
|
---|
67 | <img src="{$httpPath}/images/firefox-128.png" />
|
---|
68 | </td>
|
---|
69 | <td>
|
---|
70 | <p class="about">
|
---|
71 | <i><font size="+2">
|
---|
72 | An experiment with Firefox's Advanced
|
---|
73 | Audio Processing Extension ...
|
---|
74 | </font></i>
|
---|
75 | </p>
|
---|
76 |
|
---|
77 | <p class="about">
|
---|
78 | <i><font size="+2">
|
---|
79 | ... featuring a mash-up of
|
---|
80 | <a class="mashup" href="http://www.greenstone.org" target="_blank">Greenstone</a>,
|
---|
81 | <a class="mashup" href="http://www.omras2.org/audioDB" target="_blank">AudioDB</a>, and
|
---|
82 | <a class="mashup" href="http://seasr.org/meandre/" target="_blank">Meandre</a>.
|
---|
83 | </font></i>
|
---|
84 | </p>
|
---|
85 |
|
---|
86 |
|
---|
87 | <div id="bd-version" style="display:none;" class="about">
|
---|
88 | <p>
|
---|
89 | <img src="{$httpPath}/images/warning-icon.png" />
|
---|
90 | Version of Firefox detected:
|
---|
91 | <script type="text/javascript">
|
---|
92 | document.write(BrowserDetect.version);
|
---|
93 | </script>
|
---|
94 | </p>
|
---|
95 | <p>
|
---|
96 | You need to upgrade to a more recent version of Firefox
|
---|
97 | (v4.0 or greater) to access all the audio features of
|
---|
98 | this demonstration.
|
---|
99 | </p>
|
---|
100 | </div>
|
---|
101 |
|
---|
102 | <div id="bd-browser" style="display:none;" class="about">
|
---|
103 |
|
---|
104 | <p>
|
---|
105 | <img src="{$httpPath}/images/warning-icon.png" />
|
---|
106 | Browser detected:
|
---|
107 | <script type="text/javascript">
|
---|
108 | document.write(BrowserDetect.browser);
|
---|
109 | </script>
|
---|
110 | </p>
|
---|
111 | <p>You need to access this web-site using Firefox
|
---|
112 | (v4.0 or greater) to access all the audio features provided.
|
---|
113 | </p>
|
---|
114 | </div>
|
---|
115 |
|
---|
116 |
|
---|
117 |
|
---|
118 |
|
---|
119 | <script>
|
---|
120 | <xsl:text disable-output-escaping="yes">
|
---|
121 |
|
---|
122 | if (BrowserDetect.browser == "Firefox") {
|
---|
123 | if (BrowserDetect.version < 4) {
|
---|
124 | document.getElementById("bd-version").style.display="block";
|
---|
125 |
|
---|
126 | }
|
---|
127 | }
|
---|
128 | else {
|
---|
129 | document.getElementById("bd-browser").style.display="block";
|
---|
130 |
|
---|
131 | }
|
---|
132 | </xsl:text>
|
---|
133 | </script>
|
---|
134 |
|
---|
135 |
|
---|
136 |
|
---|
137 |
|
---|
138 | </td>
|
---|
139 | </tr>
|
---|
140 | </table>
|
---|
141 | </div>
|
---|
142 | -->
|
---|
143 |
|
---|
144 | <p class="about">This music DL embodies the following:
|
---|
145 |
|
---|
146 | <style>
|
---|
147 | li { padding-bottom: 10px; }
|
---|
148 |
|
---|
149 | </style>
|
---|
150 |
|
---|
151 |
|
---|
152 | <ol>
|
---|
153 | <li>
|
---|
154 | <i>Starting point:</i> Manual (command-line) aggregation of disparate
|
---|
155 | resources prior to building the DL. A set of bespoke Greenstone
|
---|
156 | document processing plugins corrals the heterogeneous gathered
|
---|
157 | data into a unified (homogeneous) canonical form that the DL can
|
---|
158 | access and display. Everything presented in the DL is
|
---|
159 | either pre-computed (such as the self-similarity heat-maps) or else
|
---|
160 | computed at build-time.
|
---|
161 | </li>
|
---|
162 |
|
---|
163 | <li>
|
---|
164 | <i>Audio-fingerprinting:</i> as before, but now metadata about the audio
|
---|
165 | songs is now enriched through a set of audio-fingerprinting web
|
---|
166 | services. Everything presented in the DL is still pre-computed
|
---|
167 | or else computed at build-timeâhowever, the inclusion in the document
|
---|
168 | view of a "Discovery" block allows a user to begin to
|
---|
169 | access and explore through linked-data, information related to
|
---|
170 | the song.
|
---|
171 | </li>
|
---|
172 |
|
---|
173 | <li>
|
---|
174 | <i>Client-side audio processing (and visualization):</i> the
|
---|
175 | Pre-computed self-similarity heat-maps are dropped from the
|
---|
176 | collection building process in preference for the same information
|
---|
177 | being computed through Javascript running in the user's web
|
---|
178 | browser.
|
---|
179 | </li>
|
---|
180 |
|
---|
181 | <li>
|
---|
182 | <i>Embedded Meandre workflows:</i> the Meandre Workbench is
|
---|
183 | integrated into Greenstone. Audio documents in the Greenstone
|
---|
184 | digital library can now be dispatched and be processed by the
|
---|
185 | selected Meandre, and output from the workflow returned to the
|
---|
186 | Greenstone document viewâfor example, playing audio that has been
|
---|
187 | processed and output from the workflow.
|
---|
188 | </li>
|
---|
189 |
|
---|
190 | <li>
|
---|
191 | <i>Client-side workflows:</i> the client-side Javascript
|
---|
192 | processing code is refactored to follow the same methodology as
|
---|
193 | Meandre components.
|
---|
194 | </li>
|
---|
195 |
|
---|
196 | <li>
|
---|
197 | <i>Client/Server hybrid workflows:</i> the Greenstone/Meandre
|
---|
198 | integration is extended to support the dynamic
|
---|
199 | transmission of <tt>executeCallBack()</tt> methods written
|
---|
200 | in the user's browser in Javascript to be run on the
|
---|
201 | Meandre server as part of the active workflow.
|
---|
202 | </li>
|
---|
203 |
|
---|
204 | <li>
|
---|
205 | <i>Forging ahead:</i> the next area to be worked on is upgrading
|
---|
206 | the level of Greenstone/Meandre integration so that data produced
|
---|
207 | by the workflows can be incorporated back into the underlying
|
---|
208 | digital library itself, rather than being (as it currently stands)
|
---|
209 | transitory data that only lives for the duration of the web page
|
---|
210 | being viewed. This will form an implementation stepping-stone to
|
---|
211 | a more generalized ability to have data retrieved from other
|
---|
212 | external resources (located through the Discovery block
|
---|
213 | linked-data portion of the document view) ingested in to the DL
|
---|
214 | collection.
|
---|
215 | </li>
|
---|
216 |
|
---|
217 |
|
---|
218 | </ol>
|
---|
219 |
|
---|
220 | </p>
|
---|
221 |
|
---|
222 |
|
---|
223 |
|
---|
224 | <!--
|
---|
225 |
|
---|
226 | <p class="about">This digital library collection demonstrates the
|
---|
227 | integration of a variety of sources of structural metadata from both
|
---|
228 | automatically derived content analysis and manually labelled
|
---|
229 | ground-truth data to form a rich interactive web application suitable
|
---|
230 | for musicologists (as the target end-users) to explore a collated set
|
---|
231 | of music files.</p>
|
---|
232 |
|
---|
233 | <p class="about">While the base digital library system, <a
|
---|
234 | href="http://www.greenstone.org" target="_blank">Greenstone</a>, is
|
---|
235 | designed to operate with any web browser, the audio features this
|
---|
236 | particular DL collection demonstrates relies on audio processing
|
---|
237 | features currently only available in Firefox. Using one of the other
|
---|
238 | web browsers you will be able to access most features of the digital
|
---|
239 | library collection, but when you access a particular song you may find
|
---|
240 | playing the audioâin particular the panned audio
|
---|
241 | effectâdoes not work.</p>
|
---|
242 | -->
|
---|
243 |
|
---|
244 |
|
---|
245 | <!--
|
---|
246 | <p class="about">The developed system combines the open source capabilities of
|
---|
247 | Greenstone, \cite{Greenstone}, NEMA \cite{Nema}, Salami \cite{Salami},
|
---|
248 | and AudioDB \cite{AudioDB}.
|
---|
249 |
|
---|
250 | %%
|
---|
251 | NEMA and Salami have been previously described in this article, and contribute
|
---|
252 | manual and audio content based metadata.
|
---|
253 | %%
|
---|
254 | AudioDB is a raw-audio content based searching algorithm based on Local-Sensitivity Hashing \cite{LSH}.
|
---|
255 | %%
|
---|
256 | Greenstone is a versatile digital library architecture with an
|
---|
257 | extensible service-based architecture that. In
|
---|
258 | addition to the text-based searching and browsing capability to
|
---|
259 | organize content, Greenstone provides the framework in which to harness the
|
---|
260 | structural metadata from NEMA and Salami, and the audio-content based
|
---|
261 | search functionality of audioDB.
|
---|
262 |
|
---|
263 | -->
|
---|
264 |
|
---|
265 |
|
---|
266 |
|
---|
267 | <h3>A Walkthrough</h3>
|
---|
268 |
|
---|
269 |
|
---|
270 | <p class="about">The following walkthrough is for the initial incarnation of the DL collection, where all
|
---|
271 | the information presented is either precomputed or computed at build time.</p>
|
---|
272 |
|
---|
273 | <p class="about">Taking as a starting point a set of music files identified as worthy
|
---|
274 | of study, the <a href="#browse">figure below</a> shows the result of browsing
|
---|
275 | the formed digital library collection <a href="dev?a=b&rt=s&s=ClassifierBrowse&c=salami-audioDB&cl=CL1" target="_blank">by title</a> from a web browser.
|
---|
276 | The figure is a useful snapshot in which to orientate ourselves with
|
---|
277 | the main structure and features to the digital library. Functionality that
|
---|
278 | persistently reoccurs is accessible through the header to the page.</p>
|
---|
279 |
|
---|
280 | <p class="about">This includes:
|
---|
281 | <ul>
|
---|
282 |
|
---|
283 | <li>help and preferences (top-right);</li>
|
---|
284 | <li>a quick-search option (located just
|
---|
285 | below) with links to more sophisticated searching options; and</li>
|
---|
286 | <li>pin-pointing where within the site a user is currently located
|
---|
287 | (top-left).</li>
|
---|
288 | </ul>
|
---|
289 |
|
---|
290 |
|
---|
291 | <a name="browse" />
|
---|
292 | <table style="width: 700px; margin-left: auto; margin-right: auto; margin-bottom: 6pt;">
|
---|
293 | <tr>
|
---|
294 | <td style="border: solid 1px;">
|
---|
295 | <img style="width: 700px" src="{$httpPath}/images/figs/cropped/salami-browse.png" />
|
---|
296 | </td>
|
---|
297 | </tr>
|
---|
298 | <tr style="background-color: #bbeebb">
|
---|
299 | <td>
|
---|
300 | <i>Browsing in the digital library
|
---|
301 | <a href="dev?a=b&rt=s&s=ClassifierBrowse&c=salami-audioDB&cl=CL1" target="_blank">by titles</a>.</i>
|
---|
302 | </td>
|
---|
303 | </tr>
|
---|
304 | </table>
|
---|
305 |
|
---|
306 |
|
---|
307 | The specific content to this location within the site (in this case
|
---|
308 | browsing by title) is shown beneath the main banner. Various
|
---|
309 | groupings of title can be accessed by clicking on the bookshelf icons
|
---|
310 | vertically aligned as the main part of the page: currently <a
|
---|
311 | href="dev?a=b&rt=s&s=ClassifierBrowse&c=salami-audioDB&cl=CL1#CL1.2">CâD</a>
|
---|
312 | is open, with the remaining letters to the alphabet below this,
|
---|
313 | accessed through scrolling.</p>
|
---|
314 |
|
---|
315 |
|
---|
316 |
|
---|
317 |
|
---|
318 | <p class="about">Interested in the song <i>Candela</i> our curious musicologist clicks
|
---|
319 | on <a href="dev?a=d&ed=1&book=off&c=salami-audioDB&d=D145&dt=simple&sib=1&p.a=b&p.sa=&p.s=ClassifierBrowse" target="_blank">the link for this</a>. This brings up the document view to this song:</p>
|
---|
320 |
|
---|
321 | <a name="self-similarity" />
|
---|
322 | <table style="width: 700px; margin-left: auto; margin-right: auto; margin-bottom: 6pt;">
|
---|
323 | <tr>
|
---|
324 | <td style="border: solid 1px;">
|
---|
325 | <img style="width: 700px" src="{$httpPath}/images/figs/cropped/salami-self-similarity2.png" />
|
---|
326 | </td>
|
---|
327 | </tr>
|
---|
328 | <tr style="background-color: #bbeebb">
|
---|
329 | <td>
|
---|
330 | <i>The <a href="dev?a=d&ed=1&book=off&c=salami-audioDB&d=D145&dt=simple&sib=1&p.a=b&p.sa=&p.s=ClassifierBrowse" target="_blank">musicologically enriched document view</a> for</i> Candela.
|
---|
331 | </td>
|
---|
332 | </tr>
|
---|
333 | </table>
|
---|
334 |
|
---|
335 | <p class="about">Normally in a digital
|
---|
336 | library the document view brings up a page that is strongly derived
|
---|
337 | from textual metadata. If the document viewed was a text-document,
|
---|
338 | some summary information such as title and author is typically
|
---|
339 | presented, say in tabular form, before the main text is presented.
|
---|
340 | Even in the case of multimedia digital libraries, the view presented
|
---|
341 | is still strongly derived from textual metadata: this time including
|
---|
342 | details such as the length of the video, the TV company that produced
|
---|
343 | it, whether captions are available, and so forth, accompanied with an
|
---|
344 | embedded video player for viewing the contentâessentially more
|
---|
345 | textual metadata (in this case the URL to the video content) which in
|
---|
346 | terms of the user-interface is largely divorced from the other
|
---|
347 | elements displayed on the page.</p>
|
---|
348 |
|
---|
349 | <p class="about">This contrasts sharply with the document view developed in this
|
---|
350 | digital library.
|
---|
351 | Naturally it allows the song to be played (akin to the embedded video player),
|
---|
352 | however this is largely of secondary importance to the
|
---|
353 | other functionality available this is much more closely
|
---|
354 | integrated.</p>
|
---|
355 |
|
---|
356 | <p class="about">The most striking visual component to the document view is a
|
---|
357 | self-similarity "heat map" where the duration of the song forms both
|
---|
358 | the <i>x</i>- and <i>y</i>-axis, and a red pixel located at a given <i>(x,y)</i>
|
---|
359 | co-ordinate in the map represents a location where two parts of the
|
---|
360 | song are strongly similar, proportionally shifting to blue to
|
---|
361 | represent dissimilar. Given this configuration, the leading diagonal
|
---|
362 | to the matrix (<i>x=y</i>) is always coloured red as this represents the
|
---|
363 | comparison of one part of the song with itself.</p>
|
---|
364 |
|
---|
365 | <p class="about">When the user moves the mouse cursor around the self-similarity map
|
---|
366 | a highlighting circle is shown to emphasize the area the user is over,
|
---|
367 | with a black dot at the centre (visible in in
|
---|
368 | the <a href="#self-similarity">above figure</a>); annotated vertically
|
---|
369 | and horizontally are the two time-offsets in seconds that that point
|
---|
370 | in the map corresponds to. Clicking the cursor at this point results
|
---|
371 | in the audio being played <i>simultaneously</i> from these two points.
|
---|
372 | To aid the musicologist in listening to the two parts of the song, one
|
---|
373 | part is panned to the left speaker, and the other to the right (this
|
---|
374 | was implemented using the extended audio API provided by Firefox, and
|
---|
375 | so this particular feature only work when viewing the collection with
|
---|
376 | this browserâsee implementation details below). In our figure,
|
---|
377 | the musicologist has zeroed in the location <i>x=33</i>, <i>y=97</i>
|
---|
378 | which corresponds to the start of a strong red diagonal that occurs
|
---|
379 | some distance off the leading diagonal. Listening to the two sounds
|
---|
380 | played (most reliably done with headphones on), they hear that these
|
---|
381 | two sections of the song are indeed repeating sections of the guitar
|
---|
382 | piece <i>Candela</i> with a minor variation in the latter section
|
---|
383 | where a recorder is also playing in the arrangement.</p>
|
---|
384 |
|
---|
385 | <p class="about">The structured audio time-lines (labelled A, B, ..., and 6, 5, 2,
|
---|
386 | ... in the figure) located above the self-similarity map are another
|
---|
387 | area of enriched musical content in the digital library. The upper
|
---|
388 | line shows the ground-truth data for this song generated by
|
---|
389 | the <a href="http://www.music-ir.org/?q=node/14" target="_blank">Salami project</a>;
|
---|
390 | the lower line is generated by an algorithmically based content
|
---|
391 | analysis algorithm.</p>
|
---|
392 |
|
---|
393 | <p class="about">While there is some agreement between these two lines, there are
|
---|
394 | also significant differences. The play and search buttons within the
|
---|
395 | structured time-lines (the latter represented by a magnifying glass)
|
---|
396 | allow the user to investigate further these structures. We shall
|
---|
397 | return to the search functionality shortly (which is content based,
|
---|
398 | using <a href="http://omras2.doc.gold.ac.uk/software/audiodb/" target="_blank">AudioDB</a>),
|
---|
399 | but in the meantime note that with the time-lines positioned above the
|
---|
400 | self-similarity map, there is further opportunity to study the
|
---|
401 | differences between the two structured time-line. It is certainly the
|
---|
402 | case that there are strong visual cues in the map that line up with
|
---|
403 | the algorithmic time-line, even though they do not align with a
|
---|
404 | boundary in the ground-truth data, and the user can click on these
|
---|
405 | parts of the similarity map to hear what is occurring at these
|
---|
406 | points.</p>
|
---|
407 |
|
---|
408 |
|
---|
409 |
|
---|
410 | <a name="resultspage" />
|
---|
411 | <table style="width: 700px; margin-left: auto; margin-right: auto; margin-bottom: 6pt;">
|
---|
412 | <tr>
|
---|
413 | <td style="border: solid 1px;">
|
---|
414 | <img style="width: 350px" src="{$httpPath}/images/figs/cropped/salami-audiodb-search-resultspage-halfcut.png" />
|
---|
415 | </td>
|
---|
416 | <td style="border: solid 1px;">
|
---|
417 | <img style="width: 350px" src="{$httpPath}/images/figs/cropped/salami-structured-search-resultspage-halfcut.png" />
|
---|
418 | </td>
|
---|
419 | </tr>
|
---|
420 | <tr style="background-color: #bbeebb">
|
---|
421 | <td colspan="2">
|
---|
422 | <i>Audio content based search results: left) the
|
---|
423 | <a href="dev?a=q&sa=&rt=rd&s=AudioQuery&c=salami-audioDB&startPage=1&s1.maxDocs=100&s1.hitsPerPage=20&q=D206.dir&s1.query=D206.dir&s1.offset=263&s1.length=24&mysongWindowDuration=6" target="_blank">result list</a> from
|
---|
424 | an audio content based query taken from an extract of </i>
|
---|
425 | Michelle; <i>and right)
|
---|
426 | the <a href="dev?a=q&sa=&rt=rd&s=TextQuery&c=salami-audioDB&startPage=1&s1.query=%22b+b+c+b+c%22&s1.index=JS" target="_blank">result of a structured music search</a> for
|
---|
427 | content containing "b b c b a" as a sequence.</i>
|
---|
428 | </td>
|
---|
429 | </tr>
|
---|
430 | </table>
|
---|
431 |
|
---|
432 | <p class="about">Returning to the search capability provided by the structured
|
---|
433 | time-lines, the <a href="#resultspage">above figure (left)</a> show the result of
|
---|
434 | using this feature while study the song <i>Michelle</i> by The
|
---|
435 | Beatles. In this case the user selected the section of the
|
---|
436 | ground-truth time-line corresponding to section starting "I want you
|
---|
437 | ...", but could have equally used the algorithmically calculated
|
---|
438 | time-line, or in fact paused the song playing at any point, and
|
---|
439 | started a match from there.</p>
|
---|
440 |
|
---|
441 |
|
---|
442 | <p class="about">Not surprisingly <i>Michelle</i> is returned as the top hit (at
|
---|
443 | 92.1%)âwe shall see shortly that this is because the system found
|
---|
444 | several sections of the song that matched thisâthe next hit being
|
---|
445 | <i>Bigger Than JC</i> at 74.4%, and so on down, where only one hit
|
---|
446 | per song occurs. Clicking on the top hit,
|
---|
447 | the <a href="#audiodb">figure below</a> shows the document view that is
|
---|
448 | displayed, focusing in on the key area to this screen. This time the
|
---|
449 | time-line area has an additional bar: the points within the song that
|
---|
450 | AudioDB found to be similar. Like the other time-lines, a play button
|
---|
451 | is present on these segments so the user can play the matching points
|
---|
452 | directly. In this case
|
---|
453 | clicking on them reveals the matching sections found correspond to
|
---|
454 | melodically repeating sections of the song, only with different lyrics
|
---|
455 | ("I love you ..." and "I need you ...").</p>
|
---|
456 |
|
---|
457 |
|
---|
458 | <p class="about">A further form of musical content-based searching is available through
|
---|
459 | the main header to the digital library. Instead of searching by title
|
---|
460 | or artist (which are also available) the user can click on this
|
---|
461 | "search by" menu next to the quick-search text box and select
|
---|
462 | "ground-truth structure" instead.
|
---|
463 | The <a href="#resultspage">figure above</a> (right) shows the result of using this
|
---|
464 | option, where the user has entered "b b c b c" as the query, in
|
---|
465 | other words searching for songs that have two sections in a row the
|
---|
466 | same, then a new section, then returning to the original section,
|
---|
467 | before progressing to a recurrence of the second distinct section. In
|
---|
468 | pop and rock music this is a popular sequence corresponding to verse,
|
---|
469 | verse, chorus, verse, chorus. A more intriguing framing of a query
|
---|
470 | along these linesâequally possible in the digital libraryâwould be
|
---|
471 | to go to the fielded search page (accessible through the <i>advanced
|
---|
472 | search</i> link below the quick-search box, and entered the same
|
---|
473 | music structure query, but this time combine it with other fielded
|
---|
474 | query terms (Genre=Blues OR Jazz, Year<1950). Results from this
|
---|
475 | query would let the musicologist explore potential evidence of this
|
---|
476 | pattern of sections being used in two of the key musical genres that
|
---|
477 | influenced the development of Rock and Roll music.</p>
|
---|
478 |
|
---|
479 |
|
---|
480 | <a name="audiodb" />
|
---|
481 | <table style="width: 700px; margin-left: auto; margin-right: auto; margin-bottom: 6pt;">
|
---|
482 | <tr>
|
---|
483 | <td style="border: solid 1px;">
|
---|
484 | <img style="width: 700px" src="{$httpPath}/images/figs/cropped/salami-audiodb-search2.png" />
|
---|
485 | </td>
|
---|
486 | </tr>
|
---|
487 | <tr style="background-color: #bbeebb">
|
---|
488 | <td>
|
---|
489 | <i>The document view for</i> Michelle <i><a href="dev?a=d&c=salami-audioDB&d=D206&dt=simple&p.frameOffset=133,393,264,262,392&p.frameLength=24&p.a=q&p.s=AudioQuery&hl=on&ed=1#D206" target="_blank">augmented with the locations of the matches</a> of the audio query.</i>
|
---|
490 | </td>
|
---|
491 | </tr>
|
---|
492 | </table>
|
---|
493 |
|
---|
494 |
|
---|
495 |
|
---|
496 |
|
---|
497 | <h3>Implementation details</h3>
|
---|
498 |
|
---|
499 | <p class="about">The core part of the interactive elements in the document view were
|
---|
500 | implemented using SVG combined with Javascript. The left- and
|
---|
501 | right-panning interactively available from the self-similarity map was
|
---|
502 | implemented by processing the raw audio stream, made accessible by the
|
---|
503 | Firefox Audio extension
|
---|
504 | <a href="https://wiki.mozilla.org/Audio_Data_API" target="_blank">API</a>.</p>
|
---|
505 |
|
---|
506 | <p class="about">AudioDB content based searching was integrated into Greenstone through
|
---|
507 | two components of the digital library software architecture: its
|
---|
508 | build-time document processing plugin system, and its runtime
|
---|
509 | message-passing service-base framework. The developed plugin accepts
|
---|
510 | a wide range of audio formats (including OGG and MP3), and converts
|
---|
511 | them to WAV, the format needed by AudioDB for processing. The new
|
---|
512 | search service took the form of a proxy, accepting messages in the
|
---|
513 | XML syntax used by Greenstone, turning them into the necessary calls
|
---|
514 | to the AudioDB command-line interface, and then converting the output
|
---|
515 | from AudioDB back into the XML syntax expected by the digital library
|
---|
516 | architecture. Finally, the two parts were packaged to operate as a
|
---|
517 | Greenstone extension; the software is available at:
|
---|
518 | <a href="http://trac.greenstone.org/gs3-extensions/audioDB/trunk/src" target="_blank">http://trac.greenstone.org/gs3-extensions/audioDB/trunk/src</a>.</p>
|
---|
519 |
|
---|
520 | </div>
|
---|
521 |
|
---|
522 |
|
---|
523 | </xsl:template>
|
---|
524 |
|
---|
525 |
|
---|
526 | </xsl:stylesheet>
|
---|
527 |
|
---|