Difference between revisions of "Field Query Mapping"
Dithyramble (talk | contribs) m (disambig "subject") |
Dithyramble (talk | contribs) |
||
| Line 12: | Line 12: | ||
== Requirements == | == Requirements == | ||
| − | When DiscoverEd crawls feeds and resources and saves metadata such as the page title, it converts this information into RDF triples; those triples are eventually saved on disk in a triple store, Jena. | + | When DiscoverEd crawls feeds and resources and saves metadata such as the page title, it converts this information into RDF triples; those triples are eventually saved on disk in a triple store, namely Jena. |
| − | We will create a new configuration file that stores a list of mappings from predicate URIs | + | We will create a new configuration file that stores a list of mappings from predicate URIs. For example, we might list "method:" as a shorthand for the RDF predicate <http://purl.org/dc/terms/instructionalMethod>, a.k.a. "dct:instructionalMethod". At indexing time, a Lucene column called "method" will be created in the Lucene documents corresponding to each resource that has the dct:instructionalMethod predicate set in the Jena store. |
| − | Then, at search time, Nutch's built-in query parser handles the query. | + | Then, at search time, Nutch's built-in query parser handles the query, e.g., "method:yaddayadda". |
== Implementation == | == Implementation == | ||
| Line 24: | Line 24: | ||
* Problem: The Lucene documents does not seem to show our column, so we're going back to the drawing board and carefully reading the [http://wiki.apache.org/nutch/HowToMakeCustomSearch relevant Nutch documentation] to make sure we're using the APIs correctly | * Problem: The Lucene documents does not seem to show our column, so we're going back to the drawing board and carefully reading the [http://wiki.apache.org/nutch/HowToMakeCustomSearch relevant Nutch documentation] to make sure we're using the APIs correctly | ||
| − | ==Deferred until later== | + | == Deferred until later == |
| − | * Handling provenance with regard to this. | + | * Handling provenance with regard to this. Based on the current plan for how to handle curator exclusion, and using the above example, we have to make sure that instead of adding merely the column "method", we add something like "curator1:method", "curator2:method", and so on. (This may be out of date; see the spec for Excluding curators.)` |
Revision as of 15:05, 22 June 2010
| Contact | Contact::Asheesh Laroia |
|---|---|
| Project | ,|project_name|Project Driver::project_name}} |
| Status | Status::In Development |
The people who run a DiscoverEd may wish to let users search specific metadata easily. For example, http://discovered.creativecommons.org/search/ lets users search for works "tagged" with "banana" by searching for tag:banana. (In particular, the predicate for "tag" is the term "subject" as specified by the Dublin Core.)
These prefixes, like tag:, are stored in the DiscoverEd code right now. This feature aims to move those into a configuration file.
This feature was defined and developed during the June 2010 DiscoverEd Sprint
Requirements
When DiscoverEd crawls feeds and resources and saves metadata such as the page title, it converts this information into RDF triples; those triples are eventually saved on disk in a triple store, namely Jena.
We will create a new configuration file that stores a list of mappings from predicate URIs. For example, we might list "method:" as a shorthand for the RDF predicate <http://purl.org/dc/terms/instructionalMethod>, a.k.a. "dct:instructionalMethod". At indexing time, a Lucene column called "method" will be created in the Lucene documents corresponding to each resource that has the dct:instructionalMethod predicate set in the Jena store.
Then, at search time, Nutch's built-in query parser handles the query, e.g., "method:yaddayadda".
Implementation
- Added a sample configuration file
- Added code to our IndexFilter that looks for relevant triples and stores them in the Lucene document
- Problem: The Lucene documents does not seem to show our column, so we're going back to the drawing board and carefully reading the relevant Nutch documentation to make sure we're using the APIs correctly
Deferred until later
- Handling provenance with regard to this. Based on the current plan for how to handle curator exclusion, and using the above example, we have to make sure that instead of adding merely the column "method", we add something like "curator1:method", "curator2:method", and so on. (This may be out of date; see the spec for Excluding curators.)`
