Metadata Retriever Plugins - Revision history

Nathan Yergler at 16:55, 18 June 2010

2010-06-18T16:55:50Z

Nathan Yergler at 14:22, 17 June 2010

2010-06-17T14:22:25Z

Nathan Yergler at 14:15, 17 June 2010

2010-06-17T14:15:12Z

Nathan Yergler: /* Deferred until later */

2010-06-17T13:55:57Z

‎Deferred until later

Nathan Yergler: moved Metadata Importing Plugins to Metadata Retriever Plugins

2010-06-17T13:55:11Z

moved Metadata Importing Plugins to Metadata Retriever Plugins

BrendanGuenther: /* First Phase */

2010-06-16T19:04:32Z

‎First Phase

BrendanGuenther: /* First Phase */

2010-06-15T18:23:02Z

‎First Phase

BrendanGuenther: /* First Phase */

2010-06-15T18:16:17Z

‎First Phase

BrendanGuenther: /* First Phase */

2010-06-15T18:15:33Z

‎First Phase

BrendanGuenther at 18:12, 15 June 2010

2010-06-15T18:12:33Z

← Older revision		Revision as of 16:55, 18 June 2010
Line 6:		Line 6:

	DEd sites may wish to include metadata about Resources from other sources, including web services (ie, Semantic Analysis), databases, etc. This describes a plugin system for adding sources of information at aggregation time.		DEd sites may wish to include metadata about Resources from other sources, including web services (ie, Semantic Analysis), databases, etc. This describes a plugin system for adding sources of information at aggregation time.
		+
		+	This feature was defined and developed during the [[DiscoverEd Sprint (June, 2010)\|June 2010 DiscoverEd Sprint]]

	== Requirements ==		== Requirements ==

@@ Line 4: / Line 4: @@
 |status=Complete
 }}
-Operators may wish to include metadata about Resources from other sources, including web services (ie, Semantic Analysis), databases, etc.  This describes a plugin system for adding sources of information at aggregation time.
 == Requirements ==
-Metadata retriever plugins are Nutch plugins which implement the  MetadataRetriever extension point.  MetadataRetriever extensions must implement a single method, <code>retrieve</code>.  <code>retrieve</code> takes a Resource as an argument, and may add additional metadata to it.  <code>retrieve</code> is not responsible for persisting the Resource.
+Metadata retriever plugins are [http://wiki.apache.org/nutch/PluginCentral Nutch plugins] which implement the <code>MetadataRetriever</code> extension point.  MetadataRetriever extensions must implement a single method, <code>retrieve</code>.  <code>retrieve</code> takes a Resource as an argument, and may add additional metadata to it.  <code>retrieve</code> is not responsible for persisting the Resource.
 We will implement two demonstration plugins: a "dummy" plugin which logs the URLs being passed to it, and a functional demonstration which uses the Delicious API to [http://delicious.com/help/api#posts_suggest retrieve suggested and popular tags] for a page.

@@ Line 1: / Line 1: @@
 {{DiscoverEd Specification
-|status=In Development
+|contact=Nathan Yergler
 }}
-==First Phase==
+Operators may wish to include metadata about Resources from other sources, including web services (ie, Semantic Analysis), databases, etc.  This describes a plugin system for adding sources of information at aggregation time.
-A plugin implements IMetadataImporter, which provides a single method, loadMetadata.  loadMetadata takes a Resource, and is responsible for adding any fields to it.  loadMetadata is not responsible for persisting the Resource.
+== Requirements ==
-The initial prototype will log URLs as it goes.
+Metadata retriever plugins are Nutch plugins which implement the  MetadataRetriever extension point.  MetadataRetriever extensions must implement a single method, <code>retrieve</code>.  <code>retrieve</code> takes a Resource as an argument, and may add additional metadata to it.  <code>retrieve</code> is not responsible for persisting the Resource.
-The first working plug-in could exercise the del.icio.us API to retrieve tags that delicious users have placed on a URI.  Example: <code>api.del.icio.us</code> <code>/v1/posts/suggest?url=http://ocw.nd.edu/computer-applications/applied-multimedia-technology</code>
+We will implement two demonstration plugins: a "dummy" plugin which logs the URLs being passed to it, and a functional demonstration which uses the Delicious API to [http://delicious.com/help/api#posts_suggest retrieve suggested and popular tags] for a page.
-To accomplish this...
+== Implementation ==
-**method that gets all the assertions from resource
+* Added support for storing arbitrary metadata on Resource objects
-*associate each custom metadata field with a URI-namespace to avoid cardinality
+* Added support to TripleStore for serialization an de-serialization
-*Create plug-in; call from aggregate step; use resource's save method;
+* Implemented the org.creativecommons.learn.plugin.MetadataRetriever extension point and org.creativecommons.learn.plugin.MetadataRetrievers extension loader
 ==Deferred until later==
-*Allow configuration of plug-ins
-*Alter order in which plug-ins run (to determine which are authoritative)
+* Configuration parameters for plugins (the Delicious plugin reads from discovered.xml, but the plugin.xml manifest doesn't state that it needs parameters).

@@ Line 22: / Line 22: @@
 ==Deferred until later==
-Allow configuration of plug-ins
+*Allow configuration of plug-ins
-*enable/disable individual plug-ins
+*Alter order in which plug-ins run (to determine which are authoritative)

← Older revision		Revision as of 19:04, 16 June 2010
Line 6:		Line 6:

	A plugin implements IMetadataImporter, which provides a single method, loadMetadata. loadMetadata takes a Resource, and is responsible for adding any fields to it. loadMetadata is not responsible for persisting the Resource.		A plugin implements IMetadataImporter, which provides a single method, loadMetadata. loadMetadata takes a Resource, and is responsible for adding any fields to it. loadMetadata is not responsible for persisting the Resource.
		+
		+	The initial prototype will log URLs as it goes.
		+
		+	The first working plug-in could exercise the del.icio.us API to retrieve tags that delicious users have placed on a URI. Example: <code>api.del.icio.us</code> <code>/v1/posts/suggest?url=http://ocw.nd.edu/computer-applications/applied-multimedia-technology</code>

	To accomplish this...		To accomplish this...

← Older revision	Revision as of 13:55, 17 June 2010
(No difference)

← Older revision		Revision as of 18:15, 15 June 2010
Line 6:		Line 6:

	A plugin implements IMetadataImporter, which provides a single method, loadMetadata. loadMetadata takes a Resource, and is responsible for adding any fields to it. loadMetadata is not responsible for persisting the Resource.		A plugin implements IMetadataImporter, which provides a single method, loadMetadata. loadMetadata takes a Resource, and is responsible for adding any fields to it. loadMetadata is not responsible for persisting the Resource.
		+
		+	To accomplish this...
		+	*enable resources to allow dynamic metadata to be attached (by plug-ins)
		+	*associate each custom metadata field with a URI-namespace to avoid cardinality
		+	*Create plug-in; call from aggregate step

	==Deferred until later==		==Deferred until later==