Creative Commons - User contributions [en]

Metadata Retriever Plugins

2010-06-16T19:04:32Z

BrendanGuenther: /* First Phase */

{{DiscoverEd Specification
|status=In Development
}}
==First Phase==
Operators may wish to include metadata about Resources from other sources, including web services (ie, Semantic Analysis), databases, or something else. This describes how to implement a plugin to import this information.

A plugin implements IMetadataImporter, which provides a single method, loadMetadata. loadMetadata takes a Resource, and is responsible for adding any fields to it. loadMetadata is not responsible for persisting the Resource.

The initial prototype will log URLs as it goes.

The first working plug-in could exercise the del.icio.us API to retrieve tags that delicious users have placed on a URI. Example: <code>api.del.icio.us</code> <code>/v1/posts/suggest?url=http://ocw.nd.edu/computer-applications/applied-multimedia-technology</code>

To accomplish this...
*enable resources to allow dynamic metadata to be attached (by plug-ins) through a new method on "resource"
**method that adds an assertion to resource
***checks to see if the assertion exists, uses if exists, creates if not exist
***populates assertion with data

**method that gets all the assertions from resource
*associate each custom metadata field with a URI-namespace to avoid cardinality
*Create plug-in; call from aggregate step; use resource's save method;

==Deferred until later==
Allow configuration of plug-ins
*enable/disable individual plug-ins
*add additional or custom plug-ins
*alter order in which plug-ins run (to determine which are authoritative)

Open Calais Metadata Importing Plugin

2010-06-16T18:59:58Z

BrendanGuenther:

{{DiscoverEd Specification
|status=In Development
}}

Attempt to use [http://www.opencalais.com/about|Open Calais] as a metadata importing plug-in to provide additional metadata for crawled resources.

<blockquote> Calais is a big initiative with a lot of components. What you care about is the Calais Web Service. The web service is an API that accepts unstructured text (like news articles, blog postings, etc.), processes them using natural language processing and machine learning algorithms, and returns RDF-formatted entities, facts and events. It takes about 0.5 to 1.0 second depending on how big a document you send and the size of your pipe.</blockquote>

[http://www.opencalais.com/about/developer|Open Calais Developer Documentation]

Open Calais is a Thomas Reuters, Inc product. It requires agreement to a legal terms of service. It has usage limits. It requires registration and retrieval of a key.

Reason for caution for some users...
If you syndicate, publish or otherwise transmit any content containing, enhanced by or derived from Calais-generated metadata you will use your best efforts to incorporate the correct Calais-provided Globally Unique Identifier (GUID) in that content. You specifically agree not to attach incorrect GUIDs to your content with any intent to mislead, spam, spoof, phish or otherwise deceive downstream consumers of your content.

DiscoverEd Sprint (June, 2010)

2010-06-16T13:10:41Z

BrendanGuenther:

[[Category:DiscoverEd]]

== Overview ==

* '''What:''' A sprint on development of [[DiscoverEd]]; see [[:Category:DiscoverEd_Specification|DiscoverEd Specifications]] for possible areas of work
* '''When:''' Tuesday, June 15 through Thursday, June 17, 2010
* '''Where:''' [http://vudat.msu.edu/location/ Wills House], Michigan State University, East Lansing, MI ([http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=101+Wills+House,+east+lansing,+mi&sll=37.0625,-95.677068&sspn=41.818029,58.447266&ie=UTF8&hq=&hnear=Wills+House,+East+Lansing,+Ingham,+Michigan+48823&z=14 map])

== Attendees ==

* Asheesh Laroia (OpenHatch / Creative Commons)
* Raphael Krut-Landau (OpenHatch / Creative Commons)
* [[User:Nathan Yergler|Nathan Yergler]] (Creative Commons)
* Alex Kozak (Creative Commons)
* Ali Asad Lotia (open.michigan)
* Kevin Coffman (open.michigan)
* Prabode Weebadde (MSU Global)
* Brendan Guenther (MSU vuDAT)
* Timothy Barney (MSU vuDAT)
* ''add your name and affiliation here''

== Travel & Accommodations ==

* [http://vudat.msu.edu/directions/ Directions to Wills House]
** Brendan Guenther will provide MSU Guest Parking Passes when you arrive in the morning
* Area Hotels (mention MSU for discounted rate)
** [http://www.marriott.com/hotels/travel/lants-towneplace-suites-east-lansing/ Townplace Suites by Marriott]
** [http://www.hamptoninn.com/en/hp/hotels/index.jhtml?ctyhocn=LANETHX Hampton Inn]

== Agenda ==

''This is a draft, subject to change.''

=== Tuesday ===

* 9:00 AM - Welcome
* DiscoverEd Context: Why, When, Where are we going? (10 min)
* Introductions<br/>Developers introduce themselves, give brief statement on what they've done with DiscoverEd, what they're interested in working on. (5 min ea.)
* MSU Context: AgShare, FSKN, etc (Chris Geith, 10-15 min)
* 10:30 AM - Identify Themes, Possible Blocks of Work
* 11:30 AM - Pair up and begin work
* 4:45 PM - Brief report back from each group: unexpected issues, things to bring to the group as a whole, etc.

=== Wednesday ===

No schedule; work as pairs.

=== Thursday ===

* 3:30 PM - Pairs begin to make sure work is pushed to Gitorious, determine next steps
* 4:00 PM - Full report back: pairs report on progress and state of their code. Pairs are encouraged to identify follow up steps they will take post sprint to see tasks to completion.

== Preparation ==

In order to minimize time spent configuring laptops, etc, please try to do the following before arriving at the sprint:

# Make sure you have prerequisite software installed and working:
#* git (Windows users, see http://progit.org/book/ch1-4.html and http://code.google.com/p/msysgit/)
#* Java 1.6 JDK
#* Eclipse (not required, but can make life easier)
# Generate an SSH public key (if needed; see http://progit.org/book/ch4-3.html for some instructions)
# Create a [http://gitorious.org Gitorious] account and add your SSH key to it

Open Calais Metadata Importing Plugin

2010-06-15T18:59:11Z

BrendanGuenther:

Open Calais Metadata Importing Plugin

2010-06-15T18:46:58Z

BrendanGuenther:

{{DiscoverEd Specification
|status=In Development
}}

Attempt to use Open Calais as a metadata importing plug-in to provide additional metadata for crawled resources.

<blockquote> Calais is a big initiative with a lot of components. What you care about is the Calais Web Service. The web service is an API that accepts unstructured text (like news articles, blog postings, etc.), processes them using natural language processing and machine learning algorithms, and returns RDF-formatted entities, facts and events. It takes about 0.5 to 1.0 second depending on how big a document you send and the size of your pipe.</blockquote>

[http://www.opencalais.com/about/developer|Open Calais Developer Documentation]

Open Calais Metadata Importing Plugin

2010-06-15T18:46:27Z

BrendanGuenther:

{{DiscoverEd Specification
|status=In Development
}}

Attempt to use Open Calais as a metadata importing plug-in to provide additional metadata for crawled resources.

<blockquote> Calais is a big initiative with a lot of components. What you care about is the Calais Web Service. The web service is an API that accepts unstructured text (like news articles, blog postings, etc.), processes them using natural language processing and machine learning algorithms, and returns RDF-formatted entities, facts and events. It takes about 0.5 to 1.0 second depending on how big a document you send and the size of your pipe.</blockquote>

Open Calais Developer Documentation [http://www.opencalais.com/about/developer|Open Calais Developer Documentation]

Open Calais Metadata Importing Plugin

2010-06-15T18:41:54Z

BrendanGuenther: Created page with '{{DiscoverEd Specification |status=In Development }} <blockquote> Calais is a big initiative with a lot of components. What you care about is the Calais Web Service. The web ser…'

{{DiscoverEd Specification
|status=In Development
}}

<blockquote> Calais is a big initiative with a lot of components. What you care about is the Calais Web Service. The web service is an API that accepts unstructured text (like news articles, blog postings, etc.), processes them using natural language processing and machine learning algorithms, and returns RDF-formatted entities, facts and events. It takes about 0.5 to 1.0 second depending on how big a document you send and the size of your pipe.</blockquote>

Metadata Retriever Plugins

2010-06-15T18:23:02Z

BrendanGuenther: /* First Phase */

{{DiscoverEd Specification
|status=In Development
}}
==First Phase==
Operators may wish to include metadata about Resources from other sources, including web services (ie, Semantic Analysis), databases, or something else. This describes how to implement a plugin to import this information.

A plugin implements IMetadataImporter, which provides a single method, loadMetadata. loadMetadata takes a Resource, and is responsible for adding any fields to it. loadMetadata is not responsible for persisting the Resource.

To accomplish this...
*enable resources to allow dynamic metadata to be attached (by plug-ins) through a new method on "resource"
**method that adds an assertion to resource
***checks to see if the assertion exists, uses if exists, creates if not exist
***populates assertion with data

**method that gets all the assertions from resource
*associate each custom metadata field with a URI-namespace to avoid cardinality
*Create plug-in; call from aggregate step; use resource's save method;

==Deferred until later==
Allow configuration of plug-ins
*enable/disable individual plug-ins
*add additional or custom plug-ins
*alter order in which plug-ins run (to determine which are authoritative)

Metadata Retriever Plugins

2010-06-15T18:16:17Z

BrendanGuenther: /* First Phase */

Metadata Retriever Plugins

2010-06-15T18:15:33Z

BrendanGuenther: /* First Phase */

Metadata Retriever Plugins

2010-06-15T18:12:33Z

BrendanGuenther:

DiscoverEd Sprint (June, 2010)

2010-06-07T17:16:10Z

BrendanGuenther: /* Travel & Accommodations */

[[Category:DiscoverEd]]

== Overview ==

* '''What:''' A sprint on development of [[DiscoverEd]]; see [[#Objectives]] below for possible development work
* '''When:''' Tuesday, June 15 through Thursday, June 17, 2010
* '''Where:''' [http://vudat.msu.edu/location/ Wills House], Michigan State University, East Lansing, MI ([http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=101+Wills+House,+east+lansing,+mi&sll=37.0625,-95.677068&sspn=41.818029,58.447266&ie=UTF8&hq=&hnear=Wills+House,+East+Lansing,+Ingham,+Michigan+48823&z=14 map])

== Attendees ==

* Asheesh Laroia (Open Hatch / Creative Commons)
* Raphael Krut-Landau (Open Hatch / Creative Commons)
* [[User:Nathan Yergler|Nathan Yergler]] (Creative Commons)
* Alex Kozak (Creative Commons)
* Ali Asad Lotia (open.michigan)
* Kevin Coffman (open.michigan)
* ''add your name and affiliation here''

== Travel & Accommodations ==

* [http://vudat.msu.edu/directions/ Directions to Wills House]
** Brendan Guenther will provide MSU Guest Parking Passes when you arrive in the morning
* Area Hotels (mention MSU for discounted rate)
** [http://www.marriott.com/hotels/travel/lants-towneplace-suites-east-lansing/ Townplace Suites by Marriott]
** [http://www.hamptoninn.com/en/hp/hotels/index.jhtml?ctyhocn=LANETHX Hampton Inn]

== Objectives ==

''TBD''

== Preparation ==

In order to minimize time spent configuring laptops, etc, please try to do the following before arriving at the sprint:

# Make sure you have prerequisite software installed and working:
#* git (Windows users, see http://progit.org/book/ch1-4.html and http://code.google.com/p/msysgit/)
#* Java 1.6 JDK
#* Eclipse (not required, but can make life easier)
# Generate an SSH public key (if needed; see http://progit.org/book/ch4-3.html for some instructions)
# Create a [http://gitorious.org Gitorious] account and add your SSH key to it

=== Code Preparation ===

''TBD''