Metadata Scraper - Revision history

CCID-rob: Decommissioning the scraper.

2018-01-08T18:44:19Z

Decommissioning the scraper.

Greg Grossmeier: Added the Software Project form

2009-07-30T21:51:33Z

Added the Software Project form

Nathan Yergler: New page: The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages. == What it Does == The metadata scraper is used by the license de...

2008-08-13T15:56:04Z

New page: The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages. == What it Does == The metadata scraper is used by the license de...

New page

The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages.

== What it Does ==

The metadata scraper is used by the license deeds to extract embedded metadata from pages. It currently scans the page for [[RDFa]] using [http://pypi.python.org/pypi/rdfadict rdfadict] and [http://www.w3.org/2007/08/pyRdfa/ pyRdfa].

== When it Happens ==

When a creator or copyright holder [http://creativecommons.org/license selects a license], they have the opportunity to provide additional metadata about there work. This includes information such as creator and medium, as well as how the creator would like to be attributed. If this information is provided, it is encoded in the HTML generated using [[RDFa]].

In order to provide more context relevant information for visitors, the license deeds load a script which looks at the referring page for metadata. If found, the metadata is used to update the deed and display additional attribution or CC+ details.

== Developer Information ==

Metadata Scraper is implemented in Python using the [http://cherrypy.org CherryPy] web framework. The source code is available in the <code>metadata_scraper</code> module in [[Source_Repository_Information#Subversion|subversion]]. You can [http://code.creativecommons.org/svnroot/metadata_scraper/ checkout] the source or [http://code.creativecommons.org/viewsvn/metadata_scraper/ browse it].

== Contact ==

Contact [[User:Nathan Yergler|Nathan Yergler]] with questions about the scraper.

[[Category:Developer]]

@@ Line 1: / Line 1: @@
 {{Software Project
 |Description=The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages.
-|Bug tracker=http://code.creativecommons.org/issues/
+|Bug tracker=https://github.com/cc-archive/metadata_scraper/issues/
-|Code repository=http://code.creativecommons.org/viewsvn/metadata_scraper/
+|Code repository=https://github.com/cc-archive/metadata_scraper
-|Mailing list=http://lists.ibiblio.org/mailman/listinfo/cc-devel
+|Mailing list=https://creativecommons.email/mailman/listinfo/cc-devel
 }}
-The metadata scraper is used by the license deeds to extract embedded metadata from pages.  It currently scans the page for [[RDFa]] using [http://pypi.python.org/pypi/rdfadict rdfadict] and [http://www.w3.org/2007/08/pyRdfa/ pyRdfa].
+== Decommissioned ==
-== When it Happens ==
+The Creative Commons Metadata Scraper was decommissioned on Monday 8th January 2018. Due to the changing nature of the web, in particular the widespread adoption of HTTPS, the Scraper had seen greatly reduced use. For potential future directions in metadata presentation, see Creative Commons' [https://github.com/creativecommons/open-ledger Open Ledger] project.
-When a creator or copyright holder [http://creativecommons.org/license selects a license], they have the opportunity to provide additional metadata about there work.  This includes information such as creator and medium, as well as how the creator would like to be attributed.  If this information is provided, it is encoded in the HTML generated using [[RDFa]].
+== What it Did ==
-In order to provide more context relevant information for visitors, the license deeds load a script which looks at the referring page for metadata.  If found, the metadata is used to update the deed and display additional attribution or CC+ details.
+The metadata scraper was used by the license deeds to extract embedded metadata from pages.  It scanned the page for [[RDFa]] using [http://pypi.python.org/pypi/rdfadict rdfadict] and [http://www.w3.org/2007/08/pyRdfa/ pyRdfa].
 == Developer Information ==
-Metadata Scraper is implemented in Python using the [http://cherrypy.org CherryPy] web framework.  The source code is available in the <code>metadata_scraper</code> module in [[Source_Repository_Information#Subversion|subversion]].  You can [http://code.creativecommons.org/svnroot/metadata_scraper/ checkout] the source or [http://code.creativecommons.org/viewsvn/metadata_scraper/ browse it].
+Metadata Scraper was implemented in Python using the [http://cherrypy.org CherryPy] web framework.  The source code is available on GitHub at https://github.com/cc-archive/metadata_scraper .
 == Contact ==
-Contact [[User:Nathan Yergler|Nathan Yergler]] with questions about the scraper.
+Contact [[User:CCID-rob|Rob Myers]] with questions about the scraper.
 [[Category:Developer]]

@@ Line 1: / Line 1: @@
-The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages.
+{{Software Project
+|Description=The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages.
 == What it Does ==