Metadata Scraper

From Creative Commons
Revision as of 15:56, 13 August 2008 by Nathan Yergler (talk | contribs) (New page: The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages. == What it Does == The metadata scraper is used by the license de...)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages.

What it Does

The metadata scraper is used by the license deeds to extract embedded metadata from pages. It currently scans the page for RDFa using rdfadict and pyRdfa.

When it Happens

When a creator or copyright holder selects a license, they have the opportunity to provide additional metadata about there work. This includes information such as creator and medium, as well as how the creator would like to be attributed. If this information is provided, it is encoded in the HTML generated using RDFa.

In order to provide more context relevant information for visitors, the license deeds load a script which looks at the referring page for metadata. If found, the metadata is used to update the deed and display additional attribution or CC+ details.

Developer Information

Metadata Scraper is implemented in Python using the CherryPy web framework. The source code is available in the metadata_scraper module in subversion. You can checkout the source or browse it.

Contact

Contact Nathan Yergler with questions about the scraper.