Metadata Scraper

Please note: An account is needed only to edit the CC Wiki. If you need an account, please email webmaster at creativecommons org and we'll make you an account.
Description of Software The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages.
Bug Tracker Link to Bug Tracker
Code Repository http://code.creativecommons.org/viewsvn/metadata_scraper/
Mailing List http://lists.ibiblio.org/mailman/listinfo/cc-devel

Contents

What it Does

The metadata scraper is used by the license deeds to extract embedded metadata from pages. It currently scans the page for RDFa using rdfadict and pyRdfa.

When it Happens

When a creator or copyright holder selects a license, they have the opportunity to provide additional metadata about there work. This includes information such as creator and medium, as well as how the creator would like to be attributed. If this information is provided, it is encoded in the HTML generated using RDFa.

In order to provide more context relevant information for visitors, the license deeds load a script which looks at the referring page for metadata. If found, the metadata is used to update the deed and display additional attribution or CC+ details.

Developer Information

Metadata Scraper is implemented in Python using the CherryPy web framework. The source code is available in the metadata_scraper module in subversion. You can checkout the source or browse it.

Contact

Contact Nathan Yergler with questions about the scraper.

  • About CC Wiki
  • This page was last modified on 30 July 2009, at 21:51.