Difference between revisions of "Metadata Scraper"
(Added the Software Project form) |
(Decommissioning the scraper.) |
||
| Line 1: | Line 1: | ||
{{Software Project | {{Software Project | ||
|Description=The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages. | |Description=The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages. | ||
| − | |Bug tracker= | + | |Bug tracker=https://github.com/cc-archive/metadata_scraper/issues/ |
| − | |Code repository= | + | |Code repository=https://github.com/cc-archive/metadata_scraper |
| − | |Mailing list= | + | |Mailing list=https://creativecommons.email/mailman/listinfo/cc-devel |
}} | }} | ||
| − | |||
| − | + | == Decommissioned == | |
| − | + | The Creative Commons Metadata Scraper was decommissioned on Monday 8th January 2018. Due to the changing nature of the web, in particular the widespread adoption of HTTPS, the Scraper had seen greatly reduced use. For potential future directions in metadata presentation, see Creative Commons' [https://github.com/creativecommons/open-ledger Open Ledger] project. | |
| − | + | == What it Did == | |
| − | In order to provide more context relevant information for visitors, the license deeds | + | The metadata scraper was used by the license deeds to extract embedded metadata from pages. It scanned the page for [[RDFa]] using [http://pypi.python.org/pypi/rdfadict rdfadict] and [http://www.w3.org/2007/08/pyRdfa/ pyRdfa]. |
| + | |||
| + | == When it Happened == | ||
| + | |||
| + | When a creator or copyright holder [http://creativecommons.org/license selects a license], they have the opportunity to provide additional metadata about their work. This includes information such as creator and medium, as well as how the creator would like to be attributed. If this information is provided, it is encoded in the HTML generated using [[RDFa]]. | ||
| + | |||
| + | In order to provide more context relevant information for visitors, the license deeds loaded a script which looks at the referring page for metadata. If found, the metadata was used to update the deed and display additional attribution or CC+ details. | ||
== Developer Information == | == Developer Information == | ||
| − | Metadata Scraper | + | Metadata Scraper was implemented in Python using the [http://cherrypy.org CherryPy] web framework. The source code is available on GitHub at https://github.com/cc-archive/metadata_scraper . |
== Contact == | == Contact == | ||
| − | Contact [[User: | + | Contact [[User:CCID-rob|Rob Myers]] with questions about the scraper. |
[[Category:Developer]] | [[Category:Developer]] | ||
Latest revision as of 18:44, 8 January 2018
| Description of Software | The Creative Commons Metadata Scraper is a simple crawler used by the license engine to detect metadata stored in pages. |
|---|---|
| Bug Tracker | Link to Bug Tracker |
| Code Repository | Link to Code repository |
| Mailing List | Link to Mailing list |
Decommissioned
The Creative Commons Metadata Scraper was decommissioned on Monday 8th January 2018. Due to the changing nature of the web, in particular the widespread adoption of HTTPS, the Scraper had seen greatly reduced use. For potential future directions in metadata presentation, see Creative Commons' Open Ledger project.
What it Did
The metadata scraper was used by the license deeds to extract embedded metadata from pages. It scanned the page for RDFa using rdfadict and pyRdfa.
When it Happened
When a creator or copyright holder selects a license, they have the opportunity to provide additional metadata about their work. This includes information such as creator and medium, as well as how the creator would like to be attributed. If this information is provided, it is encoded in the HTML generated using RDFa.
In order to provide more context relevant information for visitors, the license deeds loaded a script which looks at the referring page for metadata. If found, the metadata was used to update the deed and display additional attribution or CC+ details.
Developer Information
Metadata Scraper was implemented in Python using the CherryPy web framework. The source code is available on GitHub at https://github.com/cc-archive/metadata_scraper .
Contact
Contact Rob Myers with questions about the scraper.
