OER Discovery 2009 Comments

James Dalziel - 11 June 2009

• Many repository projects fail to take off because they assume educators will share their own work (which is rare), and for those who do, they are expected to add lots of metadata, which no one really wants to do. So you get one of a few possible outcomes – little sharing, garbage metadata, or lots of cost to fix the problems (librarians to fix up metadata, etc)

• Quality assurance is a regular problem – typical educators like the idea of a trusted repository/network of repositories that meets some sort of educational quality assurance level; however, this is almost impossible to ensure with community generated content, so again, you either get very low rates of submission, long delays in publication (due to quality assurance) and/or high costs to ensure quality. It would be good to hear from MERLOT on their experiences of this point – my sense is they’ve done as well as could be hoped, but in the end they only have modest usage levels, and the delays in publication are long (but I could be wrong on this!)

• Relevance ranking is the killer problem across disparate repositories of learning materials – there seems to me to be no real solution to this problem at the moment, as we don’t have methods for dealing with relative popularity/relevance across different repositories – in a sense, the only reason Goolge works is because it treats the whole web as one repository, and then uses links as a level playing-field measure between all sites for relevance.

• Educational search is complex because you may be searching for different kinds of things – an image, a website of text, a Flash animation, a chunk of courseware, a SCORM learning object, a LAMS Learning Design, etc. We’ll need to think about how to present search results for different kinds of things (hopefully some intelligence in the search can help identity different types of things automatically).

• Despite all my work on copyright, CC, etc, I’m still not certain of the boundaries of mashup/compilation in relation to license clash issues – ie, not modifying inside a resource, but just wanting to put a bunch of resources together in some structure (a series of links, etc), and where the boundaries of mashup start to impact on licenses (eg, at what point of “closeness” of mashup does the Share Alike requirement impact on my other materials, not just the one that has this as its existing license requirement.)

• With some repositories, they only provide metadata for search about an object, whereas others let the search get “inside” the object – this issue can affect relevance ranking that relies on looking inside an object.

• Some search approaches are based on a single local search of a harvest copy of everything that the search engine can get a copy of; whereas other approaches use a real time distributed search that doesn’t have a local harvest/cache, but instead queries multiple resources in real time (and this has all the problems of relative ranking, different response times, etc).

• Increasingly, the data people want during search to help them make decisions are web2/social data, like ratings, downloads, etc – so can this sort of data be aggregated to help inform relevance ranking for search.

Overall, when I think of a typical K-12 teacher searching for educational content, my sense is that they are after 3 core bits of information together:

1. Something like Google, ie, free text search with relevance ranking
2. Who else is using this? (various info about how many other teachers are using it, feedback on its use, who is using it, etc)
3. Mapping to their Curriculum framework, so they know how this item fits with their local Educational Standards/Competencies/Objectives/etc

For higher ed, I think 3 can be dropped, except for a few domains with strong existing curriculum frameworks.

Ahrash Bissell - 2 July 2009

• One of the themes we wish to explore is this issue of building solutions that actually work in the universe we live in. In other words, rather than 'wishing' that people would behave differently, how can we utilize solutions that function just fine given the way people actually behave? We believe that RDFa mark-up and other semantic solutions can get us there, but it will be interesting to hear everyone's thoughts on this.

Pieter Kleymeer - 8 July 2009

A few thoughts/questions that have come to my mind:

• An area of interest for a number of groups has been how to reduce the cost of creating structured metadata around objects. Granted, as the search engine giants begin to make use of the structured metadata available, more content providers will begin sharing that metadata and the efficiency for creating that metadata will improve. However, OER content producers/publishers need help today marking up content that will, in turn, make their resources more visible. How can we address this issue? Does this mesh with Ahrash's goal of building practical solutions, or is this still a dream?

• Building on James's comments, it would be interesting to learn more about the OER seekers and determine what set of metadata is critical to serve their search and use goals. What observations/studies already exist on information seeking behavior in the educational resources realm? Can we compile data from existing resource providers to get a clearer picture of what users want when searching and how they find it (or don't find it)?

• The ccLearn white paper did not discuss federated search as a possible solution to some of these problems. How might an enhanced search/feed aggregator (e.g., DiscoverEd) and a federated search system work in tandem using structured metadata at the object level to deliver a richer set of resources? Also, can we expect curators to self-select and submit feeds of content to not only DiscoverEd, but OERCommons, OCWFinder, ZaidLearn, etc.? Should they simply exist and be harvested by others over time? Can we designate OER sites with a certification of some sort (http://opened.creativecommons.org/Definition)? Is this, again, too idealistic to explore as a short-term solution?