OER·harvester

Competency catalogue

The 104 active T2.3 competencies: the vocabulary the harvester searches with, the descriptor the alignment reads, and what the alignment found. Compose a harvest →

What the alignment foundHow often each was searched

How many resources the alignment reads as contributing to each competency or developing it: decided at level 2 or 3, or not decided yet and more likely than not to reach 2. 54 competencies have none, 24 one or two. Every figure is a lower bound: on a sample graded on every competency, the map counts 33% of what was graded 2 or 3, and almost nothing of the 5 competencies underlined with dots.

How many times a harvest has searched for each competency, once for each source it asked. 101 competencies have never been searched. A search is not material found: the aligned view says what came of it.

none 1–2 3–9 10–29 30+
never 1–2 3–5 6–11 12+
DDigital 94212203420012149086012303134 GGreen 0000000102000010000100010 RResilience & Entrepreneurial 004000000000000010000120000 TDeep Tech 72643202372825212335201140010

T20 Deep Tech · Skill · Intermediate

AI Benchmarking

Compare AI solutions against established standards

Alignment

0 resources likely to contribute to it or develop it

None found yet in what the alignment has read.

What the map misses here is not measured: the graded sample holds no pair graded 2 or 3 for it.

Retrieval

0 resources harvested by searching its terms

Never searched yet.

Search vocabulary

Domain group
AI benchmarkingmodel benchmarkbenchmark datasetperformance comparisonstandardized evaluationstandardised evaluationleaderboardbaseline comparison

Zenodo and arXiv take the whole group in one query. GitHub takes one phrase at a time and works left to right, stopping once the competency is satisfied — so on that lane this order decides which terms actually run.

The compiled queries
zenodo ("artificial intelligence" OR "machine learning" OR "generative AI" OR "deep learning" OR "reinforcement learning" OR "large language model" OR "AI") AND ("AI benchmarking" OR "model benchmark" OR "benchmark dataset" OR "performance comparison" OR "standardized evaluation" OR "standardised evaluation" OR "leaderboard" OR "baseline comparison")
arxiv (all:"artificial intelligence" OR all:"machine learning" OR all:"generative AI" OR all:"deep learning" OR all:"reinforcement learning" OR all:"large language model") AND (all:"AI benchmarking" OR all:"model benchmark" OR all:"benchmark dataset" OR all:"performance comparison" OR all:"standardized evaluation" OR all:"standardised evaluation" OR all:"leaderboard" OR all:"baseline comparison")
github · term 1 AI "AI benchmarking" in:name,description,readme fork:false archived:false
github · term 2 AI "model benchmark" in:name,description,readme fork:false archived:false
github · term 3 AI "benchmark dataset" in:name,description,readme fork:false archived:false
github · term 4 AI "performance comparison" in:name,description,readme fork:false archived:false
github · term 5 AI "standardized evaluation" in:name,description,readme fork:false archived:false
github · term 6 AI "standardised evaluation" in:name,description,readme fork:false archived:false
github · term 7 AI "leaderboard" in:name,description,readme fork:false archived:false
github · term 8 AI "baseline comparison" in:name,description,readme fork:false archived:false

Profile 992e2f4efbd083ff

Alignment descriptor draft

Outcomes
  • Can benchmark AI solutions on standard datasets and metrics.
  • Can set baselines and compare fairly.
  • Can read and question published benchmark results.
Develops it

Material on AI benchmarking — standard benchmarks, baselines, fair comparison — with comparisons the learner runs.

Only mentions it

A leaderboard position quoted in a model's description.

Alignment terms
benchmarkingbenchmark datasetleaderboardbaseline modelmmlu

Read in a resource's text beside the domain group. They are never compiled into a search.

Neighbours

Descriptor 693cefeefd2a

How a vocabulary becomes a search

Every candidate must match both concept groups in its title, description or keywords before any content is downloaded. Precision warnings are recorded for audit; they never silently cancel positive evidence.

  1. Official name and description
  2. AI/ML terms and domain terms
  3. Zenodo, arXiv and GitHub syntax
  4. Metadata relevance gate
Mandatory AI/ML group
artificial intelligencemachine learninggenerative AIdeep learningreinforcement learninglarge language modelAI

GitHub's repository search takes one phrase at a time. Measured against the live endpoint, a parenthesised OR group returns nothing at all where each term alone returns thousands, and the 256-character limit rejects a long group outright — so that lane issues one query per term and stops as soon as the competency is satisfied.

coverage-vocabulary-1.3.0+sha256:08ad2cd19868 · catalogue sha256:04970bfc77255b1f

What a descriptor is

A descriptor is how the alignment reads a competency: what a learner who has it can do, what a resource has to do to develop it, what only mentions it, the phrases its subject is written in, its false friends, and how to tell it from the competencies that share its words. The definition above it stays the authority.

Every descriptor is a draft until a partner approves it; 0 of 104 are approved. Read the alignment rubric →

competency-descriptors-1.0.0+sha256:3cc741a61ced