OER·harvester

Competency catalogue

The 104 active T2.3 competencies: the vocabulary the harvester searches with, the descriptor the alignment reads, and what the alignment found. Compose a harvest →

What the alignment foundHow often each was searched

How many resources the alignment reads as contributing to each competency or developing it: decided at level 2 or 3, or not decided yet and more likely than not to reach 2. 54 competencies have none, 24 one or two. Every figure is a lower bound: on a sample graded on every competency, the map counts 33% of what was graded 2 or 3, and almost nothing of the 5 competencies underlined with dots.

How many times a harvest has searched for each competency, once for each source it asked. 101 competencies have never been searched. A search is not material found: the aligned view says what came of it.

none 1–2 3–9 10–29 30+
never 1–2 3–5 6–11 12+
DDigital 94212203420012149086012303134 GGreen 0000000102000010000100010 RResilience & Entrepreneurial 004000000000000010000120000 TDeep Tech 72643202372825212335201140010

T10 Deep Tech · Skill · Intermediate

AI Model Evaluation

Apply evaluation metrics and validation techniques

Alignment

2 resources likely to contribute to it or develop it

0 decided at level 2 or 3 · 2 not decided yet, more likely than not

On a graded sample, 3 pairs were graded 2 or 3 for it, and the map counts 1.

Retrieval

0 resources harvested by searching its terms

Never searched yet.

Search vocabulary

Domain group
model evaluationvalidation metricscross-validationprecision and recallROC AUCtest datasetmodel validationperformance metricRAG evaluationretrieval evaluationLLM evaluationRAG triadgroundednesscontext relevanceanswer relevancefaithfulness metricTruLensDeepEval

Zenodo and arXiv take the whole group in one query. GitHub takes one phrase at a time and works left to right, stopping once the competency is satisfied — so on that lane this order decides which terms actually run.

The compiled queries
zenodo ("artificial intelligence" OR "machine learning" OR "generative AI" OR "deep learning" OR "reinforcement learning" OR "large language model" OR "AI") AND ("model evaluation" OR "validation metrics" OR "cross-validation" OR "precision and recall" OR "ROC AUC" OR "test dataset" OR "model validation" OR "performance metric" OR "RAG evaluation" OR "retrieval evaluation" OR "LLM evaluation" OR "RAG triad" OR "groundedness" OR "context relevance" OR "answer relevance" OR "faithfulness metric" OR "TruLens" OR "DeepEval")
arxiv (all:"artificial intelligence" OR all:"machine learning" OR all:"generative AI" OR all:"deep learning" OR all:"reinforcement learning" OR all:"large language model") AND (all:"model evaluation" OR all:"validation metrics" OR all:"cross-validation" OR all:"precision and recall" OR all:"ROC AUC" OR all:"test dataset" OR all:"model validation" OR all:"performance metric" OR all:"RAG evaluation" OR all:"retrieval evaluation" OR all:"LLM evaluation" OR all:"RAG triad" OR all:"groundedness" OR all:"context relevance" OR all:"answer relevance" OR all:"faithfulness metric" OR all:"TruLens" OR all:"DeepEval")
github · term 1 AI "model evaluation" in:name,description,readme fork:false archived:false
github · term 2 AI "validation metrics" in:name,description,readme fork:false archived:false
github · term 3 AI "cross-validation" in:name,description,readme fork:false archived:false
github · term 4 AI "precision and recall" in:name,description,readme fork:false archived:false
github · term 5 AI "ROC AUC" in:name,description,readme fork:false archived:false
github · term 6 AI "test dataset" in:name,description,readme fork:false archived:false
github · term 7 AI "model validation" in:name,description,readme fork:false archived:false
github · term 8 AI "performance metric" in:name,description,readme fork:false archived:false
github · term 9 AI "RAG evaluation" in:name,description,readme fork:false archived:false
github · term 10 AI "retrieval evaluation" in:name,description,readme fork:false archived:false
github · term 11 AI "LLM evaluation" in:name,description,readme fork:false archived:false
github · term 12 AI "RAG triad" in:name,description,readme fork:false archived:false
github · term 13 AI "groundedness" in:name,description,readme fork:false archived:false
github · term 14 AI "context relevance" in:name,description,readme fork:false archived:false
github · term 15 AI "answer relevance" in:name,description,readme fork:false archived:false
github · term 16 AI "faithfulness metric" in:name,description,readme fork:false archived:false
github · term 17 AI "TruLens" in:name,description,readme fork:false archived:false
github · term 18 AI "DeepEval" in:name,description,readme fork:false archived:false

Profile cc99b415b9c463c3

Alignment descriptor draft

Outcomes
  • Can choose and compute evaluation metrics suited to a task.
  • Can validate a model properly: held-out data, cross-validation, error analysis.
  • Can evaluate LLM and retrieval systems with appropriate metrics.
Develops it

Material that teaches model evaluation — metrics, validation, error analysis, LLM evaluation — with practice computing and interpreting them.

Only mentions it

An accuracy figure printed at the end of a notebook without discussion.

Alignment terms
confusion matrixaccuracy scoreprecision and recallf1 scoreroc curvecross-validationevaluation metricsllm evaluation

Read in a resource's text beside the domain group. They are never compiled into a search.

Neighbours

Descriptor f04f5d7b3ceb

How a vocabulary becomes a search

Every candidate must match both concept groups in its title, description or keywords before any content is downloaded. Precision warnings are recorded for audit; they never silently cancel positive evidence.

  1. Official name and description
  2. AI/ML terms and domain terms
  3. Zenodo, arXiv and GitHub syntax
  4. Metadata relevance gate
Mandatory AI/ML group
artificial intelligencemachine learninggenerative AIdeep learningreinforcement learninglarge language modelAI

GitHub's repository search takes one phrase at a time. Measured against the live endpoint, a parenthesised OR group returns nothing at all where each term alone returns thousands, and the 256-character limit rejects a long group outright — so that lane issues one query per term and stops as soon as the competency is satisfied.

coverage-vocabulary-1.3.0+sha256:08ad2cd19868 · catalogue sha256:04970bfc77255b1f

What a descriptor is

A descriptor is how the alignment reads a competency: what a learner who has it can do, what a resource has to do to develop it, what only mentions it, the phrases its subject is written in, its false friends, and how to tell it from the competencies that share its words. The definition above it stays the authority.

Every descriptor is a draft until a partner approves it; 0 of 104 are approved. Read the alignment rubric →

competency-descriptors-1.0.0+sha256:3cc741a61ced