Development of AI Literacy Instruments to Map Elementary School Students' Abilities

Rexp Criteria

Based on the data in Table 3, the reliability values of the AI literacy instrument for each indicator range from moderate to strong. The Recognizing AI indicator had a coefficient of 0.486 (reliable), Understanding AI was 0.308 (unreliable), Function/Use Apply AI was 0.770 (reliable), Evaluate and Create AI was 0.631 (reliable), and AI Ethics was 0.528 (reliable). In general, all indicators show that the instrument has sufficient to strong consistency in measuring students' AI literacy. This condition proves that the instrument can provide stable measurement results and does not create differences in meaning between items. Thus, the AI literacy instrument can be declared reliable and feasible for use.

TABLE 4. AI Literacy Results

usage and comprehension that must be addressed through education. By introducing a structured instrument, educators can systematically measure and improve students’ AI-related competencies. Furthermore, AI literacy is becoming a fundamental skill comparable to traditional literacies such as reading and numeracy. Without adequate literacy, students may become passive users rather than critical thinkers. Thus, the development of this instrument responds to an urgent educational need. It ensures that students are equipped not only to use AI but also to understand and evaluate it critically. (Gökçe,

H., Nacaroğlu, O., 2026) Elementary school is identified as a crucial stage for introducing AI literacy due to its role in shaping foundational cognitive abilities. At this stage, students begin to develop critical thinking, logical reasoning, and problem-solving skills. These competencies are essential for understanding the basic principles of artificial intelligence. Early exposure to AI concepts can foster curiosity and adaptability in learning. It also helps students build a mindset that is open to technological innovation. Integrating AI literacy at the elementary level ensures that students grow alongside technological advancements. This proactive approach prevents the development of misconceptions about AI in later stages of education. Moreover, young learners are generally more receptive to new concepts when introduced through appropriate pedagogical strategies. Therefore, elementary education serves as an ideal entry point for AI literacy development. (Sabatini, J., Graesser, A. C., Hollander, J., & O’Reilly, T., 2023) The alignment of this study with prior research further strengthens its theoretical foundation. The findings of C., Carter, and Smith (2021) indicate that elementary school students possess the ability to recognize patterns and understand simple technological concepts. These skills are closely related to the basic principles underlying artificial intelligence. Pattern recognition, for instance, is a fundamental aspect of machine learning processes (Siregar, T. (2025). By leveraging these existing cognitive abilities, educators can introduce AI concepts in a more accessible manner. This alignment suggests that AI literacy is not beyond the reach of young learners. Instead, it can be integrated تدريجياً into the curriculum בהתאם to students’ developmental stages. The research also highlights the أهمية of scaffolding learning experiences to match students’ cognitive levels. Consequently, this study builds upon established findings to justify the introduction of AI literacy in elementary education. (Gökçe, H., Nacaroğlu, O., 2026) The indicator of Recognizing AI serves as the foundational level in the instrument. It focuses on students’ ability to identify AI technologies in their environment. This includes recognizing applications, tools, and outputs generated by AI systems. Such awareness is essential in helping students differentiate between human-generated and machine-generated content. In the digital age, where AI-generated media is increasingly prevalent, this skill becomes highly significant. Recognizing AI also involves familiarity with commonly used platforms and applications. This practical knowledge enhances students’ engagement with technology. Furthermore, it lays the groundwork for deeper understanding in subsequent stages. Without this foundational awareness, higher-level competencies cannot be effectively developed. (Gökçe, H., Nacaroğlu, O., 2026) Understanding AI represents the second level of the instrument and emphasizes conceptual knowledge. At this stage, students are expected to grasp how AI systems function in a simplified manner. This includes understanding that AI operates based on data and algorithms. Students also learn that AI systems can provide automated responses but are not always accurate. This awareness is crucial in preventing over-reliance on AI outputs. Additionally, understanding AI involves recognizing its capabilities and limitations. For example, students learn that AI can generate content but lacks human emotions and consciousness. This distinction helps in developing realistic expectations לגבי AI

technologies. Therefore, this indicator bridges the gap between awareness and critical comprehension. (Ng, D. T. K., Su, J., Leung, J. K. L., & Chu, S. K. W., 2024)

The Function/Use Apply AI indicator focuses on practical engagement with AI tools. It assesses students’ ability to use AI applications independently and effectively. This includes searching for information, generating ideas, and completing tasks باستخدام AI. Practical usage enhances students’ confidence in interacting with technology. It also demonstrates how AI can support learning processes. However, this indicator also highlights the importance of guided usage. Students must learn to use AI as a tool for assistance rather than dependency. The ability to apply AI in meaningful ways reflects digital competence. Consequently, this dimension plays a key role in translating knowledge into practice. (Sabatini, J., Graesser, A. C., Hollander, J., & O’Reilly, T., 2023)

The Evaluate and Create AI indicator introduces higher-order thinking skills. Students are encouraged to critically assess AI-generated outputs and compare them with other sources. This evaluation process fosters analytical thinking and reduces the risk of misinformation. Additionally, students are guided to create new content باستخدام AI tools. This includes combining their own ideas with AI-generated outputs. Such activities promote creativity and innovation. The ability to refine and improve AI outputs through prompts demonstrates advanced interaction skills. This indicator reflects the integration of cognitive and creative competencies. It prepares students for more complex uses of AI in the future. (Ng, D. T.

K., Su, J., Leung, J. K. L., & Chu, S. K. W., 2024) AI Ethics is the final indicator and emphasizes responsible and ethical use of technology. In the context of increasing AI adoption, ethical considerations become highly important. Students must understand the أهمية of protecting personal data and maintaining privacy. They are also encouraged to use AI for positive and constructive purposes. Ethical literacy includes avoiding plagiarism and academic dishonesty. Students learn to respect intellectual property and produce original work. בנוסף, polite communication with AI systems reflects digital etiquette. This dimension ensures that technological competence is balanced with moral responsibility. Therefore, AI ethics is an integral component of holistic AI literacy. (Sabatini, J., Graesser, A. C., Hollander, J., & O’Reilly, T., 2023) The use of a structured instrument allows for systematic assessment of students’ competencies across all five indicators. Each indicator is operationalized into measurable items that capture specific aspects of AI literacy. This ensures that the assessment process is both reliable and valid. The instrument can be used by educators to identify strengths and weaknesses in students’ abilities. It also provides a basis for designing targeted instructional interventions. The clarity of the instrument enhances its usability in classroom settings. בנוסף, it supports consistency in data collection and analysis. Therefore, it serves as an effective evaluation tool in educational research. (Ng, D. T. K., Su, J., Leung, J. K. L., & Chu, S. K. W., 2024) The integration of AI literacy into early education also supports the development of lifelong learning skills. As technology continues to evolve, students must be prepared to adapt to new tools and systems. AI literacy equips them with the ability to learn independently and critically evaluate information. These skills are essential in navigating the complexities of the digital world. Early exposure ensures that students develop a strong foundation that can be built upon in higher education. It also fosters a proactive attitude تجاه التكنولوجيا. This approach aligns with the goals of modern education systems. Thus, AI literacy contributes to the holistic development of students. (Su, J., Ng, D. T. K., & Chu, S. K. W., 2023) In addition, the development of this instrument has significant implications for curriculum design. Educational institutions can integrate AI literacy into existing subjects or develop specialized programs.

The instrument provides a clear framework for defining learning objectives and outcomes. It also يساعد teachers in aligning instructional strategies with students’ needs. Curriculum integration ensures that AI literacy is not treated as an isolated topic. Instead, it becomes part of a broader educational ecosystem. This approach enhances the relevance of education in the digital age. Consequently, it supports the transformation of traditional teaching practices. (Ng, D. T. K., Su, J., Leung, J. K. L., & Chu, S. K. W.,

2024. The role of teachers is also crucial in the successful implementation of AI literacy. Educators must be equipped with adequate knowledge and skills to guide students effectively. Professional development programs may be necessary to enhance teachers’ AI competencies. The instrument can serve as a reference for designing such programs. Teachers can use the results to adjust their teaching methods. בנוסף, they can provide personalized support to students based on their needs. This dynamic interaction between teaching and assessment improves learning outcomes. Therefore, teacher readiness is a key factor in the success of AI literacy initiatives. (Su, J., Ng, D. T. K., & Chu, S. K. W., 2023) Furthermore, this study contributes to the growing body of research on AI in education. By developing a structured instrument, it provides a foundation for future empirical studies. Researchers can use this instrument to explore various aspects of AI literacy. For example, they can examine its relationship with academic performance or digital skills. The instrument also enables comparative studies across different educational contexts. This contributes to the generalization of findings. בנוסף, it supports evidence-based policy development. Thus, the study has both theoretical and practical significance. (Su, J., Ng, D. T. K., & Chu, S. K. W., 2023) The development of an AI literacy instrument reflects a forward-looking approach to education. It acknowledges the transformative impact of AI on society and the ضرورة of preparing students accordingly. By focusing on elementary school students, the study emphasizes early intervention. This ensures that students develop the necessary competencies from a young age. The comprehensive nature of the instrument addresses cognitive, practical, and ethical dimensions. It aligns with global trends in digital education. Ultimately, this initiative aims to create a generation of learners who are not only technologically proficient but also critical, creative, and responsible. (Behnamnia, N., et al., 2024) Furthermore, in the design phase, the researcher formulated the theoretical construct based on these five indicators. This aligns with instrument development procedures, where variables are first broken down into indicators before instrument items are created (Sugiharni and Setiasih 2018). Each indicator was developed into statement items that are relevant, representative, and appropriate to students' experiences, ensuring the items comprehensively reflect the aspects to be measured (Syafitri et al. 2023). This design phase also refers to the principle that instruments must be built on a strong theoretical foundation to accurately measure abilities and be free from bias. Additionally, item construction referred to content validity guidelines emphasizing the importance of construct representation and item consistency with the indicator. Subsequently, the instrument items were reviewed by experts to ensure the suitability of content, construction, presentation, and language so that the instrument would not cause ambiguity for students (Arini 2020). Next, in the development phase, the expert validation results showed that all indicators obtained Vexp values above 0.80, meeting content validity criteria. In fact, the Understanding AI and Function/Use Apply AI indicators achieved perfect Vexp values (1.00), indicating that these two indicators have very high coherence with their theoretical constructs. This aligns with the view of (Rivera, Santos, and Gomez 2019) that a good instrument must have coherence between theory, indicators, and items to produce stable and unbiased measurements. At this stage, item correlations also showed significant

results, confirming that the items functioned optimally for the empirical testing phase. (Behnamnia, N., et al., 2024)

Furthermore, the empirical validity results showed that most indicators met validity criteria. The Function/Use Apply AI (0.770), Evaluate and Create AI (0.631), and AI Ethics (0.528) indicators were declared valid because they had positive and significant correlations. However, the Understanding AI indicator (0.308) was declared less reliable, so several items needed revision. This aligns with the findings of (Li, Y., Wang and Chen 2021), which state that abstract concepts regarding how AI works are more difficult for students to understand compared to aspects of use or ethics. Nevertheless, the overall reliability of the instrument ranged from 0.486 to 0.770, which is considered adequate for educational instruments in the initial development stage (Rahman and Supardi 2022).

In the context of the Merdeka Curriculum, the analysis of validity and reliability is also closely related to determining the Criteria for Achieving Learning Objectives (KKTP). Data for this analysis can be obtained through various methods such as surveys, interviews, or documentation studies. The use of questionnaires is a common choice because it can collect data in large quantities effectively and efficiently. However, the validity and reliability of the instrument must be ensured so that the analysis results can truly serve as a basis for determining the level of achievement of learning objectives (Siregar and Rhamayanti 2025).

After the instrument was declared feasible, it was implemented with elementary school students. The results showed that the average AI literacy scores ranged from 13.90 to 15.52. The lowest score was for the Recognizing AI indicator (M = 13.90), indicating that students need reinforcement in understanding basic AI concepts. Conversely, the highest score was for the AI Ethics indicator (M = 15.52), meaning that ethical aspects are easier to understand because they relate to daily behavior in using digital devices. This finding aligns with research by (Nurhayati, N., & Sari 2023), which explains that digital ethics awareness develops faster than technical AI skills. Furthermore, the relatively low standard deviation (SD = 2.169--2.861) indicates that the distribution of student abilities is relatively homogeneous, so the instrument can measure consistently. (Behnamnia, N., et al., 2024)

In the context of the Merdeka Curriculum, the analysis of validity and reliability plays a crucial role in determining the Criteria for Achieving Learning Objectives (KKTP). These criteria serve as benchmarks to evaluate whether students have successfully achieved the expected competencies in a particular learning domain. Therefore, the accuracy of the instrument used to measure students’ abilities becomes highly important (Siregar, T. (2025). Data for such analysis can be obtained through various methods, including surveys, interviews, and documentation studies, each offering unique advantages in capturing educational phenomena. Among these methods, questionnaires are widely preferred due to their practicality in collecting large-scale data within a relatively short time. However, the effectiveness of questionnaires depends heavily on their validity and reliability. Without these two qualities, the collected data may not accurately reflect students’ actual competencies. Consequently, ensuring instrument quality is a fundamental step before conducting further analysis. As emphasized by Siregar and Rhamayanti (2025), valid and reliable instruments are essential to produce trustworthy findings that can inform educational decision-making. (Behnamnia, N., et al., 2024)

After the instrument was declared feasible through rigorous validation and reliability testing, it was implemented among elementary school students to measure their AI literacy levels. The results revealed that the average scores of students’ AI literacy ranged from 13.90 to 15.52 across the five indicators. This range indicates a generally moderate to high level of AI literacy among participants. However, differences across indicators highlight specific areas that require further attention. The lowest mean score was found in the Recognizing AI indicator (M = 13.90), suggesting that students still face

challenges in identifying and understanding basic AI-related concepts and applications. This may be due to limited exposure to explicit instruction about AI in early education. On the other hand, the highest score was recorded in the AI Ethics indicator (M = 15.52), indicating that students demonstrate a stronger understanding of ethical aspects in using AI technologies. These findings provide valuable insights into the distribution of competencies among students. (Ng, D. T. K., Luo, W., Chan, H. M. Y., & Chu, S. K. W., 2022)

The higher performance in AI Ethics can be explained by the fact that ethical behaviors are closely related to students’ ყოველდღიური experiences when interacting with digital devices. Concepts such as not sharing personal information, using polite language, and avoiding plagiarism are often reinforced in both school and home environments. As a result, students tend to internalize these values more easily compared to abstract technological concepts. This finding is consistent with the study conducted by Nurhayati and Sari (2023), which highlights that awareness of digital ethics tends to develop earlier than technical AI skills. In contrast, recognizing AI requires more specific knowledge about how technology functions, which may not yet be fully integrated into elementary-level curricula. Therefore, targeted instructional strategies are needed to enhance students’ understanding in this area. This imbalance between ethical and technical competencies underscores the importance of a comprehensive AI literacy framework. It also suggests that curriculum development should address both aspects in a balanced manner. (Behnamnia, N., et al., 2024)

Furthermore, the relatively low standard deviation values, ranging from 2.169 to 2.861, indicate that the distribution of students’ AI literacy scores is relatively homogeneous. This suggests that there is no extreme disparity in abilities among students within the sample. Such consistency implies that the instrument functions effectively in capturing students’ competencies in a stable manner (Siregar, T. (2025). A homogeneous distribution also reflects that the learning experiences related to AI literacy are relatively similar across participants. This strengthens the reliability of the instrument, as it demonstrates consistent measurement across different individuals. In addition, the stability of the data supports the validity of conclusions drawn from the analysis. These findings confirm that the developed instrument is not only feasible but also reliable for assessing AI literacy in elementary education. Therefore, it can be utilized as a reference tool for evaluating and improving learning outcomes related to AI literacy. (Ng, D. T. K., Luo, W., Chan, H. M. Y., & Chu, S. K. W., 2022)

Finally, in the evaluation phase, the researcher assessed the overall development process from expert validation to field implementation. Overall, this instrument meets the criteria for expert validity, empirical validity, and reliability, making it feasible for use as a tool for measuring AI literacy in elementary school students. However, several items in the Understanding AI indicator need improvement to optimize the instrument further. Additionally, the results of this study can serve as a basis for developing more comprehensive advanced AI literacy instruments. Thus, these findings can also serve as a reference for developing AI literacy curricula in elementary schools, aligning with the AI literacy development guidelines compiled by (Touretzky et al. 2022).