Implications for teachers: using Khan Academy in the classroom
By Bogdan Yamkovenko and Kodi Weatherholtz
Our recent quasi-experimental research highlights the significant impact Khan Academy can have on student growth, offering valuable insights for teachers deciding how to integrate it into their classrooms.
Study highlights:
Incremental gains: Encourage students to focus on mastering more skills each year. Even small increases in skill proficiency can lead to noticeable improvements. For instance, mastering just one more skill than they did the previous year results in an average increase of 0.5 percentage points in the target gain percentage.
Closing learning gaps: Implementing regular practice on Khan Academy can help students narrow achievement gaps, but understand that significant improvement requires consistent effort.
Universal benefits: The study suggests that all students, irrespective of their background or external support, can benefit from increased learning engagement on Khan Academy.
Falling behind in math
We know that recent test score data shows that many students score below grade-level expectations. A recent NWEA report for the 2023-2024 school year indicates that the gains in math are still 2%-14% lower than they were pre-pandemic (Lewis and Kuhfeld, 2024). In fact, the data from our recent MAP Accelerator study suggest that students typically only grow about 86% of their target growth in one year. Results from our latest research offer insights into what is needed to help students catch up.
Results from a three-year quasi-experimental study
Our recent quasi-experimental study tracks the same students over three years and looks at how students’ gains change year to year as they increase or decrease their usage of Khan Academy. In this case, we looked specifically at how gains on the MAP Growth test change year to year when the students learn more or fewer skills in a relevant Khan Academy course than they did a year before. On our platform, learning a skill is measured by getting to proficient or mastery level for the skill.
Often studies in edtech only look at one year of achievement data because it is difficult to track students for multiple years. In such studies, it’s also only possible to compare students who use the tool to other students who do not. The main limitation is that high-usage students are usually very different from the low-usage students on many attributes that we can’t account for in the study. But in a study like the one we describe here, each student is tracked over several years, and we compare each student’s gains to their gains a year or more prior. By comparing students to themselves, we can rule out factors like student motivation or even teacher characteristics and focus on the effect of increasing or decreasing Khan Academy usage (see this blog for why this works). This is a powerful methodology that helps address one of the most common critiques of edtech studies—that edtech tools only work for motivated students. In this study, we include all students regardless of how much they used Khan Academy. Studies like these are considered to be just one step below Randomized Controlled Trials (RCT), which are often viewed as the gold standard in research methods.
NWEA publishes fall-to-spring growth expectations for students who take MAP Growth. For example, an average student in the 5th grade is expected to gain ~10 points between their fall and spring tests (see NWEA norms). But, typically, students don’t gain what is expected. This same 5th-grade student who gained eight points between fall and spring is at about 80% of their expected or target gain.
We would expect that if this same student became proficient in more skills on Khan Academy next year, they would move closer to their target growth or maybe even exceed it. That is exactly what we see in our study results. When a student becomes proficient in one more skills than they learned last year, on average, they increase their percent of target by about 0.5 percentage points. In other words, they gain more than they gained last year, and they move closer to their expected gain. In some cases, depending on how many more skills they learn, they can exceed their target gain. This means that practicing even a few more skills on Khan Academy can help students close their gaps and catch up with their peers. This finding is true for students of different demographic groups. For the technical details, see our recent paper that was accepted and presented at ECTEL 2024.
What does this mean for a student in a classroom?
Consider a 5th-grade student who only worked on one unit of Khan Academy’s 5th-grade math course and became proficient on 10 skills in that unit in addition to whatever else their teacher had them work on in class. The full course has 130 skills, so this student’s progress is very small. Suppose this student ends the school year with gains on MAP growth that are 80% of their 5th-grade target. Now suppose in the next year, they become proficient in 40 skills in 6th-grade math. That’s an additional 30 skills more than what they learned last year. Based on our research finding that each new skill learned to proficiency or mastery results in a 0.5 percentage point increase in learning, we would expect this student to move 15 percentage points closer to their target. So, this year they should be gaining ~95% of the target. If a student made even more progress in the course, they could actually meet or exceed their target and gain more than expected.
At Khan Academy, we want to accelerate student learning. We empower students to address their individual learning needs through focused practice on the skills they need to learn. If students are behind, we can help them catch up. If they are at grade level, they can go even further. Our study shows that regardless of who you are as a student—where you live, who your teacher is, how much help you get from adults—spending even a little more time or becoming proficient on even a few more skills on Khan Academy can lead to higher gains.
The post What Is the Impact of Learning a Skill on Khan Academy? appeared first on Khan Academy Blog.
Implications for teachers: using Khan Academy in the classroom By Bogdan Yamkovenko and Kodi Weatherholtz Our recent quasi-experimental research highlights the significant impact Khan Academy can have on student growth, offering valuable insights for teachers deciding how to integrate it into their classrooms. Study highlights: Falling behind in math We know that recent test score
The post What Is the Impact of Learning a Skill on Khan Academy? appeared first on Khan Academy Blog. Educators, Math, News, Research Khan Academy Blog
