The Evidence Gap

TL;DR: 

• Pupil-facing edtech is unregulated and untested. It makes its way into the classroom on the back of marketing claims, rather than independent evaluations.

• Pupil engagement metrics are easy to capture and critical for edtech investors, but they are proxies for enthusiasm and convenience, not meaningful learning gains.

• This lack of evidence puts the responsibility of judging a product’s safety and effectiveness unfairly on teachers who are unable to keep up with a fast-moving and crowded marketplace. 

Read more

What we assume

It’s natural for school leaders and teachers to assume that anything designed for use in schools has been properly tested before it reaches pupils. In many areas, that assumption is justified. Playground equipment must comply with rigorous safety and manufacturing standards before it can be installed on a school site. Food served in school canteens is subject to mandatory School Food Standards, designed to protect children’s health and wellbeing. Even something as ordinary as a classroom chair reflects years of ergonomic and safety considerations - there’s a reason they don’t have child-sized holes in the back of them! 

These safeguards matter because schools are environments built on trust. Teachers should not have to spend their time questioning whether the climbing frame is structurally sound, whether lunchtime meals are nutritionally appropriate or whether classroom furniture is safe for daily use. Robust standards and testing processes allow teachers to focus on what they are there to do: teach, build relationships and care for pupils.

That all changed with pupil-facing edtech. 

Unlike swings and slides, pupil-facing edtech is not subject to the same quality assurance. There is no regulatory framework for developing edtech tools and there is no requirement to test their safety and effectiveness before selling directly into schools.  In the UK, only 7 per cent of edtech companies have conducted a randomised controlled trial (RCT).

In practice, this means products reach pupils based on marketing claims, best-case examples and user reviews or local procurement decisions rather than independent evaluation of their impact on learning, safety or data protection in real classroom contexts. Although schools do consider safeguarding and GDPR requirements, there is no equivalent of a national ‘approval standard’ that every digital learning tool must pass before being used by children. As a result, responsibility for judging suitability and effectiveness frequently sits with teachers and school leaders, who must make decisions in a fast-moving and crowded marketplace.

Buyer beware: engagement is not the same as understanding 

Edtech companies often focus on pupil engagement metrics in their marketing materials - it reassures investors of the product’s ‘stickiness’ and it’s easy to measure at scale. Digital platforms can automatically track clicks, sessions, task completion and time-on-task in real time. Learning itself is much harder to measure reliably. 

A pupil can spend thirty minutes actively clicking through tasks while learning very little, particularly if the work is repetitive, poorly sequenced or heavily scaffolded. Equally, quieter forms of learning, such as thoughtful reading, discussion, reflection or grappling with challenging ideas, are much harder for software to capture in a dashboard.

Deep understanding, critical thinking, creativity and long-term retention are difficult to quantify, expensive to research and often only visible over months or years. Engagement therefore becomes a convenient shorthand for success, even though it is an imperfect measure of educational impact.

Ideally, we would have robust and statutory standards for edtech products sold into schools. This would create safer environments, support consistency across schools and give teachers the reassurance that the products pupils use every day have been designed with children’s best interests in mind. In the absence of this, here are some initial questions you can ask of the edtech tools you use to reassess whether you want to continue using them or roll back. 

Here are some questions to help you move beyond the marketing claims: 

1. Is the evidence independent or mainly based on vendor case studies? 

2. What evidence is there that this product improves learning, rather than simply increasing engagement or reducing my workload? Are the reported metrics indicators of learning or just measures of usage?

3. Were improvements sustained over time or only seen immediately after use?