The Difference Between AI Personalisation and AI Surveillance in Education
When an AI learning platform tells you it "personalises the learning experience," that phrase is doing a lot of work. It can mean very different things depending on who benefits from the data being collected and how it's being used.
In some cases, personalisation genuinely serves the learner: the system uses what it knows about a student's progress to adapt what they see next, so the learning experience fits where they actually are. In other cases, the data collection primarily serves the platform: improving the model, informing product decisions, or building profiles that have value beyond the student's immediate learning.
Both can be called personalisation. They're not the same thing.
What personalisation that serves the learner looks like
Effective personalisation in education is about adapting the learning experience to where a student actually is, not where the system assumes they should be.
This means: adjusting difficulty based on demonstrated understanding, not just completion. Surfacing the concepts a student is shaky on rather than moving them forward because they ticked a box. Giving an educator a clear picture of where their cohort is struggling, so they can make informed decisions about what to revisit.
The defining characteristic of this kind of personalisation is that the data flows toward the learner and their educator. The student benefits directly. The teacher uses the information to teach better.
Noodle Factory's Walter is designed around this principle: progress data is surfaced to educators through a full dashboard so they can see where each student is, where they're stuck, and what kind of support has helped. The data is in service of the teaching relationship, not extracted from it. That transparency, educators having visibility into what the AI knows about their students and how it's using that, is what genuine learner-centred personalisation looks like in practice.
What data collection that serves the platform looks like
The alternative is subtler and worth understanding.
Platforms that primarily benefit from student data tend to share a few characteristics. The data collection is broad — tracking not just learning outcomes but behavioural signals like time-on-task, click patterns, and engagement metrics that go well beyond what's needed to adapt learning content. The terms of service make it difficult to understand exactly what data is retained, how long, and who can access it. And the personalisation, on inspection, turns out to be less about adapting to the individual learner and more about optimising for engagement — keeping users in the platform rather than moving them efficiently toward learning outcomes.
This distinction matters because engagement and learning are not the same thing. A student can spend a long time in a platform without learning much. A platform optimised for engagement metrics may be producing exactly that.
Questions to ask before adopting an AI learning tool
The following checklist won't answer every question, but it will surface the most important ones.
What data is collected? Ask for a specific list, not a vague description of "usage data." This should include what behavioural data is tracked beyond assessment performance.
Who can access the data? Is it limited to the student and their educators? Can the vendor access it for product development? Is it shared with third parties?
How long is data retained? After a student leaves the institution, what happens to their data? Is there a clear deletion policy?
Does the personalisation serve the learner's goals or the platform's engagement metrics? Ask the vendor to walk you through specifically how personalisation works: what signals drive what adaptations. If the answer is vague, push harder.
What visibility do educators have? In a well-designed system, the educator can see what the AI knows about their students and how it's being used. If the system is opaque to educators, that's worth noting.
What are the terms for the institution's data? In some platform agreements, institutions effectively grant the vendor rights to use student interaction data to train their models. This is a significant commitment that should be made consciously, not by default.
The right frame
The question isn't whether AI in education should collect data — it will, and that data is what makes personalisation work. The question is whether the design of the system keeps students and educators as the primary beneficiaries of that data, or treats them as the source of a resource that primarily benefits someone else.
Tools that are transparent about what they collect, give educators meaningful visibility into how it's used, and design their personalisation around genuine learning outcomes rather than engagement optimisation are the ones worth adopting.
Those that can't answer the checklist questions clearly probably shouldn't be in your classroom.
Noodle Factory builds AI tutoring tools for universities, polytechnics, and K-12 schools across Southeast Asia. Learn more at noodlefactory.ai.
