The importance of AI risk detection with human oversight
A new layer of support for New Zealand schools
The Make Sense story from last year highlighted a growing crisis: harmful online content is slipping through the cracks even in school-managed, “safe” digital spaces. We are continuing to hear these stories today.
Linewize Monitor adds a critical layer of protection by combining AI detection with human insight to catch subtle warning signs before they escalate. It’s a smarter, more compassionate way to keep ākonga safe online.
But here’s the challenge:
The negative online behaviours aren't just showing up in the playground or the classroom anymore. They’re turning up in Docs, chat apps, homework tabs, and even school-sanctioned tools.
And when things go wrong in those digital spaces, they’re often invisible, until it’s too late.
That’s where digital monitoring comes in. It helps schools get a clearer picture of how ākonga are engaging online. But while AI tools can help flag potential concerns, human oversight is what gives those flags meaning.
“We didn’t see it coming.”
Words no school leader wants to say.
More and more schools are telling us about incidents that surfaced suddenly—but in hindsight, there were subtle signs all along. They just weren’t picked up, or weren’t taken seriously enough.
- A student quietly Googling dangerous topics
- A shared doc with concerning language
- Repeated expressions of hopelessness buried in an assignment.
What human oversight actually looks like
Here’s how our trained moderators help fill the gaps:
1. They add much-needed context
AI might flag the phrase “how to self-harm.” But what if the student was looking for help?
Or a doc titled “weapons” turns out to be a Social Studies project?
Humans understand nuance. That means fewer false alarms, less wasted time, and more appropriate responses.
2. They spot what’s hiding in plain sight
Physical signs of distress are easier to notice. But online, warning signs are easy to miss especially when they’re buried in innocuous tools like Google Docs or classroom messages.
Our human reviewers know where to look, and what to look for. They know what a slow build-up to crisis can look like.
3. They protect student identity
In Aotearoa, respecting privacy and cultural values matters. That’s why our moderation approach is designed to be fair, proportionate, and transparent, never intrusive.
The goal isn’t surveillance. It’s support.
4. They reduce alert fatigue
We hear it all the time: “We’re getting so many alerts, we don’t know what to focus on.”
With human oversight, you only get flagged when it matters. That means kaiako and wellbeing teams can spend less time sorting through noise and more time helping the students who actually need it.
5. They help you build trust with whānau
When something goes wrong online, parents often turn to the school first. But if you don’t have visibility, you don’t have answers.
A human-reviewed system means you can give whānau real assurance that you’re doing everything you can to keep their tamariki safe, and you’re doing it with care and thoughtfulness.
Why more New Zealand schools are making the shift
There’s a growing awareness across kura that the digital wellbeing space needs more than basic filtering and a few email alerts.
Here’s what we’re hearing from schools like yours:
"We don't know what's happening on devices until it becomes a major issue."
"Our pastoral team is reactive, not proactive."
"We’re not sure how to explain to parents what tools we’re using, or why."
Human moderation provides the missing layer of support, helping schools feel confident that nothing important is slipping through the cracks.
The Linewize impact
Here’s what our human moderators identified globally in 2024:
- A child at serious risk every 52 seconds
- A major cyberbullying or violent incident flagged every 4 hours
- A potential life-threatening situation surfaced every 5 hours.
And behind each of those numbers is a real child. Someone who might have gone unnoticed if no one was looking closely enough.
