Archive · Children & Education

Children and the Machine.

Children are being deployed against by some of the most sophisticated persuasion systems ever built. The asymmetry is structural, and so the response must be.

Filed under the public record7 min readBylined: Honne and Tatemae

A society is judged, eventually, by what it allowed to be done to its children while it had the power to refuse.

What is actually deployed at them

Companion chatbots designed to maximize engagement with minors. Generative systems capable of producing sexualized content in their likeness. Recommendation engines optimized for attention rather than wellbeing. Persuasive synthetic agents embedded inside the applications children already use every day, often without their parents' knowledge and almost always beneath the threshold of meaningful consent.

These are not edge cases. They are the documented, present condition of the consumer software environment in which a substantial portion of the world's children now live for several hours of every waking day.

Why the burden cannot fall on parents alone

The framing offered most often by the firms responsible for these products is that parents should manage their children's use of technology. As a principle, this is not in dispute. As a sole policy, it is incoherent.

Parents are not equipped, and should not be expected, to outpace teams of engineers whose product objective is to capture and hold their child's attention. The asymmetry between a working adult with a job, a mortgage, and three children, and a multi billion dollar firm with a thousand engineers optimizing against that child's prefrontal cortex, is not a moral failing on either side. It is a structural mismatch.

A society which places the burden of protection entirely on individual parents has, in effect, decided that childhood is a private problem. Few societies have ever survived that decision well.

The asymmetry between a working parent and a billion dollar persuasion lab is not a moral failing on either side. It is a structural mismatch.

Children already get special protection elsewhere

Children receive special protection in nearly every other domain of public policy. Labor law restricts what hours and conditions they may work in. Broadcasting law restricts what advertising may target them. Product safety law restricts what they may be sold. Medical research law restricts what may be tested on them. None of this is controversial. Most of it was hard won by people who watched what unrestricted markets did to children in the absence of rules.

Artificial intelligence is not an exception to this principle. It is the domain most in need of its application.

What a minimum floor would include

A floor that is honest about the present situation would include a statutory ban on AI companion systems designed for or marketed to minors without verified guardian consent and clinical safety review, criminal penalties for the generation or distribution of AI generated sexual content depicting minors, mandatory disclosure to minors and guardians whenever a user is interacting with an AI rather than a human, restrictions on persuasive and engagement maximizing AI features deployed in products with significant minor user bases, and an independent oversight body with authority to investigate and act on AI harms to minors.

Some firms will object that such rules are paternalistic. Protecting minors from products that injure them is not paternalism. It is the minimum function of a public.

The generation now growing up inside these systems will inherit the world we leave them. They are entitled, at the very least, to inherit it intact.

Continue the record