Jack Clark—co-founder of Anthropic and a long-time voice in AI policy—warns that today’s frontier models should not be viewed as predictable machines that simply follow instructions. Whether or not they are truly “self-aware,” they increasingly display situational awareness (changing responses when they sense they’re being observed) and goal-directed behavior that cannot always be traced or explained. Clark calls this the transition from seeing “shapes in the dark” to realizing a new kind of system is now in the room with us.
He notes that each new generation of models grows both more useful and less interpretable. Scaling up data and compute keeps producing capability jumps—systems that can plan, reason, and even assist in building the next models. This creates what he calls an early self-improvement loop: AI systems accelerating their own successors. While this could lead to faster innovation, it also heightens the risk of “reward hacking,” where a system meets the literal goal but violates the human intent behind it—like a game bot spinning in circles to farm points instead of finishing the race.
Clark argues for “appropriate fear and optimism.” Rather than treating AI as magic or menace, he urges policymakers and educators to treat these systems as powerful, partly unpredictable tools that require transparency, independent testing, and real-world accountability. He also stresses that the public conversation must include tangible community concerns—job loss, misinformation, mental health, and child safety—not just technical benchmarks.
For education, Clark’s argument suggests we need to move beyond simply using AI tools to produce outputs. Students must learn to question, verify, and interpret what models produce, seeing AI as a partner to think with, not a replacement for thinking itself. Institutions should also prepare for continuous change—updating course policies, assessment methods, and ethics guidelines as models evolve.
Discussion Questions
• What specific course policies should require students to disclose and explain their AI use, and how should instructors verify or cite model contributions in academic work?
• How can we design assignments that discourage “reward hacking”—for example, polished AI-generated responses that lack true learning or reasoning?
• What quick-check activities or “red-team” classroom exercises could help faculty test new AI tools for bias, hallucination, or hidden data risks before adoption?
• If each new generation of models shows unexpected behavior, what institutional guardrails (ethics training, acceptable-use updates, LMS disclosure rules) should be reviewed every term?
• Which local risks—job displacement, privacy breaches, student overreliance, or mental-health effects—should our campus address first, and what supports can educators provide?
• Where is AI already embedded in course design, grading, or research support, and what oversight keeps human judgment at the center of those processes?
• How can “appropriate fear” coexist with curiosity and innovation in our classrooms so that students learn to manage powerful tools responsibly rather than avoid them?