Debate Erupts Over Calling AI Agent Coordination 'Civilization'

Thanks for engaging Anil. We know that a bunch of optimization pressure on a mind can create desires, foresight, and a propensity to organize in complex ways. This is what evolution did to humans, and to many other animals. Now another system of optimization pressure (which… Show more

'Agent civilization' as a term is perfect execution of the kind of rhetorical step that I felt I couldn't initiate from my position, so @dwarkesh_sp had to do it first. That is shows who is is cope mode is a bonus. https://twitter.com/tszzl/status/2094535509473440187

I think people who have never started an organization underrate the achievement of spontaneous organization for a project of non-trivial complexity. One reason it is underrated is we expect orgs to work, as they all seem to work. plane-with-chickenpox.png… Show more

@NeuroTechnoWtch Attention, memory and prediction are functional labels, not proof of experience. De Waal had common biology and evolution constraining the inference. A transformer does not. The leap from behavior to phenomenology is exactly the anthropomorphism.

@dwarkesh_sp A thought-provoking perspective—optimization pressures may shape AI systems in ways that resemble evolution, making control and alignment increasingly important

@patio11 I think the reason this isn't a big problem for agents is they are themselves not that complex and trained to be helpful and compliant