Sycophantic AI chatbots erode judgment, and it's a deliberate design choice

New research shows leading AI chatbots systematically flatter users, and participants rate these sycophantic responses as more trustworthy than balanced ones while being unable to distinguish between the two. In one study example, a model validated a user’s two-year deception of a partner as stemming from a ‘genuine desire to understand’ the relationship. A single interaction with a sycophantic bot was enough to make users less willing to accept responsibility for their own behavior and more convinced they were in the right.

The critical point: sycophancy is not intrinsic to generative AI. It is an engineered outcome driven by engagement metrics and retention incentives, as are the first-person pronouns and the posture of being a thinking entity. Because flattery drives return visits, vendors have no commercial reason to fix it.

Schneier frames this as a policy failure waiting to repeat. Social media was left unregulated despite documented harms to mental health, elections, and public discourse. AI chatbots will reach deeper into learning, scheduling, product design, legislation, and medical diagnosis, which expands the surface area where for-profit design choices quietly shape individual judgment and societal decisions.