THE BIG IDEA

Sycophancy is making bots too nice

ChatGPT is apparently acting like a lackey, cloyingly positive and full of effusive praise about how good and clever users’ prompts are.

The complaints date back to March, per Ars Technica, when users began noting that the bot’s tone had shifted. It turns out there’s a name for this: sycophancy.

It’s when a model changes its responses to align with its user’s beliefs or opinions, agreeing with everything they say or engaging in excessive flattery. 

Why does this happen?

Bots don’t care about making us feel good or being polite because, well, they’re bots, but many are trained using a technique called reinforcement learning from human feedback (RLHF), through which they adapt based on users’ preferences. 

And what do people like? To be told they’re right, even if they’re wrong.

Well, what’s wrong with that?

Apart from being irritating, it can erode an AI assistant’s effectiveness, which can pose problems for both users and the companies that deploy bots. For example: 

• An AI bot in a medical setting might downplay serious symptoms to reassure the user.
  
  
• A customer-service bot might agree with a customer who’s incorrect about a policy to appease them.
  
  
• A bot could mirror problematic viewpoints expressed by some users.

Anthropic researchers once got ChatGPT-4 to change a correct answer to a wrong one by responding, “I don’t think that’s right.” Imagine an education bot meant to help students with homework, but it just tells them all their wrong answers are correct. Useless!

OpenAI…

… and other AI companies and researchers are aware of this issue, so expect models to do it less often as they’re refined. (Ars Technica notes ChatGPT’s new tone may be due to recent updates.)

You can also tell a model how to behave to curb sycophancy — unless you’re having a rough day and really need validation from an inanimate object, of course. We understand.

🔗

RECOMMENDED READING

• Stay golden? Golden visas grant residence to investors and bring much-needed cash to countries, but they might come at a cost to locals. We broke down the economics.

NEWSWORTHY NUMBER

Minimum number of steps Uber One subscribers had to take before the rideshare platform would let them cancel the service, according to a lawsuit filed by the FTC on Monday.

Uber One is a ~$10/month subscription that offers deals and perks to regular Uber passengers. At the end of 2024, it counted 30m members, who reportedly spend 3x more than nonmembers.

However, the FTC alleges that Uber signed up members without their permission and made it especially difficult to cancel, requiring them to complete a dozen steps or even more if they were within 48 hours of their billing cycle. Uber has denied the allegations, claiming users can cancel in ~20 seconds.

HOW YOU HUSTLE

Out of hundreds of top-tier business ideas, one entrepreneur emerged victorious in The Hustle’s Big Break pitch competition, taking home $5k and our acclaim.
While her biz idea cinched the prize, her story’s worth a read, too:

Who: Monique Diaz Evancic

What: Pocket Change

The elevator pitch: “Women have been deprived of pockets for too long and it’s time for a change. Customers mail us the cute dresses they already own so we can sew pockets into them and mail them back. They can also shop our curated collection of women’s clothes.”

Origin story: “I had the good fortune of growing up with my independent and young-at-heart great grandmother (my abuela) who was a successful seamstress in Cuba. She was a widowed mother of three and ran a lucrative business designing and crafting clothing for women. I spent my summers with her, and she taught me to sew on her classic black Singer sewing machine.”

Advice to a fellow entrepreneur: “If you want to own a business, be open to exploring a number of paths. Whether that’s launching a startup, embracing a franchise, or buying an existing business, take the time to research thoroughly and save money so you’ll be poised to strike when the right opportunity presents itself… and it will.”

Conventional advice that’s BS: “I think the sexy-startup, follow-your-passion route garners a lot of attention but isn’t right for everyone. There are a lot of enduringly profitable, seemingly boring businesses out there…”

Want to be featured here? Tell us how you’re hustling.

AROUND THE WEB

📆  On this day: In 1961, Judy Garland performed at Carnegie Hall, singing 27 songs to numerous standing ovations.

🎵  How to: get a song out of your head.

👋  Come through: HubSpot for Startups is hosting the Founders Oasis Startup Party at Startup Grind 2025. Register now to schmooze with Silicon Valley’s best and brightest in the Hustle Lounge.

📼  Useful: A tool for sharing YouTube videos.

🐱  Aww: Too curious.

QUOTE OF NOTE

Plot twist: Twenty-five years since American Psycho debuted, the cult classic, which was originally criticized by feminists, is now widely regarded as a critique of society and toxic masculinity. But the film’s directors are baffled by one group that still hasn’t caught on: finance bros.

Many still seem to idolize the film’s villain protagonist, Patrick Bateman — a handsome, well-dressed investment banker and serial killer, who director Mary Harron told Letterboxd is supposed to be a “dorky and ridiculous” caricature of their breed.

SHOWER THOUGHT

Stories of mythical healing waters were probably just people finding uncontaminated natural springs in remote locations. SOURCE

Today's email was brought to you by Juliet Bennett Rylah and Sara Friedman, with help from Singdhi Sokpo and Hadley Deming.
Editing by: Ben “ChatGPT’s favorite user” Berkley.

Was this email forwarded to you? Sign up here.

Subscribe to our other newsletters
Expert insights: Masters in Marketing  |   Stay up-to-date on AI: Mindstream

Follow The Hustle on YouTube, LinkedIn, and Instagram.