This post was written for an internal audience at Cash App. I've lightly edited it for a public audience, but it is mostly still written in that voice.
A while back, a colleague of mine wrote an internal post on experimentation that I really loved. I wish I could link to it here, but such life: it's private. And since I can only share the critical parts here and none of the good parts, I sadly cannot in good conscience credit them. (If you read this - you know who you are! Make it public!)
I worked previously at a company with an extremely strong experimentation culture, though (Instagram/Facebook), and I have one huge, huge caveat about experimentation that needs to be actively remembered by anyone working with systems like this. It is about a way of thinking that can pop up without even realizing it. It popped up completely inadvertently in my colleague's post in this statement:
Why not try a few things? Let the users decide.
Experiments allow us to do more than just make ship/no ship decisions, and in that sense this statement is totally right. But the rhetorical flourish it uses - “Let the users decide” - is wrong, in a serious way: the outcome of an experiment is not a customer’s decision.
This is not an academic point, either. If you used Facebook from 2010-2020 or so (you probably did if you’re reading this on the day I’m publishing it), and saw how the platform evolved over time, you experienced the consequences of this worldview. Over time, the platform mutated from an exciting new place where people revitalized connections with old friends, to a place many people (myself included) made the decision to excise from their lives to avoid a constant drumbeat of trolling and ragebait.
Facebook employees throughout that time sincerely believed that they were building in service of their users. At the same time, the entirety of many feature ship/no ship decisions was was, "Does this experiment move the needle on a key platform metric?" Questions like, "Is this a cohesive customer experience?" were viewed as somewhere between secondary and irrelevant: if the metric didn’t move in the right direction, it was believed that this was an indication that the customer didn’t actually want that option.
This is only my own experience, of course. Facebook is/was a big company, and I worked on a tiny part of it (Instagram video uploads). But how that experience influenced the way I thought about my work has really stuck with me.
At the end of the day, “Experimental outcomes are customer sentiment” isn’t a philosophy we buy into because it actually serves the interests of our customers. It is a philosophy we buy into because it lets us believe we are serving the interests of our customers, even as we are not. And in the long run, it’s a damaging philosophy that eventually depletes the trust our customers place in us.
Or, here’s another way of saying the same thing from Matt Levine, in June 7th's Money Stuff column:
I think a lot of the concern that people have about rogue artificial intelligence destroying the world is based on the observed behavior of social media platforms. Nobody at Meta was like “we need to build a recommendation algorithm to help people who want child pornography find child pornography.” Meta was just like “we need to build a recommendation algorithm to maximize engagement.” And the recommendation algorithm got to work and said “hey we can increase engagement by 0.01% by serving more child sexual abuse material to pedophiles,” so that’s what it did. It is a paperclip maximizer in the real world.
We're in the business of helping our customers do their own work more effectively. Experiments are a powerful tool for critical analysis of that job! But they aren't an oracle for understanding what customers want. They must always be subservient to deeper thinking about what we're building towards.