How Meta intends to use its dominant position to dominate an AI future “at the intersection of technology and culture”, and how Sarah Wynn-Williams’ memoir of Facebook should warn us against that

By Tony Curzon Price

“Careless People” is the title of Sarah Wynn-Williams’ remarkable memoir of working as Facebook’s first international diplomat. She takes her well-chosen title from the Great Gatsby (Picture HT https://kwize.com/quote/9961#google_vignette)

“Small companies are all alike; every dominant company exercises their dominance in their own particular way” … might have written Tolstoy, if he’d been interested in abuse of power by companies and not just in the human soul. And so it is, in spadefuls, for the MAGMA (Meta, Amazon, Google, Microsoft and Apple).

Take the Meta headlines these past few weeks, for example. First, a refusal to sign the EU’s AI code of conduct; then, Zuckerberg’s very public $100m+ packages for AI talent; and then this, his smoothly polished auto-queued Facebook reel in which he lays out, somewhat codedly, what he thinks Meta’s role will be in the SuperIntelligent future:

an even more meaningful impact in our lives is going to come from everyone having a personal superintelligence that helps you achieve your goals, create what you want to see in the world, be a better friend, and grow to become the person that you aspire to be […] At Meta, we believe in putting the power of superintelligence in people’s hands to direct it towards what they value in their own lives. Some of this will be about improving productivity, but a lot of it may be more personal in nature […] The intersection of technology and culture is where Meta focuses, and this will only become more important over time. If trends continue, I expect that people will spend less time on productivity software and more time creating and connecting. I think that personal devices like glasses that can see what we see, hear what we hear, and interact with us throughout the day are going to become our main computing devices […] I believe deeply in building personal superintelligence for everyone, and at Meta, we have the resources to build the massive infrastructure required

What do all of these tell us about how he’s going to deploy his dominant position for … well … more domination?

The EU AI code of practice is voluntary. It politely and toothlessly asks AI firms to do things like refrain from using pirated content, perform risk-assessments of their internal activities, maintain technical documentation that will make checking compliance with the (potentially toothy) AI Act a simple matte of handing over information. A code of practice like this is part of a regulatory dance, and to be invited to dance and refuse is quite a strong signal that you’re sitting sulkily on the edges of the dancefloor wishing you were at another party with more attractive partners. That puts Meta and X in one corner, and the surface-level cooperators, Google, Microsoft, Amazon and Apple – in another.

The superb – and very well written – tell-all memoir, “Careless People”, by Sarah Wynn Williams, Facebook’s first employee hired to look after non-US government affairs – their very first diplomat, as it were – provides some fascinating insights into what that other dancefloor might be. The whole book is really worth reading. Facebook tried to suppress it, and it says a great deal about the degree to which she must have documented and evidenced one eye-popping claim after another that they failed. Thank goodness for people with the courage and skill of Wynn Williams.

Here is Wynn Williams recounting her experiences of the Meta board and the performance review her manager Joel Kaplan – Trump ally and successor to Nick Clegg – gave her after the meeting. (She goes on to describe some very troubling sexual harassment by Kaplan … what a delightful man to have in charge of government relations…):

We [the staff team presenting to the Meta board] lay out our case that things are bad and about to get much worse, especially outside America. Politicians and governments see a borderless internet as a threat. Facebook is seen as competing against homegrown industries like publishers, telecom operators, and local tech companies. And they’ll use everything from antitrust regulation to privacy regulation to crush Facebook.

Our goal is to frighten the board into dramatic action, but they don’t seem very frightened. Their response is “deals.” What do these different players want? Especially the decision makers, the regulators, and politicians. And what can Facebook “offer them”? What will head off regulation and change the global narrative on Facebook? […]

Then the board gets into a conversation about what other companies or industries have navigated similar challenges, where they have to change a narrative that says that they’re a danger to society, extracting large profits, pushing all the negative externalities onto society and not giving back. People suggest various analogues that don’t seem to fit very well and then Elliot finally says out loud the one I think everyone’s already thinking about (but not saying): tobacco.

That shuts down the conversation and they move on to the next agenda item.

[… Now the post-board performance review with Joel Kaplan]

Kaplan: “You’re in a meeting with Mark, Sheryl, and the entire Facebook board and there were a couple of times where I saw you, in fact the whole room saw you, rolling your eyes.”

“Do you happen to know when that was?” I ask as neutrally as possible, keeping my eyes straight on.

He pauses and thinks but can’t seem to place it. We sit in silence until I decide, perhaps injudiciously, to speak.

“Was it maybe when one of our board members was suggesting to our largely Jewish leadership team that Facebook needs to get much closer to the far-right political parties in Europe because that is where the power is shifting to?”

This was the proposal that we should use the election support and campaign tools we were offering to US presidential candidates (and that Donald Trump was using aggressively in his 2016 campaign)—and offer them to the Alternative für Deutschland, the far right in Germany, and to Marine Le Pen and the Front National, the far right in France. Getting closer to these political parties and helping them into government would be the most effective way to stop governments from regulating Facebook.

I say this realizing that my poker face is slipping.

“Well sure,” Joel says. “But whatever it was, you can’t be rolling your eyes and making faces at the board.”

That’s true and I apologize for it. By way of explanation, I tell Joel that I understand that some board members think fascists could solve our regulatory issues and we need to cozy up to them right away. And that I’d expected more pushback from the rest of the board.

Other than that, Joel tells me, great review!

I do not roll my eyes in response.

Throughout the book, it is clear that Zuckerberg can’t bear to lose. Wynn Williams recounts a hilarious scene on the private jet, returning from a heads-of-state forum with him, where he invites her to a board game and she has the temerity not to let him win – something no one else in the entourage ever does. He immediately accuses her of cheating. And yet Meta is losing in the AI race at the moment. After initially impressing with its open source model strategy, the research team behind Llama was so fed up with Meta culture that they left and joined forces with Mistral, the Franco/European contender. So Zuckerberg is using all means to win again, much as he does when playing board games. Wynn Williams lets the readers guess which of the board members it might be that advocated a move of helping the AfD and Marine Le Pen (my unimaginative guess would go with Peter Thiel, whom she tells us was in the room in his role as philosopher king of the Valley) as a way of countering EU regulation … but I imagine that refusing the EU’s code of conduct is absolutely consistent with such a strategy. It is part of the business interest Meta has in undermining the legitimacy of the EU’s rule-making powers. It would have been pretty costless for Meta to sign the code of practice – after all, all it does is ask politely for firms to do things like follow the law and be responsible – but the cost is that it legitimises the notion that the EU has regulatory authority in this domain. Of course, the alignment with the AfD and Rassemblement National is, in this regard, very clear.

So Meta’s refusal to sign, I think, is a calculated decision to adopt a muscular posture, one that erodes the legitimacy of the EU, a body that has for one been the source of most forward-thinking platform regulation in recent years (for example in its Digital Market Act provisions to force interoperability), and on top of that represents a sufficiently large chunk of the world’s non-Chinese, non-American economy that platforms might actually prefer to comply with local regulations than to walk away from the market (or credibly to threaten to).

The huge pay packets are, of course, part of the AI catch-up game that Meta is now engaged in. Again, from reading Wynn-Williams, it is not all that surprising that Meta needs to offer quite so much cash to poach talent – the internal culture coming from the top seems absolutely poisonous. But for those sorts of life-changing sums of money, I guess a bit of poison is OK.

But now the big question: what are all these star coders going to be building? For that, let’s read carefully the scripted answer Zuckerberg gives on Reels. Here, to me, are the key points:

• “The intersection of technology and culture is where Meta focuses, and this will only become more important over time […] people will spend less time on productivity software and more time creating and connecting.”
• “everyone having a personal superintelligence that helps you achieve your goals”
• “personal devices like glasses that can see what we see, hear what we hear, and interact with us throughout the day are going to become our main computing devices.”

First, AI as a business productivity tool is not for Meta. Quite the opposite, as he says in the first quote, the world of “creating and connecting” – what Meta does today, perhaps call it scrolling for short – provides ample competition for the world of being productive and will only provide more. Then, to this important notion that Meta focuses on the “intersection of technology and culture”. It is interesting language. What does it mean? I think what he means is that Meta technology “intersects” and therefore has an impact on, changes the direction of, culture. Culture is that common space that brings us together and binds us; it has its roots in the “cult”, the religious practice of bringing people together in a shared understanding.

So how will the new technology, in Meta’s view, change the direction of culture? He answers that too: “everyone having a personal superintelligence that helps you achieve your goals”, and, more specifically, “personal devices like glasses that can see what we see, hear what we hear, and interact with us throughout the day are going to become our main computing devices.” So this is the marriage of the Zuckerberg Metaverse vision with the SuperIntelligence vision: we will wear Meta devices that put us into a giant connected field where each of us is guided and helped by our Meta-powered “a personal superintelligence”. Now, note the coda that says that this will “help you achieve your goals”. The text Zuckerberg read is 423 words. Words were precious in drafting this. Every one of them was carefully chosen. Why spent 1% of your word budget on saying that this tech will be help you achieve your goals?

Presumably because, these days, we rather mistrust Meta to produce technology whose prime design focus it is to help humans flourish. Wynn-Williams again is astonishing about this. She recounts from the inside the infamous new moment when a powerpoint presentation from the Facebook Sales team (the dominant force in the organisation, it seems) was leaked to the Australian press: those bringing in the advertising revenues were bragging to potential beauty product advertisers that they could identify the moment when a teenage girl deleted a selfie and was feeling bad about her appearance … the ideal moment to pop in an ad for some image-improving product into her attention space. Of course, this story makes it amply clear that Meta’s objective shareholder interest includes making teen girls feel miserable about their appearance. Let me quote again at some length from Wynn-Williams (you get the message that the book is great, I hope – there is lots more stuff like this in there):

No one in [the privacy] group, other than me and my Australian team, seems surprised that Facebook made an advertising deck like this. One person messages the group, “I have a very strong feeling that she [the Australian staffer who prepared the deck] is not the only researcher doing this work. So do we want to open a giant can of worms or not?” And they’re right. At first, we think the leaked document is one Facebook made to pitch a gum manufacturer to target teenagers during vulnerable emotional states. Then eventually the team realize, no, the one that got leaked was for a bank. There are obviously many decks like this.

The privacy staffer explains that teams do this type of customized work targeting insecurities for other advertisers, and there are presentations for other clients specifically targeting teens. We discuss the possibility that this news might lead to investigations by state attorneys general or the Federal Trade Commission, because it might become public that Facebook commercializes and exploits Facebook’s youngest users.

To me, this type of surveillance and monetization of young teens’ sense of worthlessness feels like a concrete step toward the dystopian future Facebook’s critics had long warned of.

A statement is quickly drafted and the response team debates whether Facebook can include the line, “We take this very seriously and are taking every effort to remedy the situation,” since in fact this is apparently just normal business practice. A comms staffer points out what should be obvious: that “we can’t say we’re taking efforts to remedy it if we’re not.”

This prompts other team members to confirm his take, revealing other examples they know of. Facebook targets young mothers, based on their emotional states, and targets racial and ethnic groups—for example, “Hispanic and African American Feeling Fantastic Over-index.” Facebook does work for a beauty product company tracking when thirteen-to-seventeen-year-old girls delete selfies, so it can serve a beauty ad to them at that moment.

We don’t know what happens to young teen girls when they’re targeted with beauty advertisements after deleting a selfie. Nothing good. There’s a reason why you erase something from existence. Why a teen girl feels that it can’t be shared. And surely Facebook shouldn’t then be using that moment to bombard them with extreme weight loss ads or beauty industry ads or whatever else they push on teens feeling vulnerable. The weird thing is that the rest of our Facebook coworkers seem unbothered about this.

My team and I are horrified; one of them messages me, “Also wondering about asking my apparently morally bankrupt colleagues if they are aware of any more. The Facebook advertising guy who is cited in the [Australian] article has three children—I talked him through his kid being bullied—what was he thinking?”

I’m still struggling to get a better picture of what we’re dealing with here. So I ask for an independent audit by a third party to understand everything that Facebook has done like this around the world, targeting vulnerable people, so I can try to stop it. Who has this information and how many advertisers has it been shared with? The team is not enthusiastic. Elliot nixes any audit and cautions against using the word “audit” at all, even as an ask like mine, saying that “lawyers have discouraged that description in similar contexts.” He doesn’t say why but I’m guessing he doesn’t want to create a paper trail, a report with damning details that could be leaked or subpoenaed. Years later I would learn that British teenager Molly Russell had saved Instagram posts including one from an account called Feeling Worthless before committing suicide. “Worthless” being one of the targeting fields. This only emerged due to a lawsuit that revealed internal documents acknowledging “palpable risk” of “similar incidents.”

So that is how today’s products are no doubt thought by the top brass at Meta to “help you achieve your goals”. But the big point is this: at the intersection of technology and culture, “your” goals are somewhat up for grabs. They can be shaped by the technology doing that intersection. Indeed, it seems that this might very well be the purpose of the intersection.

There is an opinion piece in yesterday’s New York Times by an 81 year-old psychologist, Dr Harvey Lieberman, who talks about his experience using chatGPT as an intimate thought-partner. He starts out sceptically – he calls out the fads of self-help, meditation, mindfulness, etc. But chatbots are different, he thinks. He is clearly someone who has thought a lot about introspection and done a great deal of it himself. He knows what to look for and how to analyse what a relationship is doing to your innards. He writes even-handedly yet is clearly surprised at how good chatGPT has been as a thinking tool for him. In some ways, he should be a poster-child for the potential for “a personal superintelligence that helps you”.

And yet this is what jumped out at me from his essay:

At the intersection of Culture and Technology, there is precisely no firm ground where a tool can simply help with pre-existing goals: this is precisely where the technology changes goals and changes the self. As Dr Lieberman, an expert introspecter, can see, this is a technology that changed how he thought. And in his expert hands, ChatGPT was a good thinking tool. But note how expert those hands had to be. As he says:

So here is the question I would like to leave you with: given the power of these tools at the intersection of culture and technology, do we want Meta, the organisation so powerfully described by Wynn-Williams to be the one “building personal superintelligence for everyone”? You won’t be surprised by my answer. I only hope that the man who can’t bear losing at board games is not allowed the latitude to exercise his dominance in this, so much more for humanity than simply a board game.

Of course, no post goes without closing reminding you that at FIDU, the First International Data Union, we are working to build tools that will put the power of this technology in the right place. And in fact, there is exciting news – we are basically ready to launch an alpha product – maybe it’s for you?