Tim Pritlove: Good morning, Linus.
Linus Neumann: Good morning, Tim.
Tim Pritlove: Linus, what watch?
Linus Neumann: Ten watch.
Tim Pritlove: Such much?
Meredith Whittaker: You'll get along beautifully in America.
Tim Pritlove: Logbuch Netzpolitik Nummer 503 vom 16.
Tim Pritlove: Oktober 2024 und ihr werdet es schon gemerkt haben.
Linus Neumann: Wir sprechen eine andere Sprache.
Tim Pritlove: Wir sprechen eine andere Sprache heute, weil wir haben heute einen Gast und
Tim Pritlove: das ist dann ein Gespräch, was wir heute auf Englisch führen wollen.
Linus Neumann: Bevor wir aber kurz unsere Episodennummer korrigieren, ist nämlich 504.
Tim Pritlove: Was habe ich gesagt?
Linus Neumann: 503.
Tim Pritlove: Oh.
Tim Pritlove: Ja, ausnahmsweise lebe ich mal nicht in der Zukunft.
Linus Neumann: Man merkt, wir sind aufgeregt.
Linus Neumann: Tim hat ja einen britischen Pass und muss jetzt mal seine Britischkenntnisse unter Beweis stellen.
Tim Pritlove: Was habe ich einen britischen Pass? Ich habe gar keinen britischen Pass, ich bin Deutscher.
Linus Neumann: Okay, aber du hattest ja nicht einen britischen Pass.
Tim Pritlove: Das ist aber abgelaufen.
Linus Neumann: Achso, der ist abgelaufen. Okay, dann das erklärt einige.
Tim Pritlove: Anyway, we say hello to Meredith. Meredith Whitaker. Hello, welcome to our podcast.
Meredith Whittaker: I'm so happy to be here. Thank you for having me.
Linus Neumann: Meredith, you are, in case somebody doesn't know you, the president of the Signal Foundation.
Meredith Whittaker: I am. I am.
Linus Neumann: And Signal is a messenger application?
Meredith Whittaker: It is the world's most widely used, truly private messaging application.
Meredith Whittaker: I always have to put the full sentence in there. A very special messaging application.
Linus Neumann: What makes it special?
Meredith Whittaker: Um we build against
Meredith Whittaker: the grain to put it in kind of flowery language we go out of our way to build
Meredith Whittaker: for privacy and to get as close to collecting no data at all as possible which
Meredith Whittaker: as you and i assume many in your audience know is a lot more work.
Linus Neumann: Than simply encrypting.
Meredith Whittaker: Then well encrypting is part of that but then you need to make sure the encryption
Meredith Whittaker: works and then encryption of message content is Das ist nicht möglich,
Meredith Whittaker: weil du also hast Meta-Data. Du hast auch libraries, du hast du in.
Meredith Whittaker: Du hast Core Services, du hast du mit. So, da ist viel ingenuity required zu
Meredith Whittaker: actually create a system das ist rejecting den normative assumptions der Tech-Industrie heute.
Meredith Whittaker: Wir alle wollen ein ton of Daten, und wir alle wollen monetize das Daten.
Meredith Whittaker: Wir wollen Sie ads mit dem Daten. Wir wollen das zu anderen Kunden.
Meredith Whittaker: Wir wollen das Daten-Broker. Wir wollen das AI-Modeln mit dem Daten.
Meredith Whittaker: Most of the infrastructure in the tech ecosystem now assumes that as a given.
Meredith Whittaker: And then we have to rewrite things too.
Linus Neumann: And you negate all these assumptions and try to build a truly private messaging
Linus Neumann: platform in a world that has a completely different business model.
Meredith Whittaker: Yeah. Well, I'm going to just flip that to be a little bit rhetorical and say
Meredith Whittaker: those assumptions negate the human right to private intimate communication.
Meredith Whittaker: And we are trying to rebuild a tech ecosystem that actually honors those.
Linus Neumann: That's an even higher goal.
Meredith Whittaker: Yeah.
Tim Pritlove: Before we dig more into the Signal project, which in itself I think is quite
Tim Pritlove: interesting, we'd like to understand how did you get there, where you are right now.
Tim Pritlove: So what was your introduction into this digital world? When did it start?
Meredith Whittaker: I don't have one of those stories where I'm like an Atari. I was four. I'm a hacker.
Meredith Whittaker: I didn't care about tech, qua tech. I wasn't interested.
Meredith Whittaker: You remember back in school, there were two kinds of nerds.
Meredith Whittaker: There were the math nerds and the book nerds.
Linus Neumann: You were the book nerd?
Meredith Whittaker: I was the book nerd. But in the story we tell now, because nerddom has been
Meredith Whittaker: overlapped with monetary success and sort of a career in math and science,
Meredith Whittaker: I think we forget about the book nerds. But hello, I'm here to remind you.
Meredith Whittaker: We exist. We're here. And that's who I was.
Meredith Whittaker: And so I studied literature and rhetoric at Berkeley, which I just thought were,
Meredith Whittaker: I mean, they're still beguiling.
Meredith Whittaker: Like being able to read and write is pretty fundamental for anything.
Meredith Whittaker: Und dann ich war poor so ich habe einen Job bei Google weil sie waren die ersten
Meredith Whittaker: Menschen zu offeren mich ein Job und dann habe ich sehr fasziniert mit was auf
Meredith Whittaker: der Erde was going on When war das?
Linus Neumann: Wenn warst du bei Google als jemand der Studierende Literatur und Rhetorik?
Meredith Whittaker: July 10, 2006 Ich habe mich über Berkeley und ich musste Geld und ich musste
Meredith Whittaker: und ich put mein Resumé auf Monster.com which ist ein precursor zu LinkedIn,
Meredith Whittaker: und ich habe mich überlegt out.
Meredith Whittaker: And then I talked to my friend Michelangelo, because at the time you still needed
Meredith Whittaker: to invite to join Gmail, as I recall.
Meredith Whittaker: And I was like, hey, Michelangelo, can you get me an invite so I can make a
Meredith Whittaker: Gmail address so I can appeal to this recruiter?
Meredith Whittaker: And now my spam-filled Gmail address dates from that moment.
Meredith Whittaker: So I was hired as something called a customer operations associate,
Meredith Whittaker: and I didn't know what that was.
Meredith Whittaker: No one knows what that is. It's just a bunch of words, But it sounded,
Meredith Whittaker: I was like, that's a business job. It sounds like a business title.
Meredith Whittaker: I had no idea. It was basically a customer support person who wrote technical
Meredith Whittaker: documentation, who are kind of user documentation, some technical documentation,
Meredith Whittaker: and answered inquiries about Google's free products.
Meredith Whittaker: And I was doing that for Rightly, which was an acquisition that Google made
Meredith Whittaker: that then they rebuilt to become Google Docs. Okay.
Meredith Whittaker: Und das ist an der Zeit, die ganze Zeit, die Google-Businessen war, was nur Search.
Meredith Whittaker: So Search war all auf der Main Campus. Es war wo die Money kam,
Meredith Whittaker: und dann war das Dinky-little-Building, wo alles, Gmail, Blogger, Rightly, Reader,
Meredith Whittaker: all of them sat und sie waren experimental projects. Und das war called Apps.
Tim Pritlove: Did you ever have exposure to actual customers or did you just write documentation?
Meredith Whittaker: Well, they would email in. I only did this for a second because I kind of figured
Meredith Whittaker: out I wanted to do other things. And that's part of the story.
Meredith Whittaker: There's a part of the story, actually. So I did answer inquiries sometimes.
Meredith Whittaker: And there were auto responses, like hotkeys you'd use in this really janky ticketing system.
Meredith Whittaker: And this is back. We didn't have laptops. There's no such thing as a smartphone.
Meredith Whittaker: Our desktops were chained to the desk. Und wenn du nach Hause bist,
Meredith Whittaker: wenn du nach Hause bist, wenn du nach Hause bist, dann musst du nachdenken.
Meredith Whittaker: Du hast du nachdenken, das klunky Lenovo oder so.
Meredith Whittaker: Ich weiß nicht genau, aber es war...
Meredith Whittaker: Ich war mir, okay, ich werde rewarded in meinem Job, based auf wie viele Tickets ich close bin.
Meredith Whittaker: Aber ich weiß nicht die Engenheer, die haben zu close diese Tickets,
Meredith Whittaker: weil ich mich reporting bugs, und ich muss sie fix.
Meredith Whittaker: Und wenn sie nicht fix die Bugs sind, dann bin ich nicht rewarded.
Meredith Whittaker: Und ich war mir, das ist wirklich silly.
Meredith Whittaker: So ich war über die Apps-Büte, weil ich in einer anderen Bühne bin bin.
Meredith Whittaker: Und ich mit der engineering team und ich war, hey, ich bin das Person,
Meredith Whittaker: die gibt es diese E-Mails, die Sie anwenden Sie.
Meredith Whittaker: Und ich war able, zu convince einem einen zu gehen, zu Costco,
Meredith Whittaker: die ist dieses große Warehouse-Store, ein Walmart-E-Store in den USA,
Meredith Whittaker: und put ein giantes Kuchen auf seine eigene Karte, und bring es in den Apps-Bilden,
Meredith Whittaker: so ich konnte sit mit dem.
Meredith Whittaker: Weil wenn ich mit dem satte, dann könnte ich mit dem, und dann würde ich mich
Meredith Whittaker: mehr möglich zu fixen, meine Bugs.
Meredith Whittaker: Und so all my bugs started getting fixed but then my manager got really,
Meredith Whittaker: upset because it was like obviously I had breached the hierarchy and I was like
Meredith Whittaker: and so this meeting appeared on my calendar that was basically about me being
Meredith Whittaker: insubordinate but I'm also like I didn't come from that class I had no I was
Meredith Whittaker: like what the fuck did I do wrong I don't know,
Meredith Whittaker: and I was like oh god I like I did something and then.
Meredith Whittaker: The day before I think that meeting was going to happen, this email hit,
Meredith Whittaker: we had this all, like some group, I don't know, it was like the group to the
Meredith Whittaker: consumer operations team.
Meredith Whittaker: And one of the engineering directors in Apps must have had like a beer or something
Meredith Whittaker: during one of the many, many, many drinking parties that happened during the
Meredith Whittaker: day at Google in that era.
Meredith Whittaker: And sent some email that was like, Meredith, this couch is a model for collaboration.
Linus Neumann: I was going to say, that's why you hired somebody from Berkeley.
Linus Neumann: To write your tickets because they come up with ideas like that.
Meredith Whittaker: And then that meeting just disappeared and I was like, well shit,
Meredith Whittaker: I need another job because I just burned someone who's going to figure out how
Meredith Whittaker: to get back at me and that's where my street sense comes in.
Meredith Whittaker: So I took a bike around campus for a couple months and asked every VP during
Meredith Whittaker: their office hours, which is a thing they used to have like, hey, I want another job.
Meredith Whittaker: And I would just drop in and be like, hi, you don't know me. I'd like a job.
Meredith Whittaker: And then I got a job doing basically Standards work, like trying to push document interop standards.
Meredith Whittaker: And so I started in standards and then standards parlays into measurement pretty
Meredith Whittaker: quickly because measurement is ultimately the methodological standard.
Meredith Whittaker: And co-founded MLab and then that became the nucleus around which the research
Meredith Whittaker: group I founded existed and then that was the...
Meredith Whittaker: That was when the fresh air of sort of these political and social complexities
Meredith Whittaker: started hitting my technical work. And I was like, oh, standardization is power.
Meredith Whittaker: Oh, creating data means you own the narrative. Oh, shit, none of this is neutral.
Meredith Whittaker: All of this is contingent.
Meredith Whittaker: I started seeing the balance sheet. I started seeing the capital that was involved
Meredith Whittaker: in running infrastructure.
Meredith Whittaker: And I remember around the time I met you, Linus, like maybe over a decade ago.
Linus Neumann: Yeah, it must have been about a decade or longer. Yeah, probably a bit longer even.
Meredith Whittaker: Yeah, aging is weird.
Meredith Whittaker: Aber ich habe diese schluss-Ballanz-Ballanz-Sheets. Ich habe aus Google gepritten,
Meredith Whittaker: die gezeigt, wie viel die Uplink-Bandwidth und die Infrastruktur und Power-Costs für MLAB war.
Meredith Whittaker: Und ich würde das zeigen, das zu all diese Civil Society funders,
Meredith Whittaker: wie gesagt, Sie können 250k verdienen.
Meredith Whittaker: Es kostet 40 Millionen a Jahr in Bandwidth, nur zu MLAB.
Meredith Whittaker: Du wirst nicht verstehen, die economics of this, denn wir sind in der Era der
Meredith Whittaker: Civic-Technik, All we need is a good idea, right?
Meredith Whittaker: And I think I was really lucky to get sensitized to the political economy and
Meredith Whittaker: the fact that we're talking about infrastructure,
Meredith Whittaker: capital, network effects, economies of scale and not some kind of brilliant
Meredith Whittaker: idea that just ephemerally transformed our world,
Meredith Whittaker: and that our side just needs to wait to have one of those to get our turn.
Tim Pritlove: That sounds like a quick upgrade from a customer support person.
Tim Pritlove: Yeah, yeah, yeah. So, you mentioned MLab, which stands for Measurement Lab.
Meredith Whittaker: It does, yeah.
Tim Pritlove: Can you explain what this is all about and how it came alive and why you were involved?
Meredith Whittaker: Yeah. It was a project that was Derek Slater, Vince Cerf, Stephen Stewart,
Meredith Whittaker: Sasha Meinrath at Open Tech Institute.
Tim Pritlove: The elders of the internet.
Meredith Whittaker: Yeah, some old internet guys and me.
Linus Neumann: Let's say the elders of the internet.
Tim Pritlove: You're one of them now.
Meredith Whittaker: And the lady with the couch. And the conceit there, which felt really simple
Meredith Whittaker: to me at the time, is everyone is buzzing about net neutrality.
Meredith Whittaker: And I was a Kool-Aid drinker. I still think the value underlying that kind of
Meredith Whittaker: mythology, let's say, like, yes, we should not have one gatekeeper deciding
Meredith Whittaker: which news source, right? This is old school common sense.
Meredith Whittaker: It goes back to Western Union, who there's a little, you know,
Meredith Whittaker: in the US would refuse to carry telegrams from political candidates the company didn't support, right?
Meredith Whittaker: So there's a real bedrock precedent here.
Meredith Whittaker: And I was like, yeah, of course we need net neutrality.
Meredith Whittaker: Neutrality itself is a really wafty, it's a loose concept.
Meredith Whittaker: It needs to be augured in some type of benchmark against which we can assess is this neutral or not.
Tim Pritlove: You need to quantify it.
Meredith Whittaker: Yes. And then you just flung into the abyss of philosophy the second you try to do that.
Meredith Whittaker: But we started and I would say it was not necessarily built to succeed.
Meredith Whittaker: It was built as a sort of hypothesis project where
Meredith Whittaker: we stood up three servers that hosted
Meredith Whittaker: open source measurement clients and the thing
Meredith Whittaker: that we were doing was putting servers that were all configured the same that
Meredith Whittaker: gave each client a dedicated slice of resources and then way over-provisioned
Meredith Whittaker: the uplink between the server and the switch so that we could all,
Meredith Whittaker: for all intents and purposes, guarantee that any artifacts that were detected
Meredith Whittaker: through the measurement methodology like some TCP,
Meredith Whittaker: RTT or something, were not interfered with by our infrastructure and weren't
Meredith Whittaker: suffering for bandwidth.
Tim Pritlove: So for practical reasons you had unlimited bandwidth and zero latency to everywhere.
Meredith Whittaker: Yeah, and let's, you know, and that's very expensive. But then,
Meredith Whittaker: we immediately DDoS the servers or like, oh shoot, because Vint's put a blog
Meredith Whittaker: post out and he was you know he's a big name,
Meredith Whittaker: and then it was like this job of years and years of getting servers in different
Meredith Whittaker: interconnection points because we wanted to measure across consumer you know
Meredith Whittaker: we're not just measuring to a server in the network exactly and we need to cross
Meredith Whittaker: interconnection points to do that because that's where you begin to see interesting
Meredith Whittaker: business relationships and kind of feuds,
Meredith Whittaker: and it was just it was very it taught me about like how difficult and contingent and,
Meredith Whittaker: ultimately subjective, creating data is, and the political process of then sort
Meredith Whittaker: of defending that data as a reliable proxy for reality.
Meredith Whittaker: Because I would be at the Federal Communications Commission and I had Comcast
Meredith Whittaker: across from me being like, no, we want to measure it multi-threaded TCP versus
Meredith Whittaker: single-threaded because you get a, let's just say, higher number that way.
Meredith Whittaker: It's more forgiving. and we would be sort of defending the methodology of our
Meredith Whittaker: tests and the openness principles.
Meredith Whittaker: So it was open source code, the database architecture was open.
Meredith Whittaker: I don't remember all the server architecture, it was old school,
Meredith Whittaker: old school ideological.
Tim Pritlove: You really don't sound like a book nerd now. You've become a real network nerd.
Meredith Whittaker: Let's just say book nerds read and network stuff is written down.
Meredith Whittaker: I read the fucking menu of it. I'm like, no one told me I was allowed to not
Meredith Whittaker: read it and still have an opinion.
Tim Pritlove: I love networking.
Meredith Whittaker: It taught me everything about so much because you get down to the network layer
Meredith Whittaker: and you're like, fuck, everything is broken. It's all tied together with string.
Meredith Whittaker: What's going on? Okay, so your promises in the pitch meeting are guaranteed by this.
Meredith Whittaker: This crusty old library that's maintained by a guy on a boat with a BitLocker
Meredith Whittaker: account. Like, what is going on?
Tim Pritlove: It's scary, isn't it?
Meredith Whittaker: It is. I mean, it's a catalyst into today for my work.
Linus Neumann: You did that for, I think, a couple of years, the MLab stuff.
Linus Neumann: And you also, at this point in time, I think, began, you know, the ties to the,
Linus Neumann: Let's say broader internet freedom community. I remember Uniprobe at the time, I think, was another.
Linus Neumann: They still exist, right? They do.
Meredith Whittaker: And we actually talked to them at Signal. They'll detect signal blocking.
Meredith Whittaker: And I believe, so one of the things MLab did was provide backend infrastructure
Meredith Whittaker: for projects like Uniprobe.
Meredith Whittaker: And I don't actually know what that relationship is today.
Meredith Whittaker: So academics and hackers and developers could write a test or kind of a measurement
Meredith Whittaker: methodology deploy a client,
Meredith Whittaker: to consumers like you test from your laptop or whatever and then we would be
Meredith Whittaker: the ones paying for the bandwidth and infrastructure that would allow that to scale Uni.
Linus Neumann: Is the Open Observatory of Network Interference.
Meredith Whittaker: If I remember the acronym Hi Arturo So...
Linus Neumann: Disclosing a whole social network here voluntarily. So, but eventually something
Linus Neumann: happened that is known as the Google Walkouts.
Linus Neumann: Was that following your MLab activities or during your MLab activities or did
Linus Neumann: you rise even higher in the Google hierarchies before that happened?
Meredith Whittaker: Well, I mean, there are many. I think, so...
Meredith Whittaker: I was not working on MLAB by that point.
Meredith Whittaker: MLAB, there's a part of this story that I didn't cover, which is around,
Meredith Whittaker: I don't know, 2014, 2015,
Meredith Whittaker: we started seeing really interesting artifacts in the data,
Meredith Whittaker: which showed essentially that at particular interconnection points between particular
Meredith Whittaker: telcos, we were seeing drastic drops in performance.
Meredith Whittaker: And what we were able to do was look at, you know, every intersection of Telco
Meredith Whittaker: 1 and Telco 2, we see this drop.
Meredith Whittaker: Intersections of Telco 3 and Telco 2 don't see this drop.
Meredith Whittaker: And so what we're able to do is say, like, there's actually a business feud
Meredith Whittaker: going on. And the interconnection point is the locus of that feud.
Meredith Whittaker: And using traceroute data, we were able to say, like, look, these aren't,
Meredith Whittaker: it's not just that they're sharing a path in some other region of the network that's slowing it down.
Linus Neumann: They're actively throttling.
Meredith Whittaker: They're actively throttling. And we put together a report on that.
Meredith Whittaker: And this is where I was sort of, I think, kind of sideloading myself into some
Meredith Whittaker: academic research work. I was like, okay, how do we do this?
Meredith Whittaker: Document our methodology. Document sort of everything was open.
Meredith Whittaker: So we pulled that together.
Meredith Whittaker: And it showed, I don't know if you remember, there was a sort of Comcast Cogent Netflix.
Linus Neumann: Oh, yeah.
Meredith Whittaker: Netflix was shoving all its traffic through Cogent. What we had done is expose
Meredith Whittaker: that and exposing that it exposed the principle that you have to if you wanted
Meredith Whittaker: to ensure net neutrality, you have to take the interconnection points and the
Meredith Whittaker: interconnection agreements into account.
Meredith Whittaker: And that led Obama to add interconnection to the reclassification under Title
Meredith Whittaker: II of these, you know, which kind of moved toward net neutrality that was nullified.
Meredith Whittaker: But, you know, that was kind of the swan song, let's say, for my MLab time,
Meredith Whittaker: because, of course, that was a huge deal, right?
Meredith Whittaker: Like, that's where the business model rubber hits the road.
Meredith Whittaker: And, you know, I'll just shorthand it. Like, that bought me a lot of capital at Google.
Meredith Whittaker: And with that, I was, you know, I was already interested in a lot of things.
Meredith Whittaker: MLab was sort of humming along and, like, it had grown from a hypothesis project
Meredith Whittaker: to, like, a global thing that was working and doing stuff.
Meredith Whittaker: And I started getting agitated by AI and a lot of these privacy and security
Meredith Whittaker: concerns, being part of the community that you were part of,
Meredith Whittaker: you know, kind of thinking around like
Meredith Whittaker: tech alternatives and getting less comfortable with the business model.
Meredith Whittaker: And from there I always had like eight different projects going on but I went
Meredith Whittaker: on to found, to co-found the AI Now Institute which was really trying to bring the conversation,
Meredith Whittaker: on what we called machine learning back then but like AI was like a flashier
Meredith Whittaker: term, like bring it down to the ground a little bit and stop talking about superintelligence
Meredith Whittaker: and stop talking about, you know,
Meredith Whittaker: Political economy, what are these technologies, how are they being deployed,
Meredith Whittaker: how do you oversee them, who uses them on whom, and what are the social and
Meredith Whittaker: political dynamics of that.
Linus Neumann: But the AI Now Institute is not hosted at Google, is it? No.
Linus Neumann: Which university is it again?
Meredith Whittaker: It was at NYU, and it's moved out of NYU for reasons of that being easier.
Linus Neumann: Okay.
Meredith Whittaker: And cheaper, because they take like 40%, those universities.
Linus Neumann: Okay.
Meredith Whittaker: Word to the wise, your money.
Linus Neumann: What do you think?
Tim Pritlove: Like whatever donations you get. Okay.
Meredith Whittaker: Yeah. So it's an expensive brand.
Linus Neumann: Well, for a reason, right? I mean, NYU Institute, that's something to have, I think, co-founded.
Meredith Whittaker: Co-founded, yeah. I mean, the work was really good. And this was,
Meredith Whittaker: again, gaining the capital at Google, cementing a reputation,
Meredith Whittaker: being able to get to a level where I had a budget.
Meredith Whittaker: And then part of what I was always trying to do is how much can I pull out of
Meredith Whittaker: Google and get into the hands of people doing work that I think is cool.
Meredith Whittaker: How do we carve tributaries in the massive river of this huge,
Meredith Whittaker: rich company and get it out?
Tim Pritlove: So you started that institute at the end of 2017.
Meredith Whittaker: Well, beginnings are sometimes hard to date.
Meredith Whittaker: It was born out of a request from the Obama administration to be the host of one of his AI summits.
Meredith Whittaker: I don't remember exactly the contours there, but it was in 2016.
Meredith Whittaker: And so it started as like, like the idea then was like, let's do this and let's do it big.
Meredith Whittaker: Like, let's make this the most polished, flashiest, like hard to ignore kind
Meredith Whittaker: of spectacle using all the tools we can get from like hiring an events agency,
Meredith Whittaker: doing good press, all of that.
Meredith Whittaker: But let's also make this the one that is the most engaged with these political topics,
Meredith Whittaker: that's actually forcing the debate in that direction and kind of making it face
Meredith Whittaker: these questions that are much more grounded and, you know,
Meredith Whittaker: hopefully much more healthy for our population.
Meredith Whittaker: So it was very successful and from there we got offers of funding and a lot
Meredith Whittaker: of encouragement and the work just kept going.
Tim Pritlove: Well, we in Europe, we are used to be a bit behind, but if I can recall 2016,
Tim Pritlove: there was no discussion about AI in Europe at all.
Tim Pritlove: And so, it's quite interesting to see that Obama has actually,
Tim Pritlove: you know, decided that it's finally time to do something about this topic nobody
Tim Pritlove: has ever really talked about.
Linus Neumann: Germans are currently making up their mind whether that is a topic.
Tim Pritlove: Ja, wir sprechen viel über Europa heute.
Tim Pritlove: Aber mein Frage ist, kann man describe was kind of discussions going on 10 Jahre
Tim Pritlove: ago in American society that this has come up to be something of a topic for the near future,
Tim Pritlove: which actually became a big one.
Tim Pritlove: Where did this discussion take place?
Meredith Whittaker: Yeah, to answer this question, I'm going to be drawing on a lot of the research
Meredith Whittaker: and the historical work, the work I've done since then.
Meredith Whittaker: Because when this dawned in my life, when it started being a thing,
Meredith Whittaker: I had basically that same question. What is this stuff?
Meredith Whittaker: Why is it kind of... At Google, you would see...
Meredith Whittaker: A shift toward a new paradigm or a new trend by the incentives that were structured
Meredith Whittaker: into the OKRs, the quarterly goals,
Meredith Whittaker: that kind of, you know, there'd be all these training modules that would pop
Meredith Whittaker: up and it's like make your software engineer into an AI developer,
Meredith Whittaker: you know, and you'd be like, there's an incentive here.
Linus Neumann: Seems like they weren't too successful though, right? I mean,
Linus Neumann: that's not like the biggest AI wave at Google.
Meredith Whittaker: Well, they were, that was DeepMind era. So, okay, okay, good.
Linus Neumann: Sorry.
Meredith Whittaker: I think they were ahead. It was them and Meta for a long time. And the history...
Linus Neumann: Didn't make it to the business though at the time, right?
Meredith Whittaker: No, but they're chaotic. It's a kind of court in decline. So that,
Meredith Whittaker: you know, actually where the... Actually the business model part has never been
Meredith Whittaker: their strong suit beyond search.
Meredith Whittaker: To be real, you know, like cloud is like the best technology presented confusingly
Meredith Whittaker: with 18 versions all deprecated, right? Like that's... Um,
Meredith Whittaker: um, but this, the, the AI stuff was, so if you look at the, if you look at the
Meredith Whittaker: sort of recent history, which is something I'm spending, I've spent a lot of time on.
Meredith Whittaker: I spend a lot of time on because I think we, you know, that gives us a really
Meredith Whittaker: different picture than the Elon Musk narrative or the kind of popular narrative.
Meredith Whittaker: There's a very important paper that was published in 2012 that introduced the
Meredith Whittaker: AlexNet algorithm. und das war Jeff Hinton und his students, you may recognize.
Tim Pritlove: Who just got the Nobel Prize. Physics, interestingly enough.
Linus Neumann: Yeah, because there is no AI.
Tim Pritlove: Yeah, because there is no computer Nobel Prize.
Meredith Whittaker: Well, if you claim that your technology is everything, then you can get a prize in anything.
Tim Pritlove: Which is a proven point now, yeah.
Meredith Whittaker: And it was Ilya Stutzkever and then Alex. I'm sorry, Alex, I'm not grabbing
Meredith Whittaker: your last name from the ether right now.
Meredith Whittaker: But nonetheless, this was a paper that kind of pulled together key ingredients
Meredith Whittaker: that became the foundation of the AI boom now.
Meredith Whittaker: So this is deep learning algorithms, which is the paradigm we're still in.
Meredith Whittaker: It doesn't matter. There's architectural sort of rejiggering,
Meredith Whittaker: but nonetheless, it's deep learning.
Meredith Whittaker: Huge amounts of data. So what I've called the derivatives of the surveillance business model.
Tim Pritlove: He found all the cats on YouTube.
Meredith Whittaker: I mean, yeah, that was Jeff Dean, I think.
Meredith Whittaker: And then, you know, powerful compute, right?
Meredith Whittaker: And they showed that sort of using gaming chips and a lot of data,
Meredith Whittaker: you could beat the benchmark.
Meredith Whittaker: So score much better against standard evaluations than past models and thus
Meredith Whittaker: sort of catalyzed the industrial industry interest in AI and why were they interested?
Meredith Whittaker: I think this is a key point because these optimist-seeking algorithms are really
Meredith Whittaker: good at curating news feeds.
Meredith Whittaker: They're really good at figuring out algorithms, right?
Meredith Whittaker: I don't think it's an accident that Jeff was immediately hired,
Meredith Whittaker: dass Jan LeCun, die die deep-learning-Algorithmen, die das Seed der Moment in
Meredith Whittaker: AI in den 80ern, in den letzten 80ern, war er mit Meta.
Meredith Whittaker: Und es war die Plattform-Companien mit der realen Investition in squeezing mehr
Meredith Whittaker: Ad-Dollars aus der Daten, better-serving, all of das.
Meredith Whittaker: Das ist Google mit DeepMind. Du sehen Meta und Google die leaders in das.
Meredith Whittaker: Als measured by different evaluation standards, the measurement question hier
Meredith Whittaker: ist eigentlich wirklich interesting und troubling.
Meredith Whittaker: Until this generative moment where I think the ChatGPT Microsoft products shifted
Meredith Whittaker: people's perception of AI and what it can do and just rearrange the leaderboard.
Meredith Whittaker: But the paradigm is still the same And the paradigm is still that AI is applying
Meredith Whittaker: old algorithms on top of the sort of massive platform monopoly business model,
Meredith Whittaker: availing itself of huge amounts of data, which is produced via this surveillance
Meredith Whittaker: business model, and really powerful compute that was designed,
Meredith Whittaker: built up, you know, I would say, like, consolidated in the hands of these platform
Meredith Whittaker: companies via the imperatives of the surveillance business model, right?
Linus Neumann: Would you say that, so clearly, I mean, you say,
Linus Neumann: so here are Google and Meta that have these massive amounts of data,
Linus Neumann: like larger amounts of data than probably ever existed or were in the hands
Linus Neumann: of anybody or any organization.
Linus Neumann: Would you say that they had amassed all this data and eventually learned,
Linus Neumann: okay, we probably can't handle this anymore, so we're interested in this new
Linus Neumann: paradigm to even monetize this data any further?
Linus Neumann: Or was it rather like, oh, we have all this data, we're monetizing it well,
Linus Neumann: here's another way to monetize it on top of that. because clearly these deep
Linus Neumann: learning algorithms are not of much use if you don't have large data.
Meredith Whittaker: Set and large compute for both training and inference but is.
Linus Neumann: It the hen egg what's the hen what's the egg there.
Meredith Whittaker: I think this
Meredith Whittaker: is almost a perpetual motion machine right like
Meredith Whittaker: every quarter you have to report progress you have to report growth the logic
Meredith Whittaker: is metastasis and so you know you're trying to squeeze more out of what you
Meredith Whittaker: have and you're trying to get more of what you have so you can squeeze more
Meredith Whittaker: right there's also these sort of laws of scale remember big data we used to call it that,
Meredith Whittaker: and so I don't I don't actually know the answer to your question but I think it,
Meredith Whittaker: Which came first? But, well, I mean, the business model came first, right?
Meredith Whittaker: You had to have the ingredients to know what they did together.
Meredith Whittaker: And I think it was deep learning and AI had sort of languished in the backwater
Meredith Whittaker: with some interesting experiments through the 2000s because its history is always
Meredith Whittaker: promising too much and disappointing since the mid-50s when it was invented.
Meredith Whittaker: And I think the goal was really to sort of supercharge their existing business model.
Tim Pritlove: I think deep learning was just the technology that perfectly served their current
Tim Pritlove: beliefs in that they have to work on the data,
Tim Pritlove: that they have to build up algorithms to somehow predict your personal future
Tim Pritlove: and be there with an ad before you even know it and we've seen this,
Tim Pritlove: as an ad everywhere and we've also seen it in political influence as we've seen in the Brexit,
Tim Pritlove: decision and also in the elections where we heard there's going to be one in
Tim Pritlove: the near future as well in the US that might be influenced as well.
Meredith Whittaker: Let me check my calendar.
Meredith Whittaker: I mean I think we can like peel back also just this concept of like what is
Meredith Whittaker: an advertisement right? It's an influence.
Meredith Whittaker: Is it trying to get you to buy something? Is it trying to get you to like something?
Meredith Whittaker: Is it trying to get you to believe something? Is it trying to get you to vote a certain way?
Meredith Whittaker: And I think that the term advertisement is usefully deconstructed when we start
Meredith Whittaker: to think about the connection between all of those.
Linus Neumann: The term advertising is one of the best tricks the devil ever pulled.
Linus Neumann: Because it's behavior manipulation. It's a stated goal. But I remember when.
Linus Neumann: The first debates came up about, for example, the Cambridge Analytica scan.
Linus Neumann: It's like, they used Facebook data to manipulate voters.
Linus Neumann: And it's like, no, there's a red line over here. You can't manipulate behavior.
Linus Neumann: I mean, Facebook and Google were basically built to manipulate users' behavior
Linus Neumann: and ideally capture their attention to change their behavior.
Linus Neumann: Wir haben wahrscheinlich dieses shirt oder das oder das oder das oder das oder
Linus Neumann: das oder das oder das oder das oder das oder das oder das oder das oder das oder das ist.
Linus Neumann: Wie können wir das so etwas machen? Das ist unheard of.
Linus Neumann: So viewing advertising as like, oh, this is just an offer.
Linus Neumann: This is, oh, we're just making our product. No, no, I have a limited amount of attention per day.
Linus Neumann: You're capturing it and you're doing it with one simple reason.
Linus Neumann: And that is changing my behavior in the sense that you aim for.
Linus Neumann: And I guess eventually it sounds like,
Linus Neumann: eventually you questioned your Google career, right? Because you were,
Linus Neumann: I mean, clearly you had an impressive career there in just a few years.
Linus Neumann: Um, but you began to question not only your, like Google and the company culture,
Linus Neumann: but apparently also a little bit your,
Linus Neumann: your, the way you want to continue with your, with your, you know,
Linus Neumann: making use of the influence and the knowledge you have.
Meredith Whittaker: Yeah. I think.
Linus Neumann: Is that the walkouts?
Meredith Whittaker: I mean, it's all interlinked and kind of periodizing your own consciousness is hard.
Meredith Whittaker: But I think I'm pretty earnest and I also don't come from that world,
Meredith Whittaker: I don't come from that class so there are often places where I just didn't,
Meredith Whittaker: you know, I would take things sincerely or be like really committed and then only realize like
Meredith Whittaker: two thirds of the way through whatever it was like oh no one else really cares
Meredith Whittaker: about this they're just networking or you know whatever it was so I think there
Meredith Whittaker: was an element there where like you know when I was doing MLab I was like I
Meredith Whittaker: really want to win net neutrality Und dann we won net neutrality.
Meredith Whittaker: Aber dann ist das, dass ich das nicht der Fall war.
Meredith Whittaker: Google hat eine größere Netzwerke als Comcast. Das ist nicht die Gatekeeper.
Meredith Whittaker: Aber das war eine Sincere Sache.
Meredith Whittaker: Und dann war ich, okay, ich kann mich,
Meredith Whittaker: um, Geld zu all diese coolen Privatsch-Hacker-Projekte. Das war Sincere.
Meredith Whittaker: Und dann wurde ich in AI und ich war, okay, und ich denke, das ist etwas,
Meredith Whittaker: das hat sich verändert für mich.
Meredith Whittaker: I think I used to have a lot, lot more faith in the power of ideas to influence real change, right?
Meredith Whittaker: And I still think, you know, I spend a lot of time in kind of thinking through
Meredith Whittaker: discourses, how do we shape them?
Meredith Whittaker: Like, how do we kindly walk people into understanding things that,
Meredith Whittaker: you know, they may have an interest in not understanding or they may have been,
Meredith Whittaker: you know, misinformed about or what have you.
Meredith Whittaker: But I began you know I began around the time I was looking at AI and sort of making all these cases,
Meredith Whittaker: that everyone loved right like I was out there giving talks that were completely
Meredith Whittaker: against the Google party line and I was getting like applauses,
Meredith Whittaker: I was getting promoted like I was like this is a this is a perfect job.
Meredith Whittaker: And then you know like I was I envied you not only once I'm the house troll,
Meredith Whittaker: but then there was I was getting more influence so I was becoming known outside
Meredith Whittaker: and inside I was like the person you'd call into your team when it was like
Meredith Whittaker: oh we want to implement this is there an ethical way to do it,
Meredith Whittaker: and you would say no I would, I would be like my dear friends let us sit down,
Meredith Whittaker: and then I was getting, I don't know, that was sort of my life and we kind of
Meredith Whittaker: took the AI Now Institute and really did a lot to reshape the debate.
Meredith Whittaker: I was very focused on that discursive intervention and how do we begin to talk
Meredith Whittaker: about AI in a more realistic way and that was working outside of Google but
Meredith Whittaker: it wasn't really influencing core decisions at Google and that was kind of the
Meredith Whittaker: thing I kept hitting up against more and more strongly until I got a signal message in...
Meredith Whittaker: Late 2017.
Linus Neumann: From Moxie?
Meredith Whittaker: No, I mean, I probably did get one from Moxie at that time, but this one was not that one.
Meredith Whittaker: It was from a friend of mine at Google who said, yo, there's a really disturbing
Meredith Whittaker: project that is hidden that I'm very close to.
Meredith Whittaker: And you should know about it because you're the AI person who cares about this
Meredith Whittaker: and you have standing at the company around it.
Meredith Whittaker: And this was the secretive contract that Google signed with the Department of
Meredith Whittaker: Defense to build AI drone targeting and surveillance systems for the drone war.
Meredith Whittaker: And of course, like I was politicized post 9-11, post Snowden.
Meredith Whittaker: Like this was, you know, the drone war and the signature strike and all of that
Meredith Whittaker: were really core in kind of my, you know, things that I ideologically rejected
Meredith Whittaker: and, you know, felt like we needed to disarm, not supercharge.
Meredith Whittaker: And I had like a, you know, like a righteous anger. I was just like, fuck this.
Linus Neumann: Because you're there running around, you know, trying to shape the discourse on AI.
Meredith Whittaker: And making them look good, right? Because they get to be like,
Meredith Whittaker: look, we platform such heterodox voices. We're surely benevolent, right?
Meredith Whittaker: And then I'm like, okay, and then you're inking this deal with the DoD behind
Meredith Whittaker: the scenes for technology that one we know doesn't sort of work for the purpose, right?
Meredith Whittaker: Like, you know, it's not going to better identify a worthy
Meredith Whittaker: target of death oder whatever the fuck it's you know it's not you
Meredith Whittaker: know we know this is bullshit but like and you
Meredith Whittaker: know this is a multinational company like more than
Meredith Whittaker: half the employees are outside the US there is an issue with yoking yourself
Meredith Whittaker: to you know not that you have one nation's government not that many tech companies
Meredith Whittaker: care that much about that and then there was just the you know what is the structural
Meredith Whittaker: danger which is deeply acute and,
Meredith Whittaker: of a massive surveillance company with more data than the world has ever seen,
Meredith Whittaker: more compromised than you can imagine, like yoking their fortunes and a key
Meredith Whittaker: dependency to the US military, right?
Meredith Whittaker: And, you know, we know from Snowden that that's already like,
Meredith Whittaker: you know, seen as, you know, as long as it's a corporation gathering it, it's not, you know.
Tim Pritlove: Was Google still running under the motto of don't be evil at the time?
Meredith Whittaker: They were, they were. And that was, we marshaled that actually.
Tim Pritlove: Kind of like the end of it, wasn't it?
Meredith Whittaker: They quietly, like the lawyers removed that from the Google Manifesto.
Meredith Whittaker: It was slowly fading. Yeah, it was like, they were like, just don't open that closet.
Tim Pritlove: Which motto? I don't recall any. It's like, do you know something about this?
Meredith Whittaker: Try not to be bad is the new motto.
Tim Pritlove: Don't be as bad as that.
Meredith Whittaker: Yeah, like lay off the evil.
Meredith Whittaker: Ja, so I mean, and this was, there were like a lot of old school people there
Meredith Whittaker: at that time who really did drink the Kool-Aid and so it was you know,
Meredith Whittaker: I just put my energy into organizing against that and that was when I turned
Meredith Whittaker: toward labor organizing and thinking through traditional,
Meredith Whittaker: methods and approaches to combating that type of corporate power or you know
Meredith Whittaker: industrial power and we, you know, and that That was the on-ramp to the walkout.
Meredith Whittaker: So the walkout was like a big, that was like a rupture, like a manifestation.
Meredith Whittaker: It got a lot of press and it was the biggest labor action in tech.
Tim Pritlove: So just to make that clear, the walkouts were actually people walking outside
Tim Pritlove: of the Google buildings for, I don't know, how long?
Meredith Whittaker: I think it was November 11th, 2018.
Meredith Whittaker: Und everyone walked out for 20 minutes at 11.11am in their local time so we
Meredith Whittaker: called it Rolling Thunder and it started in the Singapore office as I was going
Meredith Whittaker: to bed in New York and I was seeing the photos,
Meredith Whittaker: and this was chaos I hadn't slept in days there's so many meetings there's so
Meredith Whittaker: many tears it's hard to organize something like that,
Meredith Whittaker: and I remember going to bed and seeing the images from Singapore mit ein paar
Meredith Whittaker: hundert Leuten in Singapur und ich war,
Meredith Whittaker: das ist großartig.
Meredith Whittaker: Und dann habe ich New York time gewohnt, um 5 a.m. zu gehen in New York zu unserer
Meredith Whittaker: location und prepen es, um, dass wir uns da sind.
Meredith Whittaker: And then I just remember seeing, like, there was this little park near the New
Meredith Whittaker: York office, and then it just, like, grew outside the park.
Meredith Whittaker: And then no one could get into the park. And then I was looking at my home,
Meredith Whittaker: and there's my phone, and there's live helicopter feeds.
Meredith Whittaker: And we don't have bullhorns, because I'm one of the speakers.
Meredith Whittaker: There's, like, speakers standing on chairs to address the crowd.
Meredith Whittaker: And then this guy, like, some, you know, there's this sort of type.
Meredith Whittaker: I don't know if in Germany you have this type, but they're, like,
Meredith Whittaker: kind of, like, the leftist at every protest.
Meredith Whittaker: And like some guy had like found out about it and came in and I just remember this like,
Meredith Whittaker: like this man I'd never seen like handing me a bullhound from below and I picked
Meredith Whittaker: it up and like we were like bullhound like a megaphone like a megaphone yeah yeah yeah.
Meredith Whittaker: And then it was you know and then we walked over
Meredith Whittaker: to a Mexican restaurant and sat at the table and had a press operation
Meredith Whittaker: where like that we were running and what was cool was like everyone organizing
Meredith Whittaker: that was kind of a professional most of them were femme and most of them had
Meredith Whittaker: jobs at Google that were like organizing the company so organizing their comms
Meredith Whittaker: being an administrator for like 13 different directors,
Meredith Whittaker: this particular type of hyper competence at coordinating activity across a number
Meredith Whittaker: of people who you may not have direct power over and like for that much more.
Tim Pritlove: Than developers in that.
Meredith Whittaker: Respect I mean let's just say Yeah, yes. You know, you can, like somebody who
Meredith Whittaker: can write an email and get someone to respond to that email by doing a thing is kind of a witch.
Meredith Whittaker: And it was all these like these preternaturally competent femmes just turning
Meredith Whittaker: that energy toward organizing across the company,
Meredith Whittaker: which was building on this sort of base of meetings and locals and all of this
Meredith Whittaker: sort of work that we'd done to kind of form a discursive environment inside
Meredith Whittaker: Google where we're like weekly meetings to discuss the news,
Meredith Whittaker: what Google could do, what campaigns we could do.
Meredith Whittaker: So we had this sort of energy and solidarity already at that point.
Tim Pritlove: How many percent of the Google employees would you say have at one point taken part in this?
Meredith Whittaker: I don't know. we were really careful
Meredith Whittaker: not to keep lists yeah I mean you can dream up a number now you know I.
Meredith Whittaker: 20,000 was the number estimated from the Sky Photos and the local reports.
Meredith Whittaker: We had local leads at every office.
Meredith Whittaker: We sent them zip files of the kit for handing out the flyers,
Meredith Whittaker: the talking points, how to treat media, all of that.
Meredith Whittaker: They all had that, and then they organized their local. And then they had reporting
Meredith Whittaker: back in from press, reporting back into the central organizers around numbers.
Meredith Whittaker: And then we were issuing the press releases. um but
Meredith Whittaker: then you know there's employees of google and
Meredith Whittaker: then there's contractors of google which more than double employees so
Meredith Whittaker: it you know that number is and and then there's people who
Meredith Whittaker: couldn't participate but were really supportive because they
Meredith Whittaker: have you know need their health insurance or they'll die or they are on a visa
Meredith Whittaker: and so it wasn't um i think there was a huge amount of support uh i know that
Meredith Whittaker: we were able to get google to drop their military contract because there was
Meredith Whittaker: enough support and there was you know there was like a spreadsheet of people who are quitting,
Meredith Whittaker: conscientious objectors we had you know like every week at the all-town meeting
Meredith Whittaker: there was a table we would set up with like banners and something like we would
Meredith Whittaker: had questions like it was a very,
Meredith Whittaker: it was a very like rigorous campaign and it kind of laid the laid the groundwork
Meredith Whittaker: and i think it built some muscle that people are probably almost are definitely
Meredith Whittaker: using now even if they don't call at organizing.
Meredith Whittaker: It's like, how do we marshal the resources from this?
Tim Pritlove: So did Google then still love you at that point?
Meredith Whittaker: Well, I think Google does still love me, but it doesn't love itself enough to admit it.
Tim Pritlove: They have issues, I think.
Meredith Whittaker: I do think that. Come on, come on. Which side do you want to be on?
Meredith Whittaker: Which party do you want to go to? Let's be real.
Tim Pritlove: I mean, it leads to an interesting point because I would say,
Tim Pritlove: looking from the other side of the ocean, I think to us, this whole tech scene,
Tim Pritlove: The startups, the new stuff, the internet, everything that has developed in
Tim Pritlove: the last 20 years or so, always had this liberal touch to it.
Tim Pritlove: It felt as if it was mostly about an open, world-loving agenda and it's good for everybody.
Tim Pritlove: And Google kind of tuned in with their motto and some other companies did as well, some not so much.
Tim Pritlove: But it was always this feeling that liberal thinking is at the core of everything
Tim Pritlove: that is driving the internet forward.
Tim Pritlove: And I think we stopped thinking that now because it looks totally different right now.
Tim Pritlove: Currently, we have more the feeling that it's turned into a total right-wing apocalypse somehow.
Tim Pritlove: And I haven't really seen this coming.
Tim Pritlove: Can you explain what happened to this tech scene? What happened?
Meredith Whittaker: I, well, you know, I think about post 9-11 and a lot of the fights over surveillance and tech.
Meredith Whittaker: And there's a talk that, who was, Frank Rieger and Rupert Gronkripp?
Tim Pritlove: Rob Gronkripp, Rob Gronkripp.
Meredith Whittaker: Sorry, I don't, that's my American, you'll do beautifully in America.
Tim Pritlove: It's complicated for us.
Meredith Whittaker: Ja, aber ich entschuldige für nicht das richtig.
Tim Pritlove: Roppenfrank.
Meredith Whittaker: Ja, ich habe diese Talk, We Lost the War, ich glaube es war 1994. Nein, 2004.
Tim Pritlove: Ja.
Meredith Whittaker: Ich habe die gleiche Sache gemacht.
Meredith Whittaker: Und nicht jeder, ich glaube, war equating die growth und monetary success of
Meredith Whittaker: the US-based tech industry with sort of, you know, values of social progress.
Meredith Whittaker: I mean, I think we can, yeah, yes, it was liberal and then we can get back to
Meredith Whittaker: a critique of liberalism and what have you, but it was, I think there were people
Meredith Whittaker: who were looking at the infrastructure,
Meredith Whittaker: who were looking at its capabilities, who were looking at the gap between the
Meredith Whittaker: promises and the reality of what this tech did and calling that out.
Meredith Whittaker: And I feel like, you know, when I entered into this kind of,
Meredith Whittaker: Privacy, security, hacker development scene.
Meredith Whittaker: There was a lot of that skepticism there around Google.
Meredith Whittaker: That educated me a lot. There was a lot of skepticism around surveillance.
Meredith Whittaker: I immediately recognized, yes, we need privacy because it doesn't matter if
Meredith Whittaker: these people are good or benevolent.
Meredith Whittaker: What we're doing is setting up an infrastructure that could be turned over at
Meredith Whittaker: any moment to another regime. Logically, all of that made sense.
Meredith Whittaker: But I don't feel that until the Snowden revelations, any of that was anywhere
Meredith Whittaker: near in the nervous system of a kind of tech consciousness.
Meredith Whittaker: And a lot of the work I've spent in the 1990s trying to sift through the crypto
Meredith Whittaker: wars and sift through what happened with tech regulation to set up these surveillance
Meredith Whittaker: giants and to permit this monopoly platform business model,
Meredith Whittaker: has kind of looked at that gap between the rhetoric of liberal rights preserving.
Meredith Whittaker: Open, free tech, and what was actually being built, right?
Meredith Whittaker: And one of the things, if you look at, there's a scholar named Katharina Ryder,
Meredith Whittaker: who I would really suggest, for show notes, I can send some of these links.
Meredith Whittaker: But she did her dissertation looking at some of the negotiations in the crypto wars.
Meredith Whittaker: And what you begin to see is that, yeah, we, you know, and this is a thesis
Meredith Whittaker: sort of I build on top of in some of my work.
Meredith Whittaker: Yeah, we won liberalized encryption, right?
Meredith Whittaker: By 1999 in the US, it was finally legal to build, share, implement strong cryptosystems
Meredith Whittaker: without approval from the government, without some threshold that made them useless.
Meredith Whittaker: But the agreement there was basically, yeah, you can have encryption,
Meredith Whittaker: but we're going to permit mass surveillance by companies.
Meredith Whittaker: Und so you don't actually, like, you can just get the, you know,
Meredith Whittaker: we're going to permit, we're going to endorse the advertising business model.
Meredith Whittaker: We're going to endorse, and I can actually, I can start this point over, actually.
Linus Neumann: Would you say those two decisions were...
Linus Neumann: Strategically interlinked or were they just coinciding?
Meredith Whittaker: I can't say that there was a conspiracy. What I can say is that Katarina's work
Meredith Whittaker: shows that Microsoft saying liberalize encryption, don't worry we're not going
Meredith Whittaker: to encrypt all the data, we need it.
Meredith Whittaker: Just come to us quietly and we'll give it to you.
Meredith Whittaker: Instead of fighting over a backdoor, instead of doing this in sort of the public
Meredith Whittaker: domain where we're kind of losing the fight on technical and other grounds.
Meredith Whittaker: Allow companies free reign to surveil because that allows us to implement this
Meredith Whittaker: ad-supported business model.
Meredith Whittaker: And then the data agreements can happen behind the scene.
Meredith Whittaker: Now, I've like completely compressed a very complex history into basically a meme.
Meredith Whittaker: But I think the purpose is there is that there was always that gap between these
Meredith Whittaker: sort of rhetoric and what was actually going on.
Meredith Whittaker: And I think the, I don't know, like there's like a kind of internet people, right?
Meredith Whittaker: This type maybe misunderstood exactly like how this, you know,
Meredith Whittaker: who would have power over this technology, right?
Meredith Whittaker: Like encryption is liberalized, but it's not gonna be applied to protect personal
Meredith Whittaker: communication. It's applied to protect transactions, right?
Meredith Whittaker: It's not, you know, like the people who get to choose whether or not it's used aren't us.
Meredith Whittaker: In terms of, you know, actually this sort of mass infrastructure in terms of
Meredith Whittaker: the tech ecosystem that's being built by the actors who are calibrating their
Meredith Whittaker: decisions based on the surveillance business model that has been instantiated right,
Meredith Whittaker: you know, through regulatory decisions made by the Clinton administration.
Linus Neumann: So then there were a couple of hundreds of thousands of nerds worldwide that used PGP encryption,
Linus Neumann: using additional software and plugins to send an email with nonsense information
Linus Neumann: to avoid government or private sector surveillance.
Linus Neumann: But it was probably never really a significant number of individuals that...
Linus Neumann: That got to the point of, you know, having mass encryption out there.
Meredith Whittaker: Yeah, well, I tried. It's just that my friend group didn't overlap exactly with
Meredith Whittaker: those couple of thousand nerds. So what am I going to do, right?
Meredith Whittaker: And I think that's, you know, this is the network effect. This is why it's actually
Meredith Whittaker: very difficult to do that.
Meredith Whittaker: And this is why if one of those actors that controls these infrastructures doesn't
Meredith Whittaker: make the choice for us, it's really difficult to make that choice.
Linus Neumann: And none of these actors voluntarily got the idea, right? I think it's part
Linus Neumann: of the founding myth of Signal that Moxie at the time wanted to implement the,
Linus Neumann: it's now called the Signal Protocol.
Linus Neumann: I still remember it as being Axolotl or whatever it was called.
Meredith Whittaker: It used to be Axolotl. And then it was when Signal was launched,
Meredith Whittaker: which was the Red Phone Tech Secure integrated into one app for iOS in 2013. It was changed.
Linus Neumann: Right. But initially, I believe Moxie wanted to implement at Crypto for Twitter,
Linus Neumann: direct messages, if I'm not mistaken. I think it was at Twitter before.
Linus Neumann: And so his idea to roll out Twitter,
Linus Neumann: Definitely, he devoted a few years of his life to implementing and rolling out
Linus Neumann: mass end-to-end encryption, right?
Linus Neumann: But I believe it was he wanted to do it at Twitter at the time.
Linus Neumann: And maybe I'm wrong here, then our listeners will correct me two minutes into the show.
Linus Neumann: But clearly, none of the large...
Meredith Whittaker: And I don't actually know for sure that story, so don't correct me.
Linus Neumann: It's my mistake.
Tim Pritlove: Ich habe einen anderen Punkt, aber ich denke, dass Twitter nicht sich als eine
Tim Pritlove: Messergebung und eine Messergebung und eine Messergebung was,
Tim Pritlove: was wahrscheinlich ein Mistake von der Ich meine.
Meredith Whittaker: Und der Regulatory Issues. Ja. Ich meine, ich habe mich mit der Security Team
Meredith Whittaker: und da waren immer diese Ideen,
Meredith Whittaker: ich habe immer ein Projekt wo wir uns an die Keystore für andere Services und
Meredith Whittaker: du hast diese wirklich exciting conversations mit den Security Guys and then
Meredith Whittaker: it wouldn't go anywhere because then, you know, the other guys would get involved.
Linus Neumann: Point I was trying to make is none of the big platforms ever rolled out mass
Linus Neumann: scale end-to-end encryption until eventually WhatsApp did.
Linus Neumann: And it took an Edward Snowden revelations and an ongoing scanner for I think
Linus Neumann: roughly one year until Facebook at the time said, okay, we'll implement this.
Meredith Whittaker: Well, actually, I mean, that's, So Signal existed before the WhatsApp integration.
Linus Neumann: Sure.
Meredith Whittaker: And the WhatsApp integration was driven by Brian and Jan, who are the co-founders.
Meredith Whittaker: And my understanding there is that they were rushing to get that done before
Meredith Whittaker: the Facebook integration, to make sure that they weren't selling something that
Meredith Whittaker: would violate their principles.
Meredith Whittaker: And I know Moxie was working on that. I remember that period of time.
Meredith Whittaker: Und dann Facebook bought it.
Meredith Whittaker: Und ich weiß nicht, dass die deal da ist, aber es bleibt, mit der Signals-Protokolle.
Meredith Whittaker: Es war ein Post-Snowden Moment.
Meredith Whittaker: Du sahst Android und iOS implementiert Full Disk Encryption.
Meredith Whittaker: Du sahst Google encrypting. its.
Meredith Whittaker: HTPS for its networks and I think a lot of this was just like we need to distance
Meredith Whittaker: ourselves from bad government spying by adding encryption that proves that we're
Meredith Whittaker: not actually part of the problem that they have just sort of,
Meredith Whittaker: the bad government has attacked us as taxpaying corporations,
Meredith Whittaker: taken this data that we really want to protect but god how could we have known
Meredith Whittaker: and so now we're encrypting things and,
Meredith Whittaker: ultimately good, right? But it was a way of not looking at the full story of
Meredith Whittaker: like, why is that there to begin with?
Meredith Whittaker: And what else is not being encrypted? What other data is being given over?
Meredith Whittaker: And why do you have the choice to do that to begin with instead of a more socially
Meredith Whittaker: beneficial, democratic process
Meredith Whittaker: of determining how we're comfortable with technology entering our lives?
Linus Neumann: I probably would have even had an even better or even stronger perspective on
Linus Neumann: it seems like the governments didn't really attack these corporations.
Linus Neumann: The governments made their business model look bad. So now they needed to change
Linus Neumann: something to convince people no, no, no, your data is safe here.
Linus Neumann: We'll encrypt something and moving on, moving on.
Meredith Whittaker: I was, for the gentle listener, I was kind of joking because that rhetoric around
Meredith Whittaker: being attacked and being like, oh my gosh, was Very much the mood at that time.
Meredith Whittaker: I was at Google when Snowden dropped and I remember just things popped off and
Meredith Whittaker: I actually had to get, I got on a plane.
Meredith Whittaker: I don't know, you all probably have a memory similar to me of like when the
Meredith Whittaker: Guardian stories with the Verizon water, like the Glenn Greenwald stories.
Meredith Whittaker: It was night in New York. It was probably morning the next day you guys saw
Meredith Whittaker: it. I remember sitting on my couch and being like, holy fuck.
Meredith Whittaker: And realizing just how big that was because it was the kind of thing we'd been
Meredith Whittaker: talking about, speculating about in the rooms that you and I were in, Linus.
Meredith Whittaker: And then it was like, oh, receipts, shit.
Meredith Whittaker: And there was a lot of unclarity. There was that Prism slide where it was like,
Meredith Whittaker: is Google just giving them full access? That people were rioting inside.
Meredith Whittaker: Security engineers were threatening to quit.
Meredith Whittaker: And then that morning I got on a plane to TorDev.
Meredith Whittaker: And so, yeah, in Berlin, actually.
Linus Neumann: I remember that this time,
Linus Neumann: and we at CCC really had to bite our tongue and write a media communications
Linus Neumann: strategy and said, nobody says I told you so.
Linus Neumann: We need to at least act surprised as well, right? And not say,
Linus Neumann: well, we told you all along.
Meredith Whittaker: I give you all a pass, because you did tell us all along.
Linus Neumann: But of course it gave a lot of international media attention to our cause.
Linus Neumann: And we needed to play that public attention wisely.
Meredith Whittaker: Exactly.
Linus Neumann: I think we maybe did.
Tim Pritlove: I find it quite interesting how encryption as a topic has changed over time. It's just...
Tim Pritlove: More or less 10 years ago that Facebook actually changed to HTTPS on their website by default.
Tim Pritlove: And so there was a time not so long ago, you know, where most of the data was
Tim Pritlove: flowing around on the Internet, mostly unencrypted.
Tim Pritlove: And that although there was these already mentioned crypto wars,
Tim Pritlove: you know, about general encryption, but it was also always for nerds and for
Tim Pritlove: specific applications.
Tim Pritlove: Then it also got this nice paint with this whole cryptocurrency craze going
Tim Pritlove: on, which made it somehow popular and almost took the word away.
Meredith Whittaker: We're so wrestling.
Tim Pritlove: And it was also the rise of encrypted messaging that was really giving a new fuel.
Tim Pritlove: So Signal was in the middle of all of this, as we already heard.
Tim Pritlove: So, I'd like to focus on Signal for a moment as an organization that you now head.
Tim Pritlove: What's your understanding of
Tim Pritlove: what Signal is and what it's not and how the organization deals with it?
Meredith Whittaker: Well, I love Signal and I'm really...
Meredith Whittaker: Yeah, it's the only cool tech company in my view. Und ich denke...
Meredith Whittaker: So boiling down, like what is Signal in one word is a little,
Meredith Whittaker: you know, I'll just, I'll start somewhere and we'll end another place because
Meredith Whittaker: I think it's actually, it's a number of things and kind of represents even more.
Meredith Whittaker: You know, Signal started back, you know, the late 2000s, right?
Meredith Whittaker: And we can, you know, we can date it to whenever, right?
Meredith Whittaker: Signal as the integrated app was 2013, but your Red Phone, TextSecure predated that.
Meredith Whittaker: And this is, you know, there's no iPhone.
Meredith Whittaker: Jabber is the competition, right? It's like web client-based chat.
Meredith Whittaker: There's no, you know, people aren't carrying smartphones. There isn't,
Meredith Whittaker: WhatsApp doesn't exist.
Meredith Whittaker: IMessage doesn't exist. You have a very, very different marketplace.
Tim Pritlove: Were there still ICQ and AIM, I think, at the time?
Meredith Whittaker: Yeah, yeah. I mean, I remember using, like, I don't remember,
Meredith Whittaker: I'd like send text messages sometimes, but they were expensive on my BlackBerry, maybe.
Meredith Whittaker: But we're talking about a drastically different tech ecosystem.
Meredith Whittaker: And this is particularly important in the context of messaging and communications apps.
Meredith Whittaker: Because, of course, you need a network effect for those to work.
Meredith Whittaker: No one buys the first telephone because you can't use one telephone.
Meredith Whittaker: Your friend has to have a telephone.
Meredith Whittaker: All your friends have to have an app if you're going to use it.
Meredith Whittaker: Particularly, it takes two to encrypt. Group encryption, it takes everyone in the group.
Meredith Whittaker: And if everyone isn't using your app for communication, it's very different
Meredith Whittaker: in a saturated marketplace where you have WhatsApp,
Meredith Whittaker: where you have iMessage, where you have these normative models that people go
Meredith Whittaker: to just to have their regular communication to introduce a new platform.
Meredith Whittaker: For secure messaging or insecure messaging, right?
Meredith Whittaker: Because people don't switch outside of the network unless their friends switch
Meredith Whittaker: outside of the network and there's a collective action problem there and an
Meredith Whittaker: inertia problem there and all of that, right?
Meredith Whittaker: So when I think about, there's many things that are very precious about Signal,
Meredith Whittaker: but the fact that Moxie carried it on his back for that decade and was actually
Meredith Whittaker: able to keep it going and surviving without selling out,
Meredith Whittaker: without selling data and actually creating something that is now able to scale
Meredith Whittaker: to hundreds of millions of people.
Meredith Whittaker: Means that Signal actually has a position in this ecosystem that makes it useful to people.
Meredith Whittaker: That means that it's actually providing encrypted communication to people all
Meredith Whittaker: over the globe because their friends are using it.
Meredith Whittaker: And my contention here, and I'm willing to discuss this, I don't think we can
Meredith Whittaker: recreate Signal, right?
Meredith Whittaker: You could shift, you know, because it has that user base, right?
Meredith Whittaker: You can introduce a new app, but how do you get people to use it without an
Meredith Whittaker: OEM, without an existing installed user base, without some way of,
Meredith Whittaker: you know, kind of making it useful to people?
Meredith Whittaker: Because again, it's, you know, one telephone or, you know, a couple thousand
Meredith Whittaker: hackers who all use PGP, but they can't talk to their dad on PGP, right?
Meredith Whittaker: They can't talk to anyone outside of themselves on PGP. So Signal has both sort
Meredith Whittaker: of kept this form that is very heterodox in tech.
Meredith Whittaker: It's a nonprofit, so I couldn't really sell out profitably.
Meredith Whittaker: I could try to sell out, but I'm not going to get any money from it.
Meredith Whittaker: It has to be reinvested in the mission, and there are certain rules around mission alignment there.
Meredith Whittaker: So we're not pressured to monetize surveillance or to collect any data.
Meredith Whittaker: We're really able to stay focused on that mission and in an ecosystem where.
Meredith Whittaker: The diametric opposite is the norm. That's really, really important,
Meredith Whittaker: irrespective about the flaws of the nonprofit model more generally.
Meredith Whittaker: So it's achieved this pretty rare, like it's the only thing like it in the ecosystem.
Meredith Whittaker: And I think it also serves as a model for how we could think about building tech differently.
Meredith Whittaker: Like, how do we disarm, deconstruct the massive centralized power of a handful
Meredith Whittaker: of platform companies that basically control most of the infrastructure and
Meredith Whittaker: information ecosystem in our world and our jurisdiction in the US?
Meredith Whittaker: And how do we build other models that may be interoperable, that are more open,
Meredith Whittaker: that are more rights preserving, and that aren't subject to the pressures of
Meredith Whittaker: and incentives of the surveillance business model?
Linus Neumann: Now, I find it interesting, you mentioned the network effect that for a long
Linus Neumann: time worked against you, right? You said, okay, it took Moxie a decade.
Linus Neumann: Now, that network effect, you know, the EU regulatory bodies have an idea on
Linus Neumann: how to weaken it by enabling messenger interoperability,
Linus Neumann: Thus, pretty much trying to force the large messenger operators,
Linus Neumann: be it WhatsApp or whatever else people use, to offer an interoperability interface.
Linus Neumann: So any new messenger would have it easier than Signal did to reach the critical,
Linus Neumann: amount of users to actually have the network effect.
Linus Neumann: Now I know Signal's position strongly opposes this idea or you decided not to
Linus Neumann: participate in this interoperability or making use of it.
Linus Neumann: There are two.
Linus Neumann: Forces or two goals of Signal, I guess, that contradict each other here,
Linus Neumann: and that is its security and building open communication systems.
Linus Neumann: Maybe you can explain a little bit why, after, you know, having understood how
Linus Neumann: hard it is to build Signal and, you know, against a market, why you would still
Linus Neumann: oppose messenger interoperability.
Meredith Whittaker: Yeah, I like this question because I'm going to, I want to clarify our position,
Meredith Whittaker: which is a little bit of nuance.
Meredith Whittaker: I don't oppose interoperability in principle.
Meredith Whittaker: Like if the interoperability mandate of the DSA were, you all have to,
Meredith Whittaker: you know, interoperability needs to be an option and it has to happen at this rigorous security bar.
Meredith Whittaker: You have to make sure that you're implementing metadata security, sealed sender.
Meredith Whittaker: Basically, you're adopting the
Meredith Whittaker: signal privacy and security bar as the conditions for interoperability.
Meredith Whittaker: That would be really cool, right? And I think that's the issue here.
Meredith Whittaker: I mean, and I want to put an asterisk here for a moment saying,
Meredith Whittaker: like, there are a lot of other complexities around policy.
Meredith Whittaker: Like, you know, how do you take in a, you know, who deals with a law enforcement request, right?
Meredith Whittaker: Even if you have no data to give them, you can't just ignore that.
Meredith Whittaker: You know, like, if a user has a complaint, who do they write in?
Meredith Whittaker: If you're interoperating with a platform for communication that also has a social
Meredith Whittaker: media arm, there's a totally different regulatory environment for Telegram or
Meredith Whittaker: WhatsApp with channels than for a signal.
Meredith Whittaker: How does that sort of work?
Meredith Whittaker: So I want to say this isn't simple.
Linus Neumann: It's a whole can of worms.
Meredith Whittaker: There's a massive can of worms as the EU often opens. Um, but the,
Meredith Whittaker: you know, like the conditions of interoperability are actually really,
Meredith Whittaker: you know, they're really political here. Right.
Meredith Whittaker: So in order to interoperate with WhatsApp, am I going to be giving signal user data to Meta?
Meredith Whittaker: Well, that would violate the entire premise of what I'm spending my life's energy
Meredith Whittaker: doing and Signal, right?
Meredith Whittaker: Is Meta going to decide to, you know, cut off the account of one of our users?
Meredith Whittaker: Who gets to decide that, right? Are they, you know, are they collecting,
Meredith Whittaker: you know, other data because they aren't implementing some of our libraries or whatever?
Meredith Whittaker: And so I think, you know, that's where the rubber meets the road.
Meredith Whittaker: And we have to have a duty of care to the people who rely on Signal.
Meredith Whittaker: Das heißt, wir sind absolut nicht going zu verabschieden, dass wir wissen,
Meredith Whittaker: wenn wir real sind, dass es eine Frau in der Jäule in den USA ist,
Meredith Whittaker: weil Meta in den Facebook-Messagen hat, zwischen ihr und ihr daughter,
Meredith Whittaker: die sie überprüfen hat, dass sie sie erhalten,
Meredith Whittaker: in der Stadt von Nebraska, nach dem Dobbs-Dobbs-Dobbs-Dobbs-Dobbs.
Meredith Whittaker: That's the stakes of this conversation, even when we're talking about the technical
Meredith Whittaker: details of interoperability, right?
Linus Neumann: Dobbs' decision is reverting Roe v. Wade, right?
Meredith Whittaker: Yeah.
Linus Neumann: Okay, so just for some of the listeners that are not that much into the US policy.
Tim Pritlove: Basically outlawing abortion.
Linus Neumann: Allowing states to outlaw abortion.
Meredith Whittaker: Access to life-saving healthcare that more than half the population may need.
Linus Neumann: So your answer in short is interoperability is fine,
Linus Neumann: but you do not see any path currently being debated that would result in upholding
Linus Neumann: the security promises that Signal has worked long for to be able to make to its user base.
Meredith Whittaker: We will continue to advocate for a path that raises that bar,
Meredith Whittaker: meets or exceeds Signal's bar. And if it succeeds, I'm like,
Meredith Whittaker: yeah, I want to talk about that.
Meredith Whittaker: But, you know, those are the conditions under which we would interoperate.
Meredith Whittaker: So we're not, you know, we don't take a stand against it.
Meredith Whittaker: You know, we just say like, look, these are the complexities and this is,
Meredith Whittaker: you know, Signal stands with the people who rely on Signal, not with a sort
Meredith Whittaker: of, you know, vision for some muddy middle where we're all interoperating,
Meredith Whittaker: but we've sort of, you know, sold people out and made them susceptible to,
Meredith Whittaker: you know, what we describe with Meta.
Tim Pritlove: Do people in the EU understand what you're talking about if you're offering
Tim Pritlove: these technical explanations why it's complicated?
Tim Pritlove: Because we have the impression that they don't really get it.
Meredith Whittaker: Is this mean or median, people?
Meredith Whittaker: I mean, I don't think most people don't understand this at all because they
Meredith Whittaker: got laundry to do and this isn't their area, right?
Meredith Whittaker: I think some of the politicians I've talked to seem to get it, but it's not...
Meredith Whittaker: Ich denke, dass es eine Inertia über das Prozess ist, das bedeutet,
Meredith Whittaker: dass es nicht klar, wie weit diese Punkte haben, die Bedrock zu haben, so zu sprechen.
Meredith Whittaker: Ich würde auch, als Amerikaner, ich bin hier, für heute.
Meredith Whittaker: Aber ich also habe, ich würde sagen, mein Instinkt auf die Generalentwicklung
Meredith Whittaker: der EU ist nicht was ich, was ich rely auf.
Tim Pritlove: Which brings me to an interesting point because we are actually very interested
Tim Pritlove: in your view as an American knowing how things work on that continent,
Tim Pritlove: What's your impression of how Europe deals with tech these new technologies
Tim Pritlove: coming up and how it impacts society Can you just give me a feeling for how this is to you,
Tim Pritlove: in a good way, in a bad way, whatever,
Tim Pritlove: you feel just to...
Meredith Whittaker: One meta-observation, having been here for a bit of duration now, is,
Meredith Whittaker: And I've thought this before, but it's interesting to think of Europe as one thing.
Meredith Whittaker: When you go to different countries and meet different people and you're like,
Meredith Whittaker: wow, this is not one thing.
Linus Neumann: I mean, that's the same in the US, right?
Meredith Whittaker: It is in the US, yeah. I mean, you know, it's big.
Tim Pritlove: But there is a common theme somehow. I'm just focusing on what you can probably
Tim Pritlove: match to Europe in general. or at least to the kind of discussions you have
Tim Pritlove: on a political level when you face EU institutions.
Meredith Whittaker: It's almost, it's split in an interesting way.
Meredith Whittaker: Because on one part, if you go to kind of the startup ecosystem,
Meredith Whittaker: the VC ecosystem, like that world, there's a real,
Meredith Whittaker: you know, there's a lot of smart people and cool people doing cool things.
Meredith Whittaker: And there's sometimes a bit of magical thinking that I see, which is really like,
Meredith Whittaker: you know, if we wish hard enough, if we're able to figure it out,
Meredith Whittaker: we're going to be able to create competitors to the US incumbents,
Meredith Whittaker: right? And we're going to have our own thing.
Tim Pritlove: Own search engine.
Meredith Whittaker: Which often just, you know, sometimes I'm just like, okay, that's a money play, right?
Meredith Whittaker: Like, you get enough in, you know, your Series A, Series B, and then you'll
Meredith Whittaker: get acquired, and no one will do anything with it, or you'll get rich, or whatever.
Meredith Whittaker: You know, like, you may not necessarily believe that. Like, markets float on Hype.
Meredith Whittaker: So, okay. But there is this thread where it's almost a willful misunderstanding
Meredith Whittaker: of the reality of incumbent platforms, of the history that accrued that type
Meredith Whittaker: of power to US companies,
Meredith Whittaker: and of the dependencies that Europe and most of the rest of the world have on these companies.
Meredith Whittaker: The three cloud companies based in the US have 70, 70% of the global market.
Meredith Whittaker: You have five major social media platforms.
Tim Pritlove: Amazon, Google and Microsoft.
Meredith Whittaker: Yeah, AWS, Azure, GCP. And then I think the other percentage are made up by
Meredith Whittaker: US companies as well. And then there's some Chinese companies.
Meredith Whittaker: And then you have five platforms that effectively shape our global information
Meredith Whittaker: ecosystem, like our perception of reality.
Meredith Whittaker: The four biggest jurisdictions in the US, right?
Tim Pritlove: Which five platforms?
Meredith Whittaker: Das ist, was ich werde, ich werde, ich werde, vier haben, denn da ist immer eine Last in der Liste.
Meredith Whittaker: So TikTok ist eine Non-U.S. one, und das ist die eine die Leute freigte aus, um, recently.
Linus Neumann: Weil es ist Non-U.S.?
Meredith Whittaker: Ja, ich denke, so flatly, ist es.
Meredith Whittaker: Facebook, Instagram, X.
Meredith Whittaker: Und dann, es ist nicht Twitch, aber irgendwie, da ist eine andere one auf YouTube.
Meredith Whittaker: Und dann ist Twitch, too.
Meredith Whittaker: Für all intents und purposes, das ist ein huge amount of concentrated power
Meredith Whittaker: that, again, relies on network effects, relies on economies of scale,
Meredith Whittaker: relies on all kinds of global infrastructure.
Meredith Whittaker: It's trillions of dollars that can't just be interrupted by investment. This is a kind of...
Tim Pritlove: This is social media companies.
Meredith Whittaker: Social media and platform companies, right?
Tim Pritlove: And isn't Telegram one of them?
Meredith Whittaker: Well, Telegram doesn't, I think, run most of their own infrastructure.
Meredith Whittaker: They don't have a cloud business model.
Meredith Whittaker: And they also don't really have a business model. It seems like they have this crypto play.
Meredith Whittaker: But it's not clear how that money moves. There's a lot of UAE investment.
Meredith Whittaker: So you're not talking about these big tech...
Tim Pritlove: Okay, you're focusing on cloud, not so much on the social media aspect.
Meredith Whittaker: And what is it difficult? Where is the normative shape of the tech industry coming from, right?
Meredith Whittaker: Like if the cloud companies all of a sudden decided to cut off half their APIs
Meredith Whittaker: and change their infrastructure, there's like most startups in the entire world,
Meredith Whittaker: including organizations like Signal, Telegram, whoever's writing on top,
Meredith Whittaker: all their engineers' pagers go off.
Meredith Whittaker: They got to respond to that, right? It's unidirectional that way.
Linus Neumann: Right?
Meredith Whittaker: If sanctions go up, let's do a wild dictator gets elected in the US and decides
Meredith Whittaker: that Europe is now sanctioned and that Amazon can't do business with Europe. What happens?
Tim Pritlove: Data in the US, that would never happen.
Meredith Whittaker: I make things up as a creative person.
Meredith Whittaker: Remember, I come from literature and rhetoric.
Tim Pritlove: Just stories in your head. Ja.
Meredith Whittaker: Und dann sieht man die Social Media Platforms, die wir wissen,
Meredith Whittaker: dass es ein far-right ist,
Meredith Whittaker: das wirklich hat die Intelligenz und Interessenzierung auf die Alternative Media
Meredith Whittaker: Ecosystems über die Plattformen und die Affordanzer von Surveillance-Advertising-Driven
Meredith Whittaker: Media Plattformen zu schaffen.
Tim Pritlove: Okay, back to the magical thinking of Europe.
Meredith Whittaker: Yeah. I'd like to understand how you see.
Tim Pritlove: So in this world.
Linus Neumann: There are like a few Europeans that say, here's 50 million, let's compete with these guys.
Meredith Whittaker: Yeah, yeah, yeah.
Tim Pritlove: And that's probably what the European search engine is.
Meredith Whittaker: It's like 500 million dollars European sovereign AI fund.
Meredith Whittaker: And then you're like, but that's half a training run. Like, what are you buying with that?
Meredith Whittaker: Which is like which is disturbing because it's like okay well that's a lot of
Meredith Whittaker: money also let's not be flip about it and it could be going to really good things
Meredith Whittaker: it could be supporting you know interesting open projects it could be supporting
Meredith Whittaker: you know alternative interoperable alternatives are like you know smaller clouds for more,
Meredith Whittaker: heterodox open source project like there's really cool stuff that is languishing
Meredith Whittaker: without that money and I think it's where is that money going?
Meredith Whittaker: Well if you're talking about going into AI it's going to one of those three
Meredith Whittaker: cloud companies it's renting infrastructure from Microsoft Amazon or Google
Meredith Whittaker: for model development or for deployment which is inference and inference is
Meredith Whittaker: really expensive you don't just train once,
Meredith Whittaker: you use a model using it is way way more expensive than normal information retrieval
Meredith Whittaker: and so you're also it's just this massive computationally expensive,
Meredith Whittaker: Und du hast nicht die Europäische Sovereinheit, du hast eine Gefühl, die...
Meredith Whittaker: I don't know, like, not behind, which, like, you know, and feeling of not being
Meredith Whittaker: ashamed by being technologically...
Tim Pritlove: But apart from the magical thinking, is there anything else you would stick to in Europe?
Meredith Whittaker: And then there's the other side, which is I often find a much more sophisticated
Meredith Whittaker: and clear-eyed view of these problems, right?
Meredith Whittaker: Like, having this discussion about that concentrated power in the hands of infrastructure
Meredith Whittaker: and media ecosystem, way easier in Europe.
Meredith Whittaker: I mean, people feel it, right? They see it. And there's been a history of pushing
Meredith Whittaker: back against the encroachment of US
Meredith Whittaker: tech, both effectively and often very ineffectively, that I really enjoy.
Meredith Whittaker: And particularly in Germany, there's a very high sensitivity to privacy,
Meredith Whittaker: very often clear-eyed view on some of these debates, which doesn't always translate into policy.
Meredith Whittaker: But there's at least, like I find the intellectual environment around this stuff,
Meredith Whittaker: when you talk to people who are knowledgeable and have thought about it,
Meredith Whittaker: to be very, I don't know, to teach me a lot and be really sophisticated.
Tim Pritlove: Yeah, the GDPR is probably a German thing somehow in its core, for sure.
Tim Pritlove: So how does this affect the talk to European politicians and how do you see
Tim Pritlove: the trends in regulations and trying to apply new laws and regulations towards
Tim Pritlove: this whole tech industry?
Meredith Whittaker: Ja, ich meine, es gibt beide threads.
Meredith Whittaker: Du hast zwei Wolves inside European Politik.
Meredith Whittaker: Der eine will seine eigene Tech-Industrie und der eine will sie nicht zu machen.
Meredith Whittaker: Das ist nicht zu US-Tech-Colonialismus.
Meredith Whittaker: Und ich denke, du hast du some weirden Laws.
Meredith Whittaker: Du hast die AI-Act, die aufwärtigte, die aufwärtigte, kind of had this last-minute
Meredith Whittaker: brinksmanship around whether foundation models.
Meredith Whittaker: These big LLMs that are now the trendy kind should be included or not.
Meredith Whittaker: And you often see bold regulatory attempts that then get kind of shaped in odd
Meredith Whittaker: ways, trying to have it both ways, right?
Meredith Whittaker: Like how do we regulate the Americans away and get our own, right?
Meredith Whittaker: But how do we do that in a way that it is reflected in principles,
Meredith Whittaker: not in actually declaring that as an intent.
Meredith Whittaker: And I think that is, I think you're seeing a huge amount of money be spent by
Meredith Whittaker: the US companies in Brussels right now, which is also influencing things in interesting ways.
Meredith Whittaker: And then this is something I'm theorizing a lot in my intellectual work,
Meredith Whittaker: or just, and I think is really important.
Meredith Whittaker: You also see what I'm calling the politics of intellectual shame.
Meredith Whittaker: Be really pervasive in this conversation and this is not just Europe this is across the board.
Meredith Whittaker: I mean that there is a real.
Meredith Whittaker: A real fear among a lot of people who are in decision-making positions,
Meredith Whittaker: politicians or academics or whoever,
Meredith Whittaker: and not even in decision-making positions, but it matters when it's them,
Meredith Whittaker: of being stupid about tech,
Meredith Whittaker: of being behind the ball on tech.
Meredith Whittaker: And this plays right into patriarchal dynamics.
Meredith Whittaker: Men hate when someone else knows something more than them, in particular if that's a small woman.
Linus Neumann: I think we enjoy it right now.
Meredith Whittaker: Ja, ja, ja. Totally.
Linus Neumann: Wir haben eine gute Zeit.
Meredith Whittaker: Ja. Well, you all are, generally, and I don't want to gender this in such a
Meredith Whittaker: schematic, but there is an ego that can be very, very fragile here.
Meredith Whittaker: And the way I've put it before is it kind of turns uncertain men into yes men.
Meredith Whittaker: Like, they don't want to ask the dumb question. They don't want to be like,
Meredith Whittaker: what's an LLM? What's a server? Like, how does that work?
Meredith Whittaker: And that type of insecurity, The fear of being behind, the fear of being called
Meredith Whittaker: technically unsophisticated or hampering progress or putting your finger on
Meredith Whittaker: the scales of science, look the Nobels, how could you stand in the way of all this progress?
Meredith Whittaker: I think really gives the upper hand to the companies and those who have an interest
Meredith Whittaker: in creating products and growth and domination via these technologies.
Meredith Whittaker: Because people really don't want to challenge them because challenging their
Meredith Whittaker: dominance or their plans gets conflated with somehow being anti-science or being
Meredith Whittaker: stupid about tech or not being smart enough to have a position on a topic.
Meredith Whittaker: And I think that's something because I kind of came up through Google asking
Meredith Whittaker: every dumb question in the book because I had no I didn't come from that world.
Meredith Whittaker: Right. So I had to ask, like, how does a computer work? Right.
Meredith Whittaker: I'm like, I'm like, can someone diagram what a function is? I don't know any of this stuff. Right.
Meredith Whittaker: But I kind of I think I have a sensitivity to that because I also I remember feeling it.
Meredith Whittaker: I remember people being mean about it. Like if I didn't, you know,
Meredith Whittaker: like back in the day when I was trying to learn this stuff.
Meredith Whittaker: And I think that, you know, a discourse that collapses scientific progress into kind of.
Meredith Whittaker: The success of a handful of tech companies preys on that type of insecurity
Meredith Whittaker: and has created an environment in which people have no idea what AI is and are
Meredith Whittaker: still professing boldly on how to regulate it.
Tim Pritlove: So I read it as you think that the European positions might be slightly under-informed
Tim Pritlove: and probably not well thought out in the current situation.
Meredith Whittaker: I should be clear. I don't think that what I was describing and the politics
Meredith Whittaker: of intellectual shame are not unique to Europe, but I think are in Europe as well.
Meredith Whittaker: And particularly folks who feel like, you know, the Americans beat us, we got to get ahead.
Meredith Whittaker: Right. I think where I see the European position being most, let's say,
Meredith Whittaker: under-informed or perhaps just in some cases pernicious is in the chat controls
Meredith Whittaker: regulation and the desire,
Meredith Whittaker: the apex of magical thinking, Which is, you know, let's rename a backdoor client-side
Meredith Whittaker: scanning and then let's mandate scanning everyone's private messages,
Meredith Whittaker: comparing what's in those messages against some database of permissible or impermissible
Meredith Whittaker: content and then, you know, taking action on those, you know,
Meredith Whittaker: in the name of protecting children,
Meredith Whittaker: which is the justification during this instantiation of the crypto wars.
Tim Pritlove: Let's stick to this topic, because it's still an ongoing battle right now.
Tim Pritlove: We are more or less talking about this in every of our shows.
Tim Pritlove: And yeah, it's still totally unclear what's going to come out of this.
Tim Pritlove: How do you see this discussion evolve?
Meredith Whittaker: Well, I don't, you know, I see this as an ongoing power struggle, right?
Meredith Whittaker: Between who? Well, between, like this is not a misunderstanding.
Meredith Whittaker: I think a lot of the people pushing for this understand that backdoors are dangerous
Meredith Whittaker: and understand that, you know, the pretext is flimsy, but that, you know.
Meredith Whittaker: Asymmetric power constitutes itself in part through information asymmetry.
Meredith Whittaker: And there's a deep discomfort that dates back to 1976 when Diffie Hellman were
Meredith Whittaker: trying to publish their paper introducing public cryptography and the U.S.
Meredith Whittaker: Government was trying to suppress it, trying to say, don't publish this, right?
Meredith Whittaker: And then, you know, but databases weren't quite big enough, networks weren't
Meredith Whittaker: quite big enough or ubiquitous enough for it to matter, matter.
Meredith Whittaker: But they were already looking at like, oh, shit, we don't want this in the public,
Meredith Whittaker: right? And then you go through the 90s and there's the Clipper chip and key
Meredith Whittaker: escrow and you have Stuart Baker writing in Wired magazine, like PGP is just
Meredith Whittaker: for terrorists. We have proof.
Meredith Whittaker: No, PGP is for pedophiles, right? Which really echoes what we're hearing now.
Meredith Whittaker: Right? Like who even has a computer in 1994, I believe, when this op-ed is written?
Meredith Whittaker: And then we have Post 9-11 and then it's like, actually, PGP is for terrorists, right?
Meredith Whittaker: And encryption is for terrorists. All the while, our dependency on digital infrastructures
Meredith Whittaker: for communications is growing and growing and growing.
Meredith Whittaker: Our dependency on digital infrastructures generally is growing.
Meredith Whittaker: And the need for encryption to protect commerce becomes existential to the Internet industry.
Meredith Whittaker: And then what do you do about communications, right? And I think this has been
Meredith Whittaker: an anxiety that is pervasive among those who, you know, law enforcement,
Meredith Whittaker: governments, whoever, who feel that they need to constitute their power via
Meredith Whittaker: information asymmetry.
Meredith Whittaker: And any encryption that protects people, not just commerce, is a threat to that, right?
Meredith Whittaker: And so what I don't see is that we're going to win an argument,
Meredith Whittaker: right, or that we're going to win this via strength of argument.
Meredith Whittaker: I do think we can fight and we, you know, I think, I think we're in a position
Meredith Whittaker: now where we're seeing, we're seeing chat controls, you know,
Meredith Whittaker: I believe Hungary just tried to raise it and didn't get the support.
Meredith Whittaker: There was the Belgian proposal a few months ago, also didn't get the support at the last minute.
Meredith Whittaker: And we just had the Dutch law enforcement authorities writing a memo to the
Meredith Whittaker: government there saying, yo, don't support this.
Meredith Whittaker: You're talking about a very dangerous backdoor that would undermine Dutch cybersecurity,
Meredith Whittaker: right? At the same time, we have reporting in the Wall Street Journal...
Meredith Whittaker: Es gibt einen Receit für was, was all of us should have suspected all along,
Meredith Whittaker: ist, dass die Backdoors, die in den US-Telecommunications Infrastruktur,
Meredith Whittaker: für Government Intercept, haben wir von China-Intelligence und vielleicht anderen.
Meredith Whittaker: Ich denke, an dieser Moment, wir haben ein vieles...
Meredith Whittaker: Die Facts sind auf unserer Seite, und die Facts sind auf der Seite,
Meredith Whittaker: ist permeatet in diese Diskussion, and making it harder and harder for them
Meredith Whittaker: to push it forward in the European Commission.
Linus Neumann: But that usually means they just do another attempt next year.
Meredith Whittaker: Exactly. And that's why I think we're not going to win. There's going to be
Meredith Whittaker: another pretext if we win this one, right?
Meredith Whittaker: There's going to be another angle if we win this one.
Meredith Whittaker: We just have to keep building our muscle to sustain this fight probably forever,
Meredith Whittaker: because I don't think the will to power is going away.
Meredith Whittaker: I think they're just going to keep trying to rearrange the reasoning.
Tim Pritlove: Aber wie geht es? Wenn Sie sagen, dass die Strecke der Argument nicht genug ist, ist es nicht genug?
Meredith Whittaker: Ich mache Yoga every day.
Tim Pritlove: Does es helfen?
Meredith Whittaker: Ich meine, yes, es hilft.
Tim Pritlove: Ich meine, in terms of politische Diskussionen.
Meredith Whittaker: In terms of politische Diskussionen, es hilft, dass wir richtig sind.
Meredith Whittaker: We bring in, there's a huge amount of evidence that a lot of people haven't
Meredith Whittaker: seen in these political discussions.
Meredith Whittaker: I think we're on the, our side has been on the back foot for a while.
Meredith Whittaker: There has been just in civil society, there's been cutting funding to privacy
Meredith Whittaker: advocacy has happened since around, there's a sort of a history here.
Meredith Whittaker: I think there's a move toward tech accountability happened after the 2016 election.
Meredith Whittaker: There's the Cambridge Analytica scandal. There's all of this.
Meredith Whittaker: And it's like, okay, we need to hold tech accountable.
Meredith Whittaker: And then there are a number of – the way to hold tech accountable is to attack
Meredith Whittaker: the business model, is my view.
Meredith Whittaker: But there aren't that many pieces of legislation or proposals that actually do that.
Meredith Whittaker: Many of them sort of use the wrapping and the language of accountability.
Meredith Whittaker: But are actually just expanding surveillance, right?
Meredith Whittaker: It's like, we're going to hold them accountable, so we need a database,
Meredith Whittaker: so we need to know who's logging into websites so we can find the bad guy.
Meredith Whittaker: We need to know what's in your messaging so that we can make sure that these
Meredith Whittaker: tech companies aren't allowing crime on their platforms, et cetera, et cetera.
Meredith Whittaker: So it was basically a hijacking of this, in many cases, kind of righteous moment
Meredith Whittaker: where people recognize that this business model was pretty harmful,
Meredith Whittaker: to fulfill the wishes that have been pervasive since well before then.
Meredith Whittaker: At the same time that we're seeing privacy advocacy and a lot of those,
Meredith Whittaker: you know, a lot of the things Linus, you and I had been doing for a long time,
Meredith Whittaker: receiving less and less support and sort of, you know, out of the limelight.
Meredith Whittaker: And so I think it was in that environment that things like chat controls,
Meredith Whittaker: that things like the online safety bill and other, you know,
Meredith Whittaker: paradigmatic examples of this, you client-side scanning to save children meme,
Meredith Whittaker: grew up and then one of the reasons I was,
Meredith Whittaker: There are many, many reasons I
Meredith Whittaker: decided to move from being on the board of Signal to full-time at Signal.
Meredith Whittaker: One of them was that I saw this moment and I realized there weren't that many
Meredith Whittaker: people fighting it and that one of the things that I could bring was a staunch
Meredith Whittaker: willingness to fight it.
Tim Pritlove: And how do you do that? I mean, can you walk us through a day?
Meredith Whittaker: I open up my laptop. Ja, um, um, well, we, you know, it's not,
Meredith Whittaker: obviously, it's not just me, nothing like this is a singular thing.
Meredith Whittaker: We work with a pretty broad coalition of folks. I'm sure, you know,
Meredith Whittaker: many of your friends, many listeners, perhaps, are part of that.
Meredith Whittaker: Um, Signal doesn't have a policy arm. It's a very kind of lean,
Meredith Whittaker: targeted, pretty senior organization.
Meredith Whittaker: But we do work with people around the globe EDRI and the EU a number of other
Meredith Whittaker: organizations to keep tabs on what's happening we also are in a good position.
Meredith Whittaker: We're a non-profit we are very committed to rigorous communication so we don't
Meredith Whittaker: have a history of hyper-marketing we don't do hyper-marketing now and so we're
Meredith Whittaker: very careful when we make a claim When we make a statement,
Meredith Whittaker: we're backing that with citations. It's accurate.
Meredith Whittaker: We're really marshalling the technical knowledge and prowess that we have.
Meredith Whittaker: I almost think of it as clarifying the record. If there's a report that says
Meredith Whittaker: client-side scanning is actually safe, we know it's not safe.
Meredith Whittaker: Okay, well, there's an academic coalition that has written this letter.
Meredith Whittaker: Signal can write a letter. We can begin to put a bit more weight on the scales
Meredith Whittaker: that have been fairly light, given the dynamics I just outlined.
Meredith Whittaker: And then I do media, I do public speaking.
Meredith Whittaker: I think a lot about how to tell this story in a way that isn't boring or alienating
Meredith Whittaker: for regular people, particularly because the story on the other side is so arresting.
Meredith Whittaker: It's like we have to save children from abuse.
Meredith Whittaker: And that every one of us it like hits you in the heart, right?
Meredith Whittaker: Like myself, right? Like my amygdala is activated.
Meredith Whittaker: I, you know, suddenly I just want to do something. I want to help,
Meredith Whittaker: you know, give me the thing to do. How do we do that, right?
Meredith Whittaker: And then sitting across from that and being like, well, let me tell you about
Meredith Whittaker: a one-way function, right? Like you can't, that's not.
Meredith Whittaker: That's not going to work, right? And so like, how do you,
Meredith Whittaker: How do you enter into that debate in a way that isn't dismissing the very grim
Meredith Whittaker: and real problem that is being evoked and make it clear that the solution to
Meredith Whittaker: that problem that is being presented will not solve that problem, one,
Meredith Whittaker: and two, will cause drastically worse problems for many people around the world?
Meredith Whittaker: And that's the task at hand right now.
Tim Pritlove: So basically the discussion is
Tim Pritlove: led by pointing the other side to like the infeasibility of the approach.
Meredith Whittaker: Well, the infeasibility, the danger of the approach,
Meredith Whittaker: a lot of evidence around the infeasibility of the approach that is either kind
Meredith Whittaker: of willfully ignored or just not understood,
Meredith Whittaker: and then figuring out how we explain that without being either accidentally
Meredith Whittaker: or genuinely callous about the concerns that have brought people to the table.
Tim Pritlove: Then they will say, but we have to do something.
Meredith Whittaker: Well, how about funding social services?
Meredith Whittaker: How about, you know, what do you do? And I mean, like, if we're going to go
Meredith Whittaker: there, we're going to go there.
Meredith Whittaker: Prince Andrew's walking around.
Tim Pritlove: No, you're suggesting to make the world better.
Meredith Whittaker: Jimmy Savile's walking around. You know, like what are the infrastructures in
Meredith Whittaker: place to make sure that when children are going through this,
Meredith Whittaker: they're believed, they're protected.
Meredith Whittaker: You know, what happens when it's your priest? What happens when it's your teacher?
Meredith Whittaker: What happens when it's your brother, right? Like these are the questions that
Meredith Whittaker: are really hard to look in the face because they implicate social pathologies
Meredith Whittaker: and interpersonal relationships and power dynamics that are really,
Meredith Whittaker: really difficult and often relate to emotionally challenging factors outside
Meredith Whittaker: of that or people's past experiences or what have you.
Meredith Whittaker: So you're going right into very traumatic subjects.
Meredith Whittaker: But I don't think we can have that conversation without having a real conversation.
Meredith Whittaker: And then when you begin to pull back the layers there, you say,
Meredith Whittaker: oh, well, the UK has been pushing for client-side scanning as a remediation to child abuse.
Meredith Whittaker: But the UK government in 2023 funded social services at 7% of the amount recommended.
Meredith Whittaker: Right? So like there's, you know, the roofs on the schools in the UKs are collapsing.
Meredith Whittaker: Like, you know, there isn't support for this.
Meredith Whittaker: And then if you, you know, if you look at, and I don't have public numbers to
Meredith Whittaker: share, but I've had a number of personal conversations.
Meredith Whittaker: Okay, well, how many law enforcement people, how many people are tasked with
Meredith Whittaker: actually sort of pursuing the,
Meredith Whittaker: you know, like the criminality that may be reported via online imagery?
Meredith Whittaker: In some cases, it's two. In one case, it's two in one country. Two people, right?
Meredith Whittaker: So, like, you're not actually, like, if you begin to map this,
Meredith Whittaker: what you see is a story that does not add up.
Meredith Whittaker: And you see, like, what, you know, and this is where I get enraged.
Meredith Whittaker: Because I'm like, you are fucking trading...
Meredith Whittaker: On children's pain, to get your backdoor, whatever the fuck you want,
Meredith Whittaker: pretending that you're solving it, so taking up the space for actual solutions
Meredith Whittaker: that could actually help real children who are suffering now,
Meredith Whittaker: and turning no attention to every glaring problem in this massive list,
Meredith Whittaker: which is pretty obvious, even for me, and I'm not an expert here,
Meredith Whittaker: I've just sort of sifted through this.
Meredith Whittaker: So I think that's the dynamic we're walking into.
Tim Pritlove: They might only have two people, but at some point in time they might have doubled,
Tim Pritlove: by adding another one.
Linus Neumann: I agree that, I mean, it is quite telling how much emphasis is being laid on,
Linus Neumann: hey, we really need client-side scanning and then the world is going to be safe.
Linus Neumann: And, you know, if you say, well, how about we, you know, we fund support or
Linus Neumann: any kind of prevention, preventive activities in um social care it's like yeah.
Meredith Whittaker: Yeah yeah well sorry we use the prefix online so that's not our,
Meredith Whittaker: um and i think it's also like there's something people gravitate to the abstraction
Meredith Whittaker: right if this is online child abuse then we don't have to deal with it in our
Meredith Whittaker: real lives it becomes an abstraction that we can almost blame on the same platforms
Meredith Whittaker: that have been so unaccountable we can blame it as an internet phenomenon, not a phenomenon.
Meredith Whittaker: And like, oh wait, our church doesn't have the infrastructure to actually deal
Meredith Whittaker: with this in a humane way, right? And I think that's a dynamic that we're also seeing here.
Linus Neumann: This is, by the way, one thing I find so interesting about Signal as a secure messenger.
Linus Neumann: Well, it has become mainstream, but it has also managed to maintain a reputation
Linus Neumann: of goodness, right? I mean, saying like, okay, text me on signals.
Linus Neumann: Oh yeah, that's the secure messenger, blue symbol, looks nice,
Linus Neumann: you know, very friendly user interface.
Linus Neumann: Or it would be like, let's text on 3MAR, be like, oh, that's the complicated black one.
Linus Neumann: Well, how about Telegram? And that's like, okay, that's a completely different end of the internet.
Linus Neumann: And that makes me think of the curious case of Pavel Durov being detained in
Linus Neumann: France and apparently at least charged because they refused to cooperate in numerous cases.
Linus Neumann: Why do you dare to come to Europe, Meredith?
Meredith Whittaker: Well, I'm a brave person, Linus. Aber, zu sagen, ich denke,
Meredith Whittaker: This is one of the places where more public education is necessary because Telegram
Meredith Whittaker: is actually, you said the other end of the internet, it is very,
Meredith Whittaker: very, very different from Signal.
Meredith Whittaker: So Telegram is a social media platform. It allows mass broadcasts to millions
Meredith Whittaker: of people. You can go viral on Telegram.
Meredith Whittaker: You can find strangers on Telegram via directories.
Meredith Whittaker: There's a near-me feature that will geolocate things happening near you,
Meredith Whittaker: all sorts of things that are not private, are not secure,
Meredith Whittaker: are regulated completely differently from private and secure communications
Meredith Whittaker: like Signal, which is solely a private and secure interpersonal communications app.
Linus Neumann: What about Signal Stories, though?
Meredith Whittaker: Signal Stories don't go viral. If I sent all the people in my contact list one
Meredith Whittaker: by one a photo or something...
Linus Neumann: I don't get your stories.
Meredith Whittaker: They're so cute, Linus. I wish, like, it's, you know.
Tim Pritlove: I hate it.
Linus Neumann: I mean, that feature was implemented. I was like, okay, where can I turn it off?
Meredith Whittaker: I'm sorry. Well, you're all missing my stories is all I'm saying,
Meredith Whittaker: and they are pretty good.
Tim Pritlove: Have you sent us some? No.
Meredith Whittaker: Well, have you activated them to check? But we do let you deactivate them forever
Meredith Whittaker: and never bother you about them, which is, you know, part of the way that I
Meredith Whittaker: think we maintain our reputation for not being shitty is we try to literally not be shitty.
Tim Pritlove: Yeah, thanks.
Meredith Whittaker: Right? And when we're designing Signal, we're actually very,
Meredith Whittaker: very careful not to be a social media platform.
Meredith Whittaker: We think about that in the design phase so everything we do can be as encrypted as possible.
Meredith Whittaker: So that we don't know anything about you or as close to zero about the people
Meredith Whittaker: who use Signal as possible.
Meredith Whittaker: And what we do know is we can say, yes, this phone number did sign up for a
Meredith Whittaker: Signal account. We know when that phone number signed up for a signal count
Meredith Whittaker: and we know last time they accessed it.
Meredith Whittaker: Um, but we don't, you know, we, we would like to even not know that if it were possible.
Meredith Whittaker: On the other hand, Telegram is a social media platform, which retains huge amounts of data,
Meredith Whittaker: has a duty to under law to cooperate in turning over that data and was search
Meredith Whittaker: functions that has search functions, has, you know, directories so you can find new things.
Meredith Whittaker: So it's a, it's a very different beast. And I think, one, because Durov has, you know, been very.
Meredith Whittaker: I'm trying for a diplomatic word, like, has made statements that are not supported
Meredith Whittaker: by fact around Telegram being private and secure and kind of taken on this,
Meredith Whittaker: you know, like yeoman's defender of free speech and privacy position.
Meredith Whittaker: People often think Telegram is private and secure because it has a DMs feature, right?
Meredith Whittaker: But, you know, Signal is just private and secure. So the TLDR on that is,
Meredith Whittaker: there's really no danger for Signal here because we are very,
Meredith Whittaker: very far away from Telegram.
Meredith Whittaker: And we have set ourselves up so that one, such cooperation isn't required.
Meredith Whittaker: And two, such cooperation is not possible.
Meredith Whittaker: Because we literally, like you could put a gun to my head, I don't have that
Meredith Whittaker: data. Whereas Telegram has servers and servers and servers full of that data.
Tim Pritlove: Leaving out Signal completely now, what do you think happened?
Tim Pritlove: Pawelldorf was put into custody. He's now free on bail.
Tim Pritlove: And France talked to him.
Meredith Whittaker: I have no idea. I mean, this is like...
Linus Neumann: Didn't he send you like a telegram message?
Meredith Whittaker: I haven't checked.
Tim Pritlove: You missed his story.
Linus Neumann: Yeah.
Meredith Whittaker: I mean, this is like overlaid the French legal system.
Meredith Whittaker: I am not a lawyer, especially not in France. Some of the vagaries of their legal
Meredith Whittaker: system in which, you know,
Meredith Whittaker: any judge can open an investigation and the basis for the charges will not be
Meredith Whittaker: known until, you know, trial.
Meredith Whittaker: And we're looking at years and years until then, with me not speaking French
Meredith Whittaker: well at all with like weird translation, you know, like, so I want to stay away
Meredith Whittaker: from speculating there.
Meredith Whittaker: But what it looks like based on the charges that were released in the press
Meredith Whittaker: release is that it was, you know, failure to comply with requests for data and
Meredith Whittaker: then, you know, kind of a handful of other charges added on that aren't as severe as those.
Tim Pritlove: So does Signal get these requests too?
Meredith Whittaker: You can go to Signal.org slash Big Brother and every request that we have been
Meredith Whittaker: forced to comply with because we fight them and have unsealed are posted there
Meredith Whittaker: showing exactly how close to no data we are able to turn over and showing,
Meredith Whittaker: and I think this is interesting for some of your listeners probably,
Meredith Whittaker: You see what the law enforcement agencies in these requests are requesting,
Meredith Whittaker: and it's often huge lists,
Meredith Whittaker: massive amounts of data, which gives you a sense of just how much data surveillance,
Meredith Whittaker: like a telegram or another platform,
Meredith Whittaker: is commonly able to provide that Signal is not.
Linus Neumann: Oh, so it's actually every single request. I thought it would maybe be aggregated.
Meredith Whittaker: Yeah, no, it's PDFs of the request. It's not a transparency report, it's transparency.
Tim Pritlove: I like that.
Meredith Whittaker: Although it's just the ones that we can unseal, so we do have to go through that.
Linus Neumann: I would assume there are probably some with gag orders, right? be able to say?
Meredith Whittaker: There's no gag order on me being able to say.
Meredith Whittaker: And there are some, but that's the fight to unseal it. And what I don't recall
Meredith Whittaker: the answer for right now is whether we're in one of those fights right now or not.
Meredith Whittaker: But I'd have to check with my friends at the ACLU.
Tim Pritlove: So Signal as a company, how does it work?
Meredith Whittaker: It's a non-profit. So we're funded by donations only
Meredith Whittaker: only yeah um and we
Meredith Whittaker: are thinking about in the future maybe having a
Meredith Whittaker: paid tier for some features something like backups encrypted
Meredith Whittaker: backups which we're you know building right now um you know could we charge
Meredith Whittaker: people for media storage or other expensive features but um you know that would
Meredith Whittaker: be in addition to donations and squarely within the non-profit structure that
Meredith Whittaker: keeps us safe from pressure to surveil.
Tim Pritlove: And how do you get your talent? How do you get people to work for Signal?
Meredith Whittaker: We pay very well. And we are, I mean, it's a really cool mission, right?
Meredith Whittaker: So imagine the jobs in tech are kind of depressing in many cases.
Meredith Whittaker: Not everyone wants to go optimize an ad server and then Signal,
Meredith Whittaker: you know, Signal you can work for core infrastructure for dissent and human
Meredith Whittaker: rights work and journalism around the world that, you know, without which a
Meredith Whittaker: lot of those things would be deeply imperiled.
Meredith Whittaker: Like it's a real cool thing to get to do and support. And we pay well.
Linus Neumann: So not only do you get respected for your six-digit salary, but for doing good
Linus Neumann: thing, earning that money.
Meredith Whittaker: Es ist die original Silicon Valley dream.
Tim Pritlove: So ist es der letzte Platz in tech where people are actually happy?
Meredith Whittaker: Well, I would never presume to speak to the consciousness of another person without there.
Meredith Whittaker: But I think, yeah, I am very happy. I think a lot of the people who are at Signal are very happy.
Meredith Whittaker: And I think it's also like we're kind of part of a project. And this is...
Meredith Whittaker: Shows that what we have in tech, what's built in the tech industry is not inevitable.
Meredith Whittaker: There's a series of choices, a series of incentives, a business model that has
Meredith Whittaker: shaped tech into the form we have now, but it does not have to be that way, right?
Meredith Whittaker: Like we can rewrite the stack, we can build alternatives.
Meredith Whittaker: Nonprofits can work, right? We need capital, we need will, we need talent,
Meredith Whittaker: we need all of those things.
Meredith Whittaker: But the thing we have now is not inevitable. And I think of Signal as like a
Meredith Whittaker: keystone species in the ecosystem, Kind of like, you know, like setting the
Meredith Whittaker: bar, kind of regulating the rest, right? Like, you know, you can have privacy.
Meredith Whittaker: You can have, you know, the right to private communications.
Meredith Whittaker: You can subsist outside of this paradigm.
Meredith Whittaker: And I think the future I want is that it's not just Signal, right?
Meredith Whittaker: There are many, many other organizations and efforts sort of doing it differently,
Meredith Whittaker: rejecting that paradigm, you know, drawing in capital there and,
Meredith Whittaker: you know, away from the other place.
Meredith Whittaker: Und beginnt zu marscheln die type of political will that is often very shallow
Meredith Whittaker: like the 500 million AI fund but marscheln it for something that is actually
Meredith Whittaker: substantive and is actually,
Meredith Whittaker: making the kind of change to the tech ecosystem that I think we need to have a livable world.
Tim Pritlove: You mean not only a model for other communication companies but also a model
Tim Pritlove: for any kind of technology company?
Meredith Whittaker: Ja, well, I mean, we build a communications app, but we rely on telecom networks.
Meredith Whittaker: We rely on server infrastructure. We rely on core libraries.
Tim Pritlove: That's not what I meant. I mean, I understood you that you think that the modus
Tim Pritlove: operandi of Signal as a company might be something that other companies could also leverage and do.
Tim Pritlove: It's not only limited to some much-needed device.
Meredith Whittaker: I do think that there's a model there. I think I'm interested right now in researching
Meredith Whittaker: hybrid structures and tandem structures.
Meredith Whittaker: Are there sort of for-profit,
Meredith Whittaker: Are there areas of tech that aren't driven by surveillance?
Meredith Whittaker: Are there ways you could fund nonprofits, fund some of this core infrastructure,
Meredith Whittaker: libraries and other things that have been languishing for decades?
Meredith Whittaker: How do you revitalize that? And then are there, you know, are there ways to
Meredith Whittaker: build truly independent infrastructure outside of the, you know,
Meredith Whittaker: three companies, five platforms model that, like, I think it's just clearly
Meredith Whittaker: critically dangerous at this point.
Linus Neumann: So, when it's about building infrastructures in our hands, that's not going
Linus Neumann: to be easier in an AI world, right?
Linus Neumann: Where it's model data that we need, huge investments into these models just
Linus Neumann: for training, then for operating them.
Linus Neumann: Do you see any future for this whole AI thing in users' hands,
Linus Neumann: in our hands, serving our actual privacy needs and let's say private and business needs?
Meredith Whittaker: Well, I think my answer to that is that that future will rely on laying an independent
Meredith Whittaker: infrastructural bedrock and actually transforming some of the way we govern
Meredith Whittaker: and think about digital technology generally,
Meredith Whittaker: Including being really attentive to things like how is data created?
Meredith Whittaker: Who gets to decide what data we use to reflect our complex lives and realities?
Meredith Whittaker: Who gets to decide how patterns in that data are made sense of?
Meredith Whittaker: What analysis is done to that data? And then what we do with the sense we make
Meredith Whittaker: of it, right? What decisions we make, right? And so...
Meredith Whittaker: We do all that, we transform what AI is and what it means.
Meredith Whittaker: Because you're no longer just scraping all the detritus off the stupid web,
Meredith Whittaker: which was deposited or created via this surveillance business model,
Meredith Whittaker: packaging that in an LLM and calling that intelligence, right?
Meredith Whittaker: You're actually having to grapple with the epistemic process by which data becomes
Meredith Whittaker: a proxy for reality, and that proxy shapes our lives and institutions.
Meredith Whittaker: And so I think AI itself, right now we're talking about these massive models, right?
Meredith Whittaker: This laws of scale, this sort of like big American guy dream of the,
Meredith Whittaker: you know, the largest in the world.
Meredith Whittaker: But AI is a, you know, it's a very slippery term. It's not a technical term
Meredith Whittaker: of art. It can apply to many, many different things. And there are small models.
Meredith Whittaker: There are sort of, you know, heterodox approaches. There are expert systems,
Meredith Whittaker: which they're now trying to bolt onto the side of generative systems.
Meredith Whittaker: Because weight, probabilistic answers aren't true, so we need to bolt truth
Meredith Whittaker: back on and we're kind of repeating a lot of the history of AI,
Meredith Whittaker: kind of speedrunning it in the search for a business model.
Meredith Whittaker: So my answer there is that a lot of the things that need to be done to simply
Meredith Whittaker: disarm and draw down the centralized power of these surveillance and infrastructure companies
Meredith Whittaker: are the same things that would need to be done to redefine, in a sense,
Meredith Whittaker: what AI is and our relationship to how truth, how, you know,
Meredith Whittaker: decisions, how, you know, I don't want to use the word truth actually,
Meredith Whittaker: but like how information is made via analyzing data and who gets to control that.
Meredith Whittaker: And I think, you know, my sensitivity to data in that answer comes directly
Meredith Whittaker: from my measurement experience, right?
Meredith Whittaker: Like where you, you know, one upgrade to the Linux kernel across our server
Meredith Whittaker: fleet fundamentally changed.
Meredith Whittaker: The kind of data we were able to create, like how it populated the schema.
Meredith Whittaker: And meant that that data wasn't necessarily fungible with the data collected
Meredith Whittaker: on the older version of the kernel, right?
Meredith Whittaker: And in order to solve that problem, I had to get a guy to go sit with the kernel
Meredith Whittaker: maintainers for like two years to make sure that the update wasn't going to
Meredith Whittaker: fuck up the way we got TCB DOMS, basically.
Meredith Whittaker: Um so like that you know that's and
Meredith Whittaker: that's that's then think about social
Meredith Whittaker: data then think about data that reflects like who gets
Meredith Whittaker: access to resources then think about all of the other things and um
Meredith Whittaker: i think i think it's actually an exciting idea to think on like you know how
Meredith Whittaker: do we how do we create systems where we're much more attentive to that and recognize
Meredith Whittaker: that there is a you know it really matters how those choices are made how those
Meredith Whittaker: you know how data is created who gets a und wie es zu sagen ist, und wie es zu sagen ist.
Linus Neumann: Would you say in that future, so there is these large gen AI models and whatnot,
Linus Neumann: and from other discussions we've had, I know that you probably believe there
Linus Neumann: is a stronger future for specialized models, expert models, not the generative ones.
Linus Neumann: Would that be a prediction of how this whole AI thing is going to evolve?
Linus Neumann: Not from the business model perspective or from the political perspective but
Linus Neumann: from the actual technology and research perspective.
Linus Neumann: Do you think there is still going to be exponential improvement in the Gen AI
Linus Neumann: world or do you think it's now the time for the smaller speedboats?
Meredith Whittaker: Well, I think it's definitely time for smaller speedboats and I want to index
Meredith Whittaker: on that word improvement because if we scratch the surface on some of these large models,
Meredith Whittaker: some of which are generative you begin to realize that a lot of the claims to
Meredith Whittaker: improvement and accuracy are based on really narrow benchmarks and evaluations
Meredith Whittaker: that don't reflect the performance of these models That's.
Linus Neumann: How Germans optimize their diesel engines.
Meredith Whittaker: Exactly So it's there
Meredith Whittaker: are things that Gen AI models can do I don't see them realistically going away
Meredith Whittaker: But I do see the struggle for a market fit that can produce the kinds of returns
Meredith Whittaker: necessary to prop up a massively energy intensive,
Meredith Whittaker: massively infrastructurally intensive, extraordinarily capital intensive industry, right?
Meredith Whittaker: So you have billions of dollars for a training run, just huge amounts of energy
Meredith Whittaker: and effort needed to create a model.
Meredith Whittaker: But okay, who's going to keep paying for a chatbot that's wrong?
Meredith Whittaker: And so I think there is a struggle for market fit.
Meredith Whittaker: I think you see this with things like Microsoft Recall where they pushed to
Meredith Whittaker: implement this. I don't know.
Meredith Whittaker: Microsoft Recall was supposed to ship with Windows 11.
Tim Pritlove: Es ist aber nicht auf der Default.
Meredith Whittaker: Ja, wir haben das kleine Sache. Aber es ist ein AI-System, whose value proposition
Meredith Whittaker: ist, dass es will remember everything you were doing on your device for the last N months.
Linus Neumann: Das ist ein nice value proposition.
Meredith Whittaker: Ich weiß, value to your boss, maybe. Und so you can type, oh,
Meredith Whittaker: ich will find this browser, ich will find this thing, und es will...
Meredith Whittaker: I get it for you. I obviously don't use it. And how does it remember is really the key here.
Meredith Whittaker: It remembers because it's taking screenshots of your device every five seconds,
Meredith Whittaker: creating a library of those screenshots and accessing those as the data on which
Meredith Whittaker: it is able to claim intelligent memory.
Linus Neumann: That is not an efficient way to approach the problem.
Meredith Whittaker: And I don't need to know that I was doomscrolling. Like, you know,
Meredith Whittaker: that's not a proud moment of memory for me.
Meredith Whittaker: So, yeah, I mean, and to me what that says, like, that's not a very useful purpose.
Meredith Whittaker: It's probably going to be marketed to enterprises for, you know,
Meredith Whittaker: worker surveillance is my guess.
Meredith Whittaker: But it shows that Microsoft is really trying to find a market for this, right?
Meredith Whittaker: Because they clearly circumvented their QA process.
Meredith Whittaker: They clearly circumvented their security evaluation. There was a lot of things
Meredith Whittaker: that clearly didn't happen and didn't happen at a company that is actively...
Meredith Whittaker: Die haben OpenAI, sie investieren eine große Menge in Azure,
Meredith Whittaker: sie sind in der Infrastruktur überall, all of sie sind.
Meredith Whittaker: Aber Microsoft ist joked zu OpenAI, so sie hat die Leader Position für ein Moment.
Meredith Whittaker: Und ich denke, also, dass Microsoft
Meredith Whittaker: ist wirklich zu regieren eine gute Reputation in der Security-World,
Meredith Whittaker: es ist indicative of how desperate but that rush to market fit and the AI exceptionalism
Meredith Whittaker: that is driving it is that they just mess that up so egregiously.
Meredith Whittaker: And I can hear some hacker in a Microsoft hallway being like,
Meredith Whittaker: I don't fucking know. I just left that meeting.
Meredith Whittaker: Because you can kind of sense how those things happen.
Tim Pritlove: What would you say is the dividing line right now between useful applications
Tim Pritlove: for machine learning, expert systems, AI stuff, and the hype?
Meredith Whittaker: Well, I mean, the question to dig into is useful to whom?
Meredith Whittaker: Because hype is useful to investors, right? You know, yada yada.
Tim Pritlove: I'm not talking about, yes.
Meredith Whittaker: I mean, I think AI is...
Tim Pritlove: In a computer science-y way. I mean, really like making applications possible
Tim Pritlove: that haven't been before, that actually do useful stuff to people or society.
Meredith Whittaker: Yeah, I mean, one use I think about a lot that is definitely,
Meredith Whittaker: I'm sure, useful to intelligence services is, you know, I'm sure we all assume
Meredith Whittaker: that POTS telephony data is being collected en masse by every intelligence service,
Meredith Whittaker: who can and has been for many, many, many years,
Meredith Whittaker: and that data was probably not that useful for a long time because,
Meredith Whittaker: you know, you're going to have to know, you're going to have to have a human
Meredith Whittaker: review it or, you know, something like,
Meredith Whittaker: It's probably a lot more useful now that you can quickly transcribe that with
Meredith Whittaker: AI and sort of synthesize and search using these generative systems, right?
Meredith Whittaker: So that's one example where I think it's probably almost certainly very,
Meredith Whittaker: very useful and changes the calculus on like how dangerous this surveillance
Meredith Whittaker: business model is as well.
Meredith Whittaker: But who is that useful to? It's not me and you.
Tim Pritlove: Yeah, you're going right into surveillance capitalism again.
Meredith Whittaker: No, well, that's my, I'm really good at making that turn.
Tim Pritlove: I'm looking for a rosy outlook into the future, like hope.
Meredith Whittaker: Yeah, well, I mean.
Tim Pritlove: Anything in store?
Meredith Whittaker: I refuse to plant my hope in delusion.
Linus Neumann: Okay, thanks, thanks.
Meredith Whittaker: And I am probably one of the more optimistic people you'll meet.
Tim Pritlove: All right.
Linus Neumann: Meredith, it was a pleasure.
Meredith Whittaker: So fun. Always so fun. Thank you for having me. It's been a really,
Meredith Whittaker: really delightful conversation with you all.
Tim Pritlove: All right. Thank you. Thanks for listening and see you next week. Goodbye.
Linus Neumann: Bye-bye.