In short. Capability spreads, and Anthropic says three labs ran over 16 million exchanges with Claude to copy its abilities. Records about particular people spread far less readily, so they last, and anyone who gets hold of them can sort them for almost nothing. For most firms that makes them a liability first, so keep less and clean what has to leave your machine.
What this is. A report from published measurements, court records and vendor documentation, plus one worked sum at list price.
Contents
Cheap reading makes every stored copy worth mining
Personal data used to be safe partly because reading a chat log or a profile took human time, and now a model does it for almost nothing. By Epoch AI’s measure, machine reasoning at a fixed level of skill has been getting about 13 times cheaper a year since 2023.[3] The model also finds more than the writer meant to share.[7] So every stored copy of personal data is now worth mining, for whoever holds it and whoever obtains it.
Vitalik Buterin put the idea bluntly on 3 November, at Devcon 8 in Mumbai. “We really need to treat data as toxic waste by default,” Cointelegraph’s caption on a clip quotes him as saying. He called for open-source technology and zero-knowledge proofs, it says, to cut how much people have to trust.[1]
Sharing the clip, David Hoffman predicted that shrinking your data footprint will become normal practice. As AI inference becomes increasingly free, he wrote, personal data becomes increasingly useful, “and not for good reasons.”[2]
We’ll call the place where your data first becomes readable to someone else the boundary. The less that crosses it, and the less kept on the far side, the less there is to mine.
At list price, one question to a million chats costs about $86
Put one yes-or-no question, such as whether a chat mentions a health condition, to an archive of a million chats. At the list price of Liquid AI’s d1-ultra, our sum comes to about $86. A closer look at the chats it flags is cheap too.
On 15 October 2026 Liquid AI released d1-ultra, which it calls a decision model. In one pass it reads text, images or both and gives a probability for each answer to one fixed question, whether yes or no, a choice or a score. It writes no text, and Liquid bills only the input, at $0.042 per million tokens.[37]
Our sum assumes a question of about 50 tokens and chats of about 2,000 tokens each, text only, one question per call and no discount. At list price, that’s 2,050 tokens × 1,000,000 records × $0.042 per million tokens, less than a hundredth of a cent a record.[37] Liquid bills each question separately, including the text and images it asks about, so ten questions cost ten times as much.[38]
Artificial Analysis ranks d1-ultra first of 20 models on its Decision Index. Its score is 65, against a median of 53.[40] But the index launched two days before d1-ultra. The decision models it tested then, Liquid’s earlier d1 among them, were right less often than they claimed at almost every probability level. As a class they also trailed language models on knowledge and reasoning.[39]
Liquid says it trained d1-ultra to make its probabilities more reliable.[37] Its documentation tells users to calibrate any decision threshold on their own data.[38] To whoever holds the archive, we think imperfect answers matter less than they sound. The screen only has to pick out chats for a slower, costlier model to read closely.
A fixed level of skill has been getting cheaper fast. Epoch AI’s measure, the cost of reaching a fixed benchmark score with the cheapest model available, has fallen about 47 percent a quarter since 2023. By Epoch’s figures, OpenAI’s o3 reached its score on GPQA Diamond, a science benchmark, for about 30 cents a question early in 2025. Under 18 months later another model matched it for $0.0004. Epoch puts that drop at 725-fold.[3]
Epoch’s earlier per-token series found declines of 9 to 900 times a year, depending on the task. The median was 50.[5] In Stanford’s AI Index, GPT-3.5’s level on the MMLU benchmark cost $20 per million tokens in November 2022. By October 2024 it cost $0.07.[6]
In 2023 ETH Zurich researchers had language models guess the age, location, income and other traits of real Reddit writers from their comments. GPT-4 got up to 85 percent right on its first guess, and 95 percent within three. It did the job for about a hundredth of the cost of human labelers, in a 240th of the time.[7]
Other researchers went from traits to names. In February 2026 a team including Nicholas Carlini and Florian Tramèr gave a web-search agent summaries of 338 Hacker News users’ activity, stripped of names, handles and links. The agent named the person behind 67 percent of the accounts, at 90 percent precision. Each profile cost $1 to $4.[4]
A second test matched accounts to each other. There the team’s language-model pipeline reached up to 68 percent recall at the same precision, while the best non-LLM method found close to none. The researchers knew who owned every account, and they say their edited profiles are much easier to identify than most pseudonymous accounts.[4]
In 2006 Arvind Narayanan and Vitaly Shmatikov showed that knowing a little about a Netflix subscriber was enough to find them among 500,000 whose ratings were released without names.[8] Tested on the same data in 2026, language-model agents rebuilt 79.2 percent of identities in the sparsest setting. Classical matching rebuilt 56.0 percent.[9]
A court ordered OpenAI’s chat logs kept and sampled, with no breach
A court can get at stored chats without any breach, if the chats exist and matter to a lawsuit. In the New York Times’s copyright suit against OpenAI and Microsoft, Magistrate Judge Ona Wang gave the first order on 13 May 2025. From then on, OpenAI had to preserve and segregate all output logs it would otherwise have deleted, whether a user asked for the deletion or a privacy law required it.[18]
On 9 October 2025 Judge Wang approved the parties’ agreement ending the duty to keep new logs as of 26 September, except for accounts tied to domains the plaintiffs named. Logs already set aside stayed preserved, except for users in the European Economic Area, Switzerland and the UK.[19]
On 7 November 2025 Judge Wang ordered OpenAI to give the news plaintiffs a sample of 20 million consumer ChatGPT logs.[20] OpenAI objected, and on 5 January 2026 District Judge Sidney Stein upheld the order. His ruling notes that OpenAI has kept tens of billions of such logs in the ordinary course of business. Users gave these chats to OpenAI voluntarily and OpenAI keeps them, it reasons, so their privacy interest is weaker than in a wiretapped phone call.[21]
Judge Wang weighed three safeguards, which Judge Stein upheld: the cap on the sample’s size, de-identification, and a protective order limiting who could see the logs and how.[21] Each limited what happened to the data after it was kept. None asked whether it should have been kept at all.
Retention varies widely. OpenAI keeps API abuse-monitoring logs for up to 30 days by default.[22] Anthropic keeps consumer chats for five years if the user lets them train models, and 30 days if not.[23] Google keeps Gemini chats for 72 hours even with activity saving off, and human-reviewed chats for up to three years, even after you delete your activity.[24] Each of those copies is within a court’s reach.
No cryptographic route yet hides a prompt at chat speed
The cleanest fix would be a server that never holds a readable copy. As we read the evidence, none of four techniques for computing on private data, three of them cryptographic, yet keeps a chat prompt from the server at chat speed. We leave out hardware enclaves here; our essay on private-AI verification covers them.
The first, fully homomorphic encryption or FHE, lets a server compute on data it can’t read, so the boundary never opens on the server. Buterin treats it as a long-run goal.[11] The cost is speed. IEEE Spectrum reported in March 2026 on an Intel chip built to speed up FHE. On today’s CPUs and GPUs, it said, FHE runs thousands to tens of thousands of times slower than unencrypted computing.[28]
A January 2026 paper ran Llama-3-8B under FHE at about 18 seconds per output token. That was with 128 encrypted input tokens on eight RTX PRO 6000 GPUs.[25] An implementation its authors describe as open source took 366.4 seconds on one H100 to process a single 128-token prompt.[26]
In Zama’s library, the server runs only a model’s linear layers, on encrypted data, while the user’s side runs attention and activations unencrypted. That reaches about 11 seconds a token for GPT-2 on a GPU.[27]
The second, multi-party computation or MPC, splits the work across several servers, and under a semi-honest model no server learns anything unless they collude. It’s slow too. PUMA, a 2023 system, took about 200 seconds to produce one LLaMA-7B token on three servers on a fast local network.[29]
The third, zero-knowledge proofs, aims at honesty rather than secrecy, proving a computation ran correctly without showing what went in. The zkLLM system took about 803 seconds on one A100 to prove a LLaMA-2-13B run over 2,048 tokens.[30] Lagrange’s DeepProve proves a 512-token GPT-2 run in 7.6 minutes.[31] EZKL can prove, for example, that a public model ran on private data.[32]
Such proofs can hide an input or a model’s weights from whoever checks them, but not from whoever runs the computation. In zkLLM that’s the company running the model, which computes the answer from the prompt.[30]
The fourth, differential privacy, limits how much a model can learn from any single piece of its training data. Google trained VaultGemma, a one-billion-parameter model, with a guarantee for each 1,024-token sequence. It sets epsilon at most 2.0 and delta at most 1.1e-10.[33]
Google says a fact in one sequence is essentially unknown to the model, while one repeated across many sequences can still be learned. Where data maps to people, it says, user-level privacy would be the better choice.[33] At worst, its technical report adds, one document can be split across sequences or sampled up to seven times, each copy counted separately.[34]
So VaultGemma’s sequence-level guarantee protects a rare passage in the training data. It doesn’t protect someone who mentions the same diagnosis in a hundred chats. Nor does it protect a prompt as the model reads it.
Local models trail the best, and cleaners still leak
Buterin’s answer for today is to run AI on your own machine, which keeps the boundary on your desk. It has two limits. Local models trail the best remote ones, and cleaning a prompt before it leaves still lets some details slip.
Crypto Briefing’s report of his Devcon 8 keynote makes trust his central problem, in areas from verified operating systems and open silicon to trustworthy AI.[10] His April post describes running models locally first and keeping his files there. His laptop’s RTX 5090 runs a 35-billion-parameter Qwen model at about 90 tokens a second. For remote models he’d ideally like unlinkable requests, mixnets, attested enclaves and a local model that strips private details before anything leaves.[11]
The first limit is capability. Artificial Analysis’s new index, launched in late October, leans on private test sets. At launch the leader scored 58, and the best open-weight model 46, though its weights weren’t out yet. The best models you can already download scored 32, which the index puts 26 points behind.[16]
Buterin concedes that local AI is still far from sufficient for writing code and other intellectual work.[11] So we expect most people to keep sending hard tasks to remote models.
The second limit is leakage. PAPILLON, a system presented at NAACL 2025, came with a benchmark of 901 real user prompts containing personal details. On 237 held-out prompts, a small local model, Llama-3.1-8B, rewrote each one for GPT-4o-mini, a stronger remote model. The best version in that study kept answer quality on 85.5 percent of queries and leaked 7.5 percent of the personal details.[35] That beats sending everything, and it’s still a leak.
The Reddit study’s team also tested a commercial cleaner. After Azure’s language service had scrubbed the text, GPT-4 could still guess 66.3 percent of personal traits, against 86.7 percent on the original. Five rounds of adversarial rewriting brought that to 41.6 percent, but GPT-4 did the rewriting, and it’s a remote model too.[36]
We’d add one idea of our own, and it’s untested. Liquid publishes some small decision models as open-weight, so they can run on your own hardware.[38] One of them, d1-2B, has 1.92 billion parameters, and Liquid’s model card pitches it for routing, triage, moderation and guardrails.[41]
A model that size could sit on your device as a gate, checking each outgoing prompt and holding back one that names a person, a diagnosis or an account. A gate can stop a prompt but can’t rewrite one, and nobody has measured how well d1-2B would do the job. Our page of proposed studies sets out how we’d count its misses.
How little an assistant needs to see in the first place is the question of our essay Can AI work with sensitive information without exposing it?
We think personal records are the lasting moat, and a liability first
We think records about real people are the one part of the data moat that lasts. For most firms, they’re a liability before they’re an asset.
The strongest case for the moat comes from firms that hold the data. In February 2024 Mark Zuckerberg told Meta’s investors that learning from unique data and feedback loops was the next key part of Meta’s playbook. He estimated that its public images and videos exceed Common Crawl.[12]
The evidence for the moat is mixed, because models can learn from other models. Microsoft used synthetic data throughout Phi-4’s training and reported that Phi-4 beat its teacher, GPT-4o, on graduate-level science and competition math tests.[13] Anthropic said in February 2026 that three labs had set out to distill Claude’s abilities. By Anthropic’s account, they ran more than 16 million exchanges through about 24,000 fraudulent accounts.[14]
On Epoch’s Capabilities Index, open models have trailed the best closed ones by about four months on average since January 2026.[15] Artificial Analysis’s new index scores models instead of dating them, and at launch it put downloadable models well behind the leader.[16]
Distillation and open weights spread skill far more readily than they spread records about particular people. That’s why those records last, and why they’re a risk. A court can order them handed over without any breach if they matter to a lawsuit. Anyone who gets hold of them can sort them for almost nothing at decision-model prices. The GDPR already limits personal data to what’s necessary for the purpose.[17]
If you use AI, find the retention setting on each service and pick the shortest one you can live with. Use a local model for personal material wherever it’s good enough. When work has to leave your machine, clean it first, and expect some of it to get through anyway.
Providers should make the shortest retention the default and publish it in plain numbers next to every privacy claim. Someone deciding what to type needs to know whether a chat lasts 72 hours or five years.
Scope and limits
- What we did
- We read Epoch AI’s and Stanford’s price series, Artificial Analysis’s two indexes, Liquid’s documentation, the Reddit, Hacker News and Netflix studies, the court orders on OpenAI’s logs, the cryptography papers and docs, and three providers’ retention pages. We priced one yes-or-no screen of a million chats at d1-ultra’s list price.
- What we didn’t do
- We ran no model, tested no cleaner and measured no decision model’s accuracy. We didn’t test d1-2B as a privacy gate; that is a proposal. We know Buterin’s Devcon 8 remarks only through a clip’s caption and a news report, and checked the GDPR wording against an unofficial reproduction.
- What this can’t tell you
- How fast local models will close the gap, whether cleaning will get much better, or how a court would treat chats a provider can’t read. Epoch lists its own caveats, from benchmark gaming to thin data. The $86 sum assumes text only, one question per call and no discount.
- Confidence
- High that reading got cheap and that a court can compel relevant stored chats without a breach; price series and orders are public. Moderate on how well a cheap screen sorts, and on how far identification carries beyond the researchers’ edited profiles. Low on the gate, which nobody has measured.
- What would change our view
- A cryptographic route that serves chat at usable speed, or a cleaner that leaks far less than 7.5 percent of personal details without losing quality. Evidence that the identification results don’t hold beyond researcher-prepared accounts. A higher court giving retained chats the privacy of a phone call.
- Corrections
- We log every correction, with its date, on How we work. Send one through a research enquiry.
The open question
How little can a request carry across the boundary before a remote model’s answer stops being worth having?
We propose one idea of our own, not yet tested. A small open decision model, such as Liquid’s d1-2B, could sit on the device as a yes/no gate that holds back a prompt naming a person, a diagnosis or an account. Our page of proposed studies sets out how we’d count its misses.
References
A selected review of published sources, with Lugano’s analysis and proposed work identified in the text. Source dates and conditions matter; historical results are not current-model rankings.
- 1
Vitalik: “We really need to treat data as toxic waste by default” (Devcon 8 clip)
A video clip of about two and a half minutes from Devcon 8 in Mumbai, posted on 3 November 2026 at 08:07 UTC. Its caption quotes Buterin and summarizes his call for open-source technology and zero-knowledge proofs to protect privacy and reduce reliance on trust.
Scope. A news outlet’s social-media clip and caption. We read the caption and cite the quoted line to Cointelegraph; a full transcript of the talk wasn’t available when we wrote this.Back to the essay
- 2
Post sharing Cointelegraph’s Devcon 8 clip
A short opinion post, published on 3 November 2026, about six hours after the clip, arguing that as AI inference becomes increasingly free, personal data becomes more useful for the wrong purposes, and predicting that reducing one’s data footprint will become a normal practice.
Scope. An opinion, cited for the argument it makes. It reports no evidence of its own.Back to the essay
- 3
The plunging price of thought
Independent analysis published 22 September 2026 of the cost per task, reasoning tokens included, to reach fixed scores on five benchmarks since 2023: about 47% per quarter, about 13x per year. Gives the o3 GPQA Diamond example ($0.30 to $0.0004 per question, which Epoch calls a 725-fold drop).
Scope. About three years of noisy data, no standard errors; benchmarks are imperfect proxies for useful work and assume users switch to the cheapest model. For GPQA Diamond, other averaging choices give 42.9% to 58.0% per quarter. Epoch dates the o3 result to January 2025.Back to the essay
- 4
Large-scale online deanonymization with LLMs
Independent preprint (February 2026; ICLR 2026 workshop poster). A web-search agent identified 226 of 338 Hacker News users (67%) from anonymized activity summaries at 90% precision, at $1 to $4 per profile. In a separate closed-world matching test, an LLM pipeline reached up to 68% recall at 90% precision, against near 0% for the best non-LLM method.
Scope. Experiments used accounts whose identities were known to the researchers; the authors say their edited profiles are much easier to identify than most pseudonymous accounts, no truly pseudonymous accounts were tested, and larger-scale figures are extrapolations.Back to the essay
- 5
LLM inference prices have fallen rapidly but unequally across tasks
Independent analysis of the cheapest price per million tokens for models reaching fixed scores on six benchmarks, published 12 March 2025. Reports declines of 9x to 900x per year, median 50x.
Scope. Per-token prices, reasoning models excluded; the authors say the fastest recent drops may not persist.Back to the essay
- 6
Artificial Intelligence Index Report 2025, Chapter 1: Research and Development
Independent report (p. 41, Figure 1.3.22): price for a model scoring at GPT-3.5 level on MMLU fell from $20.00 per million tokens (November 2022) to $0.07 (October 2024).
Scope. Built on Epoch AI and Artificial Analysis data. The report describes the span as about 18 months; the dates it gives span about 23 months, so this essay uses the dates.Back to the essay
- 7
Beyond Memorization: Violating Privacy Via Inference with Large Language Models
Peer-reviewed (ICLR 2024). On 520 real Reddit profiles, GPT-4 inferred personal attributes with up to 85% top-1 and 95% top-3 accuracy, at about 100x lower cost and 240x less time than human labelers.
Scope. One dataset of public Reddit text and eight attribute types; measures inference from text, not harm in practice.Back to the essay
- 8
How To Break Anonymity of the Netflix Prize Dataset
Independent research (first posted October 2006, revised 2007). Showed that an adversary who knows a little about a subscriber can find that subscriber's record among the ratings of 500,000 Netflix subscribers released without names.
Scope. Historical case; the abstract does not give the number of ratings needed, which this essay does not state.Back to the essay
- 9
From Weak Cues to Real Identities
Independent preprint (March 2026, listed as accepted at ICML 2026). In the sparsest Netflix Prize setting, LLM agents reconstructed 79.2% of identities against 56.0% for classical matching.
Scope. A research setting on a historical dataset; we read the abstract and summary, not every experiment.Back to the essay
- 10
Vitalik Buterin calls for trustworthy hardware at Devcon 8
Secondhand report of the 3 November 2026 opening keynote: trust as the central problem, verified operating systems, open silicon and trustworthy AI, with formal verification, open source and cryptographic proofs as tools. Its one direct quote is used in this essay.
Scope. News report of a talk with no transcript yet. It does not mention data as toxic waste, local models or sanitization.Back to the essay
- 11
My self-sovereign / local / private / secure LLM setup, April 2026
First-party post of 2 April 2026: local-first inference, local files, sandboxing; for remote models, an ideal multi-layer defense of unlinkable (ZK) API calls, mixnets, TEE inference with local attestation checks and local input sanitization, with FHE as a long-run goal. Reports Qwen3.5 35B at about 90 tokens per second on an RTX 5090 laptop and says local AI is far from sufficient for writing code and intellectual work.
Scope. One person's setup and threat model; he warns readers not to assume it is secure.Back to the essay
- 12
Meta Q4 2023 earnings call transcript
First-party transcript of 1 February 2024. Mark Zuckerberg describes learning from unique data and feedback loops as the next key part of Meta's playbook and estimates Meta's public images and videos exceed the Common Crawl dataset.
Scope. A company's statement to investors about its own strategy; not evidence that data is a lasting moat.Back to the essay
- 13
Phi-4 Technical Report
First-party report of 12 December 2024: Phi-4 uses synthetic data throughout training. The abstract says it substantially surpasses its teacher model on STEM-focused QA; the introduction names the teacher as GPT-4o and the benchmarks as GPQA and MATH.
Scope. A developer's report on its own model; benchmark results, not deployment outcomes.Back to the essay
- 14
Detecting and preventing distillation attacks
First-party report of 23 February 2026: three labs ran over 16 million exchanges with Claude through about 24,000 fraudulent accounts to distill its capabilities.
Scope. Anthropic's own account of activity against its service; the named labs' responses are not reflected.Back to the essay
- 15
Open models lag state-of-the-art closed models by 4 months
Independent analysis of 29 May 2026 using Epoch's Capabilities Index: since January 2026 open models trail the best closed models by about four months on average.
Scope. One composite index; a time lag is a different measure from a score gap.Back to the essay
- 16
Artificial Analysis Intelligence Index v5
Benchmark operator’s launch note of 27 October 2026: the leader scores 58; the best open-weights model scores 46 but its weights are not yet released; the best models with released weights score 32, which the note puts 26 points behind the leader. The operator says the open-weights gap widened from 3 to 12 points.
Scope. Scores as of the 27 October launch; rankings move. The 3-point gap was measured on v4.3, so the widening spans a change of index; 60% of v5 uses private or held-out test sets, and the operator does not say how much that explains. The best open-weights model at launch had not yet released its weights.Back to the essay
- 17
Regulation (EU) 2016/679 (General Data Protection Regulation), Article 5(1)(c)
Official text: personal data shall be adequate, relevant and limited to what is necessary for the purposes of processing (data minimization).
Scope. We checked the wording against an unofficial reproduction, not the official EUR-Lex page.Back to the essay
- 18
Order directing OpenAI to preserve and segregate output log data (ECF 551, No. 1:23-cv-11195)
Court record of 13 May 2025 directing OpenAI to preserve and segregate all output log data that would otherwise be deleted, going forward, whether deletion was requested by a user or required by privacy law.
Scope. A discovery order in copyright litigation; it does not decide the merits.Back to the essay
- 19
Stipulation and Order to terminate OpenAI's ongoing obligations under the preservation order (ECF 922)
Court record filed 9 October 2025: the ongoing preservation duty ended as of 26 September 2025; data already segregated stays preserved except for users in the EEA, Switzerland and the UK; logs for accounts tied to domains the plaintiffs listed stay preserved going forward. Approved and signed by Magistrate Judge Ona T. Wang.
Scope. Covers the segregated tables only and reserves the parties' other rights.Back to the essay
- 20
Order granting motion to compel production of 20 million de-identified consumer ChatGPT logs (MDL ECF 734, No. 1:25-md-3143)
Court record of 7 November 2025 ordering OpenAI to produce the 20 million de-identified consumer logs to the news plaintiffs, citing Concord Music v. Anthropic, where a 5-million-record sample was ordered produced.
Scope. A discovery order; production is under a protective order and of de-identified logs.Back to the essay
- 21
Order affirming the 20-million-log production orders (MDL ECF 1021)
Court record of 5 January 2026. Notes OpenAI has retained tens of billions of consumer logs in the ordinary course; recounts the safeguards Magistrate Judge Wang weighed (the 20 million cap, de-identification and the protective order); holds that users’ privacy interest in chats voluntarily disclosed to and retained by OpenAI is weaker than in wiretapped calls; affirms production.
Scope. A discovery ruling; later filings in the case after its 2026 schedule were not reviewed.Back to the essay
- 22
Data controls in the OpenAI platform
First-party documentation read 3 November 2026: API abuse-monitoring logs retained up to 30 days by default; Zero Data Retention needs prior approval; some state persists under ZDR, such as prompt-cache tensors up to 24 hours and images flagged for possible CSAM review.
Scope. Undated page.Back to the essay
- 23
Updates to our consumer terms
First-party announcement of 28 August 2025: consumer chats retained five years if the user allows training, 30 days if not.
Scope. Consumer plans only.Back to the essay
- 24
Gemini Apps Privacy Hub
First-party page (updated 26 October 2026): chats with Keep Activity off retained 72 hours; human-reviewed chats retained up to three years and not deleted when activity is deleted.
Scope. Policy page.Back to the essay
- 25
Scaling up FHE-based Privacy-Preserving ML: Higher Throughput, Longer Inputs for LLama-3-8B
Independent preprint (January 2026, revised June 2026; system called Sylph): Llama-3-8B with 128 encrypted tokens runs at about 18 seconds per output token on eight NVIDIA RTX PRO 6000 GPUs.
Scope. Research system; no plaintext comparison ratio reported.Back to the essay
- 26
An Open-Source End-to-End FHE Implementation for Privacy-Preserving Llama 3 8B Inference
Independent preprint (September 2026): one 128-token prefill of Llama-3-8B under FHE takes 366.4 seconds on one H100, 4.51x faster than the THOR baseline.
Scope. Prefill only; no decode figure. The abstract and HTML body name the system differently (Odin, Themis). The authors call it open source, but we found no repository link, and the paper reports no accuracy figure for the encrypted model.Back to the essay
- 27
Concrete ML documentation: LLM inference
First-party documentation: hybrid FHE where the server runs linear layers on encrypted data and the client runs attention and activations in the clear; GPT-2 at about 11 seconds per token on GPU, with about 2.2 MB of traffic per token.
Scope. Undated page read 3 November 2026; hardware for the timing is not named.Back to the essay
- 28
Intel Demos Chip to Compute With Encrypted Data
Secondhand report of 10 March 2026 on Intel's Heracles accelerator: FHE on current CPUs and GPUs takes thousands to tens of thousands of times as long as unencrypted computing.
Scope. Journalistic summary of vendor claims; no LLM benchmark.Back to the essay
- 29
PUMA: Secure Inference of LLaMA-7B in Five Minutes
Independent research: three-party MPC produced one LLaMA-7B token from an 8-token input in about 200.5 seconds on servers with 1 TB RAM over a fast local network.
Scope. LAN-like setting; slower over wide-area networks.Back to the essay
- 30
zkLLM: Zero Knowledge Proofs for Large Language Models
Peer-reviewed (ACM CCS 2024): proved LLaMA-2-13B inference over 2,048 tokens in about 803 seconds on one A100; the prover is the AI company holding the weights and computes the output from the verifier’s prompt.
Scope. The 803 seconds is proving time; a one-time commitment to the weights adds more.Back to the essay
- 31
DeepProve README
First-party: end-to-end zero-knowledge proofs of LLM inference; a 512-token GPT-2 run proved in 7.6 minutes on a 24-core CPU.
Scope. Undated repository page; says nothing about input privacy.Back to the essay
- 32
EZKL README
First-party: lists statements it can prove, including that a public neural network was run on private data, or a private model on public data.
Scope. Undated repository page; not an LLM private-inference product.Back to the essay
- 33
VaultGemma: The world's most capable differentially private LLM
First-party post of 12 September 2025: a 1B model trained with sequence-level DP (epsilon at most 2.0, delta at most 1.1e-10, sequences of 1,024 tokens). A fact in one sequence is essentially unknown to the model; facts in many sequences can be learned; user-level DP is better where data maps to users.
Scope. Training-time privacy for a small model; says nothing about who sees prompts at inference.Back to the essay
- 34
VaultGemma: A Differentially Private Gemma Model (technical report)
First-party report: sequences formed after sampling and packing; at worst a document can be split across sequences or sampled up to seven times, with repeats treated as separate privacy units.
Scope. Developer's own report.Back to the essay
- 35
PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles
Peer-reviewed (NAACL 2025): PUPA, a benchmark of 901 real user queries with personal information. On the 237 held-out PUPA-TNB queries, a local Llama-3.1-8B rewriting prompts for GPT-4o-mini kept quality on 85.5% of queries with 7.5% leakage of personal details.
Scope. Leakage judged by an LLM (86% agreement with humans on a checked sample); one benchmark.Back to the essay
- 36
Large Language Models are Advanced Anonymizers
Independent research: GPT-4 inferred 86.7% of personal attributes from original text, 66.3% after Azure Language Service anonymization, and 41.6% after five rounds of adversarial LLM anonymization.
Scope. One dataset with GPT-4 as the adversary; the strongest anonymizer was also GPT-4, a remote model. The PDF header gives ICLR 2025 as the venue.Back to the essay
- 37
d1-ultra: Introducing the World’s Best Decision Model
First-party announcement of 15 October 2026: a multimodal decision model that reads text, images or both in one forward pass and returns probabilities for yes/no, choice or score questions without generating tokens; $0.042 per million input tokens, billed on input only; says it outperforms all other decision models on the Artificial Analysis Decision Index by at least 8 points. It says Liquid trained d1-ultra to produce more reliable probabilities.
Scope. A vendor’s description of its own model; the index score appears only in a chart. Liquid’s pricing page lists the same input price and $0.00 for output.Back to the essay
- 38
Decision models (documentation)
First-party documentation read 3 November 2026: decision models return probabilities over a fixed set of outcomes in a single call with zero generated tokens; hosted models are billed on input tokens only, and each question is billed separately, including its text and images; d1-2B, d1-omni-2B and d1-omni-600M are open-weight under the LFM Open License; advises calibrating any decision threshold on your own data; not for free-form text generation.
Scope. Undated page; a vendor’s own documentation.Back to the essay
- 39
Artificial Analysis Decision Index v1.0
Benchmark operator’s launch article of 13 October 2026: decision models choose from a fixed set of options with a probability for each, in one call; across almost all probability levels, accuracy fell below the stated probability for all decision models tested; decision models offer speed and cost advantages over language models but trail on knowledge and reasoning.
Scope. Results as of launch, two days before d1-ultra was released; it tested Liquid’s earlier d1, not d1-ultra. Calibration for language models was not yet measured.Back to the essay
- 40
d1-ultra: Release, Decision Index Score, Performance & Price
Benchmark operator’s model page read 3 November 2026: Decision Index score 65, ranked first of 20, against a median of 53 for decision models; $32 per million index tasks; 93 ms per task; $0.042 per million input tokens; text and image input; released October 2026.
Scope. Undated live page; the score is shown rounded, and component scores appear only in charts. Some secondary listings give 65.25, a figure this page doesn’t show.Back to the essay
- 41
d1-2B model card
First-party model card read 3 November 2026: an open-weight decision model of 1.92 billion parameters, built on LFM3-VL-2B, under the LFM Open License v1.0; pitched for routing and triage, moderation and guardrails, and classification; says it is not a chat model, that knowledge-heavy questions favor larger models, and that decision thresholds should be calibrated on your own data.
Scope. A vendor’s model card. It does not mention privacy or personal-data screening; using it as a local privacy gate is our proposal, untested.Back to the essay