(Updated: September 20, 2023)
It's been more than four years since the last regular publication of documents from the Snowden trove. Last year, however, some new snippets of information from the Snowden documents appeared in the PhD thesis of hacktivist Jacob Appelbaum.
The new information isn't very spectacular and also quite specialistic, but still worth to make it more easily accessible. Also for the record I added some corrections and additions to Appelbaum's discussion of NSA surveillance methods.
NSA headquarters - Appelbaum's thesis - Eindhoven University of Technology
Jacob Appelbaum
Jacob R. Appelbaum was born in 1983 in California and became a well-known hacker and activist for digital anonymity. He was a member of the Cult of the Dead Cow hacker collective and a core member of the Tor project, which provides a tool for anonymous internet communications.
In 2012, Appelbaum moved to Berlin, where he worked closely with Laura Poitras on the NSA documents which she had received from Edward Snowden in May and June 2013. However, he was also involved in the story about the eavesdropping on German chancellor Merkel and the publication of the NSA's ANT Product Catalog.
In both cases the documents were not attributed to Snowden and apparantly came from a still unidentified "second source". In his thesis, Appelbaum seems to refer to this source when he mentions "documents exposed by whistleblowers, known and unknown, or other anonymous insiders."
In 2015, several women accused Appelbaum of sexual abuse and he subsequently lost his position at the Tor project and various other organizations. Appelbaum denied the allegations, but an investigation ordered by the Tor project determined that they appeared to be true.
Meanwhile Appelbaum had moved to The Netherlands, where he started as a PhD student at the Eindhoven University of Technology (TU/e). There he finished his thesis and received his PhD on March 25, 2022. Currently he works as a postdoc at the Coding Theory and Cryptology group at TU Eindhoven.
Appelbaum's PhD thesis
The full title of Appelbaum's thesis is "Communication in a world of pervasive surveillance. Sources and methods: Counter-strategies against pervasive surveillance architecture". His promotors were prof.dr. Mark van den Brand, prof.dr. Daniel J. Bernstein and prof.dr. Tanja Lange.
The thesis was published on March 25, 2022 and became available for download as a 24.3 MB pdf-document on September 27, 2022. The contents of this 327-page thesis are as follows:
- Chapter 1: Introduction.
- Chapter 2: Background on network protocols common to all research.
- Chapter 3: Background on cryptography common to all research.
- Chapter 4: Review of historical, political, economic, and technical adversarial capabilities (including previously published leaked documents that are from works which Appelbaum has written about in his role as a journalist).
- Chapter 5: Review of the Domain Name System and an explanation of alternative methods to improve the security and privacy of domain name lookups.
- Chapter 6: Examination of a tweak to the WireGuard VPN protocol to protect historic encrypted traffic against future attacks by quantum computers.
- Chapter 7: Introduces the Vula protocol, which is a suite of free software tools for automatically protecting network traffic between hosts in the same Local Area Network.
- Chapter 8: Introduces REUNION, a privacy-preserving rendezvous protocol.
In the preface, Appelbaum writes that his thesis is the culmination of more than a decade of research into the topic of surveillance. He expresses a political and activist aim by saying that the "machinery of mass surveillance is simply too dangerous to be allowed to exist" and that "we must use all of the tools in our toolbox – economic, social, cultural, political, and of course, cryptographic – to blind targeted and mass surveillance."
He says more has to be done than simply criticize surveillance practices. Cryptography for example, "allows for resistance in a non-violent manner to the benefit of everyone except the ones who are spying on us." From this perspective Appelbaum's thesis discusses various cryptographic implementations to "protect individual liberty, while aspiring to a broader goal of achieving societal liberty."
New information from the Snowden documents
Throughout his thesis, Appelbaum reveals some new information from Snowden documents that has not been published, but which he had access to during his research that resulted in various publications in media outlets like Der Spiegel, NDR and Le Monde. The new information is only described, so no new original documents were released.
According to Appelbaum: "Many journalists who have worked on the Snowden archive know significantly more than they have revealed in public. It is in this sense that the Snowden archive has almost completely failed to create change: many of the backdoors and sabotage unknown to us before 2013 is still unknown to us today." (page 71)
Appelbaum also provides some new information about the Snowden documents in general, by saying that The Intercept "closed their Snowden archive and reportedly it has been destroyed." (page 63, note 17)
Below, I provide exact quotes from Appelbaum's thesis, including his sources, which are in square brackets, while I added some additional links for further information.
1. BULLRUN: manipulating protocol security
"How do they accomplish their goals with project BULLRUN? One way is that United States National Security Agency (NSA) participates in Internet Engineering Task Force (IETF) community protocol standardization meetings with the explicit goal of sabotaging protocol security to enhance NSA surveillance capabilities." "Discussions with insiders confirmed what is claimed in as of yet unpublished classified documents from the Snowden archive and other sources." (page 6-7, note 8)
2. Selecting entropic internet traffic
"There are various rules governing what is selected for long-term data retention in [the NSA's] corporate repositories. One example is that some traffic which is considered entropic by a standard Shannon Entropy estimate is selected from the network in real time and saved to a database, preserving it for cryptanalysis using future technology." "This statement is based in part on an analysis of as of yet unpublished XKeyscore source code that performs a Shannon Entropy estimate. Some kinds of Internet traffic that is considered entropic is recorded for later analysis." (page 9, note 16)
3. Compromised lawful interception systems
"As part of our research, we uncovered evidence that the telecommunications infrastructure in many countries has been compromised by intelligence services. The Snowden archive includes largely unpublished internal NSA documents and presentations that discuss targeting and exploiting not only deployed, live interception infrastructure, but also the vendors of the hardware and software used to build the infrastructure. Primarily these documents remain unpublished because the journalists who hold them fear they will be considered disloyal or even that they will be legally punished. Only a few are available to read in public today." (page 41)
"Targeting lawful interception (LI) equipment is a known goal of the NSA. Unpublished NSA documents specifically list their compromise of the Russian SORM LI infrastructure as an NSA success story of compromising civilian telecommunications infrastructure to spy on targets within reach of the Russian SORM system." (page 41)
"The NSA slides have "you talk, we listen" written in Cyrillic on the jackets of two Russian officers." "Review of unpublished Snowden documents about NSA’s activities compromising deployed, lawful interception systems and as well as additional success against the vendors of such hardware or software. Needless to say, a compromised interception system is anything but lawful in the hands of an adversary." (page 41, note 4)
4. Compromised computer hardware
"While working on documents in the Snowden archive the thesis author learned that an American fabless semiconductor CPU vendor named Cavium is listed as a successful SIGINT "enabled" CPU vendor. By chance this was the same CPU present in the thesis author's Internet router (UniFi USG3). The entire Snowden archive should be open for academic researchers to better understand more of the history of such behavior." (page 71, note 21)
Update:
More information about whether Cavium CPUs may have a backdoor, as well as additional comments by Jacob Appelbaum can be found in an article published by Computer Weekly on September 19, 2023.
More information about whether Cavium CPUs may have a backdoor, as well as additional comments by Jacob Appelbaum can be found in an article published by Computer Weekly on September 19, 2023.
5. PRISM
"The PRISM slide deck was not published in full, and the public does not fully understand aspects of the program such as the retrieval of voice content data as seen in Figure 4.24. Domains hosted by PRISM partners are also subject to selector based surveillance. Several pages of the PRISM slides list targets and related surveillance data, and a majority of them appear to be a matter of political surveillance rather than defense against terrorism. One example that is not well-known except among the journalists who had access to the full PRISM slide deck is the explicit naming of targets. An example shows a suggestion for targeting of the Tibetan Government in Exile through their primary domain name. The tibet.net domain is named as an unconventional example that analysts should be aware of as also falling under the purview of PRISM. The email domain was hosted by Google Mail, a PRISM partner, at the time of the slide deck creation and it is still currently hosted by Google Mail as of early 2022." (page 76)
> See also: What is known about NSA's PRISM program
6. MYSTIC: Country X
"MYSTIC was revealed to impact a number of countries by name at the time of publication: the Bahamas, Mexico, the Philippines, Kenya and one mystery country: country X. The Bahamas, and country X are subject to SOMALGET full take data and voice collection. The publisher WikiLeaks observed that the monitoring of an entire country of people is a crime when done by outside parties, essentially an act of war by the surveillance adversary. WikiLeaks then revealed that the country in question, Country X, was Afghanistan [Yea14]. Through independent review of the Snowden archive, we confirm that this is the identity of Country X, and that WikiLeaks was correct in their claim." (page 78)
(Strangely enough, the source provided by Appelbaum ("Yea14") actually shows that already four days before Wikileaks' revelation, collaborative analysis by Paul Dietrich and the author of this weblog had already pointed to Afghanistan as being Country X. In his bibliography, Appelbaum attributes this source document to "John Young and et al." (the owners of the Cryptome website), while it was actually written by and first published on the blog of Paul Dietrich)
7. Manipulation of DUAL_EC_DRBG
"Many documents released in public from the Snowden archive and additional documents which are still not public make clear that this type of bug is being exploited at scale with help from NSA’s surveillance infrastructure. It is still unclear who authored the changes at Juniper and if bribery from the NSA was involved as with RSA’s deployment of DUAL_EC_DRBG to their customers as is discussed in Section 4.4." (page 81)
8. Software backdoors
"Example from the Snowden Archive of an as of yet unreleased backdoor in fielded software that is most certainly not an exclusively exploitable backdoor by NSA. The software’s secret key generation is sabotaged by design to ensure surveillance of the community of interest. There is a corresponding XKeyscore rule that has not yet been published. The goal of that rule is to gather up all ciphertext using this sabotaged system; it is clearly part of a larger strategy. As a flag in the ground for later, the thesis author presents the following SHA256 hash: [...]. There are additional examples from other sources that this is the general shape of the game being played with more than a few acts of sabotage by the NSA." (page 83, note 27)
Some corrections and additions
Chapter 4 of Appelbaum's thesis is about "the adversary" and describes a wide range of digital surveillance methods which are used by intelligence agencies. He writes a little a bit about the capabilities of Russia and China, but the biggest part is about the methods of the NSA as revealed through the Snowden documents.
In general, this chapter is very similar to for example Glenn Greenwald's book No Place to Hide and Snowden's memoir Permanent Record as it reads like a one-sided accusation against the NSA without much context or the latest information. Chapter 4 also contains small errors which could easily have been prevented. Here I will discuss some examples:
- Page 20, note 12: "An example is Suite-A cryptography or Type-1 cryptography, so designated by the NSA. The NSA now calls this the Commercial National Security Algorithm Suite (CNSA)"
> Comment: Actually CNSA isn't the new name for the highly secure Suite A, but for the less secure Suite B algorithms.
- Page 41: "The BND and the CIA held secret co-ownership of CryptoAG until 1993, and then the CIA held sole ownership until 2018. The devices were vulnerable by design, which allowed unaffiliated intelligence services, such as the former USSR’s KGB, and the East German Ministry for State Security [MfS], to independently exploit CryptoAG’s intentional flaws."
> Comment: This exploitation by the KGB and the MfS was apparently suggested in a German television report, based upon claims by a former Stasi officer, but so far there are no documents that support this claim. See for more information: Operation RUBICON.
- Page 41: "It does not appear that those party to the Maximator alliance are using their agreement and relative positions to spy on the entire planet – in stark contrast to the Five-Eyes agreement."
> Comment: The Five Eyes and especially NSA and GCHQ have massive capabilities, but spying on "the entire planet" is still rather exaggerated: their collection efforts are limited by national priorities, the locations of where they can access satellite and cable traffic, as well as by technical constraints. While the five members of the European Maximator alliance have/had much smaller capabilities, they could nonetheless intercept and decrypt diplomatic communications from over 60 countries where the weakened encryption devices from Crypto AG were used (see the map below).
> See also: Maximator and other European SIGINT alliances
The countries that bought and used manipulated Crypto AG devices
(graphic: The Washington Post - click to enlarge)
- Page 47, note 8: "Narus mass surveillance and analysis systems were deployed by the NSA inside AT&T facilities to intercept all traffic flowing through their large capacity network cables as documented [KB09] by whistleblower Mark Klein."
> Comment: This suggests that the NSA is intercepting American communications, but actually this is part of Upstream collection, which is aimed at foreign targets and therefore the NSA applies various filter systems to select traffic from countries of interest and discard purely domestic communications.
- Page 52: "The Foreign Intelligence Surveillance Court (FISC) is largely considered to rubber stamp requests from the FBI. The FBI has routinely misled the FISC, and from the little that is known, the FISC has neither the technical knowledge, nor the general temperament to actually act as a safeguard"
> Comment: Since the start of the Snowden revelations, numerous Top Secret documents from the FISC have been declassified, showing that the court examines the NSA's activities in great detail. The idea of being a "rubber stamp" is based upon the fact that the FISC denies just 0.5% of the applications, but later it became clear that American criminal courts only deny a tiny 0.06% of the requests for regular (so-called Title III) wiretaps.
- Page 53: "The CIA meanwhile, operates their own surveillance capabilities including capabilities that are entirely outside of the purview of the FISC, even now [cia22]."
> Comment: At least one of these cases is about the CIA's use of bulk datasets with financial information, which can of course contain information about Americans, but when the CIA obtained them in ways other than by intercepting communications, the FISC simply has no jurisdiction. It's up to lawmakers to impose privacy safeguards for creating and exchanging such bulk datasets.
- Page 56: "In the Snowden archive, we see lots of hacking and hacking related programs run by NSA, such as the TURBULENCE [Wik21u] program which is made up of modular sub programs [Amb13]. Those programs include TURMOIL [Gal14b], TUTELAGE [AGG+15a], TURBINE [GG14, Wik20d], TRAFFICTHIEF [Wik20c], and XKeyscore [Gre13d, Unk13, AGG+14b, Unk15a] as shown in Figure 4.12 and Figure 4.13, as well as data that was pilfered during those break-ins."
> Comment: This suggests that TURBULENCE and its sub-programs are about hacking operations, but actually, TURBULENCE is defined as "a next generation mission environment that created a unified system for MidPoint and Endpoint SIGINT", or in other words, an overarching framework for bulk and targeted tapping systems. Only the TURBINE sub-program can automatically trigger the implantation of malware into target computer systems. Furthermore, none of the sources mentioned in the thesis indicate that XKEYSCORE is a sub-program of TURBULANCE and XKEYSCORE is not a hacking tool either. A detailed explanation of the TURBULENCE system is given in an article by Robert Sesek, which was apparently not consulted by Appelbaum.
- Page 72: "US-984XN is the classified SIGAD while the program name PRISM is unclassified"
> Comment: There are no indications that "PRISM" is less secret than any other coverterm which the NSA uses for its collection, processing and analysis programs. That was likely also the reason that the big internet companies involved in this program initially denied that they had ever heard of something called PRISM.
- Page 91: "the NSA's Equation Group (EQGRP), which was later renamed Tailored Access Operations (TAO)"
> Comment: The name Equation Group was actually coined in February 2015 by the Russian cybersecurity firm Kaspersky for "one of the most sophisticated cyber attack groups in the world". Later on it became clear that this group was part of the NSA's hacking division TAO.
Given how many aspects of the NSA's operations Appelbaum mentions in chapter 4 of his thesis, one could say that it's inevitable that some mistakes are made and some sloppiness occurs. On the other hand, however, this is an academic publication for which the highest standards of accuracy should apply.
Finally, Appelbaum's activism is illustrated by the back cover of his thesis, which shows a logo very similar to that of the German terrorist organization Rote Armee Fraktion (RAF) from the 1970s, except that the original image of an AK-45 is replaced by that of a computer keyboard:
Comments at Hacker News and Schneier on Security
