July 14, 2018

Collection of domestic phone records under the USA FREEDOM Act

(Updated: March 2, 2020)

One of the most controversial NSA programs revealed by Edward Snowden was the bulk collection of domestic telephone records under the authority of Section 215 of the USA PATRIOT Act. A detailed analysis of the workings of this program was published on this weblog earlier.

In 2015, Section 215 was replaced by the USA FREEDOM Act, which prohibited the collection in bulk and provided more safeguards. The NSA became much more transparant about this program, which gives the opportunity for the following explanation of how the domestic phone records program currently works.

NSA is also more transparant about things going wrong: last month it revealed that it had to delete all the telephone records collected since 2015 due to technical irregularities.

Screenshot from 60 Minutes from December 15, 2013, showing an NSA contact chaining tool
used for the telephone records collected under Section 215.

Collection under Section 215 USA PATRIOT Act

The NSA started its bulk collection of domestic telephone metadata as part of the President's Surveillance Program (PSP), which president George W. Bush authorized in secret right after the 9/11 attacks. Its purpose was not to spy on random Americans, but to find connections between foreign terrorists and conspirators inside the US.

In May 2006, this bulk collection was brought from the president's authority under that of the FISA Court, based upon a very extensive interpretation of Section 215 of the USA PATRIOT Act. Internally, NSA refers to this kind of collection as BR FISA, with BR for Business Records.

Under Section 215, NSA collected domestic phone records from the three biggest American telecommunication companies: AT&T, Verizon and Sprint. According to government officials, the data provided by these companies consisted mostly of landline phone records, which meant that NSA actually got less than 30% of the total amount of US telephone metadata.

However, as of August 29, 2011, AT&T started to provide cell phone metadata too: ca. 1,1 billion records a day, which would make over 30 billion records each month. Before these records were handed over to NSA, AT&T stripped off the location data, to comply with the FISA Court orders that don't allow the collection of location data. Verizon was apparently not able or not willing to strip the location metadata, so their cell phone records could not be acquired by NSA.

To put these numbers in perspective: with a wireless communications market share of 32% for AT&T, the total number of cell phone metadata for the US would equal roughly 94 billion a month. During the first half of 2012, the NSA's total collection of foreign telephone metadata was 135 billion records a month. In January 2013, mobile phone calls in the Netherlands generated some 7.65 billion records a month.

At NSA, the domestic phone records were forwarded to MAINWAY, which is a centralized system for "contact chaining to identify targets of interest." MAINWAY not only contains domestic telephone metadata, but also foreign telephone and internet metadata, collected both inside and outside the US. Putting both foreign and domestic metadata in one system, allows finding as many connections as possible.

See for more:
- How NSA contact chaining combines domestic and foreign phone records
- Section 215 bulk telephone records and the MAINWAY database

Collection under the USA FREEDOM Act

Because the bulk collection under Section 215 was often regarded unconstitutional, the program was terminated as of November 2015 and replaced by the USA FREEDOM Act (USAFA or UFA), which was incorporated in Title V of the Foreign Intelligence Surveillance Act (FISA). Under this new authority, bulk collection of domestic phone records is not allowed anymore.

Instead, NSA can request only those records that contain phone numbers that have been in contact with an approved "seed" number. This means that all the American telecoms now have to hand over the matching results from both landline and cellphone calls, so it's a much larger pool compared to the situation under Section 215.

How this current domestic phone records program works is explained in remarkable detail in the transparancy report of the NSA Civil Liberties and Privacy Office (CLPO) from January 2016, as well as in the Annual Statistical Transparancy Report from the Office of the Director of National Intelligence (ODNI).

The statistical report for 2017 was published last April and also contains a lot of information about traditional FISA and Section 702 FAA (PRISM and Upstream) collection.

Overview of NSA's collection of domestic phone records under the USA FREEDOM Act
(source: NSA Transparancy Report - click to enlarge)

Seed numbers

The process starts with selecting specific targets and the phone numbers ("selectors") they use. Through the FBI and the Department of Justice, these selectors are submitted to the FISA Court (FISC), which determines whether there's a Reasonable, Articulable Suspicion (RAS) that these numbers are associated with foreign intelligence agents or people engaged in international terrorism. Under Section 215, the RAS was determined by one of 22 designated NSA officials.

After the FISC has approved these numbers, it issues individual orders approving the submission of these specific selectors to the telecommunications providers, and directing those providers to hand over the associated metadata to the proper government agency. According to the ODNI statistical report for 2017, the FISC issued orders for 42 targets in 2016 and for 40 targets last year.

The report doesn't mention the total number of selectors used by these targets. It's these selectors, phone numbers and maybe similar identifiers, that NSA uses as a "seed" to start creating a so-called contact chain. For earlier years, the total numbers of seed selectors were as follows (it's not known how many of these belonged to Americans):

2012 2013 2014 2015
288 423 161 56

Business records

At NSA, the RAS-approved selectors are entered into what is publicly called the "Enterprise Architecture", but which actually must be the MAINWAY contact chaining system. This returns any selectors from NSA's existing metadata collection that have been in direct contact with the RAS-approved seed selector.

Both the RAS-approved seed selectors and the connected ones from NSA's existing collection are then submitted to the telecommunications providers. They will query their databases of business records for those that contain any of the submitted phone numbers. The results are returned to the NSA, which lets them pass various validation steps, applies data tags and forwards them to the MAINWAY system.

Because a FISC order is valid for up to 180 days, the selectors can be submitted multiple times during that period in order to caputure any new matching records. These business records, or Call Detail Records (CDRs) are defined as "session identifying information" and include:
- Originating telephone number
- Terminating telephone number
- International Mobile Subscriber Identity (IMSI) number
- International Mobile Station Equipment Identity (IMEI) number
- Telephone calling card number
- Time and duration of a call
NSA is not allowed to receive the content of any communication, the name, address, or financial information of a subscriber or customer, or the cell site location or Global Positioning System (GPS) coordinates.

Contact chaining

The ODNI statistical transparancy report from April has a nice graphic that shows how to count the number of business records that the telecoms provide to the NSA:

Example of contact chaining of telephone metadata under the USA FREEDOM Act
(source: ODNI Transparancy Report - click to enlarge)

We see that the RAS-approved seed phone (number) can be in direct contact with a certain number of other phones, which is called the "first hop". Additionally, the providers also have to look for the phones that have been in contact with those first hop phones. This step is called the "second hop". A third hop is prohibited by law, but NSA analysts also determined that a third step is not analytically useful.

This way of contact chaining by linking phone numbers that have been in contact with each other may already be familiar from the reportings about the Section 215 program.

But the graphic also shows something that was rarely made clear: the business records collected by NSA are not just the phone numbers. Two phone numbers that have been in contact with eachother will usually have done so more than once (except for so-called "burner phones" that are intentionally used for one call only).

So for each pair of phone numbers, there can be a lot of records, at least one record generated per phone call or text message, both for the person calling and the person called. The example in the graphic shows 7 phones that produce 6000 call detail records (CDRs) during a certain period of time. This is something to keep in mind when it comes to the huge numbers of metadata collected by NSA.

Number of records

The ODNI transparancy report also provides the real numbers of telephone records collected by NSA under the authority of the USA FREEDOM Act. Although NSA is required by law to provide the annual number of "unique identifiers", the agency doesn't has the technical ability to isolate these unique identifiers within records received from the providers. This means that every single record is counted, even if the same record is received multiple times from one or multiple providers.

The report also explicitly says that the results of contact chaining will likely include both foreign and domestic phone numbers: "while the records are received from domestic communications service providers, the records received are for domestic and foreign numbers." Also, the targeted seed number could be a foreign number, which in the first hop could have called a foreign number, that in its turn could have called another foreign number in the second hop.

With that in mind, the report says that in 2016, the telecommunications providers handed over 151.230.968 phone records to NSA. In 2017 they did so for 534.396.285 records, which is not only a dramatic increase compared to the previous year, but also a probably unexpectedly high number for the just 40 targets approved by the FISA Court.

However, if each of these 40 targets called 50 numbers, and those numbers were also in contact with 50 numbers, we get some 100.000 phone numbers. Let's assume each pair of numbers was involved in 500 calls (or text messages), we already have 50.000.000 records. And this is still without duplicate records, like from multiple providers or recurring requests.

The large increase compared to 2016 may have been caused by a variety of factors, according to Alex Joel, ODNI's chief civil liberties officer: changes in the amount of historical data companies are choosing to keep; the number of phone accounts used by each target and changes to how the telecommunications industry creates records based on constantly shifting technology and practices.


These domestic call detail records may not be stored for more than 5 years after they were initially delivered to NSA. In addition, the minimization procedures require NSA to destroy promptly any records that are determined not to contain foreign intelligence information. Phone records that have been "the basis of a properly approved dissemination of foreign intelligence information" may be retained by NSA indefinitely.

After these records have been received and stored, they may also be queried, including using search terms associated with US persons. In 2016, NSA used ca. 22.360 search terms for such queries, while in 2017 that number had risen to 31.196.


Recently, it turned out that the practical implementation of the collection of domestic phone records under the USA FREEDOM Act is apparently not that easy: in a remarkable public statement from June 28, 2018, NSA revealed that several months earlier, "analysts noted technical irregularities in some data received from telecommunications service providers."

These irregularities occurred in a number of Call Detail Records (CDRs), which meant that NSA was not legally authorized to receive them in that form. It appeared infeasible to identify and isolate the properly produced data, so NSA concluded that it should not use any of these records.

Subsequently, the agency began deleting all the phone records they had acquired since 2015. According to the statement, NSA meanwhile addressed the root cause of the problem for future CDR acquisitions. Civil liberties blogger emptywheel suggests that the records may have contained content or location data, but NSA spokesman Chris Augustine said that the problem did not result in any collection of location records from cellphone towers.

According to the NSA's general counsel, Glenn S. Gerstell, the irregularities were caused by one or more providers who sent NSA data sets that also included some numbers of people the targets had not been in contact with. When the agency then fed those phone numbers back to the telecoms to get the "second hop" records, NSA acquired metadata of people with no connection to the approved targets.

Senator Ron Wyden, a longtime NSA critic who for years tried to get the Section 215 program disclosed, now blamed the providers instead of NSA for the technical problems: "Telecom companies hold vast amounts of private data on Americans," Wyden said. "This incident shows these companies acted with unacceptable carelessness, and failed to comply with the law when they shared customers’ sensitive data with the government."

Former assistant attorney general for national security David Kris said that these "errors illustrated how new problems can sometimes crop up when the government makes systems more complex in an effort to better balance security and privacy."


In the public statement it is said that the massive metadata deletion follows from the NSA's "core values of respect for the law, accountability, integrity, and transparency" but outsiders speculated about other motives: were these records destroyed before the Trump administration could misuse them? President Trump also tweeted about this issue and saw it as part of the "Witch Hunt" against him:

David Kris, former assistant attorney general for national security, replied to Trump that "This NSA program is only for international terrorism, not spying or clandestine intelligence activity, so unless your collusion included terrorism, it should be no problem for you personally!"


In early 2019, NSA suspended its collection of domestic phone records under the USA Freedom Act "after balancing the program’s relative intelligence value, associated costs, and compliance and data-integrity concerns caused by the unique complexities of using these provider-generated business records for intelligence purposes."

In February 2020, the Privacy and Civil Liberties Oversight Board (PCLOB) issued a report about the NSA's metadata collection under the USA Freedom Act. The Board considered the program "constitutional under settled Supreme Court precedent" and found "no abuse of the program; nor did it find any instance in which government officials intentionally sought records that they knew were statutorily prohibited." During its four years of operation, the program had cost 100 million USD (part of which was paid to the telecom providers). Metadata from the program were used in 15 intelligence reports, only two of which provided the FBI with unique information.

Links and sources
- Privacy and Civil Liberties Oversight Board: Report on the Government's Use of the Call Detail Records Program Under the USA Freedom Act (2020)
- TheMarketsWork.com: A Strange & Unsettling Day (2018)
- NYTimes.com: N.S.A. Purges Hundreds of Millions of Call and Text Records (2018)
- Emptywheel.net: AT&T Pulled Cell Location for its "Mobility Cell Data" (2015)
- HuffingtonPost.com: The NSA’s Telephone Metadata Program Is Unconstitutional (2014)


Unknown said...

Good work mate

Anonymous said...

In 2003 Admiral Poindexter’s Pentagon FutureMap program (a sub-project of the NSA Total Informational Awareness) was exposed as a futures betting operation on terrorist attacks and assassinations. Congress allegedly shut down the program but in actuality the program only went deeper into DARPA. Astute observers noted insider trading patterns hiding in plain site in the media associated with subsequent terrorist attacks, assassinations/assassination attempts and so called “natural” disasters during the Bush and Obama administrations. Numerous accurate warnings where thus sent to numerous government agencies (eg. NSA, CIA, FBI, Homeland Security, ect) via phone, FAX, email and C.R.R. mail. The response was NOT to use the massive collection of data to protect the American people and others from harm but quite the opposite. A Stasi like network (private sector merged with the gov’t/military/intel) was used/weaponized to target/terrorize/torture any whistleblower (and their children) that reported the aforesaid corruption, due to having a moral compass. Authorities who knew of this were gagged and also threatened. (See July 2016 Sen. Grassely letter to F.B.I. Comey chastizing him about gagging agents). Thus the Orwellian collection of data was used to PROTECT terrorists and others playing it both ways for financial gain and sadistic amusement (eg. similar to WWII Prescott Bush & IBM helping the NAZI’s) while targeting/terrorizing/torturing those the alleged Government of “We the People” was supposed to protect. (con’t in next post)

Anonymous said...

(con’t from previous post) Whistleblowers (and their children) were stalked, drugged, poisoned with biological agents and toxins, assaulted with sonic/microwave weaponry, sexually assaulted, had orchestrated car “accidents” occur along with incidents of arson, breaking & entering and theft. So called Verizon directory assistance operators (IDEARC) were used to harass and threaten. Schools, hospitals and CPS were weaponized to target whistleblower children, resulting in the destruction of the family/children’s lives. One such child was hospitalized, another one was kidnapped via false allegations, placed in a foster home that was burnt to the ground for insurance (Fire Chief later arrested for arson), then put in a Bible cult Church with a sign that said, “Even A Fish Can Stay Out of Trouble If It Keeps Its Mouth Shut”, then shot up with heroin and impregnated by a known criminal. Every conceivable method was employed to destroy their life and the lives of their children. (Sen. Schumer’s statement, “If you cross the intel agencies, they have six ways to Sunday, to get back at you.”) Every attempt made to discredit whistleblower. Doctor gave letter stating whistleblower was of sound mind then he was threatened that he should not have done that and found hanged, although his friends, family and colleagues stated he was not suicidal. Whistleblowers medical records were stolen out of the doctors office, like Watergate. Examples of media (and gov’t) corruption include the following: (recent) Superbowl Toyota Highlander Commercial when searched says it includes a biological disaster, Mercedez Benz FORTUNE teller commercial with reference to SNEEZE (among other interesting references), GEICO Pinnochio commercial (see before 9-11-01 Epidemic Marketing Commercial by DNA Productions, Geppetto Group (Italian puppet maker/master of Pinnochio, where person is playing it both ways by not spreading germs/washing hands then sneezing into hands/germs on hands and getting paid for each) All of these aired BEFORE the pandemic announced. Comfort Inn/Choice Hotel commercial “Badda Book Badda Boom” (Godfather/Sopranos/MOB) then Adriana Grande Terrorist attack when she was singing with Childish GAMBINO then subsequent attack on COLOMBO, Sri Lanka (Gambino and Colombo are two N.Y. MOB families), Knives Out movie then knife terrorist attack on London bridge and other similar attacks, Oct. 2005 Quaker Oatmeal Commercial of two cement mixer brothers with cement mixer then Cement mixer full of explosives bombed Sheraton and Palestine hotel in Baghdad, 2002 communique on Waaqiah.com by Ayman Al Zawahiri (OBL’s right hand man) stating Greetings from Paradise. See 10:10 (Two years before Oct. 12, 2000 Yemeni terror attack code call on U.S.S. Cole was 10:10:10 stated in book Holy War, Inc. by Peter Bergen). Paradise Ranch is pseudonym for Area 51 (Black Ops). Then communique by OBL entitled “Letter to the American People” with reference to Bounty like Mutiny on the Bounty (infighting between intel agencies and Pentagon at the time). Oct. 1, 2002 cover of National Geographic showed Bali with words Bali. Is it still a Paradise? Warning sent then Bali terrorist attack occurred on Oct. 12, 2002 next to Bounty nightclub...many more examples. NO COINCIDENCE. Payback by intel with a moral compass to use same sonic/microwave weaponry (used on whistleblowers to torture and discredit them) on embassy personnel in Cuba and China making it more difficult to use this weaponry on American Citizens in the future to torture/discredit them. See John LeCarre on NPR Fresh Air Show Sept. 5, 2017. Thomas Jefferson once said, “When the people fear the government, there is tyranny. When the government fears the people, there is liberty.” A patriot, Edmund Burke stated, “All that evil needs in order to triumph, is for good men to do nothing.”

In Dutch: Meer over het wetsvoorstel voor de Tijdelijke wet cyberoperaties