In early 2006, I sat in the Pashtun tribal areas on the Af/Pak border. When not rinsing grit out of my teeth from the thick fog in the air, I awaited information from capricious “allies,” on whether their informant could identify a senior al Qaeda leader who I’ll call “The Moroccan.” At the time, he was al Qaeda’s regional operational commander and well worth removing from the board.

I lusted for certainty on the whereabouts of the Moroccan. He remained stubbornly elusive, while I chafed with impatience aggravated by living conditions halfway between a prison and an armed bunker. I watched tantalizing video from “eyes in the sky” of vehicles associated with the Moroccan heading towards the funeral of a militant. The funeral was attended by hundreds of men, all sporting traditional bushy beards and the long, baggy shalwar-kameezs Pashtuns favored, topped by either a turban or pakol cap. Making a positive ID in that crowd was like locating a particular stalk of wheat in a bumper crop. Was the Moroccan there? Or in a safehouse 100 miles away?

Because we tried not to create more enemies with indiscriminate attacks than we killed, we didn’t take a shot that day. This was only a few months after the Damadola strike, intending to kill al Qaeda No 2, Ayman al-Zawahiri, but instead killing 19 locals. The Moroccan remained alive until a few years later, when he was killed by a Hellfire missile.

Unfortunately, before his subsequent demise, the Moroccan allegedly helped assassinate a major political reformer hoping to modernize a destitute and corrupt country.

The face recognition genie

Identification tech has come a long way since the hunt for the Moroccan, but not always to society’s benefit. Face recognition is still not on most Westerners’ list of top 100 issues to worry about. And it should be.

Even Google, years before it quietly axed the motto, “Don’t Be Evil,” opted out of face recognition tech. Then-Google CEO Eric Schmidt explained it was the only tech Google had built and then decided not to use because it could be used “in a very bad way.”

Clare Garvie, an attorney at the Georgetown Center for Privacy and Technology, testified before Congress in 2019 about the misuse of face recognition technology in the US. It wasn’t pretty.

Garvie told the House Committee on Oversight and Reform, “Face recognition presents a unique threat to the civil rights and liberties protected by the US Constitution.”

After her testimony, I asked Garvie why unique? She told me face recognition chills First Amendment free speech, and as used by many police departments, outright violates Fourth Amendment rights on unreasonable search, and 14th Amendment rights on due process. She obtained over 20,000 pages of documents from Freedom of Information Act requests and lawsuits, and found the majority of US police departments that had face recognition tech were not forthcoming about how it was used, or even whether they had it.

The NYPD, for example, claimed it had no documents on face recognition. It later turned over 2,000 pages on it when forced to. The NYPD was found to have fabricated evidence repeatedly. In one case, it tried to catch a beer thief that face recognition software couldn’t ID by using a photo of Cheers actor Woody Harrelson. The detectives thought the thief looked like him, so they fed in Harrelson’s photo to try to force a match from the software. Police routinely used falsified “probe photos” when they couldn’t get a match from the software.

Privacy laws and civil rights lawsuits, if they ever catch up to surveillance overreach, take years to get through the courts, at tremendous expense. Clearview AI, one of the biggest vendors of face recognition data and services to police, private companies, and even rich individuals, faces multiple class action lawsuits for a litany of legal and regulatory violations. Clearview’s clients include 2,200 domestic and foreign law enforcement agencies, the DOJ, DHS, universities, Walmart, Best Buy, Macys, and on and on. Did Clearview AI ask your permission to use photos you posted to Facebook or Twitter or Instagram in a criminal line up? No? Clearview AI lets police, or a random corporate cog at Walmart, do it anyway, even in localities where that has been specifically ruled illegal.

To make matters worse, face recognition tech can do a horrible job in identifying non-white suspects, putting non-Caucasians at higher risk for “false positives,” e.g., being arrested for a crime they did not commit because of a bad match. Per 2019 testing by the National Institute of Standards and Technology (NIST), “False positive rates often vary by one or two orders of magnitude (i.e., 10x, 100x),” depending on the racial or ethnic group being tested.

Amazon also sells a face recognition service, Rekognition. Rekognition has been excoriated for providing inadequate, incorrect, or non-existent training and documentation to clients, which inevitably leads to misuse. In a Washington Post article on Rekognition, the police department that helped develop it, the Washington County Sherriff’s Office in Oregon, stated it never had a complaint from defendants or defense attorneys on the use of face recognition software. But detective Robert Rookhuÿzen also said: “Just like any of our investigative techniques, we don’t tell people how we catch them. We want them to keep guessing.”

Isn’t that obstruction of justice? US police are required to share their evidence with defense attorneys, particularly exculpatory evidence, e.g., a search for a criminal suspect generated multiple matches in the face recognition software. As Georgetown’s Garvie pointed out, refusing to share those matches is a breach of the 14th Amendment.

Clearview AI and Rekognition are but two of dozens of developers of face recognition technology in the US. While much of it has been improving, it is still a mixed bag. Some versions of it can achieve a 99 percent match rate under ideal conditions. Under less-than-ideal conditions, that number can fall to 50 percent. And the rate of accurate matches from composite sketches is unbelievably low, with as few as 20 percent valid matches.

None of these shortfalls are highlighted to the public, defendants, and quite frequently, even the police using the face recognition systems. Instead, cops are told by vendors like Amazon that it works like magic. Per Rekognition’s website, “Deep learning uses a combination of feature abstraction and neural networks to produce results that can be … indistinguishable from magic.” Perhaps because “magic” is involved, some police don’t seem to grasp that face recognition has to be used within pre-existing rules of evidence and precedent.

Some police agencies do require face recognition matches to be reviewed by trained human experts, as the FBI does. But as Steven Talley can attest, human face recognition experts are not infallible either.

Talley was falsely arrested twice for bank robbery even after police had video footage showing Talley at his job while the robbery occurred. The second arrest, disregarding both fingerprint evidence and a rock-solid alibi, was based on the opinion of an FBI face recognition expert.

The arrests cost Talley his job, home, marriage, and access to his children. Talley also suffered a broken sternum, broken teeth, ruptured discs, blood clots, nerve damage, and a fractured penis during the first brutal false arrest.

Face recognition is a shinier, sexier iteration of the implicit promise justifying mass installation of CCTVs years ago. It ran, “If only we unleash unblinking and ever-present surveillance technology, we’ll reap a harvest of safety.”

The facts show otherwise. The largest ever meta-analysis on surveillance cameras, including 40 years of data, is that they reduce car theft and other property crime by 16 percent. And violent crime not at all. They can help catch violent criminals and terrorists after the fact, but surveillance cameras do not prevent violent crime.

Jetson

In 2019, the MIT Technology Review reported the US Special Operations Command (SOC) had asked for a new system for positive identifications on the battlefield. The goal was to help snipers ID terrorist/insurgent targets.

The gadget was dubbed Jetson. It is an infrared laser that, in 30 seconds, at distances of 200 meters (close to 700 ft), claims to identify an individual via his or her heartbeat, a “heartprint,” if you will, with 95 percent or greater accuracy, using laser vibrometry. The 200 meter range figure is likely an underestimate of what Jetson can do, as is the 95 percent accuracy figure. Steward Remaly, the program manager who ran Jetson development for the DOD’s Combatting Terrorism Technical Support Office (CTTSO)* told MIT that longer ranges and greater accuracy should be possible (and the DOD rarely publicizes the full capability of new gear).

The first mention of Jetson I found was the CTTSO’s 2017 Review Book, which said it will ID targets within five seconds. That five-second figure was absent in the CTTSO’s 2018 Review Book. When the 2019 MIT story appeared a year later, it said 30 seconds, as did all subsequent coverage. We don’t know which figure is more accurate.

In February 2021, with help from DOD public affairs, I interviewed Remaly, the Jetson program manager at CTTSO, about Jetson. Remaly was amiable but vague in most of his responses. The main takeaways from the Remaly interview were that Jetson was developed by Ideal Innovations Incorporated, aka I3, working with Washington University in St Louis. And that Jetson development started in 2010.

A start-date of 2010 puts a different spin on Jetson’s status. Instead of a pilot project cobbled together in 2017–2018, Jetson is the fruit of eight years of R&D, suggesting a mature piece of tech was delivered to the Army by the CTTSO. The CTTSO’s 2018 Review Book designated Jetson a “completed project” almost a year before the MIT article ran. The Army has now had that completed project for at least two years, possibly longer.

And it was impressive when it was handed over. The CTTSO’s Remaly told me: “It was great equipment; I’ll tell you that. And Ideal innovations and Washington University frankly did a great job on the parts they were asked to work on, and they got it to the stage that we needed it.”

A win for the good guys?

If the team hunting the Moroccan had Jetson technology, could we have taken him out? Possibly. Certainly, it could have helped figure out who wasn’t the Moroccan, and ruling out targets is huge in a manhunt. For want of a nail, the shoe was lost, etc. The team I served on couldn’t identify the Moroccan in a sea of Pashtun fighters. This meant he lived on to continue to wreak havoc.

You don’t join the military or intelligence community, and I’ve been in both, without endorsing the sentiment behind the Orwell quote, “We sleep soundly in our beds because rough men stand ready in the night to visit violence on those who would do us harm.” My natural impulse is to want to put more effective tools in the hands of those “rough men.”

Reducing unintended battlefield deaths is good. It is not just a reduction in needless killing; one of the biggest creators of motivated insurgents/terrorists is the death of innocents. Jetson would be a boon on asymmetrical battlefields like Afghanistan and Iraq, where only the Americans and our allies wear uniforms and follow formal codes like “rules of engagement.” While we abide by self-imposed constraints, insurgents callously use civilians as human shields, and then gleefully make hay from the collateral damage.

If Jetson works as advertised (or quite possibly, much better), it will make that tactic less effective and hopefully reduce our enemies’ reliance on it.

So, hooray for our SOC warriors and intel operators getting Jetson.

Unfortunately, we have no reason to think only those two groups will get it. For a surveillance technology as game-changing as Jetson, the battlefield is a tiny fraction of the potential market.

You can’t hide from Unique

A challenge in reporting on Jetson was finding cardiologists to examine its central claim—the mysterious ability to ID people via remote measurement of their heartbeat. The majority of cardiologists I contacted declined to comment about Jetson, and wouldn’t say why.

Dr. Ronald Berger of Johns Hopkins is a cardiologist and a biomedical engineer and a professor of both. Despite Dr. Berger’s stellar credentials, he was also reluctant to comment. I told Dr. Berger that Steward Remaly said Jetson produced an ECG-like reading, but I speculated that Remaly might have used the term ECG because it is a common term for a device that measures the heart.

Dr. Berger emailed back: “Seems possible (for Jetson) to get an assessment of heart rate, but not ECG … Even if you had an ECG, it’s not quite like a fingerprint. Even if it were, you would need to know their individual pattern ahead of time in order to ‘recognize’ the individual. If all you have is a pulse reader, then I cannot see how that would allow for individual identification.”

“Sorry,” Dr. Berger wrote, “I don’t know anything about the technology… I don’t really want to guess.”

Dr. Berger was right to be hesitant. In trying to pierce the veil of claims surrounding Jetson, and figure out just what the laser is measuring, I started to think it might work via “computer aided auscultation” (i.e., a fancy laser stethoscope). The hypothesis fit many known facts and suppositions.

But I was wrong.

Further digging invalidated the auscultation hypothesis. A tip identified the engineer (who did not respond to requests for comment) that ran Jetson development at Washington University. A review of his work turned up the 2009 research paper, ‘Laser Doppler Vibrometry Measurements of the Carotid Pulse: Biometrics Using Hidden Markov Models.’

Jetson development began the year after this paper was published. And the smoking gun, Jetson’s mechanism of action, was in the abstract.

Small movements of the skin overlying the carotid artery, arising from pressure pulse changes in the carotid during the cardiac cycle, can be detected using the method of Laser Doppler Vibrometry (LDV) … these variations are shown to be informative for identity verification and recognition … The resulting biometric classification performance confirms that the LDV signal contains information that is unique to the individual. [emphasis added]

This was expanded on in a 2012 paper, ‘Hidden State Models for Noncontact Measurements of the Carotid Pulse using a Laser Doppler Vibrometer.’ A 2016 paper, ‘Cardiorespiratory interactions: Noncontact Assessment Using Laser Doppler Vibrometer‘ noted LDV was useful on a non-contact basis, required no skin preparation, and could be used in harsh environments.

Harsh environments like the battlefield.

Dr. Berger was more forthcoming after I sent him the abstracts. While noting he didn’t have time to do a full review of the literature, he wrote, “This seems like a credible explanation of how it works and what it does.”

I3 refused to discuss Jetson, but I3’s website does offer this “Standoff Biometric Service.”

I3 designed, developed and demonstrated [sic] man-portable system for heart biometrics using a standoff laser vibrometer to measure pulse response at distances up to 200m. This system enables the operator to covertly collect heart biometrics from multiple body locations, including some that might be covered by clothing. The matching algorithm uses unique aspects of the heart pulse shape to create biometric signatures that cannot be disguised by the subject. [emphasis added]

There is that word again—unique. First used in the 2009 academic paper describing what might become Jetson, and now used by I3 advertising Jetson tech as a service. What does a unique identifier mean? It means far better than fingerprints. It means when Jetson gets a clean heartprint from a target (clothes, smoke, rain, or movement could all make that harder), it will be as good or better than a DNA fingerprint. And no messy lab-work necessary.

Jetson is a “win” for the US military and intelligence community, and both fortunately operate under federal oversight.

But there is no better advertisement for a new surveillance tech than that the “Tier One” operators of US Special Operations Command have it. What police department or intelligence agency in the world wouldn’t want the new tech? And what biometric company wouldn’t want to sell it to them? (Note: much of I3’s other business comes from selling face recognition services.)

We should probably think a dozen times before any organization not subject to strict federal oversight, e.g., local law enforcement, gets access to an identification technology as powerful and unaccountable as Jetson.

Google’s China Ambitions Threaten U.S. National Security

What can we do about China’s potential future ability to harm American politicians through influencing Internet companies that operate in both China and the United States?

QuilletteJames D. Miller

Where did Jetson go?

Steward Remaly told me twice he’d transferred responsibility for the tech to the Army Night Vision Lab when CTTSO concluded development. As an engineer and former Army colonel himself, it is unlikely he mixed that up.

When I reached out to the Army Night Vision Lab to get Jetson’s status, Richard Levine from Army Public Affairs advised me,

I checked with the U.S. Army Combat Capabilities Development Command, and the Command, Control, Communications, Computers, Cyber, Intelligence, Surveillance and Reconnaissance Center, which runs the Night Vision and Electronic Sensors Directorate—which I think is what you meant when you wrote the “Army Night Vision lab.”

No one has heard of this program.

I was subsequently told, unofficially, it might have gone to the Air Force, then that the Army did indeed have Jetson, somewhere.

Then, per a DOD spokesman Mike Howard,

The Army was asked to conduct a standard technical assessment when we sent the research to them in 2019. However, because this is proprietary information, it would be neither appropriate nor prudent to publicly discuss any aspect of the assessment.

But it could also be that the DOD opted not to discuss Jetson further when they saw my follow-up questions to the Remaly interview. They included, “To whom will Jetson be licensed? Will it be shared with allies? What policy work has DOD done on ‘rules of engagement’ for Jetson? Will it be made available to domestic law enforcement?”

My questions as to possible restrictions on who might get to use it, or how it gets used, were ignored. But I was assured Jetson’s intellectual property would be protected.

According to a DOD spokesman,

The United States welcomes fair and open market competition. We stand against policies or actions that undermine international norms. This includes theft of commercial intellectual property, forced joint venture and licensing requirements, and state-funded acquisitions to transfer sensitive technology.

International norms will be followed, for an ID technology that has never before existed. Ergo, any norms of use will arise from how the DOD, or anyone DOD authorizes it be sold to, decides how to use it.

Who has Jetson and who else is likely to get it?

My best guess of who has custody of the Jetson program is the Army’s Defense Forensics and Biometrics Agency (DFBA), whose motto is “Deny the Enemy Anonymity.” The guess rests on a few points. Jetson is a new biometric device for IDing enemies. I3 has three different contracts with the DFBA worth over $76 million. DFBA lists counterinsurgency as part of its mission, explaining on its website, “Terrorists, foreign fighters, and insurgents utilize anonymity to shield themselves from US Forces.” One of I3’s contracts include support to the DFBA’s Biometrics Operations Division (a division not mentioned on the DFBA’s website).

What we don’t know and can’t guess is who will be allowed, or has already been allowed, to buy Jetson as a device or service. We know per CTTSO’s 2018 Review Book that the Special Operations Command and the Intelligence Community will get access to it first. But both of those groups have a network of local and international tech and data sharing arrangements. Additionally, the 2018 Review Book notes, “CTTSO directly manages bilateral agreements with five partner countries: Australia, Canada, Israel, Singapore, and the United Kingdom … In addition to CTTSO’s bilateral partners, CTTSO cooperates with other countries when appropriate.”

Also, the Intelligence Community is not just the 17 US intelligence agencies. It potentially includes terrorism-info sharing arrangements, like the “Fourteen Eyes” partnership, which includes the US, UK, Canada, Australia, New Zealand, Germany, Spain, Sweden, Belgium, Italy, Denmark, Norway, France, and the Netherlands. We can’t know if some, all, or none of the various partnered law enforcement and intelligence agencies will get access to Jetson.

Or if your local cops are already buying Jetson or testing it right now.

More than halfway done

On the battlefield, Jetson could be employed with a streamlined process. If you are a US soldier, you don’t need to know the name or face of your enemy, just that he’s participated in hostile activity. Once Jetson has remotely surveyed an armed enemy combatant and captured their “heartprint,” the next time Jetson encounters that enemy, the soldier will know from the heartprint match the person is an enemy combatant, even if the combatant is at that moment unarmed and hiding among civilians.

In the civilian world, Jetson will have to be paired with another ID method that already has a name attached to it. That could be fingerprints, DNA, gait analysis, an iris scan, but far and away the most likely candidate is a face recognition scan, as face photos comprise the largest biometric databases by far (the MIT article noted Jetson was likely to be paired with face recognition).

Assume you are James Doe, standing in a crowd at a demonstration (or sitting on a park bench, or at a stoplight in your car, or standing in line at a DMV). Jetson’s optical tracker is aimed at you, which kicks on a beam director, which gimbals Jetson’s laser into alignment, and shoots the invisible infrared beam at you from a box about 8×8″ (20 cm) square. You won’t see the box, let alone the invisible laser, because it is hundreds of feet away from you. The interferometers sensing reflections of the IR laser quantify the individual-specific movements in your skin from the combination of your carotid pulse and breathing.

Now your unique “heartprint” is in a Jetson database.

To usefully identify and track you, that heartprint will have to be attached to a name. A camera linked to a face recognition database will search for a picture of James Doe. If it finds one, or one “close enough” according to the face recognition algorithm, the heartprint and face match will be a permanently linked record. Thereafter, no further use of camera or face recognition software is necessary; you will be considered positively IDed any time another Jetson laser is aimed your way and matches the heartprint on file.