Continuing from part 1, in part 2 we will continue to go back in time to the life of Woody Bledshoe to explore the history of face recognition technology.
In 1960, Woody struck out with Browning and a third Sandia colleague to found a company of their own. Panoramic Research Incorporated was based, at first, in a small office in Palo Alto, California, in what was not yet known as Silicon Valley. At the time, most of the world’s computers – massive machines that stored data on punch cards or magnetic tape – resided in large corporate offices and government labs. Panoramic couldn’t afford one of its own, so it leased computing time from its neighbors, often late in the evenings, when it was cheaper.
Panoramic’s business, as Woody later described it to a colleague, was “trying out ideas which we hoped would ‘move the world.’ ” According to Nels Winkless, a writer and consultant who collaborated on several Panoramic projects and later became a founding editor of Personal Computing magazine, “Their function was literally to do what other people find just too silly.”
The company attracted an odd and eclectic mix of researchers – many of whom, like Woody, had grown up with nothing during the Great Depression and now wanted to explore everything. Their inclinations ranged from brilliant to feral. Browning, who came from a family of poor farmers and had spent two years of his youth eating almost nothing but cabbage, was a perpetual tinkerer. At one point he worked with another Panoramic researcher, Larry Bellinger, to develop the concept for a canine-powered truck called the Dog-Mobile. They also built something called the Hear-a-Lite, a pen-shaped device for blind people that translated light levels into sound.
Bellinger, who had worked as a wing-walker as a teenager (he kept the pastime secret from his mother by playing off his bruises from bad parachute landings as bicycle injuries), had also helped design the Bell X-1, the sound-barrier-breaking rocket plane made famous in Tom Wolfe’s The Right Stuff. Later he created the Mowbot, a self-propelled lawnmower “for cutting grass in a completely random and unattended manner.”
Then there was Helen Chan Wolf, a pioneer in robot programming who started at Panoramic a couple of years out of college. She would go on to help program Shakey the Robot, described by the Institute of Electrical and Electronics Engineers as “the world’s first robot to embody artificial intelligence”; she has been called, by one former colleague, “the Lady Ada Lovelace of robotics.” In the early 1960s, when Wolf’s coding efforts could involve stacks of punch cards a foot and a half high, she was awed by the range of ideas her Panoramic colleagues threw at the wall. At one point, she says, Woody decided that he “wanted to unravel DNA, and he figured out that it would take 30 or 37 years to do it on the computers that we had at the time. I said, ‘Well, I guess we won’t do that.’ ”
Perhaps not surprisingly, Panoramic struggled to find adequate commercial funding. Woody did his best to pitch his character-recognition technology to business clients, including the Equitable Life Assurance Society and McCall’s magazine, but never landed a contract. By 1963, Woody was all but certain the company would fold.
But throughout its existence, Panoramic had at least one seemingly reliable patron that helped keep it afloat: the Central Intelligence Agency. If any direct mentions of the CIA ever existed in Woody’s papers, they likely ended up in ashes in his driveway; but fragments of evidence that survived in Woody’s archives strongly suggest that, for years, Panoramic did business with CIA front companies. Winkless, who was friendly with the entire Panoramic staff – and was a lifelong friend of Browning – says the company was likely formed, at least in part, with agency funding in mind. “Nobody ever told me in so many words,” he recalls, “but that was the case.”
According to records obtained by the Black Vault, a website that specializes in esoteric Freedom of Information Act requests, Panoramic was among 80 organizations that worked on Project MK-Ultra, the CIA’s infamous “mind control” program, best known for the psychological tortures it inflicted on frequently unwilling human subjects. Through a front called the Medical Sciences Research Foundation, Panoramic appears to have been assigned to subprojects 93 and 94, on the study of bacterial and fungal toxins and “the remote directional control of activities of selected species of animals.” Research by David H. Price, an anthropologist at Saint Martin’s University, shows that Woody and his colleagues also received money from the Society for the Investigation of Human Ecology, a CIA front that provided grants to scientists whose work might improve the agency’s interrogation techniques or act as camouflage for that work. (The CIA would neither confirm nor deny any knowledge of, or connection to, Woody or Panoramic.)
But it was another front company, called the King-Hurley Research Group, that bankrolled Woody’s most notable research at Panoramic. According to a series of lawsuits filed in the 1970s, King-Hurley was a shell company that the CIA used to purchase planes and helicopters for the agency’s secret Air Force, known as Air America. For a time King-Hurley also funded psychopharmacological research at Stanford. But in early 1963, it was the recipient of a different sort of pitch from one Woody Bledsoe: He proposed to conduct “a study to determine the feasibility of a simplified facial recognition machine.” Building on his and Browning’s work with the n-tuple method, he intended to teach a computer to recognize 10 faces. That is, he wanted to give the computer a database of 10 photos of different people and see if he could get it to recognize new photos of each of them. “Soon one would hope to extend the number of persons to thousands,” Woody wrote. Within a month, King-Hurley had given him the go-ahead.
Ten faces may now seem like a pretty pipsqueak goal, but in 1963 it was breathtakingly ambitious. The leap from recognizing written characters to recognizing faces was a giant one. To begin with, there was no standard method for digitizing photos and no existing database of digital images to draw from. Today’s researchers can train their algorithms on millions of freely available selfies, but Panoramic would have to build its database from scratch, photo by photo.
And there was a bigger problem: Three-dimensional faces on living human beings, unlike two-dimensional letters on a page, are not static. Images of the same person can vary in head rotation, lighting intensity, and angle; people age and hairstyles change; someone who looks carefree in one photo might appear anxious in the next. Like finding the common denominator in an outrageously complex set of fractions, the team would need to somehow correct for all this variability and normalize the images they were comparing. And it was hardly a sure bet that the computers at their disposal were up to the task. One of their main machines was a CDC 1604 with 192 KB of RAM – about 21,000 times less working memory than a basic modern smartphone.
Fully aware of these challenges from the beginning, Woody adopted a divide-and-conquer approach, breaking the research into pieces and assigning them to different Panoramic researchers. One young researcher got to work on the digitization problem: He snapped black-and-white photos of the project’s human subjects on 16-mm film stock. Then he used a scanning device, developed by Browning, to convert each picture into tens of thousands of data points, each one representing a light intensity value – ranging from 0 (totally dark) to 3 (totally light) – at a specific location in the image. That was far too many data points for the computer to handle all at once, though, so the young researcher wrote a program called NUBLOB, which chopped the image into randomly sized swatches and computed an n-tuple-like score for each one.
Meanwhile, Woody, Helen Chan Wolf, and a student began studying how to account for head tilt. First they drew a series of numbered small crosses on the skin of the left side of a subject’s face, from the peak of his forehead down to his chin. Then they snapped two portraits, one in which the subject was facing front and another in which he was turned 45 degrees. By analyzing where all the tiny crosses landed in these two images, they could then extrapolate what the same face would look like when rotated by 15 or 30 degrees. In the end, they could feed a black-and-white image of a marked-up face into the computer, and out would pop an automatically rotated portrait that was creepy, pointillistic, and remarkably accurate.
These solutions were ingenious but insufficient. Thirteen months after work began, the Panoramic team had not taught a computer to recognize a single human face, much less 10 of them. The triple threat of hair growth, facial expressions, and aging presented a “tremendous source of variability,” Woody wrote in a March 1964 progress report to King-Hurley. The task, he said, was “beyond the state of the art of the present pattern recognition and computer technology at this time.” But he recommended that more studies be funded to attempt “a completely new approach” toward tackling facial recognition.
Over the following year, Woody came to believe that the most promising path to automated facial recognition was one that reduced a face to a set of relationships between its major landmarks: eyes, ears, nose, eyebrows, lips. The system that he imagined was similar to one that Alphonse Bertillon, the French criminologist who invented the modern mug shot, had pioneered in 1879. Bertillon described people on the basis of 11 physical measurements, including the length of the left foot and the length from the elbow to the end of the middle finger. The idea was that, if you took enough measurements, every person was unique. Although the system was labor-intensive, it worked: In 1897, years before fingerprinting became widespread, French gendarmes used it to identify the serial killer Joseph Vacher.
Throughout 1965, Panoramic attempted to create a fully automated Bertillon system for the face. The team tried to devise a program that could locate noses, lips, and the like by parsing patterns of lightness and darkness in a photograph, but the effort was mostly a flop.
So Woody and Wolf began exploring what they called a “man-machine” approach to facial recognition – a method that would incorporate a bit of human assistance into the equation. (A recently declassified history of the CIA’s Office of Research and Development mentions just such a project in 1965; that same year, Woody sent a letter on facial recognition to John W. Kuipers, the division’s chief of analysis.) Panoramic conscripted Woody’s teenage son Gregory and one of his friends to go through a pile of photographs – 122 in all, representing about 50 people – and take 22 measurements of each face, including the length of the ear from top to bottom and the width of the mouth from corner to corner. Then Wolf wrote a program to process the numbers.
At the end of the experiment, the computer was able to match every set of measurements with the correct photograph. The results were modest but indeniable: Wolf and Woody had proved that the Bertillon system was theoretically workable.
Their next move, near the end of 1965, was to stage a larger-scale version of much the same experiment – this time using a recently invented piece of technology to make the “man” in their man-machine system far more efficient. With King-Hurley’s money, they used something called a RAND tablet, an $18,000 device that looked something like a flatbed image scanner but worked something like an iPad. Using a stylus, a researcher could draw on the tablet and produce a relatively high-resolution computer-readable image.
Woody and his colleagues asked some undergraduates to cycle through a new batch of photographs, laying each one on the RAND tablet and pinpointing key features with the stylus. The process, though still arduous, was much faster than before: All told, the students managed to input data for some 2,000 images, including at least two of each face, at a rate of about 40 an hour.
Even with this larger sample size, though, Woody’s team struggled to overcome all the usual obstacles. The computer still had trouble with smiles, for instance, which “distort the face and drastically change inter-facial measurements.” Aging remained a problem too, as Woody’s own face proved. When asked to cross-match a photo of Woody from 1945 with one from 1965, the computer was flummoxed. It saw little resemblance between the younger man, with his toothy smile and dark widow’s peak, and the older one, with his grim expression and thinning hair. It was as if the decades had created a different person.
And in a sense, they had. By this point, Woody had grown tired of hustling for new contracts for Panoramic and finding himself “in the ridiculous position of either having too many jobs or not enough.” He was constantly pitching new ideas to his funders, some treading into territory that would now be considered ethically dubious. In March 1965 – some 50 years before China would begin using facial pattern-matching to identify ethnic Uighurs in Xinjiang Province – Woody had proposed to the Defense Department Advanced Research Projects Agency, then known as Arpa, that it should support Panoramic to study the feasibility of using facial characteristics to determine a person’s racial background. “There exists a very large number of anthropological measurements which have been made on people throughout the world from a variety of racial and environmental backgrounds,” he wrote. “This extensive and valuable store of data, collected over the years at considerable expense and effort, has not been properly exploited.” It is unclear whether Arpa agreed to fund the project.
What’s clear is that Woody was investing thousands of dollars of his own money in Panoramic with no guarantee of getting it back. Meanwhile, friends of his at the University of Texas at Austin had been urging him to come work there, dangling the promise of a steady salary. Woody left Panoramic in January 1966. The firm appears to have folded soon after.
With daydreams of building his computer person still playing in his head, Woody moved his family to Austin to dedicate himself to the study and teaching of automated reasoning. But his work on facial recognition wasn’t over; its culmination was just around the corner.
To be continue
Source: WiredRelated posts: