A Brief History of Autonomous Vehicles

Hey, Frank came back with you here, and I know you're probably itching to get into some code here, but let's take a few minutes just to talk about the history of self driving cars and where you fit into that history. It's really a fascinating story and someone should make a documentary out of it. The idea of driverless cars actually goes all the way back to 1925. There was this guy named Francis Houdina, and he modified a 1926 Chandler car to be remote controlled. Basically, he stuck that big antenna on the top there that you see and a car driving behind it actually controlled the car remotely using a wireless connection. So although it wasn't really an autonomous vehicle that was making its own decisions, it was the first vision of a driverless car that we know of. And pretty exciting concept there, right? Unfortunately, it didn't really end well. This was in New York. And basically there was a loose connection to the steering column that made it lose control toward the end of its demonstration. It ended up crashing into a car full of cameramen. So not the most happy ending. Houdina was actually clinging to the running board of the car as it ran so he could jump in and take over control if he had to in case of an emergency. Just like we have a backup. Human drivers in self-driving cars today that are being developed. Unfortunately, he wasn't quite fast enough and it ended in a collision. But we think there may have been more successful tests of this following this event. But the documentation from back then is kind of spotty. It's hard to really know what happened. The story gets even more interesting because you might have noticed that Francis Houdina sounds an awful lot like Harry Houdini, who was the famous escape artist and magician at the time. Right. And it turns out Houdini had this big feud going on with Houdini because he wasn't happy that he was using a name that sounded like his. There are actually stories of Houdini breaking into Houdini's office and trashing it. There's probably a whole movie there, but it's just fascinating stuff. Fast forward to 1939, where we had the New York World's Fair and the New York City. One of the exhibits there was called City of the Future, and it featured this exhibit. This is real here of actual self driving electric cars. Sound familiar in this little virtual city here that they constructed and the cars were actually running by radio control and they were guided by these circuits embedded in the road. So they were basically running on these invisible tracks, if you will. But still, they were self driving cars being displayed at the World's Fair in 1939. And the guy behind that exhibit, his name was Bel Geddes, and he published a book the next year called Magic Motorways. It's fascinating stuff. It's actually available in the public domain. You can go to archive.org and look it up if you're interested. He has a whole chapter just talking about how the future is basically the modern interstate highway system with divided highways. This is 25 years before that existed. And he also predicted that those highways would have driverless cars on them. And he has this whole chapter devoted to just talking about how much better the world would be if human beings didn't drive from a safety standpoint because humans are messy creatures. Right. And we don't always make the right decisions. And he was really the first guy that we know of that really envisioned the safety benefits of self driving cars on a large scale. He actually predicted this would be reality by 1960. And obviously he was a little bit early with that prediction. But it's always fun to look at these predictions of the future from way back then and how they compare to what actually happened in the 50s and 60s. Some research continued to go on. It started to get a little bit more serious. In 1957, RCA set up a 400 foot length of public highway in Nebraska. That's in the middle of the United States, where, again, they embedded detectors in the road that sent impulses to guide the car. And it actually worked. They repeated that experiment again in 1960 in New Jersey, and they expected commercialization of the system in 1975, which obviously didn't happen. This picture on the left here is an actual picture of that system. You can see they actually blacked out the driver's side window there just to prove to people that they weren't cheating. The car was really driving itself just using a pretty simple electromechanical system there. But it worked. At the same time. The UK was doing their own experiments around 1960. They had these modified citroens that could steer themselves again using magnetic cables embedded in the roadway itself. So basically the state of the art in the 50s and 60s was putting stuff inside the roadway to guide these cars automatically. And also it worked. But unfortunately funding for that was cancelled in the 1970s, partially because they can never figure out the problem of how to change lanes safely with a system like that. Unfortunately, nothing really happened in the 70s worth talking about and that seems to be the case in a lot of fields. But by the late 80s things got really interesting. In 1987, h.R. Laboratories, which used to be known as Hughes research laboratories, built this system using lidar, just like we do today. Computer vision systems, just like we do today. An autonomous robotic control to create a truly self driving autonomous vehicle. Now, this was all developed for a DARPA funded autonomous land vehicle project. That's Alvey for short. And for those of you who aren't familiar with DARPA, that's a US military research organization that exists to this day, and it's been largely forgotten, which is really kind of weird. I mean, this was kind of really pioneering stuff, and we hear a lot about the DARPA Grand Challenge, but people forgot about the Alvey project. Now, it only traveled at 1.9mph, but it worked. It actually did this over 2000ft of a course over steep slopes, rocky terrain and through vegetation. And this actually proved to be a very challenging feat to reproduce even more than a decade later. Also in 1989, we start to see the first use of neural networks to steer vehicles. So that was pioneered by Carnegie Mellon University or CMU, and they continued to be a driving force in the field of self-driving car research going forward to this day. Let's jump ahead to the 90s. So the United States Congress actually had a bit of a vision. In 1991, they directed the Department of Transportation to quote, demonstrate an automated vehicle and highway system by 1997, unquote. And it actually happened again. You don't hear about this very much, but this is 1997 here, guys. The project involved a bunch of different organizations and administrations and companies, including the Federal Highway Administration, General Motors, Caltrans, CMU, again, University of California at Berkeley, Lockheed Martin and a bunch of others. They called the culmination of this project, Demo 97. And in San Diego on Interstate 15, they actually demonstrated about 20 automated vehicles, including cars and buses and trucks, all self driving on an actual interstate highway. They demonstrated something called platooning, as you can see here on the left here. That's the idea of having these cars kind of traveling in a convoy together. And that's the display they actually saw inside their cars. And they also had what they call free agent vehicles that were able to operate in mixed traffic conditions as well. Now, unfortunately, the funding for this project was subsequently cut in 1998 just due to federal budget constraints. So that was the end of this program. And unfortunately, they never published a single technical report on how they did it. So a lot of that work was kind of lost to the sands of time, believe it or not, even after they chewed through $90 million of public funding. So exciting results in 1997, but not really a happy ending because the program was terminated and nothing really came out of it, apart from know how that hopefully people involved in the project took on to their subsequent careers in the field. Also during the 90s there were several semi autonomous demonstrations, including another CMU project that was called No Hands Across America that drove cross country across the United States. That is with autonomous steering over 98% of the time. There was also a demo in Paris that involved driving in free lanes, lane changes and passing other cars at over 80mph. But they did have some human intervention involved in that. In 1996, the University of Parma, which is also been a big player in self-driving car research, modified a vehicle to follow Painted lane markings on unmodified highways. And they did this for 1200 miles through northern Italy. Now, to be fair, the longest autonomous stretch where no human intervened was only 34 miles. But hey, that's pretty darn good for a 1996. And all they use for technology was two black and white video cameras and a stereoscopic analysis in a computer of the input from those very inexpensive cameras. That's all they used technically. The first driverless vehicle was the park shuttle in the late 90s in the Netherlands, and that used magnets embedded in the road to guide it. It was only used as a pilot project in like an airport there and a little research park that they had. But technically since it actually did drive real people around and had no driver at all that qualified as the first actual driverless vehicle. Now in the 2000s, things get really interesting. This is the age of the DARPA grand challenges. And again, DARPA is a US military research branch. They offered a $1 million prize to any team who could create an autonomous car capable of finishing a 150 mile course in the Mojave Desert. Now, they had three different grand challenges throughout the 2000. The first 1 in 2004 did not yield any winners. Everybody either crashed or failed or caught fire. But in 2005, the next year, five vehicles actually completed the desert course. And you can see one of those vehicles here, Terramax is what it was called, very beefy machine. And they made extensive use of things like LiDAR and computer vision obviously played a big role in that, which is what this course is about. And in 2007, they did it a third time, this time in an urban environment which you see on the right here. And CMU won. Our friends Carnegie Mellon University with a modified 2007 Chevy Tahoe and a lot of the participants in the 2007 Grand Challenge published their research in algorithms and even the entire source code for computer vision. And that really jumpstarted the field. A lot of new LiDAR sensors came out of this work, and ultimately Google's self-driving car team was quietly assembled in 2008 using many of the participants in the DARPA Grand Challenge. Today, Google's self-driving car project is called Waymo, and it's still a pioneer in creating new computer vision algorithms and technologies in self-driving cars. That brings us to the present decade. And today, more and more autonomous vehicle technology is making its way into real cars on the real roadways. I personally own a Tesla, which is advertised as containing all of the hardware needed for full self-driving. It includes an array of cameras surrounding the car radars and a dedicated computer for processing all of that data in real time. Pretty much every vehicle manufacturer is incorporating semi-autonomous features such as adaptive cruise control, lane change warnings and assisted steering into their lineups. And those are all the same components that will make up full self-driving capabilities. Today I can drive my Tesla on the highway with no intervention by me. In most cases, it uses computer vision to follow lane markings, even in bad weather very effectively, and its cameras and radars keep me a safe distance from other vehicles in the road, even in heavy traffic. It can even change lanes automatically just as smoothly as a human driver could. So the goal of full self-driving seems within reach, but it's still eluding us. There are times when I need to take control back from my Tesla due to emerging traffic or obstacles in the road or road construction, it can't yet read traffic signs or stoplights or deal with urban driving at all. Much of the underlying technology exists in the lab, but getting it to work reliably enough in the real world under every conceivable situation remains a challenge. And we must get it right as human lives are at stake here. Already there have been some high profile news cases about fatalities from an Uber self-driving car and from Teslas with their autopilot mode engaged. Even if autonomous vehicles are safer than human drivers. Accidents involving autonomous vehicles will attract much more media attention than the over 15,000 car crashes that happen every day in the US alone. But with the underlying technology making it onto real roadways more and more every day, it seems inevitable that full self-driving will be a reality soon. And this course will teach you the computer vision and machine learning algorithms these technologies rely on so you can be a part of this revolution as it happens.