A Brief History of Autonomous Vehicles

Sundog Education by Frank Kane
A free video tutorial from Sundog Education by Frank Kane
Founder, Sundog Education. Machine Learning Pro
4.5 instructor rating • 22 courses • 492,887 students

Lecture description

We'll cover the history of self-driving cars, which starts in 1925 and includes a lot of exciting progress that's been largely forgotten!

Learn more from the full course

Autonomous Cars: Deep Learning and Computer Vision in Python

Learn OpenCV, Keras, object and lane detection, and traffic sign classification for self-driving cars

12:44:32 of on-demand video • Updated April 2021

  • Automatically detect lane markings in images
  • Detect cars and pedestrians using a trained classifier and with SVM
  • Classify traffic signs using Convolutional Neural Networks
  • Identify other vehicles in images using template matching
  • Build deep neural networks with Tensorflow and Keras
  • Analyze and visualize data with Numpy, Pandas, Matplotlib, and Seaborn
  • Process image data using OpenCV
  • Calibrate cameras in Python, correcting for distortion
  • Sharpen and blur images with convolution
  • Detect edges in images with Sobel, Laplace, and Canny
  • Transform images through translation, rotation, resizing, and perspective transform
  • Extract image features with HOG
  • Detect object corners with Harris
  • Classify data with machine learning techniques including regression, decision trees, Naive Bayes, and SVM
  • Classify data with artificial neural networks and deep learning
English [Auto] Hey, Frank came back with you here, and I know you're probably itching to get into some code here, but let's take a few minutes just to talk about the history of self-driving cars and where you fit into that history. It's really a fascinating story. And someone should make a documentary out of it. The idea of driverless cars actually goes all the way back to 1925. There was this guy named Francis Udine and he modified a 1926 Chandler car to be remote control. Basically, he stuck that big antenna on the top there that you see, and a car driving behind it actually controlled the car remotely using a wireless connection. So although it wasn't really an autonomous vehicle that was making its own decisions, it was the first vision of a driverless car that we know of. And pretty exciting concept there, right? Unfortunately, it didn't really end well. This was in New York and basically there was a loose connection to the steering column that made it lose control. Toward the end of its demonstration, it ended up crashing into a car full of cameramen. So not the most happy ending. Houdini was actually clinging to the running board of the car as it ran so he could jump in and take over control if he had to. In the case of an emergency, just like we have backup human drivers in self-driving cars today that are being developed. Unfortunately, he wasn't quite fast enough and it ended in a collision. But we think there may have been more successful tests of this following this event. But the documentation from back then is kind of spotty. It's hard to really know what happened. The story gets even more interesting because you might have noticed that Francis Hudner sounds an awful lot like Harry Houdini, who was the famous escape artist and magician of the time. Right. And it turns out Houdini had this big feud going on with Houdini because he wasn't happy that he was using a name that sounded like his are actually stories of Houdini breaking into Houdini's office and trashing it. There's probably a whole movie there, but it's just fascinating stuff. Fast forward to 1939, where we have the New York World's Fair in the New York City. One of the exhibits there was called City of the Future, and it featured this exhibit and this is real here of actual self-driving electric cars. Sound familiar? In this little virtual city here, they constructed and the cars were actually running by radio control and they were guided by these circuits embedded in the road. So they were basically running on these invisible tracks, if you will. But still, they were self-driving cars being displayed at the World's Fair in 1939. And the guy behind that exhibit, his name was Belle Geddes, and he published a book the next year called Magic Motorways. It's fascinating stuff. It's actually available in the public domain. You can go to archive dog and look it up if you're interested. He has a whole chapter just talking about how the future is basically the modern interstate highway system with divided highways. This is 25 years before that existed. And he also predicted that those highways would have driverless cars on them. And he has this whole chapter devoted to just talking about how much better the world would be if human beings didn't drive from a safety standpoint, because humans are messy creatures. Right. And we don't always make the right decisions. And he was really the first guy that we know of that really envisioned these safety benefits of self-driving cars on a large scale. He actually predicted this would be reality by 1960. And obviously he was a little bit early with that prediction. But it's always fun to look at these predictions of the future from way back then and how they compare to what actually happened in the 50s and 60s. Some research continued to go on. They started to get a little bit more serious. In 1957, RCA set up a 400 foot length of public highway in Nebraska. That's in the middle of the United States, where, again, they embedded detectors in the road that sent impulses to guide the car. And it actually worked. They repeated that experiment again in 1960 in New Jersey and they expected commercialization of the system in 1975, which obviously didn't happen. This picture on the left here is an actual picture of that system. You can see they actually blacked out the driver's side window there just to prove to people that they weren't cheating. The car was really driving itself, just using a pretty simple electromechanical system there. But it worked. At the same time, the UK was doing their own experiments around 1960. They had these modified psytrance that could steer themselves again using magnetic cables embedded in the roadway itself. So basically the state of the art in the 50s and 60s was putting stuff inside the roadway to guide these cars automatically. And also it worked. But unfortunately, funding for that was cancelled in the 1970s, partially because they can never figure out the problem of how to change lanes safely with a system like that. Unfortunately, nothing really happened in the 70s worth talking about, and that seems to be the case in a lot of fields, but by the late 80s, things got really interesting. In 1987, HRO laboratories, which used to be known as Hughes Research Laboratories, built this system using Lider just like we do today, computer vision systems, just like we do today, and autonomous robotic control to create a truly self-driving autonomous vehicle. Now, this was all developed for a DARPA funded autonomous land vehicle project. That's Alvie for Short. And for those of you who aren't familiar with DARPA, that's a U.S. military research organization that exists to this day. And it's been largely forgotten, which is really kind of weird. I mean, this was kind of really pioneering stuff. And we hear a lot about the DARPA Grand Challenge, but people forgot about the ALV project. Now it only traveled at one point nine miles per hour, but it worked. It actually did this over 2000 feet of a course over steep slopes, rocky terrain and through vegetation. And this actually proved to be a very challenging feat to reproduce even more than a decade later. Also in 1989, we start to see the first use of neural networks to steer vehicles that was pioneered by Carnegie Mellon University or CMU, and they continue to be a driving force in the field of self-driving car research going forward to this day. Let's jump ahead to the 90s. So the United States Congress actually had a bit of a vision in 1991. They directed the Department of Transportation to, quote, demonstrate an automated vehicle and highway system by 1997, unquote. And it actually happened again. You don't hear about this very much, but this is 1997 here, guys. The project involved a bunch of different organizations and administrations and companies, including the Federal Highway Administration, General Motors, Caltrans, CMU, again, University of California, Berkeley, Lockheed Martin and a bunch of others. They called the culmination of this project Demo 97. And in San Diego on Interstate 15, they actually demonstrated about 20 automated vehicles, including cars and buses and trucks, all self-driving on an actual interstate highway. They demonstrated something called platooning, as you can see here on the left here. That's the idea of having these cars kind of traveling in a convoy together. And that's the display they actually saw inside their cars. And they also had what they call free agent vehicles that were able to operate in mixed traffic conditions as well. Now, unfortunately, the funding for this project was subsequently cut in 1998 just due to federal budget constraints. So that was the end of this program. And unfortunately, they never published a single technical report on how they did it. So a lot of that work was kind of lost to the sands of time, believe it or not, even after they chewed through 90 million dollars of public funding. So exciting results in 1997, but not really a happy ending because the program was terminated and nothing really came out of it, apart from know how that hopefully people involved in the project took on to their subsequent careers in the field. Also during the 90s, there were several semi-autonomous demonstrations, including another CMU project that was called No Hands Across America that drove cross-country across the United States. That is with autonomous steering over 98 percent of the time. There was also a demo in Paris that involved driving in free lanes, lane changes and passing other cars at over 80 miles per hour. But they did have some human intervention involved in that. In 1996, the University of Parma, which has also been a big player in self-driving car research, modified a vehicle to follow painted lane markings on unmodified highways. And they did this for hundred miles through northern Italy. Now, to be fair, the longest autonomous stretch where no human intervened was only 34 miles. But, hey, that's pretty darn good for 1996. And all they use for technology was too black and white video cameras and a stereoscopic analysis and a computer of the input from those very inexpensive cameras. That's all they used. Technically, the first driverless vehicle was the park shuttle in the late 90s in the Netherlands, and that used to magnets embedded in the road to guide it. It was only used as a pilot project and like an airport there and a little research project that they had. But technically, since it actually did drive real people around and had no driver at all that qualified as the first actual driverless vehicle. Now, in the 2000s, things get really interesting. This is the age of the DARPA grand challenges and again, DARPA is a U.S. military research branch. They offered a one million dollar prize to any team who could create an autonomous car capable of finishing a 150 mile course in the Mojave Desert. Now, they had three different grand challenges throughout the 2000s. The first one in 2004 did not yield any winners. Everybody either crashed or failed or caught fire. But in 2005, the next year, five vehicles actually completed the desert course. And you can see one of those vehicles here. Tarmac's is what it was called, very beefy machine. And they made extensive use of things like LIDAR and computer vision, obviously played a big role in that, which is what this course is about. And in 2007, they did it a third time, this time in an urban environment, which you see on the right here, and CMU, one of our friends, Carnegie Mellon University, with a modified 2007 Chevy Tahoe, and a lot of the participants in the 2007 grad challenge published their research and algorithms and even the entire source code for computer vision. And that really jumpstarted the field. A lot of new light. Our sensors came out of this work. And ultimately, Google's self-driving car team was quietly assembled in 2008 using many of the participants in the DARPA Grand Challenge. Today, Google's self-driving car project is called WHAMO, and it's still a pioneer in creating new computer vision, algorithms and technologies and self-driving cars. That brings us to the present decade and today more and more autonomous vehicle technology is making its way into real cars on the real roadways. I personally own a Tesla, which is advertised as containing all of the hardware needed for full self-driving. It includes an array of cameras surrounding the car radars and a dedicated computer for processing all that data in real time. Pretty much every vehicle manufacturer is incorporating semi-autonomous features such as adaptive cruise control lanes, change warnings and assisted steering's into their lineups. And those are all the same components that will make up full self-driving capabilities. Today, I can drive my Tesla on the highway with no intervention by me. In most cases, it uses computer vision to follow lane markings, even in bad weather very effectively, and its cameras and radars keep me a safe distance from other vehicles in the road. Even in heavy traffic, it can even change lanes automatically, just as smoothly as a human driver could. So the goal of full self-driving seems within reach, but it's still eluding us. There are times when I need to take control back from my Tesla due to merging traffic or obstacles in the road or road construction. It can't yet read traffic signs or stoplights or deal with urban driving at all. Much of the underlying technology exists in the lab, but getting it to work reliably enough in the real world under every conceivable situation remains a challenge. And we must get it right as human lives are at stake here. Already, there have been some high profile news cases about fatalities from an Uber self-driving car and from Tesla's with their autopilot mode engaged. Even if autonomous vehicles are safer than human drivers, accidents involving autonomous vehicles will attract much more media attention than the over 15000 car crashes that happen every day in the U.S. alone. But with the underlying technology making it onto real roadways more and more every day, it seems inevitable that full self-driving will be a reality soon. And this course will teach you the computer vision and machine learning algorithms these technologies rely on. So you can be a part of this revolution as it happens.