Ok, but how do you make sure the car sees things? Even if you have a camera with a very high resolution, what about the rain, snow, dirt?
This is actually a really challenging problem. We use multiple types of sensors, not only cameras but also lidars
, laser sensors which illuminate objects and then measure the reflected light. By fusing all the sensors' data regardless of the weather conditions and time of day – rain, snow, day, night, we see a complete 360-degree picture all the time. So should a camera fail, you have other sensors to rely on.
Yes, and safety is our first priority, we constantly monitor all the sensors, if there is a problem and something fails, we safely stop the car. What is your plan for dealing with zero-day security vulnerabilities that will be inevitably discovered in your software?
First, all the calculations happen on board of the car. The software components responsible for navigation and driving are not contacted to any outside online service, which by default makes it impossible to take control over the vehicle without direct intervention to hardware. Moreover, Yandex has more than 20 years experience of successfully providing information security for company's multiple services (mailing, online payment etc). Our self-driving systems are regularly tested by Yandex cyber security team to make sure they meet company's high standards. What's the hardest and yet unsolved problem for your Lab in Israel right now?
The biggest challenge for us now is to hire the best people. Once we've done that, we are going to start solving very complex machine learning problems. We will be responsible for one of the very important areas of the autonomous car technology, I cannot specify which one exactly, but we will build a large scale machine learning system, which will ultimately improve our machine learning processes and will help us get better, faster and more robust results.