Researchers at McAfee Advanced Threat Research (ATR) have hacked the machine learning algorithms in a MobilEye camera chip used in Tesla cars.
The team looks at model hacking, the study of how hackers could target and evade artificial intelligence, with a focus on the broadly deployed MobilEye camera system. This is used in over 40 million vehicles, including Tesla models that implement Hardware Pack 1.
The team looked at ways to cause misclassifications of traffic signs and were able to reproduce and significantly expand upon previous research that focused on stop signs, including both targeted attacks, which aim for a specific misclassification, as well as untargeted attacks, which don’t prescribe what an image is misclassified as, just that it is misclassified. The team were successful in creating extremely efficient digital attacks which could cause misclassifications of a sign,
The team looked at ways to cause misclassifications of traffic signs and were able to reproduce and significantly expand upon previous research that focused on stop signs, including both targeted attacks, which aim for a specific misclassification, as well as untargeted attacks, which don’t prescribe what an image is misclassified as, just that it is misclassified. The team were successful in creating extremely efficient digital attacks which could cause misclassifications of a sign,
They used physical stickers, shown below, that model the same type of perturbations, or digital changes to the original photo, which trigger weaknesses in the classifier and cause it to misclassify the target image.
Targeted physical white-box attack on stop sign, causing custom traffic sign classifier to misclassify the stop sign as an added lane sign
This set of stickers has been specifically created with the right combination of colour, size and location on the target sign to cause a robust webcam-based image classifier to think it is looking at an “Added Lane” sign instead of a stop sign.
The team then repeated the stop sign experiments on traffic speed limit signs.
.
Physical targeted black-box attack on speed limit 35 sign resulting in a misclassification of the sign to a 45-mph sign
Targeted physical white-box attack on stop sign, causing custom traffic sign classifier to misclassify the stop sign as an added lane signThis set of stickers has been specifically created with the right combination of colour, size and location on the target sign to cause a robust webcam-based image classifier to think it is looking at an “Added Lane” sign instead of a stop sign.
The team then repeated the stop sign experiments on traffic speed limit signs.
.
Physical targeted black-box attack on speed limit 35 sign resulting in a misclassification of the sign to a 45-mph sign
Black-box attack on the 35-mph sign, resulting in a misclassification of 45-mph sign. This attack also transfers on state-of-the-art CNNs namely Inception-V3, VGG-19 and ResNet-50After testing in the lab using a high resolution webcam, the team took the technology out onto the road. A 2016 Model “S” and a 2016 Model “X” Tesla with MobilEye's EyeQ3 camera chip were tested. The adversarial stickers convinced the Tesla Head Up Display (HUD) that the speed limit was 85mph.
These adversarial stickers cause the MobilEye on Tesla Model X to interpret the 35-mph speed sign as an 85-mph speed signThe lab tests developed attacks that were resistant to change in angle, lighting and even reflectivity to emulate real-world conditions, reducing stickers from 4 adversarial stickers in the only locations possible to confuse our webcam, all the way down to a single piece of black electrical tape, approximately 2 inches long, and extending the middle of the 3 on the traffic sign.
A robust, inconspicuous black sticker achieves a misclassification from the Tesla model S, used for Speed Assist when activating TACC (Traffic Aware Cruise Control)Even to a trained eye, this hardly looks suspicious or malicious, and many who saw it didn’t realise the sign had been altered at all. This tiny piece of sticker was all it took to make the MobilEye camera’s top prediction for the sign to be 85 mph.
The vulnerability comes from the fact that the Tesla Automatic Cruise Control (TACC) can use speed limit signs as input to set the vehicle speed. A software release for TACC shows that the data is fed into the Speed Assist feature, which was rolled out by Tesla in 2014.
McAfee ATR’s lead researcher on the project, Shivangee Trivedi, partnered with another vulnerability researcher and Tesla owner Mark Bereza to link the TACC and Speed Assist technologies. On approaching the hacked sign, the Tesla started speeding up to the new speed limit.
The number of tests, conditions, and equipment used to replicate and verify misclassification on this target were published by McAfee in a test matrix.
The team points out that this was achieved on an earlier versions (Tesla hardware pack 1, mobilEye version EyeQ3) of the MobilEye camera platform. A 2020 vehicle implementing the latest version of the MobilEye camera did not appear to be susceptible to this attack vector or misclassification. The newest models of Tesla vehicles do not implement MobilEye technology any longer, and do not currently appear to support traffic sign recognition.
However the vulnerable version of the camera continues to account for a sizeable installation base among Tesla vehicles.
The video of the testing is at www.mcafee.com
The video of the testing is at www.mcafee.com
No comments:
Post a Comment