Eliminating latency in embedded multi-camera compression systems

By Nick Flaherty www.flaherty.co.uk

Researchers at the Fraunhofer HHI  have adapted video compression in such a way that a latency is almost no longer perceivable so that it can be used for road traffic systems or for autonomous driving. 

Up to 12 cameras are currently installed in new vehicle models, mostly in the headlights or taillights or the side mirrors and this is set to grow, putting more pressure on the embedded compression technology, particularly for the latency.

"If autonomous driving catches on as quickly as predicted, the number of cameras will increase further," said Prof. Benno Stabernack of the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institut, HHI in Berlin.

This means even more strain on the internal data networks of vehicles. Currently, these can process a data volume of around one gigabit per second. In HD quality, this data quantity is already reached with one camera. "Compression methods help here," says Stabernack. 

The Fraunhofer HHI already contributes to the H.264/Advanced Video Coding (AVC) and H.265/MPEG High Efficiency Video Coding (HEVC) video standards. "With these methods, the data quantities can be sharply reduced. In this way, more than ten times the quantity of data can be transmitted," he said.

Normally, 30 to 60 images per second are sent from a camera to the vehicle’s central computer. By compressing the image data, a small delay in transmission occurs, known as the latency. "Usually, this is five to six images per second," explains Stabernack. The reason for this is that the methods compare an image with those that have already been transmitted in order to determine the difference between the current image and its predecessors. The networks then only send the changes from image to image. This determination takes a certain amount of time.

"However, this loss of time can be of decisive importance in road traffic," says Stabernack. In order to avoid latency, the professor and his team only use special mechanisms of the H.264-coding method, determining the differences in individual images no longer takes place between images, but within an image. This makes it a low latency method. 

"With our method the delay is now less than one image per second, almost real time. We can therefore now also use the H.264 method for cameras in vehicles," he said. The technology was implemented in the form of a special chip in the camera that compresses the image data, and decompresses the data in the on-board computer.

"During development we combined our know-how from work on the video compression standards and our hardware expertise. The transmission of image data in real time is a precondition for the video compression of video data from car cameras becoming established. With it, the use of devices with a higher image repetition frequency and resolution would then be possible. For camera models which produce even more data and are therefore more precise and faster," he added.

In the next stage, the researchers also want to transfer their method to the HVEC standard and put their experience to good use in upcoming standardization formats.