"Certainly we think a big part of the things we do with computer vision will be based on some form of neural networking," Roni Sadeh, vice president of video technology for Ceva, told Electronics360 in a telephone interview.
Ceva Inc. (Mountain View, Calif.) is a leading licensor of DSP cores and platform solutions for video, audio and communications and has design centers in Israel, Ireland and France. The adoption of neural networks is likely to be a gradual development that integrates with the familiar software-programmable DSP, according to Ceva.
Eran Briman, vice president of marketing at Ceva, pointed out that the latest DSP core platform available for license from Ceva, the MM3000, already provides support for convolutional neural networks (CNNs) in the form of hardware accelerators and software. "In the past year we have seen increased interest in deep learning," he said.
Right now the MM3000 is being used to for variety of tasks including sensor fusion, gesture and voice recognition, eye-tracking – tasks that extend the natural user interface to equipment – and doing so at lower power consumption than on a host application processor, he said. "We've licensed the MM3000 to more than a dozen licensees who are using it along with the software libraries," said Briman.
Roni Sadeh, vice president of video technology at Ceva.
Sadeh said: "The Convolutional Neural Network (CNN) is the leading algorithm right now. But we are also investigating other complex algorithms that are similar to how the brain works. The CNN gets good results but it is limited compared to with the human brain. We are looking for a [neural network] processor architecture that will have much better performance than anything we can do today."
The convolution form of neural networks is a multi-layered network in which individual neurons at the input layer are connected so as to respond to overlapping regions in the input data, typically the pixels of a visual field. CNNs are modelled on biological processes and have been used with success on visual classification, object tracking, recognition and similar tasks, although they can also be used for quite different applications.
However, like all neural networks CNNs are set up based on the simple capability of each processing element, or neuron, the most appropriate number of layers, the connectivity and setting of connection weights and this is done by a process of training rather than programming. When that is done properly such CNNs can be highly accurate and rapid.
Until recently such networks have most often been the subject of academic interest or used in back-end big data functions where they can run as software simulations on banks of uniprocessor computers in a data center. This provides flexibility but is inefficient in terms of energy consumption. But with applications such as vision processing, robotics and automotive driver-assist systems coming to the fore could mean the time is right for optimized hardware implementations.
FPGA fabrics and DSPs – as Ceva has shown – can be used to implement neural networks but the extreme levels of parallelism and connectivity required by CNNs suggests dedicated hardware is the way to go.
Sadeh said any given choice of CNN architecture would likely be influenced by the generality or specificity of the applications the architecture is trying to address and be optimized for. In Ceva's case they are starting with vision processing.
"IBM has their way, the TrueNorth chip," said Sadeh (see IBM Seeks Customers For Neural Network Breakthrough).
DSP vector processors do quite a good job at running CNNs but they don't have the inherent parallelism. If you can build the parallelism you don’t necessarily need a processor but something more like a hardware accelerator, said Sadeh. "But, if you want to make changes to the CNN it would be better suited to a novel processor. That way it can do several things; a new processor architecture that follows a biological model."
Choosing the right CNN or variant on CNN for a neural network processor to cover multiple applications well, is therefore a key to future economic success. But such processors do not run C code and training them will be outside the experience of most SoC developers and programmers. So how they are invoked and linked in to conventionally programmed processors will also be key.
According to Sadeh, the answer is already present in the MM3000. "The way to look at it is as part of the processor that you access through a high-level application programming interface (API). Through the API you can direct this neural network processor core to learn, to do this, to do that."
"You need the C and assembler style programming for other things anyway because there are many things a brain-like processor is not good at. It's not very good at registration, 3D objects, stereo, they are done better in a vector architecture," he said.
Structurally speaking
Structurally what does the CNN look like. Each processing element is likely to be relatively simple ALU and there may be thousands of these but more significantly is the connectivity which ideally would be almost universal – in other words each PE would be able to connect to any and all other PEs. "The issue is how to manage the connection in two-dimensional silicon," said Sadeh.
"IBM has a method. At Ceva we are seeking alternatives. Would the PE offer 16- or 32-bit data resolution or is that overkill for networks where sometimes the output is a binary – match or no-match or a go/no-go decision. Sadeh declined to be drawn on data resolution but indicated that, like the MM3000, it would probably be necessary to support multiple data types from 8-bit up to 32-bit. Sadeh said power efficiency could be enhanced by switching off most of the network for most of the time. "Assume that in any cycle only a very small proportion of the network is active and there are means to dynamically adapt the region of interest."
So how soon will a neural network processor have an impact on the commercial market?
Sadeh: "In the next several years. It will be part of the solution for facial analysis but it will be more about integration in a total processing approach than a stand-alone neural network."
And Sadeh reckons a preliminary architecture will be forthcoming from Ceva in about three years. "But there's going to be a number of steps along the way." Of which support for CNNs within the licensable MM3000 DSP core is the first.
Questions or comments on this story? Contact dylan.mcgrath@ihs.com
Related links and articles:
More on convolutional neural networks
News articles:
IBM Seeks Customers For Neural Network Breakthrough
"BrainCard" Maker Project Marks Startup's Market Entry
Ceva Researches Platform to Read Body Language, Emotion
