实现全自动化车辆离我们有远，它的可行性有多大？(英)

One could argue that there is a fourth level—that of monitoring the driver. This concept is used in commercial vehicles to monitor the activities of the driver and or vehicle while it is in use. But we will not consider this further here.

Occupant monitoring. If asked to define the particulars of automotive vision, the average person would not likely suggest that the car should observe its occupants. But this could easily be one for the first automotive vision applications. It is expected to yield impressive safety results; and since the car’s interior is a predictable environment, it is also a relatively straightforward implementation with the possible exception of the inference software required.

Monitoring drivers for drowsiness, inattentiveness, or intoxication is likely to roll out in commercial fleet vehicles as early as 2006. A more difficult aspect of occupant monitoring calls for the vehicle to observe the position and posture of the passengers and, in the case of an accident, deploy airbags to account accordingly. Still in the research phase, vision-based smart airbag deployment will take several years to perfect.

Infrastructure monitoring. A vehicle’s ability to recognize and interact with stationary objects that exist around it or have been added as part of the highway infrastructure can add to the safety of the occupants of the vehicle and the convenience of the driver. Such things as embedded RFID chips in the ubiquitous reflectors set in traffic lanes, for example, could enable the car to alert the driver that it has drifted out of its lane. Or, with that RFID or vision-based information, the car could actually steer itself back into the proper lane. Similarly, sensors such as cameras or radar units mounted on the car could detect objects such as median barriers, trees, buildings, and people, in addition to those equipped with sensors.

Other specific applications that fall into the category of infrastructure monitoring include: parking assist proximity sensors for parallel parking (already in production in Europe); rear view cameras instead of mirrors; and blind spot cameras, which are also useful in monitoring other vehicles. Still another aspect of infrastructure monitoring is the interaction of automotive vision technology with existing technologies such as GPS. Knowing a vehicle’s position could be helpful in accident avoidance if, for example, it could advise of an accident or road hazard in the vicinity.

Monitoring other vehicles. This category is distinguished from the previous category by the inherent difficulty of tracking other moving objects. Examples include intelligent cruise control (to maintain a safe distance between vehicles), blind spot monitoring, rear view observation, and night vision.

Safety and forensics. Some applications do not fit easily fit into the three categories mentioned above. Black-box recorders can be used for accident reconstruction, for example, are already being used in some commercial applications. Technologies that will help autonomous vehicles learn how to avoid an accident or how to react in a crash situation are being developed in research labs.

Data fusion
The term automotive vision suggests mimicking the visual perception of the human eye with cameras. But vehicles encounter many conditions in which a “few extra eyeballs” distributed around the vehicle are not enough to provide all the safety and convenience possible.

There is general agreement among engineers in the automotive and associated industries that a variety of sensors can be used to collect a more complete set of useful information. These sensors could possibly include radar, laser, and infrared in addition to digital cameras. Data from all of these sensors will be combined and interpreted in a concept called data fusion.

In the near term, however, the focus will be on cameras because they offer good price/performance and can effectively collect-and-display information for the driver. Data fusion will be an important aspect of automotive vision in the future; but for this article the types of data are not as important as how the data are used. The industry is still grappling with several questions about the location of sensors, their resolution, and where the intelligence that will interpret the data collected should reside.Cameras are the obvious answer to the question of automotive vision. Further it is assumed that the cameras will be digital. But that is about the end to the simple answers. Beyond whether the camera’s sensors will be CMOS or CCD are more important issues. These issues include:

How many bits of dynamic range will each of the camera’s pixels need? The dynamic range of most sensors today is about 8 to 12 bits. This will not be enough to adequately handle the full dynamic range of light conditions to which the automotive vision camera will be subjected. The dynamic range that seems to be necessary to handle the breadth of light conditions encountered is in the range of 16 to 24 bits. If this sounds excessive, think about the need to discern the washed out lane markers on a road while driving directly into the morning sunlight. This might change your mind. (Editor’s Note: Click here for an article on algorithms used to expand the effective dynamic range of automotive vision cameras.

How many pixels does the camera’s sensor need to have and what is their best aspect ratio? Optimal picture resolution (i.e., the number of pixels) and aspect ratios (i.e., the number of vertical and horizontal elements) are also being debated. Using cameras with standard aspect ratios (4 x 3 or 16 x 9) may reduce cost in the short term as they could be the same ones used in digital cameras. But custom aspect ratios may prove to be more effective for data collection in an automotive application.

Also, today’s digital cameras are typically five to eight million pixels in resolution. Future digital cameras will most likely continue to increase the number of pixels to tens of millions. This resolution is far beyond that needed for automotive vision. For automotive vision, the resolution could easily be less than one million pixels depending on whether the information is to be displayed for the driver’s use or used as an input for image recognition.

Does the camera need to be color or black and white? The imager probably does not necessarily have to distinguish colors – black and white should be fine for many applications. Although it would be nice to know that the bright light on the car just ahead is red rather than white.

How many frames per second (frame rate) need to be captured by the camera? Early systems may only need to have the ability to capture a few frames per second, perhaps five or 10. But, future applications may very well require as many as 100 frames per second in order to make correct decisions in a timely manner.

Do the cameras need to be monovision or stereovision? The question of stereo versus mono vision has to do with the camera’s ability to see three dimensions. Seeing three dimensions allows better position and velocity information. Both are valuable to making driving decisions. Given all that we know now, it is most likely that cameras used in automotive vision will be stereo-vision cameras.

Where do the cameras need to be placed? One reasonable configuration calls for two cameras inside for front seat passenger observation and maybe two more to observe the passengers in the back seat; two mounted in the vicinity of the rear view mirror and aimed forward for lane and obstacle detection; four mounted on the sides (two on each side) for blind spot detection; and two aimed out the rear of the vehicle. This adds up to a lot of cameras with an enormous amount of data to collect and interpret.

Do the cameras need to be smart or dumb? How to best use the automotive vision data gathered presents considerably more complex choices. Phrased as an either/or question, the choice is between central processing of image and other data (dumb camera) and distributed processing that happens at a smart camera module. Not surprisingly, the answer depends on the application—and the application depends on the evolutionary path of automotive vision system roll out.

Network bandwidth, latency, and reliability all argue for pushing as much intelligence as possible to the system’s perimeter by including DSP and perhaps other processing capability in or near the cameras—thus requiring smart cameras.

Executing image processing algorithms inside the camera and sending pertinent information rather than raw data seems to be the better solution. It reduces overall network bandwidth and latency. It also increases inherent reliability when compared to the situation of the entire vehicle depending on a single processor. Intelligence inside the camera also makes sense if the cameras are expected to adapt to their environment – correct focus, eliminate the effect of a dirty lens, or search for a specific object by panning and zooming.

Essential programmability
It is clear that as automotive vision progresses from stage to stage the only constant will be change. This means that regardless of where the DSPs are located programmability is essential. Hardwired solutions will not be capable of adapting to changing system architectures, standards, and operating environments.

Embedding intelligence at the network nodes also makes sense for autonomous or semi-autonomous vehicles because machine vision typically requires a different set of data than does human vision. In many cases it could actually be less data than for human vision.

On the other hand, an inference engine that takes all possible scenarios and possibilities into account would most likely require a master central processor. The balance between distributed and central processing is perhaps best seen by anticipating the phased introduction of systems.

One step at a time
To be successful, today’s CMOS imaging and DSP technologies must align themselves with the sophistication level of today’s prototype automotive vision systems, their cost, and, perhaps most importantly, driver acceptance. Systems will offer visual information to the driver. This initially means implementing stand-alone applications that use "dumber" cameras linked to a central processing unit. Off-the-shelf CMOS imagers with 8- to 12-bit resolution will be used to keep system cost reasonable. Image processing will almost certainly be executed inside the camera to conserve system bandwidth and reduce latency. Eventually, rudimentary image recognition will be introduced as the DSPs performance increases. But two things are certain, more performance will be needed to make it happen and DSPs will increase their performance.

So in five years or so down the road, the technology mix will change. DSPs capable of 10 MMACS/s and designed specifically for the automotive vision market will appear. Simple object identification algorithms will be added to object recognition. At the system level, automakers will field systems with smart cameras that communicate with a master central processor. Imagers designed specifically for automotive vision, will offer 18- to 20-bit of resolution, and have application-specific aspect ratios. Data fusion will make its appearance and software reliability will have reached a point where the system will be capable of some autonomous intervention—but drivers will have the option to turn the system off or ignore its inputs.

Ten years from now, DSPs capable of 20K MMACS/s will integrate all of the functions of the CMOS imager and deliver more than 20 bits of dynamic range. Although there will still be a central processor, distributed processing will become just as important. Multiple systems in the vehicle will communicate with each other and the host. Data fusion will be more or less complete and the car will become autonomous in the sense that it will be able to take over to prevent driver from making an error. Whether we, as drivers, will allow that to happen is the big question.

In order to reach this level of system sophistication, the algorithms running on the DSPs distributed around the vehicle will be capable of identifying objects with very high reliability. The decision-making process inherent in the ability to correct driver errors will need to have extremely high reliability.

As mentioned earlier, many technical challenges must be met for automotive vision to become a reality in the next 10 years. Among these challenges are the reliability and tolerance requirements of the automobile industry. Automobiles qualify as a harsh environment for semiconductors, imagers and software. But as both the automotive and semiconductor industries have demonstrated in the past, the chief barrier to progress is lack of imagination. And, by the way, it is not a question of “will it happen;” it is a question of whether it is ten years or 20 years. But, it will happen.

Gene Frantz is a Principal Fellow and Pascal Dorster is New DSP Development Manager at Texas Instruments