Sensor fusion optimally combines the measurements from multiple sensors, providing higher quality than any single sensor can alone. This approach is already being employed in driver assistance systems and autonomous driving for on-road vehicles, where fault tolerance and accuracy are a must. While still a relatively new field in agriculture and the off-highway sector, sensor fusion is expected to quickly make inroads into mobile off-highway equipment such as bulldozers and tractors.
From November 12 to 18, Systems & Components in Hanover, Germany, will showcase the evolution of mobile machinery, driven by rapid advances in sensor technology. Powerful assistance systems that unburden the driver and increase efficiency will be presented at the exhibition grounds. Working alongside their OEM customers from the outset, the exhibitors at the B2B platform, taking place in parallel to Agritechnica, are developing market-relevant solutions that meet their customers’ automation needs.
The need for reliable technologies in the off-highway sector is growing rapidly with sensors playing an increasingly important role in the monitoring and control of mobile machinery used in agriculture, forestry, construction and mining. However, choosing the most appropriate sensors is of crucial importance, as, in contrast to industrial automation, much higher demands exist for mobile applications. Agricultural and construction machines must collect their data under adverse conditions and providing reliable measurements where factors like dirt, moisture and extreme temperatures may impact results.
Different sensors often measure the same scenario
Smart sensors that not only measure but also process data into a usable form, are already used extensively on off-highway commercial vehicles. Where several measurements of the same scenario, for example the state of a crop in the field, are taken using different sensors, “sensor fusion” algorithms can combine and compare this data from different sources and offer more reliable recommendations. A typical scenario could use a near-infrared sensor detecting nitrogen levels, a mounted visible light camera that can also see the greenness of a field, and a far-infrared satellite for field mapping. The combination of the sensor information provides more accurate, robust and up-to-date information than any one approach on its own. Sensor fusion is already employed for driver assistance systems in cars, improving driver safety through warnings and even automatic braking, and its use in off-highway applications promises the needed robustness and accuracy.
Harsh conditions, robust instrumentation
As an example, where a threshing unit is to work continuously, staying level on an uneven field, robust inclination sensors must detect deviations laterally and in the direction of travel. The driver then has all the information needed to both correct and avoid overloading. Strong vibrations lead to measurement noise – errors in angle and position measurement – and the task for the technology providers is to adapt the sensors to the conditions that prevail off-road. Robust measurement systems that correct inclination sensors using a gyroscope can effectively block out such jolts.
Sensor systems for mobile automation are of particular importance for driver assistance. The assistance functions currently available, are the first step on the way to an autonomous mobile machine that carries out its work automatically without manual intervention. At the Systems & Components trade fair, this progress will be reflected in the phased and demand-driven approach that technology providers are taking to development – starting with warning functions in the cockpit, through semi-autonomous systems that support the driver, to complete telematics packages that increase the efficiency of machines even in remote regions. Engineers, developers and fleet managers will be able to find a complete toolbox for almost all sensor applications in mobile machinery at Systems & Components.
High integrity sensor systems improve mobile machinery safety
A particularly interesting example is high-resolution radar, operating around 77GHz, that detects both moving and stationary objects. The radar sensor can accurately monitor objects in the vehicle’s surroundings and, in combination with assistance systems, can aid the driver and warn of any potential collisions. Solutions such as those offered by automotive and off-highway electronics supplier Hella use a frequency-modulated continuous-wave radar with a 75 degree field of view, which functions reliably at extreme temperatures, at night and in conditions of poor visibility, such as fog, rain or snow. A system of eight sensors can provide 360 degree monitoring of the objects in the vehicle’s immediate vicinity, precisely measuring distance, relative speed, and the angle to the objects.
The exhibitors at Systems & Components draw on a broad technology portfolio from the automotive sector, which they are continuously developing for off-highway use, above all, 24 Volt compatibility and the IP69K rating. They thus provide OEMs of mobile machinery with a practical modular system for cost-effective production applications, with which they can design their own “plug & play” assistance systems. In addition to radar sensors, systems using other sensors like ultrasonics or cameras can be flexibly configured and tailored to the requirements of the specific application. A simple sensor system might only detect obstacles and measure their distance, whereas high-end solutions combine radars with cameras leading to much improved object localization and detection reliability.
Detailed interpretation of the environment
By “fusing” the data from different sensors, the advantages of the different measurement technologies can be optimally combined, thus increasing the potential of the assistance and safety functions. Powerful “data fusion” algorithms that are increasingly using artificial intelligence, enable the interpretation of the sensor data, providing a detailed representation of the environment around the vehicle. In advanced steering system solutions, sensor fusion facilitates autonomous operation in the field, such as precision farming with modern plant row recognition or precise section control. Using GPS, the machines follow specified routes with high accuracy, reducing overlapping and the associated excess use of pesticides or fertilizers.
For object detection, distance measurement, and collision avoidance, Bosch uses a combination of ultrasonic and radar sensors for its “Off-Highway Surround Sensing” offering. The combination of ultrasonic technology, optimized for close-range applications, with radar systems for both near- and far-field monitoring can be used for collision warning and avoidance as well as for distance control, for example with spray booms or grape harvesting. The system ensures that the optimal distance between the harvesting machine and the vines is maintained. Using precise localization, the system not only warns the driver of a possible collision, but also provides the exact distance and position of other objects, making it easier, for example, to drive two machines in parallel. The concept received the “Systems & Components Trophy – Engineers’ Choice” award from DLG at Agritechnica in 2022. “With Off-Highway Surround Sensing, we are creating the basis for the efficient development of more advanced assistance functions. In the future, agricultural machines will be able to perform many work steps completely automatically,” explains Stefan Schenk, responsible for the Off-Road division at Robert Bosch GmbH.
Equipped for the trends of tomorrow
Even though it will probably be a few years before autonomous driving machines dominate construction sites and agricultural fields, networking and digitization are already making their way into the off-highway sectors. Sensors make it clear what is happening in and around the machines, and allow drivers to optimize their machines and avoid being overloaded. From November 12 to 18, Systems & Components will present a wide range of solutions for measuring every aspect of agricultural processes that will prepare commercial vehicles for the automation requirements of tomorrow.
