Virtual testing of a fully autonomous tractor for sustainable agriculture

Nowadays, farmers face multiple challenges. One of these is to feed an ever-growing population by producing food and fibre. To do this, a lot of fieldwork has to be performed within short time frames. The lack of capacity, labour and time has resulted in farmers using bigger and heavier tractors to get the work done in the limited time they have. Nevertheless, the lack of capacity and labour keeps on growing. Another problem is that these heavy machines compact the soil and make it hard for new crops to grow. The yield per hectare of the farm field is reduced, which makes the challenge of producing enough food even harder.

AgXeed [1], a Dutch company founded in 2018, believes the solution to these challenges lies in the introduction of fully autonomous tractors. With today’s technologies it builds and sells AgBots (Figure 1), fully autonomous robots, made for doing all kinds of fieldwork. Together with these machines, it also offers a cloud portal to plan the tasks for these AgBots, as well as gather data during the process, and provides the user with more insights than ever about the status of their farm fields and crops.

As the AgBot has been designed from a blank sheet of paper, it does not carry legacy from existing traditional tractors. It does not have a cabin, seat, steering wheel, suspension system, transmission or air-conditioning, and hence is a lot less complex without these systems. Furthermore, this saves weight, while AgBots can even be fitted with tracks to distribute ground pressure and further reduce soil compression. Since the robots are able to cultivate, plough, drill and perform many other field operations without user intervention, farmers or workers save valuable time which otherwise would be spent sitting in the tractor.

The current team behind AgXeed has over 70 years of combined knowledge and experience in the development of agricultural equipment. Moreover, they recently joined forces with Claas [2], a well-known German agricultural machinery manufacturer. AgXeed partners up with best-in-class companies to develop their solutions and has selected Nobleo Technology for their broad knowledge and experience in the development of autonomous machines. Nobleo Technology has been involved in the project since 2019, to develop the AgBot’s autonomy software and make sure the AgBots are able to successfully complete the tasks that they receive from the cloud portal.

The AgBots are fully electrified, although a diesel engine is the main power source of the machine. The diesel engine directly drives a generator to generate 700 VDC. Battery packs or hydrogen fuel cells are too big and heavy for the machine’s power demand at the moment, but might be a future replacement for the diesel engine.

Traction inverters in turn create 700 VAC to drive the electric motors in the tracks. The electrification allows for precise control of the speed and direction of the vehicle and thus accurate path tracking. In addition, it also enables more control of the implements connected to the tractor. The 700 VDC can be shared with attached implements that are also electrified and may be regarded as ‘smart’.

A communication bus between tractor and implement allows for accurate actuation of the equipment, while variables such as actual speed can be communicated to facilitate a good quality of work every time. A worldwide standard known as ISOBUS (ISO 11783) is commonly used and hence is implemented in an AgBot, which allows for quick integration with existing agricultural machinery.
ISOBUS is delivered over a Controller Area Network (CAN) bus. CAN, a popular, robust communication bus used in the automotive and other industries, is also the main communication bus in the AgBot itself.

Another option available on the AgBot is an electrified power take-off (PTO). A PTO on a conventional tractor allows for a mechanical coupling between the tractor’s gearbox and the implement to actuate it. With an electrified PTO, the speed of rotation can be set precisely, independent of the diesel engine’s rotation speed, while still providing the conventional mechanical coupling for existing implements.

The AgBot is also equipped with standard 3-point hitch systems on the front and back. This allows a farmer to use the equipment he already has in his shed, for example a spading machine (Figure 2), avoiding the need to invest in new implements.

For localisation, one of the main sensors is an RTK GNSS system. Where a traditional GNSS (global navigation satellite system) at best can achieve sub-meter precision, an RTK GNSS system allows for centimeter-level precision. RTK here stands for real-time kinematic positioning and uses several correction techniques to correct for common errors in a GNSS. US-owned GPS is one of the best known GNSS systems. Glonass (Russian), BeiDou (Chinese) and Galileo (EU) are similar GNSS systems. Modern receivers are able to utilise all these systems simultaneously. In addition, using multiple receivers enables even greater accuracy, fail safety and improved orientation estimation. In addition to localisation sensors, sensors for collision avoidance are also installed to make sure collisions with trespassers or unplanned obstacles are prevented.

As well as the tracked version of the AgBot, a 3-wheeled, front-steered version (a tricycle) has also been built for full autonomous spraying in fruit orchards (Figure 3). While the application might seem completely different to performing fieldwork, the purpose of autonomously following a pre planned path very accurately is also key here.

While solving some challenges of the farmer, developing complete autonomous machines poses a lot of challenges on its own. Testing is of vital importance during this development but also thereafter. And testing is a very broad concept. Consideration can, of course, be given to the testing of an assembled AgBot as a system: does it work, is it able to fully autonomously cultivate a field? But testing modules should also be considered. Efficient running of the diesel engine, maximum torques in a gearbox, and lifetime of a track are a few of the hardware-related tests. Communication with the cloud portal, correct start-up of all electronic control units (ECUs), processing GNSS and encoder and inertial sensors for precise localisation in the field are a few of the tests involving software.

Here, the focus is on the development of a ‘digital twin’ by Nobleo Technology to assist with testing. All the testing mentioned above consumes a lot of valuable time and hardware. In software development, automated tests are a well-known concept. When a programmer creates even the smallest changes to the source code, typically a lot of tests will automatically evaluate this ‘delta’ and notify the programmer of errors. These tests can evaluate small pieces of the source code in so-called ‘unit tests’, but also multiple blocks of source together in so-called integration tests.

Even further down the road, a system test could be defined when a test engineer puts an AgBot on the field and tests the complete set of hardware and software systems together. The final user acceptance test (UAT) could then be considered a final test when a customer accepts the delivered system. This ‘testing pyramid’ is depicted in Figure 4.

Figure 4 also shows the relationships between cost, speed, quantity and the level of testing. Unit tests are small and can be executed rapidly in multiples with little cost since a unit test can run on a cloud computer automatically. While a final test with the complete system takes a lot of time and money, because it requires a test engineer, a field and time to complete work on this field.

One way of saving resources can be to use a virtual version of this system for testing. This is where the digital twin comes into play. A digital twin is a virtual copy of an AgBot, in this case consisting entirely of software. Such a digital twin allows for testing at the level of a system test with the cost and speed of unit tests.

When an engineer wants to test the latest software release on a digital twin, he can do so while sitting at his desk. The engineer will start to plan a complete task in the cloud portal (Figure 5). This cloud portal was developed by another partner of AgXeed, i.e. Phact [3].

This is exactly the same tool as used for planning tasks of real AgBots. When the task is now sent to the machine, the digital twin will receive it and start executing the task in a simulation. Doing so, it will run an exact copy of a real AgBot’s software, thereby testing every software component, as well as communication between cloud and (virtual) AgBot. The simulation will also simulate sensor input for GPS sensors, inertial sensors, encoders and anti-collision sensors.

To the extreme, Nobleo’s digital twin of the AgBots even runs a complete virtual CAN bus. Small digital copies of the ECUs send and receive data on this virtual CAN-bus. This means that for the largest part of the software stack, there is no notion of whether it is running on a real AgBot or in a simulation environment. All low-level component drivers are thus also included in a digital twin test. While executing its task, the digital twin will send progress and sensor data back to the cloud, exactly in the same way that a real AgBot does so. A test engineer can thus monitor progress while still sitting at his desk. If desired, this engineer can even analyse CAN-bus traces from a digital twin as if it was from a real AgBot.

Since digital twins are software copies of AgBots, they scale easily; multiple digital twins, running in the cloud, can easily be created. This allows for simultaneous testing of several scenarios, for example a Phact engineer testing a new feature in the cloud portal, one AgXeed engineer testing a task on a tracked version of the AgBot and another testing task execution of a tricycle AgBot in an orchard.

A schematic overview of the software architecture is depicted in Figure 6. It clearly shows the identical software stack ‘Nobleo Autonomy Core’ being used on both a real AgBot and a digital twin. Sensor data comes into the software stack that is used for precise localisation. A task comes in via the cloud interface, which is processed by the management module that orchestrates correct navigation.

Finally, output is given to the actuators to make the AgBot move. Several signals and states are continuously monitored and sent to the cloud for remote monitoring, while they are also processed locally, for example in an anti-collision and geofence module.

Connecting a real AgBot or a digital twin occurs at the level of the hardware interface, which is the CAN-bus here. On the other hand, cloud software is connected to the Nobleo Autonomy Core in either a real or a virtual AgBot identically. Task planning, monitoring of the task’s progress, all is available in the cloud without a distinction being made between a real AgBot and a digital twin. Figure 7 gives an impression of a digital twin in action.

Currently, such a digital twin is in use by Wageningen University & Research for academic explorations in path planning. This digital AgBot facilitates quick evaluation of planning algorithms with the full cloud portal integration but without the need for real hardware on the field. At the same time, how ever, testing on a real AgBot is just the click of a button away.

In the near future, it will also be easy to test collaborative tasks between multiple digital twins. This could be a feasible scenario when very large fields need to be cultivated within a short time for example, or when AgBots need to work together with harvesters.

It can be concluded that testing with digital twins has a lot of advantages. Many tests can be performed quickly, saving time and requiring no engineers on site. Tests are run virtually, so there is no need for expensive hardware or the availability of fields, thus saving costs as well. And it is even possible to test tasks globally, without the need to actually transport an AgBot globally.

Obviously, real-life system tests are still needed in the end to test for things that are not easily tested digitally, such as mechanical load, electrical wiring and lifetime. But using digital twins can save a tremendous amount of time and money and identify issues early on before experiencing them in the field.

At the moment, some of the digital twins are actually performing virtual fieldwork. During development of the AgBot, the digital twin has proven to be extraordinarily useful. Roughly nine months before the first AgBot started its engine, the digital twin was already driving in the cloud, testing the complete software stack and cloud integration. In these nine months it already performed approximately 400 hectares of virtual fieldwork. The result was a first real world test with a real AgBot within two weeks of the engine being initially started. At the moment, the first series of AgBots is being built. They are scheduled for their first autonomous jobs at the customer this summer.

Without all the digital testing beforehand, this would not have been possible in this amount of time. Continuously testing every bit of software up to the level of a virtual CAN bus, ensured that a lot of bugs were solved long before the first field test took place. And today, the digital twin gives confidence in rolling out software changes to real AgBots when these have been successfully tested virtually.

Ferry Schoenmakers (mechatronics engineer), Tim Clephas (senior software architect), César Lopèz (senior robotics designer) and Frank Sperling (technical director) are all associated with Nobleo Technology in Eindhoven (NL), an engineering firm specialised in autonomous intelligent systems.

ferry.schoenmakers@nobleo.nl
www.nobleo-technology.nl

References

[1] www.agxeed.com
[2] www.claas-group.com
[3] www.phact.nl

Read the full Mikroniek article here