The author went to AWS Re:Invent 2024 as he had been studying the art of AWS Deep Lacers in the physical world for the past few years. How was that?
AWS Deepracer: Master Master Physical Racing? , I’ve written in detail about some aspects related to Racing AWS Deepracer in the physical world. We investigated the differences between virtual and physical worlds and how simulators and training approaches can be adapted to overcome the differences. Previous posts were left open-ended. With the final Championship Final still there, it was too early to share all my secrets.
With AWS Re:Invent finished, it’s time to share my strategy, how I prepared, and how it ultimately went.
strategy
As the 2024 season begins, we looked back at our performances in 2022 and 2023. In 2022, they were unable to perform fast laps on the new Re:Invent 2022 Championship track, failing to score the final 32.
Specifically, the following models were required:
- Make a straight, straight and accurate corner
- They have a survival instinct, and keep track of them out even in tight spaces
- You can ignore visual noise seen around the track
Combined with the ability to test the model before displaying it at the expo, it’s now possible to succeed!
implementation
In this section I will explain my thoughts on why physical races are so different to virtual races, and my approach to training models that overcome those differences.
How difficult is it to go straight?
If you’ve seen Deepracer over the years, you’ve probably seen most models struggling to go straight and vibrate left and right. The questions are always: Why is that so? This behavior causes two problems: distance-driven increase (results: slow lap times), which could allow the car to enter the next turn in a way that is potentially unhandled (results: off-track).
Several theories emerged:
- Real problems from SIM – The steering response is not consistent with the simulator in terms of both steering geometry and latency (the time from the photo to the servo command and the time it takes to actually operate). Therefore, when the car tries to adjust the direction of the straight, it does not get the expected response.
- Model problems – A combination of models that do not actually use linear actions, and do not have access to the angles needed to dampen vibrations (2.5-5.0 degrees).
- Calibration issues – If the maximum value on 0 degrees/right is too high (oversteer tendency) or too low (understeer tendency) then if the car is not calibrated to be straight, you may get control issues and unstable behavior.
My approach:
- Uses the Ackermann steering geometry patch. This will make the car more realistic and the turning radius decreases at a certain angle. As a result, the action space can be limited to an angle of up to about 20 degrees. This roughly matches the steering angle of your actual car.
- It stabilizes steering angles (2.5 and 5.0) in action space, allowing for straight, minor corrections.
- Avoid slipping of the simulator using a relatively slow speed (0.8-1.3 m/s). My theory is that a 15 FPS simulator and a 30 FPS car actually converts 1.2 MP of a simulator to 2.4 MP effectively in a practical world.
- Instead of vibrating the reverse right action, the chevron action space is faster, so fine tune the car to use straight actions.
- Try out the physical models of V3, V4, and V5. Test it on a real truck to see what works best.
- Otherwise, the reward function was the same progression-based reward function that we also use in virtual races.
The following diagram shows a view of the test in the garage, moving straight through at least one frame.
Become flexible
Virtual races are (almost) deterministic, with models converging over time, cars walk on narrow roads and reduce the variety of situations seen. Early in training, it is often in strange positions, almost out of the track, remembering how to get out of these situations. Once convergence occurs, the frequency of processing these decreases, and the theory is that memory declines and at some point forgets how to get out of tight places.
My approach:
- Diversify training to teach your car to handle different corners.
- Train your model consistently, both clockwise and counterclockwise.
- It uses much more complex tracks (mainly 2022 Championship tracks) than Forever Raceway.
- Final optimizations are performed with Forever Raceway.
- Take some snapshots during training. Entropy should not fall below 0.5.
- Tested on a truck that the car never saw before. The simulator has many suitable narrow tracks. A feature of the generalized model is that it can handle tracks that you have never seen before during training.
Keep focused on the track
In my last post, I saw the visual difference between the virtual world and the real world. The question is what to do about it. The goal is to trick the model into ignoring noise and focusing on what’s important: the track.
My approach:
- Train in environments where visual noise is significantly improved. The tracks in the Custom Track Repository have added noise (and shadows) to the additional lights, buildings and various walls.
- Change your environment during training to avoid overfitting additional noise. Custom tracks were created in a way that makes various objects (buildings, walls, lines) invisible at runtime. There was Kron’s job to randomize the environment every five minutes.
The following diagram illustrates the different training environments.
What I wasn’t thinking about this year was to simulate blurry during training. Before inference, I tried this by averaging the current camera frame with the previous camera frame. It didn’t seem to help.
Lens distortion is a topic I have observed, but has not been fully investigated. The original camera has a clear fish eye distortion and the gazebo can replicate it, but it will require some work to actually determine the coefficients. Similarly, I have never tried to replicate the actual car rolling movement.
test
The test was conducted in a narrow, trapezoidal truck garage. The track is obviously basic, but I had to do my job because there are two straights and two 180 degrees turns different. The garage truck also had enough visual noise to check if the model was robust enough.
This method was easy. Try all models both clockwise and counterclockwise. Using logs captured by a custom car stack, I looked up the video for each run to determine which model I like most with stability, handling (straight and straight and precise cornering), speed, and speed.
Re: Invention 2024
The track for Re:Invent 2024 was Forever Raceway. The truck shape is not new. They share the centerline with the 2022 Summit Speedway, but only shares its width at 76 cm (the original was 1.07 cm). The turns become more pronounced and become a very difficult track.
environment
The environment is a classic Re:invention: a smooth track with a smooth sparkle and a sparkle combined with the smooth, rather tall walls surrounding the track. The background is often a problem. This year, a large, bright display is hung beneath the ceiling at the far end of the track. As the following image shows, it attracted considerable attention from Gradcam.
Similarly, the pit crew cages, where the cars are maintained, attracted attention.
result
So, where did I end and why? In Round 1, he came in 14th with a highest lap time of 10.072 seconds and a highest average of 9.335 seconds. It’s not great, but not bad. The outside of the top 8 is almost 1 second.
Using the overhead camera provided by AWS via Twitch stream, you can create a graphical view showing the paths taken by the car, as shown in the following diagram.
Comparing this to the way that the same model likes to drive in training, there is a slight difference.
What’s soon revealed is that I managed to go straight (above), but the car didn’t turn the corner as closely as it was during training, causing a bit of confusion over the bottom half of the track. Nevertheless, the car displayed a desirable survival instinct and remained on track even when faced with unexpected sharp angles.
Why did this happen:
- It’s too much to rotate 20 degrees using the Ackermann steering. Real cars can’t do that in the real world
- The turning radius increases as the speed increases due to slipping.
- Reaction times play more roles as speed increases, and my model is too slow to run around the corner
The combination of turning radius and reaction time effects also caused problems at the start. As the car moves slowly it changes much faster and goes outside the truck inside, causing problems for me and others.
My takeaway:
- Overall, the training approach seemed to work well. The well-tuned car went straight and straight, and the background noise seemed to have been less annoyed my model.
- By increasing the maximum speed and reducing the maximum angle of action space, you need to approach the actual handling characteristics of the car at speed during training.
- Physical races are not yet well understood. That’s a lot about model meat cars. Some models thrive with objectively fully calibrated calibrated cars, while others work well when matched with a particular car.
- The truck is king. I have access to trucks through my employer or have built them at home, and no matter which model most people say they were surprised at in the end, there was a great advantage.
Re: Enjoy the internal view of the car on Invent and see if it can detect any of the issues I have discussed. This video was recorded after being knocked out of the race using a car using custom car software.
Closed Time: Where are you going from here?
This section is best enjoyed as a soundtrack for Semisonic closing times.
After a fierce week of race, we were all wrapped up in an exposition, so Re: inventions are literally dismantled around us, so the question was:
This was the last Deepracer Championship, but the general sentiment was that no one really missed the virtual race, but that’s a problem solved – physical races are still very fun and the community is not ready to go yet. re: Several initiatives invented have gained traction with the common goal of making deep lacers more accessible.
- By registering your car with Deepracer Event Manager with Deepracer Custom Car Software Stack, you can capture car logs directly during the event and generate analytical videos as shown in this article.
- Combine off-the-shelf components and custom circuit boards to build both options for the 1:28 scale Deepracer Pi Mini.
The Deepracer Custom Console is a drop-in replacement for the current car UI with a beautiful UI designed with CloudScape, designing DESING and AWS consoles.
Prototype Deepracer Pi Mini – 1:28 scale
Close the word
Deepracer is a great way to teach AI in a very physical and visual way, and is suitable for older children, students and adults in corporate environments. It will be interesting to see how AWS, its corporate partners and communities continue their journey in the coming years.
Thank you for mentioning all the people, and so many, who have been involved in Deep Lacers from the previous inception until today. A big congratulations to this year’s winners!
Closure times, all new beginnings come from other beginnings…
About the author
Lars Lorenz Ludwigsen He was featured on AWS Deepracer in late 2019 and was quickly engrossed in the tech enthusiast. Lars works as Managing Director at Accenture, helping clients build the next generation of smart connected products. In addition to his role in Accenture, he is an AWS Community Builder focused on developing and maintaining software solutions for the AWS Deepracer community.
