Tesla is creating San Francisco in a simulation to assist practice Autopilot/FSD
September 21, 2022
By Kevin Armstrong
Tesla may be ramping up how it uses simulation to train its Autopilot system. A report by Electrek asserts that it has sources claiming that the company is concentrating on a reproduction of San Francisco. The article includes an image of the recreation and states that Tesla is working with Real Engine on its simulation.
According to Electrek, the image below is part of Tesla’s simulation of San Francisco.
Tesla gave the world a look at how it uses simulation to advance the Autopilot program during the first AI Day in August of 2021 (recap).
AI Day
At the first AI Day Tesla talked about the use of using simulations to help train Autopilot. The video below is cued up to where they discuss a simulation.
Ashok Elluswamy, the Director of the Autopilot Program, showed a video that, at first glance, looked real other than an appearance by a Cybertruck. “I may say so myself. It looks very pretty,” said Elluswamy. He explained that the company is investing heavily in using simulation. “It helps when data is difficult to source. As large as our fleet is (FSD Beta users), it can still be hard to get some crazy scenes,” the director explained while showing a rendering of two people and a dog running in the middle of a busy highway. “This is a rare scene, but it can happen, and Autopilot still needs to handle it when it happens,” said Elluswamy.
It appears that Tesla has jumped on Fortnite’s Battle Bus by teaming up with Epic Games and its development platform — Unreal Engine. Fortnite is one of the most popular games of all time, with 80 million subscribers and 4 million daily users, and it was created with Unreal Engine. Epic flexed its creative muscles when it gathered experts to create The Matrix Awakens: An Unreal Engine 5 Experience. The goal was to “blur the boundaries between cinematic and game, inviting us to ask — what is real?” The project spotlight on Unreal Engine shows just how incredibly realistic a simulation can be.
https://www.youtube.com/watch?v=WU0gvPcc3
After Elluswamy explained that the company is investing in simulation, it makes sense that Tesla would be hiring several positions with simulation in the job description. Electrek pointed out one posting for Autopilot Rendering Engineer. The posting states the successful candidate “will contribute to the development of Autopilot simulation by enabling and supporting the creation of photo-realistic 3D scenes that can accurately model the driving experience in a wide range of locales and conditions.” Tesla prefers the candidates to have experience working with Unreal Engine.
While not new, this does show that Tesla is doubling down on efforts to improve Autopilot. It has recently rolled out Full Self Driving to 60,000 more users, bringing the FSD Beta program to 160,000 in North America.
We can only guess how many thousands of simulations the Autopilot team is conducting to add to the data the Beta testers are collecting. It seems unlikely that Tesla has only created the City by the Bay in its simulations. Perhaps Elluswamy will show more renderings at the second AI Day on September 30th.
November 3, 2022
By Kevin Armstrong
Shiny new Model 3s started appearing last month, missing the small, round polka dots on their bumpers. The circles were gone after the company announced it was removing the ultrasonic sensors (USS) to move to strictly Vision. However, after a few weeks in the wild, evidence suggests the company will mimic what USS used to perform and display.
Twitter user @EZebroni tweeted a helpful video demonstrating the difference between a Tesla with and without USS. It’s noticeably different then what Tesla drivers are used to seeing. The distance lines are gone, and even the images of what is beside the non-USS Tesla are not there, just a lot of white screen.
As seen in the video, a vehicle without USS will not display any information to the driver regarding nearby objects. The distance to nearby objects is not displayed on the screen nor are the colored arcs that usually display when an object is nearby.
However, update 2022.40.4 appears to reintroduce some functionality to owners of these vehicles, at least in shadow mode.
Twitter user @greentheonly, a well-known Tesla hacker, tweeted: 2022.40.4 merges no-ultrasonics code into “mainline,” now receives parking distances from Autopilot (where not equipped with proper ultrasonics).
From Green’s findings, it looks like Tesla is preparing to mimic ultrasonic sensors with Tesla Vision, which essentially means that the vehicle is likely to display not only the arcs when an object is nearby, but also the distance to the given object.
Greentheonly was asked several questions after tweeting this information. Someone wanted to know if the Tesla no-ultrasonics code was working in shadow mode, a blend or pure AutoPilot. Green said that it is not a blend, but he needed to figure out what is displayed on USS-less cars, encourage someone with a new Model 3 or Model Y to try 2022.40.4.
Although the code is available in 2022.40.4, it’s not clear yet whether Tesla is running this code in shadow mode, meaning that the code is run in the background without notifying the user, or if distances are now being shown on vehicles without USS.
Blind spot
One of the biggest questions that remains is how Tesla will solve the roughly 3 foot blind spot in front of the vehicle where cameras can’t see.
Some speculate that Tesla will fill in this gap by taking the information the cameras saw before the vehicle got closer to the object, but this may not work in all circumstances.
Others think that Tesla may be able to have the car back up in the rare situations where objects in front of the vehicle may have changed, such as when summong a parked vehicle. As these USS-less vehicles make it into more hands and update to 2022.40.4 we should have answers to some of these questions, however Autopark, Summon and Smart Summon remain unavailable on these vehicles, even with the latest update.
Will Tesla stop utilizing ultrasonic sensors like they did radar?
Another Twitter user asked if this would make the USS on the existing vehicles “ornaments?” This has been a widespread concern since the non-USS cars started showing up. Tesla addressed this matter when it announced it was removing USS. It states on the company website: At this time, we do not plan to remove the functionality of ultrasonic sensors in our existing fleet.
At this time USS have only been removed from the Model 3 and Model Y, but they will also be removed from the Model S and Model X in 2023. Green also believes the existing USS will remain operational, tweeting: not any time soon, I imagine. Also, the front ones still cover an area where the cameras don’t see.
Another user questioned, does it only take information from the moving/static object networks, or is it able to use the occupancy, and road edge networks as well? To which Green responded that he had not looked into that kind of detail yet.
The new world of non-USS is upon us, and there will be many questions. This revelation is likely the first of a long line of updates to assist vision in making up what was lost with ultrasonic sensors.
November 2, 2022
By Gabe Rodriguez Morrison
Since the Cybertruck unveiling in 2019, Tesla has pushed production dates back numerous times: from late 2021 to late 2022, to early 2023, and most recently to the mid-2023 target for initial low-volume production.
Tesla is reportedly planning to begin Cybertruck high-volume production at the end of next year. The company is preparing to set production dates according to a report from Reuters, indicating mass manufacturing of the Cybertruck to begin at the end of 2023.
It still seems that Tesla is on track to start low-volume production in mid-2023, with mass production beginning at the end of the year.
During the Q3 2022 earnings call, Tesla stated that it was working on preparing the Austin, Texas plan to build the Cybertruck, with “early production” set to begin mid-2023. During the call, Elon Musk said that Tesla is “in the final lap for Cybertruck.” The Q3 2022 Shareholder Deck also listed Cybertruck’s production status as in the “tooling” phase which means it’s preparing for production.
This aligns with timelines from IDRA Group, the die-casting company making the 9,000-ton Giga Press that will be used for the Cybertruck. The Giga Press has reportedly arrived in Houston, Texas and can be expected to reach the Gigafactory in Austin, Texas soon.
Reuters points out that gradually ramping up Cybertruck production starting mid-2023, reaching high volume production by late 2023 would mean that some of the roughly 1 million reservation holders may have to wait more than a year before their truck is delivered.
Elon stated that the company had more orders “than we could possibly fulfill for three years after the start of production.” For these reasons, Tesla stopped taking orders for the Cybertruck outside North America in May 2022.