The worker “Optimus” is activated.
The
Tesla humanoid robot has unlocked new skills!
Yesterday, the official Tesla Optimus released a new demo video, showing the latest progress of the second-generation Optimus humanoid robot.
This time, Optimus has started working in the factory. It has learned to assemble batteries in the Tesla battery factory, and it walks faster, farther, and more stable than before.
Let’s first take a look at Optimus’ latest skills and training details.
Now, Optimus’s end-to-end neural network has been trained and is capable of accurately sorting the battery units in the Tesla factory.
Optimus runs in real time on the robot’s FSD computer, relying only on 2D cameras, hand tactile and force sensors. Optimus maintains balance with its legs while the network drives the entire upper body.
The insertion process requires very precise actions, and the tolerance for error is very low. The neural network automatically locates the next available slot. Optimus can also recover autonomously from failures.
Optimus’s training data is collected through human remote operation and has been expanded for various tasks.
Optimus is tested in the factory, and the rate of human intervention continues to decrease.
Of course, Optimus can now walk around the office regularly, not only walking faster, but also going farther and farther.
In response to this, Tesla Optimus (Tesla Bot) engineer Milan Kovac gave an interpretation of more training details of Optimus.
In the past few months, Tesla’s excellent manufacturing team has made more robots for research and collecting artificial intelligence data.
The team has trained and deployed a neural network that allows Optimus to begin performing useful tasks, such as picking up battery units from the conveyor belt and precisely inserting them into trays.
This neural network runs entirely end-to-end, which means Optimus consumes only data from the robot’s 2D camera and onboard proprioceptors and directly generates a sequence for joint control.
Optimus runs totally on the robot’s embedded FSD computer powered by an onboard battery. This design allows a single neural network to perform multiple tasks by adding more diverse data to the training process.
Although Optimus is not perfect at the moment and is a bit slow, we are seeing higher and higher success rates and fewer and fewer errors. Tesla also trained Optimus to recover from failure cases and saw spontaneous corrections occurring.
Tesla has deployed several Optimus humanoid robots in a factory, testing them at real workstations every day and continuously improving them.
The team is working further to make Optimus move faster and handle more adverse terrains without sacrificing its human-like characteristics. The team will also focus on repeatability, training the neural network to handle dynamic calibrations and minor differences between robots. There will be more updates soon.
In addition, Jim Fan, the head of
AI Agents and a Senior Research Scientist at NVIDIA, gave high praise for the skills shown in the new Optimus video.
He believes that the video offers a glimpse into the real scenarios of human data collection, which is Optimus’s greatest strength. What does it take to build such an assembly line? Optimus has achieved the following points:
Optimus’s hand is one of the top five-fingered dexterous robot hands in the world. It has tactile perception, has 11 degrees of freedom (DOF), while many competitors only have 6-7 DOF, and has strong robustness, capable of handling a large amount of object interaction without the need for continuous maintenance.
Remote operation software: We can see human operators wearing VR glasses and gloves. Setting up software to achieve real-time transmission of first-person video and precise control output, while keeping extremely low latency, is quite amazing. Humans are very sensitive to the slightest delay between their movements and the robot’s movements. Optimus has a smooth full-body controller that can execute human postures in real time.
Large scale fleet: You need more than one robot to collect data in parallel, need well-trained humans on multiple shifts each day (ideally around the clock), and an on-call maintenance team to ensure the robots are always busy. This is a lot of operational complexity that an academic research laboratory would never even consider.
Tasks and environment: It is equally important to figure out what to remote-control. Currently, most of this kind of work is demonstration-driven: collecting task data that you want to put into social media videos. But to solve the problem of general-purpose robots, we need to carefully consider the distribution of tasks and environment. From 43 to 51 seconds in the video, we can see factory and home environments, such as moving batteries, handling laundry, sorting everyday items onto shelves.
This is an open research question: If you only have a budget to collect training data for a thousand tasks, what would you choose to maximize skill transfer and generalization?
Conclusion: Remote operation is a necessary condition to solve the problem of humanoid robots, but it is not enough to fully solve the problem. It fundamentally cannot scale.
In the comment section, Musk previewed the latest goal of the Optimus humanoid robot’s hand, which will have 22 degrees of freedom (DoF) later this year. That is something to look forward to.
Previously, according to Musk’s disclosure in the earnings call, the Tesla Optimus humanoid robot may have the ability to complete “useful factory tasks” by the end of this year, and plans to use it in their own factories first by the end of this year. At the same time, Tesla might sell Optimus externally before the end of 2025.
“The Evolution of Tesla’s Humanoid Robot“
The earliest reference to Tesla’s humanoid robot dates back to the “Tesla
AI Day” event in August 2021, when Musk first revealed plans to create a humanoid robot (Tesla Bot).
This humanoid robot stands at approximately 1.72 meters tall, weighs about 112 pounds, has a balanced physique, and also carries a facial screen. It is capable of moving at a speed of approximately 8 kilometers per hour.
In October 2022, at that year’s “Tesla AI Day”, the Optimus humanoid robot made its first appearance. At that time, it could not perform stunts like dancing. The power consumption of the robot at work is 500W, the weight reaches 73 kilograms, and the freedom of hand movement has 27 degrees.
However, in the demo presentation, Optimus demonstrated scenes of it moving boxes, watering plants, and working in the Tesla Super Factory.
In May 2023, after more than half a year of dormancy, Tesla’s Optimus finally gained the ability to walk smoothly and grab objects.
In September 2023, Tesla’s Optimus evolved again, being able to independently classify objects. The highlight is the complete end-to-end training of the neural network: inputting video and outputting control. In its spare time, it can also do yoga for recreation.
In December 2023, the second generation of Optimus arrived, indeed resembling something from the movie “I, Robot“.
With the release of new videos, the progress speed of Tesla’s humanoid robot has once again amazed people.
© Copyright notes
The copyright of the article belongs to the author, please do not reprint without permission.
