Tao Chen

I am a Ph.D. student in EECS at MIT CSAIL, advised by Prof. Pulkit Agrawal. My research interests revolve around the intersection of robot learning, manipulation, and navigation.

I received my master's degree from the Robotics Institute, Carnegie Mellon University (CMU RI) in May 2019, advised by Prof. Abhinav Gupta.

Prior to this, I was a research engineer in Shanghai LX Robotics, where I conducted research on object detection, image segmentation, deep reinforcement learning in robotics, SLAM, etc.

I earned my bachelor's degree from Shanghai Jiao Tong University (SJTU) in June 2016, majoring in mechanical engineering and automation. I was also an exchange student (GEARE program) at School of Mechanical Engineering, Purdue University.

Email  /  CV  /  Google Scholar  /  GitHub /  LinkedIn


PyRobot: An Open-source Robotics Framework for Research and Benchmarking
Adithya Murali*, Tao Chen*, Kalyan Vasudev Alwala*, Dhiraj Gandhi*, Lerrel Pinto, Saurabh Gupta, Abhinav Gupta [* Equal contribution]
Press Coverage: WIRED, VentureBeat, THE ROBOT REPORT, SiliconANGLE, IB Times, SD Times, Medium
arXiv / bibtex / project page / code GitHub Star / facebook AI blog @article{pyrobot2019,
author = {Adithyavairavan Murali* and Tao Chen* and Kalyan Vasudev Alwala* and Dhiraj Gandhi* and Lerrel Pinto and Saurabh Gupta and Abhinav Gupta},
title = {{PyRobot}: An Open-source Robotics Framework for Research and Benchmarking},
journal = {CoRR},
volume = {abs/1906.08236},
year = {2019},
url = {https://arxiv.org/abs/1906.08236},
archivePrefix = {arXiv},
eprint = {1906.08236}

This paper introduces PyRobot, an open-source robotics framework for research and benchmarking. PyRobot is a light-weight, high-level interface on top of ROS that provides a consistent set of hardware independent mid-level APIs to control different robots. PyRobot abstracts away details about low-level controllers and inter-process communication, and allows non-robotics researchers (ML, CV researchers) to focus on building high-level AI applications. PyRobot aims to provide a research ecosystem with convenient access to robotics datasets, algorithm implementations and models that can be used to quickly create a state-of-the-art baseline. We believe PyRobot, when paired up with low-cost robot platforms such as LoCoBot, will reduce the entry barrier into robotics, and democratize robotics.

Learning Exploration Policies for Navigation
Tao Chen, Saurabh Gupta, Abhinav Gupta
International Conference on Learning Representations (ICLR), 2019
arXiv / bibtex / project page / video / code / poster @inproceedings{chen2018learning,
author = "Chen, Tao and Gupta, Saurabh and Gupta, Abhinav",
title = "Learning Exploration Policies for Navigation",
booktitle = "International Conference on Learning Representations",
year = "2019",
url = "https://openreview.net/forum?id=SyMWn05F7"

Numerous past works have tackled the problem of task-driven navigation. But, how to effectively explore a new environment to enable a variety of down-stream tasks has received much less attention. In this work, we study how agents can autonomously explore realistic and complex 3D environments without the context of task-rewards. We propose a learning-based approach and investigate different policy architectures, reward functions, and training paradigms. We find that use of policies with spatial memory that are bootstrapped with imitation learning and finally finetuned with coverage rewards derived purely from on-board sensors can be effective at exploring novel environments. We show that our learned exploration policies can explore better than classical approaches based on geometry alone and generic learning-based exploration techniques. Finally, we also show how such task-agnostic exploration can be used for down-stream tasks.

Hardware Conditioned Policies for Multi-Robot Transfer Learning
Tao Chen, Adithya Murali, Abhinav Gupta
Advances in Neural Information Processing Systems (NeurIPS), 2018
arXiv / bibtex / project page / video / code / poster @inproceedings{chen2018hardware,
title={Hardware Conditioned Policies for Multi-Robot Transfer Learning},
author={Chen, Tao and Murali, Adithyavairavan and Gupta, Abhinav},
booktitle={Advances in Neural Information Processing Systems},

Deep reinforcement learning could be used to learn dexterous robotic policies but it is extremely challenging to transfer them to new robots with vastly different hardware properties. It is also prohibitively expensive to learn a new policy from scratch for each robot hardware due to the high sample complexity of modern state-of-the-art algorithms. We propose a novel approach called Hardware Conditioned Policies where we train a universal policy conditioned on a vector representation of robot hardware. We considered robots in simulation with varied dynamics, kinematic structure, kinematic lengths and degrees-of-freedom. First, we use the kinematic structure directly as the hardware encoding and show great zero-shot transfer to completely novel robots not seen during training. For robots with lower zero-shot success rate, we also demonstrate that fine-tuning the policy network is significantly more sample efficient than training a model from scratch. In tasks where knowing the agent dynamics is crucial for success, we learn an embedding for robot hardware and show policies conditioned on the encoding of hardware tend to generalize and transfer well.

Development of a Soft Elastomeric Gripper for Dexterous Grasping
Tao Chen, Guo-Ying Gu
Bachelor's Thesis, 2016
Awarded 2016 Excellent Bachelor Thesis (Top 1%) of Shanghai Jiao Tong University
project page

Development of robotic grippers that can grasp universal objects analogous to humans hands is still a challenging and open problem in the robotics society. I designed a four-fingered soft gripper, in which each finger is a bending multi-cavity pneumatic elastomer actuator (MCPEA). The fingers are modularized and entirely composed of soft elastomer material that is inherent compliant. I also developed a finite-element analysis (FEA) model to analyze the kinematics of the MC-PEA. The influences of the geometric parameters on the bending performance in terms of the bending angle and output force at the distal tip are investigated as well. The four-fingered soft gripper is able to grasp universal unknown objects with different sizes and shapes even frangible objects, such as vegetables, fruits, eggs, noodles, pens, candles, electronics, and cups.

Work Experience

Automatic Exploration for Dense Map Building
Tao Chen
Shanghai LX Robotics, 2017
project page

A key step for robots to get popularized into our daily life is that robots should be able to automatically explore the new environment when they are deployed in new houses or buildings. In this project, I combine the strength of motion planning (OMPL and SBPL), frontier-based exploration, SLAM (ORB-SLAM2), and object recognition and segmentation (FCIS) techniques to build an automatic mapping system that can autonomously explore the new houses, recognize daily objects and remember their locations while keep building the dense map as it moves. After the map is built, the robot can be asked to find and move to the objects it has seen (like cup, monitor) autonomously.


CMU 16-824 Visual Learning and Recognition - Spring 2019
Teaching Assistant (TA)

Course Projects

Robot Construction via Planning and Learning
Tao Chen, Xianyi Cheng
Learning for Manipulation, 2018
Instructor: Oliver Kroemer
video1 / video2 / video3 / code / report

We combined the symbolic planning and supervised learning to efficiently learn to move a set of blocks from an initial configuration to a goal configuration (a.k.a, robot construction problem). The symbolic planning module plans the sequence actions (path) to move the blocks (a block or a sub-assembly) to reach the goal configurations. The supervised learning module (stability checker) predicts whether the state (RGB image) is stable or not so that the planning module only plans with the actions that lead to stable states. We used domain randomization techniques to generate more diversified visual data to make the stability checker more robust. These two modules combined lead to an effective way to solve the robot construction problem.

Design and Manufacturing of a Tennis Ball Collecting Robot
Tao Chen, Matthew Stouder, Zhedong Han, Duankang Fu, Sara Lyons, Zhishang Xu
GEARE program, Purdue University, 2015
video / code

This project was the Senior Engineering Design Capstone project at Purdue University. We built an aesthetically pleasing tennis ball collecting robot which can collect tennis balls dispersed on a tennis court. I was fully responsible for all the programming and control tasks for the robot.

2015 RoboMaster Robotics Competition
Tao Chen, Mechanical Team Leader
hosted by DJI , 2015

This competition was a real-life version of Counter-Strike game with real mobile robots. I led and managed the mechanical group. We won the second prize in 2015 National RoboMaster Robotics Competition of east China and the third prize in 2015 National RoboMaster Robotics Competition Final.

An Active Rehabilitation Device for Elbow Joints
Tao Chen, Leader
Advisor: Prof. Hua Shao
Engineering Design, 2015
China Patent, CN105148460B
project page

We designed and built an inexpensive yet effective elbow joint rehabilitation device. The device is only composed of mechanical parts such as a lead screw, and a four-bar mechanism. It can help patients exercise their elbow joints in an inexpensive way, and it is also very easy to use and portable. We have applied a patent (Application Number: CN201510472161.5, Publication Number: CN105148460B) for this rehabilitation device.

A High-adaptability Rescue Robot
Tao Chen, Leader
Advisor: Prof. Qinghua Liang
Design and Manufacturing, 2014
code 1 / code 2

We built a high-adaptability track robot with two separate frames. I led and managed the team and I was responsible for electronic control, programming, and part of manufacturing.

This guy makes a nice webpage.