|
|
|
I am a Ph.D. student in Robotics at Carnegie Mellon University, working with Prof. Sebastian Scherer and Dr. Wenshan Wang at AirLab, Robotics Institute, CMU. My research focuses on multi-modal perception for robotics — specifically building self-supervised methods that fuse RGB and thermal imagery for robust day/night perception. I am currently also a Robotics Research Intern at Field AI, pushing state-of-the-art off-road semantic segmentation by fusing RGB, Thermal, and Language into a unified multi-modal alignment framework. Find more at Experience and Publications. My broader research interests span multi-modal perception, self-supervised learning, and motion planning — with a focus on making robots robust to challenging real-world conditions like night, fog, and off-road terrain. In my free time apart from the usual movies, I love to play badminton and being near water — fountains, rivers, and beaches. Feel free to check out my resume and drop me an e-mail if you want to chat! |
|
May '26 |
Started as Robotics Research Intern at Field AI, Irvine, CA, under Dr. Shehryar Khattak and Prof. Sebastian Scherer. |
|
Jan '26 |
AnyThermal accepted at ICRA 2026! |
|
Oct '25 |
Awarded the Uber Presidential Fellowship at the Robotics Institute, CMU. |
|
May '25 |
ThermalDiffusion accepted at the Thermal Infrared in Robotics Workshop, ICRA 2025. |
|
Aug '24 |
Started Ph.D. in Robotics at AirLab, Robotics Institute, Carnegie Mellon University. |
|
May '24 |
Graduated from IIT Kharagpur with an Integrated M.Sc. in Mathematics and Computing (GPA: 9.29/10). |
|
Feb '24 |
TartanDrive 2.0 accepted at ICRA 2024! |
|
Sep '23 |
Submitted a manuscript for ICRA 2024. |
|
May '23 |
Returned to Airlab, CMU for the summers as a Research Associate. |
|
Jul '22 |
Manuscript of our solution in the IROS Challenge accepted at 2022 IEEE International Conference on Control and Robotics (ICCR). |
|
May '22 |
Spending the summer at AirLab, CMU as part of the Robotics Institute Summer Scholar Program 2022. |
|
Apr '22 |
2 out of 3 submissions accepted as part of the MLRC 2021 and published under 2022 ReScience C Journal. |
|
Feb '22 |
Led 3 teams and submitted 3 manuscripts as part of the 2021 Machine Learning Reproducibility Challenge. |
|
Sep '21 |
Won 1st place at the Navigation and Manipulation Challenge at IROS 2021! |
|
Sep '21 |
Submitted a manuscript for ICRA 2022. |
|
May '21 |
Our manuscript accepted at 2021 ICRA OCAR Workshop. |
|
May '21 |
Started working under Dr. Bharadwaj Amrutur at ARTPARK, IISc Bangalore in collaboration with NOKIA. |
|
Apr '21 |
Our work done in the Indy Autonomous Challenge submitted as manuscript at 2021 ICRA Workshop on OCAR. |
|
Apr '21 |
Awarded the IASc-INSA-NASI Summer Research Fellowship for working under Dr. Bharadwaj Amrutur. |
|
Mar '20 |
Joined the Autonomous Ground Vehicle Research Group as Planning team member. |
|
Jul '19 |
Started my undergrad at IIT Kharagpur! |
|
May '26 – Present Under Dr. Shehryar Khattak and Prof. Sebastian Scherer Project: Pushing state-of-the-art off-road semantic segmentation — day and night — by fusing RGB, Thermal, and Text into a unified multi-modal alignment framework for closed- and open-vocabulary perception.
|
|
August '24 – Present Under Dr. Wenshan Wang and Prof. Sebastian Scherer at AirLab, Robotics Institute, CMU Project 1 — AnyThermal: Improved RGB-Thermal perception via a multi-modal self-supervised backbone, open-source RGB–T platform, and diverse dataset that together set new state-of-the-art and enable scalable, community-driven research.
Project 2 — TartanStar: 100+ hours of time-synchronized multi-modal data (Stereo RGB, Thermal, Event, RADAR, LiDAR, IMU) across diverse environments and embodiments to advance foundation models for uncommon modalities.
|
|
|
May '22 – July '24 Working under the supervision of Prof. Sebastian Scherer and Dr. Wenshan Wang at Airlab, Robotics Institute, CMU on developing generalisable and accurate vehicle models. My primary mission was to enhance the autonomous navigation capabilities of a modified Yamaha All-Terrain Vehicle by developing an accurate dynamics model which considers the effect of both the control input and the ATV's environment. |
|
May '21 - September '21 Worked under the supervision of Dr. Bharadwaj Amrutur and Dr. Mukunda Bharatheesha . Worked on a centralised task allocation module and motion planning of warehouse robots. Extended this work to a networked setting where robots could cimmunicate with each other to actively avoid colliding. This work was submitted at ICRA, 2022. |
|
March '20 – May '24 Working under the supervision of Dr. Debashish Chakravarty at AGV, IIT Kharagpur in the motion planning team as a team meber in my inital years. Here I got to worlk on local planners. In my junior year , I led the team and our performance was tremendous in IROS Navigation and MLRC challenge. |
|
P. Maheshwari et al. Accepted at ICRA 2026 [paper, project page, code] We present AnyThermal, a task-agnostic thermal backbone trained via cross-modal self-supervised distillation from DINOv2-based visual foundation models. By distilling 90k+ RGB-T pairs across 5 diverse datasets (aerial, urban, indoor, off-road), AnyThermal achieves up to 36% gains over prior task-specific thermal backbones on segmentation, depth estimation, and cross-modal place recognition — all without task-specific training. We also introduce TartanRGBT, the first open-source RGB-T data collection platform and a diverse 16k-pair dataset to lower barriers for thermal research. |
|
P. Maheshwari et al. Accepted in Elsevier Book — Artificial Intelligence Modeling for Dynamical Problems [paper, video] We address the challenge of modeling off-road vehicle dynamics by introducing PIAug, a physics-informed data augmentation method. In scenarios where well-distributed datasets are hard to obtain, PIAug leverages nominal models to generate high-speed motion predictions from low-speed data. Our approach incorporates a physics-inspired loss function during neural network training, resulting in up to 67% less mean error in trajectory prediction compared to a standalone nominal model. In real-life navigation experiments, our model outperforms the Kinematic Bicycle Model (KBM) by achieving 4x tighter waypoint tracking constraints at out-of-domain velocities, especially during aggressive maneuvers. |
|
M. Sivaprakasam, P. Maheshwari et al. Accepted at ICRA 2024 [paper] TartanDrive 2.0 extends our prior TartanDrive dataset with additional sensor modalities and improved infrastructure for self-supervised learning in off-road driving. The dataset spans diverse terrain types and includes synchronized multi-modal recordings to advance learning-based approaches to off-road navigation. |
|
Published at 2022 IEEE International Conference on Robotics and Automation (ICRA) [paper, code] We introduce an indoor motion planning system for navigating agents through unknown cluttered environments. Our approach employs a novel probabilistic global planner for waypoint selection, a local planner for path planning, and an adaptive Model Predictive Control strategy for robust system control and swift maneuvers. Evaluation results demonstrate a noteworthy decrease in computational costs while maintaining high accuracy and robust control. |
|
Presented at ICRA 2021 Workshop on Opportunities and Challenges with Autonomous Racing [paper, code] We present a strategy for the control of anautonomous racing car on a pre-mapped track. Using a dynamic model of the vehicle, the optimal racing line is computed, taking track boundaries into account. With the optimal racing line as areference, a local nonlinear model predictive controller (NMPC) is proposed, which takes into account multiple local objectives like making more progress along the race line, avoiding collision with opponent vehicles, and use of drafting to achieve more progress. |
|
Published in ReScience C 2022 Journal and Presented at 2022 NeurIPS Journal Track [paper] The following paper is a reproducibility report for "Social NCE: Contrastive Learning of Socially-aware Motion Representations" published in ICCV 2021 as part of the ML Reproducibility Challenge 2021. The original code was made available by the author. We attempted to verify the results claimed by the authors and reimplemented their code in PyTorch Lightning. |
|
S. Bansal, W. Wang, Y. Liu, P. Maheshwari. Accepted at Thermal Infrared in Robotics Workshop, ICRA 2025 [paper] We present ThermalDiffusion, a method for visual-to-thermal image-to-image translation using conditional diffusion models. In nighttime or degraded environments (fog, mist, dust), thermal cameras provide valuable information due to heat signatures. ThermalDiffusion augments existing multi-modal datasets with synthetically generated thermal data, facilitating widespread adoption of thermal cameras in robotics and autonomous systems for navigation in degraded visibility conditions. |
|
Published at 2024 IEEE International Conference on Robotics and Automation (ICRA) [paper] This study introduces an efficient approach to designing costmaps for off-road driving tasks. Instead of relying on deep neural networks, we employ an inverse reinforcement learning-based method that considers uncertainty. Leveraging parallel model-predictive control and robotic risk estimation, our approach demonstrates a 44% improvement in expert path reproduction and 57% fewer interventions compared to a geometric baseline. We also highlight the impact of varying risk tolerance on navigation behaviors, particularly in challenging scenarios like slopes and tall grass. |
|
Presented at 2023 ICRA Workshop on Pretraining4Robotics Lightning [paper] This work focuses on enhancing our TartanDrive dataset to address challenges in robotics related to real-world complexity. We aim to provide a comprehensive dataset for off-road driving by incorporating diverse terrain and multiple modalities across approximately 225 acres. Additionally, we offer scripts for data manipulation, enabling users to filter and format the data for specific needs. Our framework supports both utilizing our dataset and collecting new data, aiming to contribute to robotics advancement and lower barriers to accessing large-scale data. |
|
Published in ReScience C 2022 Journal and Presented at 2022 NeurIPS Journal Track [paper] This report details our attempt to replicate the study on "Differentiable Spatial Planning using Transformers" by Chaplot et al. The paper explores spatial path planning through Spatial Planning Transformers, demonstrating superior performance over previous models. Our reproduction involves conducting experiments, testing on new data, and assessing planning accuracy under heightened obstacle complexity. |
|
[challenge] [code] The task was to develop (in simulation) a mobile robot with a manipulator capable of multiple waypoint navigation and exploration. As the winning solution, we created a probabilistic route planner, which finds near-optimal solutions while respecting the computational capabilities of the robot and integrated this with an adaptive MPC. We presented this work at the International Conference on Control and Robotics 2022 in our paper "Multiple Waypoint Navigation in Unknown Indoor Environments" |
|
[challenge] Member of the IIT Kharagpur - IUPUI, Indiana - USB Colombia collaborative team. I focused on motion control algorithms for full-scale autonomous racing. I first developed a Non-linear Model Predictive Controller (NMPC) with a dynamic bicycle model, incorporating a high-fidelity Pacejka Model for precise tire force representation at high speeds. I then optimized the reference racing line and designed a tailored cost function to encourage our vehicle to execute real-world racing maneuvers like overtaking and drafting. My contributions helped my team secure the 8th position in the challenge's simulation round and also resulted in a first-authored work accepted at the OCAR Workshop, ICRA 2021 |
|
[GitHub] Co-organised by IIT Kanpur and Drona Aviation, as part of the Inter IIT Tech Meet 11.0. This challenge aimed to develop a vision based state feedback control for an indoor multi-drone system handling socket communication with the flight controller without the use of ROS. Being a core team menber, I played a pivotal role in leading the communication and motion planning team. We used a adaptive PID controller which received the current pose estimate on basis of precise detection of Aruco Tag using our vision module. Our team secured the first place. |
|
Airlab, CMU In spring 2023, my aim was to enhance dynamics modeling for out-of-distribution (OOD) scenarios while maintaining in-domain accuracy. I explored Physics-Informed Neural Networks (PINN), which merge accurtacy of learning based methods with the generalization capability of traditional physics models. PINNs yielded significant improvements in modeling vehicle dynamics compared to solely data-dependent neural networks. However, during the summer of 2023, I encountered a key limitation of PINN: it required well-spread data for physics evaluations, a challenge in off-road driving due to data collection complexities. To address this limitation, I introduced "PIAug - Physics Informed Augmentation," accepted in the Elsevier Book — Artificial Intelligence Modeling for Dynamical Problems. This approach involved using a physics priior for both augmenting a dataset while also introducting the phsyics principles to NN. I then optimised the deployment of the NN-based vehicle models by reducing the inference runtime by 6x and GPU memory requirements by 1.9x. As a usecase PIAug allowed the vehicle to navigate with 4x tighter tracking constraints at out-of-domain velocities. |
|
Airlab, CMU During my first summer at CMU in 2022, I focused towards developing an online prediction system for a dynamics-based vehicle model. This approach involved harnessing information from the robot's history, such as odometry and IMU data, and incorporating additional inputs about the future's environemnt like FPV images and processed top-down terrain maps. As a result of these efforts, we achieved a noteworthy 25% improvement in modeling accuracy for the vehicle's dynamics. However, the approach revealed limitations in handling out-of-distribution (OOD) scenarios, potentially leading to unsafe predictions of parameters. Despite these setbacks, this project introduced me to the stimulating field of aiding robots in navigating challenging environments. While we couldn't deploy this method on the ATV, it strengthened my determination to continue working towards enhancing robotic navigation in demanding conditions. |
|
ARTPARK, IISc Bangalore in collaboration with NOKIA The objective of this project was to develop a high-performance network infrastructure to create a safe network-aware autonomy for multiple robots. Being one of the earliest members in the team, I was responsible with developing the Multi Robot Task Allocation module. I did this using a market auction based approach which considers the time required to complete the already queded tasks of each robot before finding the robot which would optimize the overall time required by the fleet to execute the tasks. After this, I developed a Multi Robot Path Planning module by extending the existing NAV2 library to support multirobot navigation. At last, I proposed and showcase how sharing ego-footprint information with other robots in vicinty over a shared network can help in proactively avoiding the other robots. This also allows robots to avoid collisions even when other robots cannot be detected using sensors due to occluded sensing range. Example - when 2 robots are coming from round the corner and are not detected by each other's lidar or camera due to obstruction in sensing. This work was submitted at ICRA 2022. |
|
AGV, IIT Kharagpur [code | demo] Led a team of 11 to develop an Open Source Python and C++ implementation for a sampling based planner in Frenet Frame with support for both ROS1 and ROS2 support along with simulation support of Gazebo and LGSVL. OpenMP was used for increasing the speed of the sampling and map updation step while planning. In its early days, this project acted as the first step towards parallelized computing and later this also became the first reasearch project where I led a team :) |
|
Term project under Prof. Soumyajit Dey [code | report] Implemented an optimized tiled and batched Cholesky factorization in CUDA using only single-precision arithmetic. Extensive experiments were then conducted on advantages of chunking while tuning the parameters of tile size, and order of looking. This project helped me get aquantied to CUDA and helped me understand how to efficiently exploit its use by efficient use of continuous threads and blocks. |
|
Term project under Prof. Nirupam Chakraborti, IIT Kharagpur [code | report] Developed a primary healthcare disease checker bot as a combination of a traditional multi-layer perceptron neural netowork and steps of evolutionary/genetic algorithms like – Selection, Crossover, and Mutation. Used SymCat, a public dataset mapping diseases to various symptoms, as the training and validatiaon dataset. Further compared this hybrid method against standard classifiers based on Naïve Bayes Classifier, neural networks and random forest. |
Graduate | |
|
Oct 2025 |
Awardee, Uber Presidential Fellowship — Robotics Institute, CMU |
Undergraduate | |
|
2022 |
Planning Team Lead, Machine Learning Reproducibility Challenge — Online |
|
2021 |
Winner, IROS Navigation and Manipulation Challenge — Prague, Czech Republic |
|
2021 |
Participant, Indy Autonomous Challenge — Indiana, USA |
|
2021 |
Awardee, IASc-INSA-NASI Summer Research Fellowship — IAS/INSA/NASI |
|
August 2019 - May 2023 [website,certificate] GYWS is a registered voluntary non-governmental organization run by students of IIT Kharagpur which ensures the socio-economic welfare of underprivileged sections of society. We run a school Jagriti Vidya Mandir (JVM) near the village of Gopali focusing on providing good quality free-of-cost education to 250+ children. I was a part of the Sponsorship team which is responsible for planning and executing fundraising events and programs. I started as a junior executive member in my freshman year and eventually led the entire team in my sebnior year as the Chief Fundrasing Officer. During the latter, I spearhead a team of 30 and helped to increase the raised funds from $57000 (2019-20) to $130,000 (2022-23). This was due to enhanced fund-raising methods for various initiatives of our NGO by contacting companies for Corporate Social Responsibility(CSR), Alumni of various premier institutes, and introducing gift-matching programs of MNCs like Google, Microsoft, Apple, NVIDIA etc. |
This template is a modification to Jon Barron's website. Find the source code to my website here.