Professional Experience | Akshay Antony

Curate scenario-specific datasets by analyzing model failures and identifying coverage gaps, ensuring stronger training signals and more representative evaluation.
Fine-tune Wayve’s end-to-end world foundation model with curated data and additional input signals to improve performance across highway and urban driving situations.

Enhanced MapTR for HD map prediction, optimizing inference speed and extending the model to output novel map change classes; trained across multiple RTX 3090 GPUs and achieved an average IoU of 0.91, ensuring robust mapping in complex driving environments.
Curated and refined multi-sensor datasets, filtering mislabeled/failed samples and designing new augmentation methods to maximize performance with limited data, enabling reliable perception in dense urban driving scenarios.
Integrated improvements into multi-class road and lane feature detection, providing high-fidelity map updates that strengthened prediction accuracy and system adaptability in dynamic environments.

Migrated the point cloud processing pipeline from CPU to GPU, significantly enhancing performance and efficiency.
Engineered calibration methods for aligning multi-sensor systems, including lidar-lidar, camera-lidar, and lidar-robot configurations, ensuring high spatial accuracy in complex environments.
Developed a ground segmentation system utilizing LiDAR and camera data, combined with advanced algorithms, to detect low-lying obstacles that are challenging to identify with LiDAR data alone, thereby improving navigation.
Designed and implemented a dimensioner system for determining pallet dimensions, achieving 1-2 inch accuracy in cluttered environments using LiDAR data.

Research Assistant

Machine Learning and AI Lab, Carnegie Mellon University

Feb 2023 - April 2023

During my time at MAIL CMU I worked on the intersection of large sequence modelling and chemistry. Some of my notable works are mentioned below.

Developed GPCR-BERT, a protein language model, to interpret and design G protein-coupled receptors (GPCRs), which are key targets in drug design. Fine-tuned Prot-Bert to predict variations in GPCR motifs and analyzed 3D structures to understand higher-order interactions within receptor conformations. (publication link)
Developed a self-supervised multimodal learning approach, combining graph neural networks (GNNs) and transformer-based language models, to improve the prediction of adsorption energy in catalysis. This method, called graph-assisted pretraining, reduces prediction errors by 10% and enhances model fine-tuning, showcasing the potential of language models in energy prediction without relying on atomic spatial coordinates. (publication link)
Developed MOFGPT, a GPT-2 based model generating Metal-Organic Frameworks (MOFs) from SMILES strings with a perplexity of 1.2. Fine-tuned the base model for adsorption energy prediction and implemented the reinforce algorithm ensuring generated MOFs are fine-tuned to meet energy, validity, and novelty criteria.

Deep Learning Perception Engineer

Thordrive

May 2022 - Aug 2022, Jan 2023 - Feb 2023

During my time at Thordrive, I focused on advancing perception systems through the development of deep learning and image processing techniques.

Built a deep learning-based semantic segmentation pipeline for LiDAR data, modifying the Cylinder 3D model to achieve high accuracy, specifically an IoU of 93.4% for the aircraft class, and integrated it into existing perception modules.
Improved model processing speed by 66.6%, enabling real-time operation at 30 fps using the MinkowskiEngine library for fast sparse convolution, and integrated this optimized model with the robotics stack.
Developed an image processing algorithm for lane detection, transforming front-center camera images into a bird’s eye view using geometric transformation and homography, to enhance navigation accuracy.