Pratik Kunapuli

Learning-based control approaches like reinforcement learning (RL) have recently produced a slew of impressive results for tasks like quadrotor trajectory tracking and drone racing. Naturally, it is common to demonstrate the advantages of these new controllers against established methods like analytical controllers. We observe, however, that reliably comparing the performance of such very different classes of controllers is more complicated than might appear at first sight. As a case study, we take up the problem of agile tracking of an end-effector for a quadrotor with a fixed arm. We develop a set of best practices for synthesizing the best-in-class RL and geometric controllers (GC) for benchmarking. In the process, we resolve widespread RL-favoring biases in prior studies that provide asymmetric access to: (1) the task definition, in the form of an objective function, (2) representative datasets, for parameter optimization, and (3) “feedforward” information, describing the desired future trajectory. The resulting contributions are the following: our improvements to the experimental protocol for comparing learned and classical controllers are critical, and each of the above asymmetries can yield misleading conclusions. Prior works have implied that RL outperforms GC, but we find the gaps between the two controller classes are much smaller than previously published when accounting for symmetric comparisons. Geometric control achieves lower steady-state error than RL, while RL has better transient performance, resultingin GC performing better in relatively slow or less agile tasks, but RL performing better when greater agility is required. Finally, we open-source implementations of geometric and RL controllers for these aerial vehicles, implementing best practices for future development.

Tracking controllers enable robotic systems to accurately follow planned reference trajectories. In particular, reinforcement learning (RL) has shown promise in the synthesis of controllers for systems with complex dynamics and modest online compute budgets. However, the poor sample efficiency of RL and the challenges of reward design make training slow and sometimes unstable, especially for high-dimensional systems. In this work, we leverage the inherent Lie group symmetries of robotic systems with a floating base to mitigate these challenges when learning tracking controllers. We model a general tracking problem as a Markov decision process (MDP) that captures the evolution of both the physical and reference states. Next, we prove that symmetry in the underlying dynamics and running costs leads to an MDP homomorphism, a mapping that allows a policy trained on a lower-dimensional "quotient" MDP to be lifted to an optimal tracking controller for the original system. We compare this symmetry-informed approach to an unstructured baseline, using Proximal Policy Optimization (PPO) to learn tracking controllers for three systems: the Particle (a forced point mass), the Astrobee (a fullyactuated space robot), and the Quadrotor (an underactuated system). Results show that a symmetry-aware approach both accelerates training and reduces tracking error after the same number of training steps.

We demonstrate the capabilities of an attention-based end-to-end approach for high-speed quadrotor obstacle avoidance in dense, cluttered environments, with comparison to various state-of-the-art architectures. Quadrotor unmanned aerial vehicles (UAVs) have tremendous maneuverability when flown fast; however, as flight speed increases, traditional vision-based navigation via independent mapping, planning, and control modules breaks down due to increased sensor noise, compounding errors, and increased processing latency. Thus, learning-based, end-to-end planning and control networks have shown to be effective for online control of these fast robots through cluttered environments. We train and compare convolutional, U-Net, and recurrent architectures against vision transformer models for depth-based end-to-end control, in a photorealistic, high-physics-fidelity simulator as well as in hardware, and observe that the attention-based models are more effective as quadrotor speeds increase, while recurrent models with many layers provide smoother commands at lower speeds. To the best of our knowledge, this is the first work to utilize vision transformers for end-to-end vision-based quadrotor control.

We developed and validated a gait phase estimator for real-time control of a robotic hip exoskeleton during multimodal locomotion. Gait phase describes the fraction of time passed since the previous gait event, such as heel strike, and is a promising framework for appropriately applying exoskeleton assistance during cyclic tasks. A conventional method utilizes a mechanical sensor to detect a gait event and uses the time since the last gait event to linearly interpolate the current gait phase. While this approach may work well for constant treadmill walking, it shows poor performance when translated to overground situations where the user may change walking speed and locomotion modes dynamically. To tackle these challenges, we utilized a convolutional neural network-based gait phase estimator that can adapt to different locomotion mode settings to modulate the exoskeleton assistance. Our resulting model accurately predicted the gait phase during multimodal locomotion without any additional information about the user’s locomotion mode, with a gait phase estimation RMSE of 5.04 ± 0.79%, significantly outperforming the literature standard (p <; 0.05). Our study highlights the promise of translating exoskeleton technology to more realistic settings where the user can naturally and seamlessly navigate through different terrain settings.

Lower limb exoskeletons provide assistance during the gait cycle using a state variable, one in particular is gait phase. This is crucial for the exoskeleton controller to provide the user accurate assistance. Conventional methods often utilize an event marker to estimate gait phase by computing the average stride time. However, this strategy has limitations in adapting to dynamic speeds. We developed a sensor fusion-based neural network model to estimate the gait phase in real-time that can adapt to dynamic speeds ranging from 0.6 to 1.1 m/s. Ten able-bodied subjects walked with an exoskeleton using our estimator and were provided with corresponding torque assistance. Our best performing model had RMSE below 29 ms and 4% for real-time estimation and torque generation, respectively, reducing the estimation error by 36.0% (p < 0.01) and torque error by 40.9% (p < 0.001) compared to conventional methods. Our results indicate that creating a general user-independent model and additionally training on user-specific data outperforms the user-specific model and user-independent model. Our study validates the feasibility of using a sensor fusion-based machine learning model to accurately estimate the user’s gait phase and improve the controllability of a lower limb exoskeleton.