Avik De
Adjunct Associate Professor, Electrical and Systems Engineering, University of Pennsylvania
Chief Science Officer, Ghost Robotics
Nov 2026

I like to talk about intelligence having two parts, an athletic part and a cognitive part.
— Marc Raibert
SoTA performance (EKA-Robotics, 2026) (Sleiman et al., 2026)

![ZEST uses residual joint-space corrections to reference positions with a fixed impedance controller [@sleiman2026zest]](zest.webp)
RL facilitates flexible, robust, embodied intelligence, but is weak at generalization

![pi0.7 uses a scratchpad for subtask reasoning [@intelligence2026pi]](pi0.7.webp)

Observed:
Missing: RL’s robustness!

![Composed examples needed for combined commands [@lake2018generalization]](lake.webp)
RNNs need training examples in context
![Test-time extrapolation of diffusion policies, ACT [@he2026demystifying]](diffusion.webp)
Diffusion policies, ACT behave like lookup tables or interpolators, with poor OOD results
![Einstein's Experience -> Jump -> Action Diagram [@zahavyposition]](eja.webp)
LLMs cannot form novel hypotheses (“jumps”) when there is no error signal gradient for hillclimbing

![Modular memory architecture [@dorovatas2026position]](modular_mem.webp)
We cannot yet continuously acquire new skills without catastrophic forgetting (IWL) or blowing up inference compute (ICL)
!["Last layer" adaptation [@lupu2024magic]](magic-vfm.webp)
![The species relying on a strongly innate strategy will outcompete the species employing a mixed strategy [@zador2019critique]](zador.webp)
Animals are able to function well so soon after birth because they are born with highly structured brain connectivity

![Pattern-recognition scaling vs. compositional, modular causal models (and physics/motor primitives) for sample-efficient generalization [@lake2016building]](learn_and_think.webp)
![Modular, hierarchical organization of biological motor systems (spinal primitives, brainstem loops, cortical planning) [@merel2019hierarchical]](merel.webp)
Is it because Biology just doesn’t have powerful enough computers?

![Constructing Skill Trees [@konidaris2012robot]](cst.webp)
O(N_\text{skills}) vs. O(N_\text{sequences}) demonstrations with explicit composition
![Linear improvements in downstream "zero-shot" capability require exponentially more pretraining data on those concepts [@udandarao2024no]](udanrao.webp)

An intelligence architecture should be generative, adaptive, efficient, and embodied — able to understand and use the body’s dynamics, synthesize novel behaviors, and efficiently adapt to new tasks and environments.


![Strategies for grouping bytes into patches [@pagnoni2025byte]](llm_tokenization.webp)
Letters (fine) vs. words (coarse)
E.g. medical journal, English-Italian dictionary
![Loss (BPB) vs. bytes/token (compression rate) shows an optimum at each training budget [@limisiewicz2026compute]](error_vs_compression_rate.webp)

![Bouncing like a monopode [@blickhan1993similarity]](blickhan.webp)
![Neural decoding of synergistic behavior of spinal motoneurons [@capsi2025merging]](synergies.webp)
![Motor neuron activities exhibit a synergistic organization in complex tasks; large dimensionality reduction in motor neuron outputs [@tanzarella2021synergistic]](synergistic_coordination.webp)

![With only energy and the motor performance in the reward, synergies (via PCA on joint torques) emerge [@li2025motor]](synergies_rl.webp)
![Learning planning abstractions in sequential decision-making domains from language-annotated demonstrations [@liu2024learning]](training_primitives.webp)

![Fish with no active control [@beal2006passive]](fish.webp)
Cannot learn from imitation/pre-training!

![SayCan [@ahn2022can], similar to pi0.7 [@intelligence2026pi]](saycan.webp)
walk -> long jump not feasible; walk -> run -> long jump is
![Sony Ace elite-level table tennis robot [@durr2026outplaying].](sony_ace.webp)


“Buegler” robot palming, catching, juggling (Burridge et al., 1999)


“Control in 3 parts” (Raibert, 1986)

Specific to a particular type of design – would not work with a high-inertia leg.


![Virtual biped permutations [@de2018vertical]](minitaur_gaits.webp)

![Parametric robustness [@de2018vertical]](vh_robustness.webp)

![Hybrid averaging theory to find asymptotic behavior of composed cyclic systems [@de2018hybrid]](hybrid_averaging.webp)

![Gallopping cat image from [@raibert1986legged]](cat.webp)



![Subsumption architecture [@brooks1991intelligence]](brooks.webp)
![Template and anchors to describe the behavior of animals and robots [@full1999templates]](templates_anchors.webp)
![Hybrid Zero Dynamics formalism of dimension reduction with posture control [@westervelt2003hybrid].](hybrid_zero_dyn.webp)
Sense -> Plan -> Act, or a linear VLA architecture
![In low-dimensional sagittal-plane models, posture control results in necessary dimension reduction to a specific form of CoM dynamics
[@de2022mode]](mode_reactive.webp)

![Mode-reactive templates [@de2022mode]](monkey_bars.webp)
Analysis: Full model -> postural dimension reduction -> CoM dynamics
Synthesis: Robot control <- posture control <- CoM task control
“Automated” \Pi function in certain settings

![Hierarchical MPC for RoboBee [@de2022efficient]](robobee_mpc_model.webp)

![Traj opt -> RL tracking -- Deep Tracking control (DTC) traj opt -> RL tracking [@jenelten2024dtc]](dtc.webp)
![Intent model -> RL tracking -- LUCID [@gupta2026lucidlearningembodimentagnosticintent]](lucid.webp)

![Leaping behaviors built by composing sequences of hardcoded transitions [@brill2015tail]](rhex_jerb_leap.webp)



Hopping, leaping, running, etc.; utilizing passive dynamics
Online recombination sequentially, parallelly
Traj opt -> cheap online discrete symbol-space planning
Robust to modeling errors; swappable body params
We (still) need an architecture capable of synthesizing, as well as autonomously tasking and composing primitives


Semantic “common sense”, task decomposition, perception
![VioLA hierarchical connection of a "frozen encoder" with a latent (designed) interface [@albaba2026viola]. Zero-shot locomotion success 100%, versus 16.7% for GR00T N1.7.](viola.webp)
Open challenge: Aligning discrete/latent foundation model interfaces with formal composability when not trained end-to-end

![Sequential composition formalism provides a preimage condition for composing controllers [@burridge1999sequential]](funnels.webp)
![ContractionPPO adds a contraction loss during RL training for robustness [@zinage2026contractionppo]](contraction_ppo.webp)
Idea: Add composability criteria to training loss for guaranteed online recombination

| Optimization-based \min J(x, u)~~\text{s.t.} ~ g(x) = 0 | RL a = \pi_\theta(s) | |
|---|---|---|
| Online generalization | Strong | Weak |
| Computational requirement | High | Low |
![MPC-Net "compiles" an MPC controller [@reske2021imitation]](mpc_net.webp)

![Systolic arrays make DNN computation faster and more efficient [@kung1979systolic]](systolic.webp)
![Robotics codesign successes: Navion [@suleiman2019navion], Robomorphic ASIC [@neuman2021robomorphic]](robomorphic.webp)
Huge perf/eff gains (e.g. 2 mW VIO; 40× dynamics grad), but inflexible

![Plasticine [@prabhakar2017plasticine], SambaNova RDU compile whole compute graphs into spatial pipelines on-chip (up to 77× perf/W)](plasticine.webp)
The opportunity for robotics:


More information: avikde.github.io









