Differentiable Contact Dynamics for Legged Locomotion: Policy Learning and Contact-Implicit Trajectory Optimization

Gijeong Kim, Seungbin Oh, Dongyun Kang, and Hae-Won Park

KAIST

Paper (arXiv)

Trained with 100,000 samples in under 2 min on a desktop PC

Trotting and pacing policies trained with the proposed gradient using 100,000 samples in under 2 min on a desktop PC, deployed on hardware.

Abstract

This paper proposes a smooth analytic gradient for frictional contact and applies it to policy learning and contact-implicit trajectory optimization for legged locomotion. The exact contact sensitivity is available in closed form but describes changes within the current contact mode, providing no first-order information toward neighboring contact modes. The proposed gradient replaces this sensitivity only in the backward pass, leaving the hard-contact forward solve unchanged and the realized rollout feasible. A central-path relaxation of the de Saxcé-corrected second-order-cone complementarity problem for Coulomb friction smooths the conditions defining the separating, clamping, and sliding modes. The relaxed sensitivity is evaluated at a point on the central path obtained by a Newton iteration initialized from the hard-contact solution. The sensitivity remains continuous through contact-mode transitions and converges to the exact sensitivity as the relaxation vanishes. Using 100,000 samples and less than 2 min of training on a desktop PC, policy learning with the proposed gradient produces a policy that transfers to hardware without fine-tuning. On hardware, the robot under a trotting policy withstands pushes and traverses debris and slippery ground. In trajectory optimization, the proposed gradient reaches contact-rich solutions missed by exact and Signorini-relaxed gradients.

Method

Smooth gradients for frictional contact

Analytic contact sensitivity via relaxed de Saxcé‑corrected conic complementarity

Under hard contact, the exact sensitivity holds the contact mode fixed and gives first-order methods no direction toward a neighboring mode. The proposed sensitivity relaxes the complete de Saxcé-corrected frictional contact law, λ̃ ∘ σ̃ = 0, along its central path, λ̃ ∘ σ̃ = ρe, in the backward pass only. It provides directions toward separation, sliding, and clamping in every contact mode, so policy learning and trajectory optimization can change contact modes through gradient-based updates.

Evaluating sensitivity at a relaxed frictional contact solution

The exact sensitivity is evaluated at the hard-contact solution, which lies on a cone boundary set by the contact mode, so it holds that mode fixed; for example, dσ = 0 for a clamped contact and dλ = 0 for a separating one. The proposed sensitivity is evaluated instead at a relaxed pair inside both cones, found by a Newton iteration from the hard-contact solution.

Relaxed contact solutions along the central path in the clamping, separating, and sliding regimes
Fig. 2 in the paper. Central-path relaxation of the exact contact solutions in the clamping, separating, and sliding regimes for a two-dimensional contact with μ = 0.8. The top row shows the contact impulse λ ∈ Kμ, and the bottom row shows the de Saxcé-corrected contact velocity σ + Γμ(σ) ∈ Kμ*. At ρ = 0, the clamping regime has σh = 0, the separating regime has λh = 0, and, in the sliding regime, both λh and σh + Γμ(σh) lie on their respective cone boundaries. For ρ > 0, the corresponding relaxed variables lie in int Kμ and int Kμ*. Black markers denote the exact solution. Blue and orange markers denote the relaxed impulse λ* and corrected contact velocity σ* + Γμ(σ*), respectively, for ρ ∈ (0, 0.5]. Darker markers correspond to larger ρ, and dotted curves trace the relaxed-solution loci as ρ increases.

Sensitivity across contact-mode transitions

In this single-contact example, increasing the free normal velocity bN moves the contact from clamping through sliding to separating. The exact sensitivity jumps at each mode boundary and contains zero components from differentiating with the mode held fixed, whereas the proposed sensitivity is continuous across these transitions.

Sensitivities of a single contact versus free normal velocity under the exact, randomized-smoothing, and proposed gradients
Fig. 3 in the paper. Sensitivities of a single contact under the exact, randomized-smoothing, and proposed gradients: the impulse and contact-velocity components with respect to the free normal velocity bN. The point-mass model has diagonal A and nonzero free tangential velocity bT. As bN increases, the hard-contact solution changes mode from clamping to sliding and then to separating. The exact sensitivity (black, ρ = 0) jumps at the mode boundaries and contains zero components resulting from differentiation of the contact constraints with the mode held fixed. Randomized smoothing with perturbation standard deviation 0.6 yields zeroth- and first-order estimates using N = 128 perturbed solves per point. The proposed sensitivity (ρ = 0.3) is evaluated deterministically at a single Newton-solved central-path point for each bN and remains continuous across these mode transitions.

Overview of the proposed differentiable contact framework

The forward pass determines the realized transition with the hard-contact solve, and the relaxation is applied only to the backward contact sensitivity. The proposed sensitivity replaces the exact one in the transition Jacobians, fx = ∂f/∂xt + (∂f/∂λ)(∂λ/∂xt) and likewise fu, which enter the BPTT recursion for policy learning and the DDP backward pass for trajectory optimization.

Framework overview: hard-contact forward solve, relaxed backward pass along the central path, and the resulting Jacobians used for policy learning and trajectory optimization
Fig. 5 in the paper. Overview of the proposed differentiable contact framework. Given (xt, ut), the hard-contact forward solve determines the contact impulse λ, contact velocity σ, and next state xt+1 under the realized sliding, clamping, or separating mode. For the backward pass, the Signorini condition, Coulomb friction cone, and maximum-dissipation principle are expressed as a single conic complementarity condition using the de Saxcé correction. The hard-contact solution at ρ = 0 is relaxed along the central path to ρ > 0, and analytic differentiation yields the relaxed contact sensitivity and transition Jacobians (fx, fu). These Jacobians are used for BPTT policy learning and DDP-based trajectory optimization.
Policy learning

Hardware demonstrations

Each policy is trained by BPTT with the proposed gradient in a single simulation environment and deployed on the 45-kg quadruped without fine-tuning. On a desktop PC, the trotting and pacing policies are trained on 100,000 samples in 108 s, and the low-friction gait-free policy on 160,000 samples in 172 s.

Trot, pace, and gait-free locomotion Trot and pace track commanded velocities on a treadmill with their prescribed contact patterns; the low-friction gait-free policy walks without a prescribed gait.
Traversing debris The trotting policy traverses scattered debris while maintaining stable locomotion.
Push recovery The trotting policy withstands external pushes.
Low-friction terrain On a boric-acid-coated surface with μ ≈ 0.25, outside its training range, the trotting policy recovers from intermittent foot slips and keeps walking without falling.
Policy learning

Comparison with PPO

On the trot command-tracking task, BPTT with the proposed gradient learns the trot with one environment (about 105 samples, 1.8 min). PPO with 8 to 32 environments does not track the commanded velocities, PPO with 64 does not learn the trot, and trotting first emerges with 128 (about 107 samples, 17.7 min).

PPO

PPO with 8, 16, or 32 environments does not track commanded velocities, and PPO with 64 environments does not learn the trot. The onset of trotting occurs at 128 environments.

Proposed (BPTT)

A single environment is sufficient for BPTT with the proposed gradient to learn the trot.

Contact-implicit trajectory optimization

Base-pose reaching via CITO

From the nominal standing pose, CITO reaches a reference base pose without prescribed contact modes or foothold locations. The proposed gradient attains the lowest costs in translation, yaw, and pitch because it provides search directions toward separation, sliding, and clamping in every contact mode.

No Relaxation

All four feet remain clamped, and the base twists relative to the fixed feet.

Signorini-Relaxed

The base pivots about the clamped front-right foot, retaining leg twist.

Proposed

The feet are repositioned as the body turns, avoiding residual leg twist.

Contact-implicit trajectory optimization

Recovery from unmodeled external pulls

CI-MPC at 40 Hz holds a standing pose while 1-s pulls, absent from its prediction model, act on the base. Averaged over eight directions, the proposed gradient withstands pulls up to 272 N, against 81 N (No Relaxation) and 107 N (Signorini‑Relaxed), because the planned feet leave clamping and slide or step with the base: their impulses oppose the pull, and the new footholds extend support.

No Relaxation

The front feet remain clamped, and the base tips forward.

Signorini-Relaxed

The front feet remain clamped, and the base tips forward.

Proposed

The front feet slide forward with the pull and keep the robot balanced.

BibTeX

@misc{kim2026differentiablecontactdynamicslegged,
      title={Differentiable Contact Dynamics for Legged Locomotion: Policy Learning and Contact-Implicit Trajectory Optimization},
      author={Gijeong Kim and Seungbin Oh and Dongyun Kang and Hae-Won Park},
      year={2026},
      eprint={2610.35852},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2610.35852},
}