[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"public-people-navigation":3,"public-output-v1-tech":13,"output-content-0-zh-tech":33},{"data":4,"meta":8},{"visible":5,"photos":6,"people":7},false,[],[],{"source":9,"releaseId":10,"releaseVersion":10,"contentRevision":11,"checksum":12},"static",null,0,"838a5b62d9a4bad8485720b81d5524140cb1e1b230aa9b32177c8299aef29090",{"data":14,"meta":31},{"output":15,"markdown":26,"resources":29},{"slug":16,"titleZh":17,"titleEn":18,"type":19,"status":20,"summaryZh":21,"summaryEn":22,"coverImage":23,"linkUrl":24,"sortOrder":25},"tech","TeCH：基于对比表征学习的时间距离建模，用于人形机器人全身控制","TeCH: Temporal Distance Modeling via Contrastive Representation Learning for Humanoid Whole-Body Control","Paper","published","人形机器人凭借其环境适应能力，成为理想的通用 AI 平台，但其高自由度使得可扩展的运动学习变得复杂。","Humanoid robots are ideal general AI platforms owing to their environmental adaptability, while their high degrees of freedom complicate scalable motor learning.","\u002Fimages\u002Foutputs\u002Ftech\u002Ftech-demo-single.png","\u002Foutputs\u002Ftech",10,{"zh":27,"en":28},"::article-video-grid\n---\nvideos:\n  - src: \"\u002Fvideos\u002Foutputs\u002Ftech\u002Ftech-under-disturbance.mp4\"\n    label: \"TeCH · 抗扰动实验\"\n  - src: \"\u002Fvideos\u002Foutputs\u002Ftech\u002Ftech-motion-tracking.mp4\"\n    label: \"TeCH · 运动跟踪实验\"\n---\n::\n\n## 摘要\n\n人形机器人凭借其环境适应能力，成为理想的通用 AI 平台，但其高自由度使得可扩展的运动学习变得复杂。传统的 on-policy 强化学习方法（如 PPO）依赖手工设计的奖励函数，存在训练成本高昂、状态覆盖有限、泛化能力差等问题。为解决这些缺陷，无监督表征学习已被广泛应用于人形机器人控制。现有方案主要包括 Forward-Backward（FB）表征学习与基于对比度量学习的 TemporaL Distance Representation（TLDR）。FB 受线性 MDP 假设的限制，在富接触复杂场景下表现不稳定且表征退化。本文提出一种新的无监督人形机器人控制方法 TeCH，基于 TLDR 的对比学习框架构建。该方法通过隐空间时间距离建模构造密集奖励，引导结构化探索，提供简洁稳定的训练流程与组织良好的隐空间。它能够实现高精度的零样本运动跟踪与目标到达，具备出色的样本效率，并可无缝部署到物理机器人上。大量仿真与真机实验验证了本方法在跟踪精度和泛化能力上的优越性，展现出其在实际人形机器人控制中的巨大潜力。\n\n## 引言\n\n人形机器人因与人类环境的天然兼容性，成为通用 AI 的理想物理平台。其高自由度支持灵活的全身 loco-manipulation，但也使可扩展的运动学习复杂化。使用手工设计奖励的传统方法仅在有限任务上表现良好，缺乏在复杂场景中的鲁棒性与泛化能力。为此，研究者转向学习通用的行为先验以获取可复用的底层运动技能，而非为单独任务进行优化。\n\n近年来，强化学习推动了人形机器人 sim-to-real 的运动技能迁移。以全身运动跟踪为核心，这些方法使机器人能够学习用于 locomotion 和 loco-manipulation 任务的动态、可复用技能。然而，主流方法大多采用 PPO 等 on-policy 优化器，并结合手工设计的跟踪奖励或任务特定奖励，存在两个关键缺陷。其一，它们需要海量高维运动数据和大规模并行环境，导致训练成本难以承受。例如，SONIC [23] 需要 128 块 GPU 连续训练三天。其二，受限的数据集覆盖无法涵盖高自由度人形机器人的完整控制空间。缺乏结构化且平滑的表征，现有方法限制了技能的泛化与复用，难以针对外部扰动高效生成鲁棒响应。\n\n为应对上述局限，基于表征学习的无监督强化学习方法开始应用于人形机器人控制。其中一个代表性分支采用基于 successor feature 的值分解，例如 Forward-Backward（FB）表征学习 [35]，通过学习可组合的隐表征实现无监督技能发现和行为复用。off-policy 无监督强化学习具有高样本效率、强复用性以及与大规模 replay buffer 的兼容性，在构建人形机器人行为基础模型方面极具前景。BFM-Zero [18] 将 FB 学习引入人形机器人控制，通过结构化探索获取平滑、可组合的运动基元，借助行为流形建模提供良好的技能插值、鲁棒性与泛化能力。然而，FB 在高维人形机器人系统中存在关键局限：它建立在隐嵌入空间的线性 Markov Decision Process（linear MDP）假设之上，在复杂接触动力学下失效，导致表征不稳定与退化。总体而言，现有的无监督技能学习方法不可避免地要在表征表达力、训练稳定性与任务可扩展性之间权衡。\n\n另一类无监督强化学习分支利用对比度量学习来学习状态表征，通过对时序相关性和行为距离建模，产生密集、稳定的优化信号。作为通过对比学习习得的代表性范式，TemporaL Distance Representation（TLDR）[1] 与 FB 方法不同：它放弃值函数分解，直接使用时间距离目标学习时间可达性与系统动力学。\n\n在此基础上，本文提出 TeCH（如图 1 所示），一种基于时间距离表征 [1] 构建的人形机器人无监督控制框架。该方法利用隐空间中的距离变化构造密集的过程奖励，赋予自主探索更清晰的方向性，并自然建立状态与目标之间的对应关系。得益于更直接的优化目标和简洁稳定的训练流程，TeCH 在全局目标控制任务中展现出更好的可扩展性和优化稳定性。我们还观察到，TeCH 能够构造结构良好的隐空间，以更高精度支持零样本运动跟踪和目标到达。它在训练阶段表现出卓越的数据效率，并在部署到物理人形机器人时展现出有前景的性能。\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-01-overview-tech.png\"\nalt: \"TeCH 隐流形与下游人形机器人控制任务概览\"\ncaption: \"图 1. TeCH 概览：机器人自探索状态经状态编码器嵌入到隐流形中，隐空间距离作为转移步数的代理度量；所学表征同时支持连续运动跟踪与不连续目标到达。\"\nwide: true\n---\n::\n\n作为两大主流无监督强化学习范式，FB 与 TLDR 框架都已被证明能够将习得的能力迁移到真实人形机器人。本文的主要贡献概括如下：\n• 方法创新：我们提出一种基于时间距离学习的新型人形机器人控制方法 TeCH，通过产生密集的隐奖励进行结构化探索，提升无监督强化学习的跟踪精度。\n• 对比基准：我们首次在人形机器人上开展无监督强化学习的系统性对比，为未来设计提供实用洞见。\n• 实验验证：大量仿真与真机实验验证了 TeCH 在实际部署中实现高精度全身跟踪与目标到达的能力。\n\n## 相关工作\n\n### 人形机器人全身控制\n\n近年来，基于学习的方法在人形机器人全身控制方面取得显著进展。尽管强化学习可以在仿真中有效生成复杂动态行为 [28, 21, 22, 34, 33]，sim-to-real 迁移仍是真机部署的关键瓶颈。研究者提出了包括域随机化和系统辨识在内的多种策略，但多数现有工作聚焦于行走、奔跑、起立等特定行为的单任务学习 [29, 30, 3, 32, 39, 15, 27]。自 2025 年起，多任务与通用人形机器人控制受到越来越多关注，旨在用单一策略处理多样任务 [12, 13, 41, 40, 38, 4, 11, 14, 5, 6, 8]。这些方法通常在仿真中训练跟踪策略，再通过基于 VAE 的隐学习或扩散模型 [12, 38, 40, 4, 41, 19] 蒸馏为统一的多任务策略。然而，它们本质上受限于运动数据集质量，且普遍采用 on-policy 强化学习。与此不同，本方法利用 off-policy 强化学习直接学习通用策略，得到更丰富、更结构化的技能空间，且不受运动数据集约束。\n\n### 无监督强化学习\n\n无监督强化学习使智能体能在没有任务特定奖励的情况下学习多样、可控的行为。基于互信息的技能发现方法（包括 DIAYN [7]、VIC [9] 和 VALOR [16]）通过最大化隐变量与状态轨迹之间的互信息来提升技能可区分性。但这些方法主要在简单的 locomotion 场景下验证，且通常采用 on-policy 训练，限制了它们在高维人形机器人系统中的可扩展性。\n\n常见的对比与表征学习方法（如 CIC [20] 和 Proto-RL [37]）通过 InfoNCE 与原型匹配学习结构化的状态与技能嵌入，获得更稳定的表征。与此同时，内在动机类方法（包括 RND [2]、ICM [26] 和 SMM [17]）通过奖励状态新颖性和覆盖率来促进探索。然而，这些范式缺乏显式的目标进度建模，对目标到达行为的监督较弱。\n\nForward-Backward（FB）表征学习 [36] 通过将值函数分解为前向状态嵌入和后向技能嵌入，提供结构化的行为建模，支持可组合的技能表征与零样本泛化。尽管有这些优势，FB 依赖严格的 successor feature 分解假设，并需要对偶映射的联合优化，且在高维、富接触的人形机器人控制场景中验证不足。\n\n作为另一种有效的对比学习范式，TemporaL Distance Representation（TLDR）[1] 放弃了 forward-backward 分解，通过时间距离感知编码器 ϕ_ψ 显式建模时间可达性，将时间相邻的状态嵌入到相近位置，并产生与过程对齐的密集奖励 r_tldr。这一设计带来更简单、更稳定的 off-policy 优化目标。在本工作中，我们将 TLDR 扩展到真实人形机器人，并对通用运动跟踪与目标到达任务进行系统评估。\n\n## TeCH 预训练\n\nTeCH 由两个核心组件构成：无监督预训练阶段和零样本推理流程（第 IV 节）。本章概述其无监督预训练流程，包括仿真环境中的训练过程以及面向真机迁移的设计细节。在第 III-A 节给出问题建模，并在第 III-B 与 III-C 节介绍基于时间距离表征框架的 TeCH 无监督强化学习框架。\n\n对 TeCH 而言，学习目标有两个方面：1) 一个能够捕捉行为轨迹时间结构的隐表征空间。在该空间中，具有时间邻近性和相似行为模式的状态或轨迹被映射到相邻位置，使策略能够基于轨迹级别的行为语义进行泛化（第 III-B 节）；2) 一个以隐任务表征为条件的策略。我们可将宏观机器人姿态映射到该结构化空间，实现对多样运动目标和行为模式的统一控制（第 III-C 节）。\n\n为实现策略的零样本 sim-to-real 迁移，我们遵循 [18] 在训练期间引入对抗目标，使人形机器人行为与人类数据分布对齐（第 III-D 节）；我们还加入辅助训练目标以强制施加 sim-to-real 迁移所需的关键约束，例如关节限位合规性和有界的动作变化率。TeCH 的完整预训练流程与伪代码在第 III-E 节进一步展开。\n\n### 问题建模\n\n我们将真实世界的人形机器人控制问题形式化为部分可观测 Markov Decision Process（POMDP），定义为五元组 (S, O, A, P, γ)，其中 S 表示完整状态空间，O 为观测空间，A 为动作空间，P(s_{t+1} | s_t, a_t) 为状态转移动力学，γ∈(0,1) 为折扣因子。\n\n对于一个具有 29 个自由度的人形机器人，动作 a∈A⊂R^29 由所有自由度的 PD 控制器目标值组成。特权状态信息 s∈R^463 包括根部高度、本体位姿、本体朝向、线速度与角速度。可观测量定义为 o_t = {q_t − q̄, q̇_t, ω^root_t \u002F 4, g_t} ∈ R^64，其中 q_t ∈ R^29 表示相对于标称位姿 q̄ 归一化的关节位置，q̇_t ∈ R^29 表示关节速度，ω^root_t ∈ R^3 表示根部角速度，g_t ∈ R^3 表示根部坐标系下的投影重力向量。我们进一步定义观测历史 o_{t,H} = {o_{t−H}, a_{t−H}, …, o_t} ∈ R^{93·H+64}，由本体感知状态和动作组成。除根部高度外，所有状态分量都相对于当前朝向和根部位置进行归一化。\n\n在预训练阶段，智能体可访问无标签运动数据集 M={τ}，其中每条轨迹同时包含观测序列和特权状态序列，即 τ = (o_1, s_1, …, o_{l(τ)}, s_{l(τ)})，l(τ) 表示轨迹 τ 的长度。\n\n### 时间距离表征学习\n\n形式上，给定状态转移 (s_t, s_{t+1})，我们通过时间滚动构造伪目标：\ng_t = roll(s_{t+1}),(1)\n\n该时间滚动操作将未来状态扩展到单步转移之外，超越瞬时相邻性，建模更长距离的时间依赖。因此，所学表征在对比学习范式下被优化以捕捉多步环境动力学，而非仅仅拟合瞬时状态转移。\n\n随后，我们通过共享编码器将所有观测状态和构造的伪目标映射到统一的隐空间：\nϕ_x = ϕ_ψ(s_t), ϕ_y = ϕ_ψ(s_{t+1}), ϕ_g = ϕ_ψ(g_t).(2)\n\n编码器使用双目标对比损失进行训练，以正则化隐空间几何：\nL_TE(ψ, λ) = −J_dist(ϕ_x, ϕ_g) + λ C_step(ϕ_x, ϕ_y),(3)\n\n其中核心项 J_dist 作为对比目标，强制在隐空间中时间相距较远的状态对之间进行显式特征分离，区分可达与不可达的时间状态关系；作为补充，步级平滑项 C_step 作为连续状态之间的正则约束，防止特征突变并保持隐演化的连续性。两者联合优化塑造了物理上一致的隐几何，使隐空间距离显式反映真实机器人轨迹的时间可达性与多步转移逻辑。\n\n### 基于时间距离的目标进度奖励\n\n对策略 π_θ 而言，关键目标是衡量朝向隐目标 z 的进度。令 z̄(·) 表示投影后的隐表征，奖励定义为：\nr_tldr(s_t, s_{t+1}, z) = ‖z − z̄(s_t)‖_2 − ‖z − z̄(s_{t+1})‖_2.(4)\n\n该形式具有清晰的直观含义：若下一状态在隐空间中更接近目标，则奖励为正；否则为负。因此，策略被持续驱动朝目标接近的方向优化，得到密集且具方向信息的学习信号。\n\n### TeCH 真机迁移\n\n将无监督强化学习部署到人形机器人面临真实观测有限、sim-to-real 动力学鸿沟、运动不稳定以及无约束探索导致的异常行为等问题。为解决这些问题并增强 sim-to-real 迁移，我们采用四个训练组件：使用非对称训练处理部分可观测性；利用域随机化降低仿真过拟合；引入辅助目标保证运动稳定性与安全性；使用以隐变量为条件的风格判别器，正则化策略生成类人行为。上述模块的详细实现均遵循 BFM-Zero [18]。\n\n### 预训练流程\n\n如图 2 所示，预训练流程由三个迭代模块组成：在线交互（online interaction）、经验回放（experience replay）和 multi-objective optimization。这些组件构成一个闭环训练回路，在并行仿真环境中持续运行。\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-02-pre-training-pipeline.png\"\nalt: \"TeCH 预训练流程\"\ncaption: \"图 2. 该预训练流程使用三个并行 critic 联合优化策略：一个判别性 critic 用于人形机器人运动正则化，一个辅助 critic 提供 sim-to-real 迁移奖励，以及一个表征 critic 正则化状态嵌入以对齐隐目标。\"\nwide: true\n---\n::\n\n在在线交互阶段，策略以隐变量 z 为条件执行动作，并收集形如 (s_t, o_{t−k:t}, a_t, s_{t+1}, z_t) 的转移，存入在线 replay buffer。重要的是，隐变量 z 并非来自单一来源，而是由三种互补信号混合构造，共同平衡探索多样性、目标可达性和行为先验：\n1) 随机采样的隐变量（例如来自高斯或超球面先验），扩大隐任务空间覆盖并鼓励行为多样性；\n2) 与在线样本相关联的隐表征，将学习对齐到当前策略可达的目标；\n3) 来自专家轨迹的隐表征，注入人类运动先验并在隐空间中提供稳定锚点。\n\n值得注意的是，TeCH 通过利用所学的时间距离度量实现更灵活的探索策略。除 [18] 采用的混合目标采样方案（包括随机目标、回放高价值目标和专家目标）外，TeCH 还估计当前状态与候选目标状态之间的时间可达性，并在整个训练过程中根据该可达性估计动态调整目标采样概率。\n\n具体而言，在训练早期阶段，TeCH 为时间相距较远且不常访问的目标分配更高的采样概率，从而鼓励更广泛的探索。随着策略逐步稳定，采样分布逐渐转向与更高期望回报相关联的区域，使学习聚焦于精炼与任务相关的技能，提升整体策略性能。\n\n在经验回放阶段，学习器同时从在线 replay buffer 中采样转移批次，并从无标签专家演示中采样轨迹片段，分别用于构造判别器学习信号和表征学习目标。\n\n随后训练进入 multi-objective optimization 阶段，参数按 判别器 → 编码器(ϕ_ψ) → 主 Critic 与辅助 Critic → Actor → 目标网络软更新 的顺序依次更新。该闭环过程重复直至收敛。\n\n从优化角度看，Actor 由三个互补学习信号驱动：由 r_tldr 诱导的表征目标 Q_REP、风格目标 Q_D，以及辅助目标 Q_R。将这些项组合得到最终 Actor 目标：\nL(π) = − E_{(o_{t,H}, s_t)∼D, a_t=π(o_{t,H}, z), z∼ν} [λ_REP Q_REP + λ_D Q_D + λ_R Q_R].(5)\n\nTeCH 的完整预训练过程见算法 ??。为简洁起见，与并行环境仿真和分布式训练相关的实现细节被省略。\n\n## TeCH 零样本推理\n\n在测试阶段，TeCH 可以 zero-shot 方式求解多种任务，无需任何额外的任务特定学习、规划或微调。我们首先将不同的下游任务嵌入到隐空间，并考虑两类代表性任务。\n• 目标到达（Goal Reaching）：给定不连续的目标状态，将其映射为多个策略条件 z，使机器人能够在不同目标姿态之间切换。\n• 轨迹跟踪（Trajectory Tracking）：在参考轨迹窗口内聚合未来状态，并将其映射为隐条件序列 {z_t}，使策略能够逐步跟踪目标运动。\n\nTeCH 不属于基于 successor feature 的无监督强化学习范式，因此不提供统一的基于值的推理接口。这导致它通常无法直接支持如 [18] 中那样的奖励优化任务。相比之下，TeCH 引入了时间距离感知表征，显式编码相对于当前状态的时间可达性关系。因此，在零样本跟踪场景下，TeCH 常常表现出更快、更紧致的轨迹跟踪行为。\n\n目标到达。对于目标到达任务，对应的隐向量形式化为：\nz_g = ϕ(s_g),(6)\n其中 ϕ(·) 表示 TeCH 的时间距离编码器。\n\n跟踪。对于运动轨迹跟踪，给定轨迹 τ = {s_1, …, s_n}，策略的隐变量序列 {z_t} 定义为：\nz_t = Σ_{t′=t}^{t+H} w_{t′} ϕ(s_{t′}),(7)\n其中 H 表示前瞻时间窗口，w_{t′} 为满足 w_{t′} ≥ 0 且 Σ_{t′=t}^{t+H} w_{t′} = 1 的折扣权重。\n\n## 实验\n\n### 实验设置\n\n本节在仿真和真机平台上全面评估 TeCH。我们基于 IsaacLab [25] 在 Unitree G1 人形机器人仿真环境 [43] 中训练两种方法。仿真频率设为 200 Hz，控制频率为 50 Hz。运动数据集采用 LAFAN1 [10]，已重定向到 Unitree G1 机器人，包含约 40 段各数分钟的运动片段。\n\n### 仿真结果\n\n#### 运动跟踪\n\n该评测衡量运动序列的模仿精度。我们在训练数据集（LAFAN1 [10]，40 段片段，总计约 2–3 小时）和测试数据集（100-Style [24]，3–4k 段片段，总计约 18–19 小时）上评估策略，所有运动均重定向到 G1 机器人。我们使用关节位置误差作为评估指标：\nE_mae(e, m) = (1\u002F|e|) Σ_{t=1}^{|e|} ‖q_t(e) − q_t(m)‖,(8)\n其中 m 表示目标运动轨迹。最终结果报告为所有运动片段的平均误差。\n\n不同于 SONIC 等依赖任务特定运动跟踪奖励和模仿学习的框架，BFM-Zero 和 TeCH 探索无监督强化学习进行人形机器人控制，学习统一的可复用技能隐空间，从而支持不连续的目标到达和分层规划。\n\n尽管如此，我们仍与最先进的通用运动跟踪控制器 SONIC [23] 进行对比。我们在相同环境和测试集上评估公开发布的模型，报告 E_mae 及其在运动片段上的标准差。为公平起见，BFM-Zero、TeCH 和 SONIC [23] 均不启用 early termination，即指标在完整轨迹上计算。由于 SONIC 在严重失败或跌倒后通常难以快速恢复，我们额外报告 SONIC (TER.)，其指标仅在终止前的片段上计算。\n\n**表 I. TeCH 与基线方法在训练集和测试集上的运动跟踪误差（E_mae）（↓）（均值 ± 标准差）。**\n\n| Method | LAFAN1 | 100-Style |\n| --- | ---: | ---: |\n| SONIC\u003Csup>b\u003C\u002Fsup> | 0.2916 ± 0.1887 | 0.1522 ± 0.1049 |\n| SONIC (TER.)\u003Csup>a\u003C\u002Fsup> | 0.1081 ± 0.0213 | 0.1355 ± 0.0351 |\n| BFM-Zero\u003Csup>b\u003C\u002Fsup> | 0.1510 ± 0.0255 | 0.1674 ± 0.0459 |\n| TeCH\u003Csup>b\u003C\u002Fsup> | 0.1318 ± 0.0329 | 0.1474 ± 0.0405 |\n\n\u003Csmall>\u003Csup>a\u003C\u002Fsup> SONIC (TER.) 仅在终止前的片段上评估。 \u003Csup>b\u003C\u002Fsup> SONIC、BFM-Zero 与 TeCH 均在完整轨迹上评估。\u003C\u002Fsmall>\n\n从跟踪结果可以看出，SONIC [23] 在跌倒恢复和地面过渡期间表现较差，这显著降低了其在测试集上的整体跟踪表现。在完全分布外条件下（例如外部扰动、跌倒或随机初始化），BFM-Zero 和 TeCH 通常能够更自然地恢复平衡并继续稳定跟踪，而 SONIC 往往无法回到稳定的运动状态。我们将这一优势主要归因于 off-policy 的无监督探索。与依赖大规模运动模仿监督的方法相比，像 TeCH 这样的无监督 RL 框架在训练期间覆盖了更广泛的非标准状态和动态转移，从而得到更平滑、更连续的技能空间。\n\n即便仅就跟踪精度而言，TeCH 也达到了与 SONIC (TER.) 相当的水平，如表 I 所示。值得注意的是，SONIC 在其原始评估中已经优于 GMT [4]、Any2Track [42] 和 BeyondMimic [19]。我们还要强调训练效率上的显著差异：SONIC [23] 使用 128 块 GPU、大规模运动数据集和海量并行环境进行训练；而本方法仅用单卡 GPU 和数小时的 LAFAN1 数据完成训练，将 GPU 小时和环境样本数降低了近两个数量级。\n\n为更细致地比较这两种无监督 RL 方法，我们在图 3 中给出定性的运动跟踪结果。TeCH 实现了高精度的运动跟踪，生成的轨迹在关节层面与参考运动紧密对齐，表明所学表征有效捕捉了局部姿态结构。此外，TeCH 始终优于 BFM-Zero，尤其是在缓解全局根部旋转漂移方面，如图 4 的定量结果所进一步支持，特别是在快速旋转场景下。这一性能差距源于 BFM-Zero 的局限：它通过间接的 forward-backward 分解学习表征，在 successor measure M 中引入近似误差，限制了其对全局朝向、长时序相位信息和累积根部漂移的建模能力。相比之下，TeCH 更直接地构造跟踪目标，在保持精确局部关节精度的同时获得更优的全局姿态一致性。\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-03-motion-tracking-simulation.png\"\nalt: \"BFM-Zero 与 TeCH 在仿真中的定性运动跟踪对比\"\ncaption: \"图 3. BFM-Zero 与 TeCH 在仿真中运动跟踪的定性对比：可见两种方法都能有效跟踪域内运动，且 TeCH 略胜一筹。\"\nwide: true\n---\n::\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-04-root-global-rotation-error.png\"\nalt: \"分布外快速 360 度旋转任务下的根部全局旋转误差对比\"\ncaption: \"图 4. 分布外快速 360 度旋转任务下 BFM-Zero 与 TeCH 的根部全局旋转误差对比。\"\nwide: true\n---\n::\n\n#### 目标到达\n\n我们从训练数据集（即 LAFAN1）中人工提取 21 个“稳定”姿态，定义为速度为零的状态。为衡量智能体与目标姿态之间的接近程度，我们将关节误差定义为：\nE_mae(e, g) = (1\u002F|e|) Σ_{t=1}^{|e|} ‖q_t(e) − q(g)‖,(9)\n其中 e 表示一个 episode，q(·) 表示关节配置（即 29 维向量）。最终结果报告为所有目标姿态的平均值。每个 episode 的固定 horizon 为 H = 500。\n\n图 5(a) 比较了 BFM-Zero 与 TeCH 在目标到达任务上的关节级平均绝对误差。结果显示两种方法精度相当，而 TeCH 在精度略有下降的情况下能更高效地到达目标。图 5(b) 展示了三个随机选取的目标姿态，体现其多样性与复杂度。图 5(c) 给出定性结果，机器人成功从躺下过渡到坐立再到站立，展现出鲁棒的恢复行为。\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-05-goal-reaching-comparison.png\"\nalt: \"目标到达的关节误差与定性示例\"\ncaption: \"图 5. 目标到达的关节误差与定性结果对比：(a) 21 个目标姿态上的平均关节 MAE；(b) 示例目标姿态；(c) TeCH 在目标到达任务上的定性结果。\"\nwide: true\n---\n::\n\n值得注意的是，即便在具有挑战性的恢复场景中，TeCH 也能产生连贯的轨迹到达目标。相比之下，如 SONIC [23] 等基于跟踪的方法在不连续目标过渡上表现挣扎，凸显出纯跟踪驱动目标在不连续目标到达上的局限。\n\n### TeCH 真机实验结果\n\n如图 6 与图 7 所示，TeCH 部署到真实硬件后在运动跟踪和目标到达上同样表现出色。当大型外力扰动推搡机器人时，TeCH 能像 BFM-Zero 一样在跌倒后自主站起，而基线 SONIC [23] 则无法从跌倒状态恢复。这是因为我们的统一隐目标空间在预训练中编码了全身恢复运动，使策略无需单独的跌倒恢复微调即可生成自扶正轨迹。\n\n然而，两种无监督 RL 方法在真机测试中表现出不同的行为特性。与 BFM-Zero 相比，TeCH 倾向于更直接地执行目标运动，响应更激进，有时会导致行为略微不平滑。这是因为我们的表征损失以最小化隐目标之间的转移步数为优化目标，将快速状态切换优先于运动平滑性；而 BFM-Zero 的运动模仿判别器损失则使策略偏向从 mocap 数据集中提取的更缓慢、渐进的运动。\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-06-real-robot-motion-tracking.png\"\nalt: \"TeCH 真机运动跟踪示例\"\ncaption: \"图 6. TeCH 在真机测试中对高动态运动实现鲁棒的运动跟踪。\"\nwide: true\n---\n::\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-07-real-world-goal-reaching.png\"\nalt: \"TeCH 真机目标达示例\"\ncaption: \"图 7. TeCH 以稳定的运动和流畅的行为过渡完成真机目标到达任务。\"\nwide: true\n---\n::\n\n### 随机采样运动\n\n我们进一步研究从隐动作空间 Z 中采样的无条件随机 rollout 的质量，如图 8 所示。我们观察到，完全随机的隐样本即可在没有任何任务引导或运动先验的情况下产生连贯、类人的人形机器人运动序列。人形 avatar 在多样平衡姿态之间平滑过渡，从深蹲到大开站位重心转移以及四肢伸展，展现出自然的人类般流畅性，避免了在正则化不足的隐表征中常见的关节痉挛或漂浮伪影。\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-08-random-latent-samples.png\"\nalt: \"随机隐样本产生自然的人形机器人运动序列\"\ncaption: \"图 8. 在没有任务监督或运动先验的情况下，随机隐样本产生自然、无伪影的人形机器人运动序列。\"\nwide: true\n---\n::\n\n### 隐空间插值\n\n如图 1 右侧绿色虚线轨迹所示，在两个目标隐嵌入之间进行线性隐空间插值，可产生平滑、物理一致的人形机器人运动序列。沿插值路径的中间机器人姿态连续且渐进地演化，没有扭曲或破碎的异常运动。这验证了我们学到的隐目标空间具有有效的连续结构：嵌入空间中直接的线性插值即可映射到机器人平滑的顺序状态转移，同时支持运动跟踪和目标到达规划。\n\n## 消融实验\n\n### 隐空间维度\n\n无监督行为基础模型的性能在很大程度上取决于隐空间 Z⊆R^D 的表征能力：当 D 过小时，任务嵌入和 successor feature 可能无法区分不同技能；当 D 过大时，优化变得更困难、样本效率下降，并可能因冗余表征和估计方差增大而出现训练不稳定。\n\n为分析该超参数的影响，我们固定其他所有训练设置（网络架构、学习率、域随机化和判别器奖励权重），在 Unitree G1 仿真环境中评估 BFM-Zero 和 TeCH。我们扫描隐维度 D∈{32, 64, 128, 256, 384, 512, 1024}，并以 LAFAN1 数据集上的 Earth Mover's Distance（EMD）[31] 作为动作分布匹配的指标（越低越好，表示与参考运动分布更接近）。\n\n如图 9 所示，两种方法的 EMD 都随 D 增大而下降，表明动作分布拟合改善，但收益递减。性能在 D≥256 时基本饱和。在所有设置下，TeCH 始终取得低于 BFM-Zero 的 EMD，表明在相同容量下它能更准确地捕捉目标运动分布。当 D 足够大时，采用 successor feature 分解的 BFM-Zero 也能达到相当的性能。综合考虑性能、计算成本和训练稳定性之间的权衡，我们在所有主实验中采用 D=256 作为默认隐维度。\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-09-latent-dimension-ablation.png\"\nalt: \"不同隐维度下的 EMD 对比\"\ncaption: \"图 9. BFM-Zero 与 TeCH 在不同隐维度 D 下的 EMD 对比。\"\n---\n::\n\n### TeCH 编码器学习率\n\n对 TeCH 中的编码器学习而言，过大的学习率可能导致隐几何不稳定，而过小的学习率可能导致进度信号稀疏和策略更新滞后。为研究其对运动分布拟合的影响，我们固定 D=256 及其他所有超参数，扫描编码器学习率 lr∈{5×10^{−5}, 3×10^{−6}, 10^{−5}, 3×10^{−6}, 10^{−6}, 8×10^{−7}, 10^{−9}}。我们报告训练过程中的 EMD（越低越好）。\n\n如图 10 所示，较大的学习率（lr≈3×10^{−5}）使 EMD 在早期迅速下降，但很快在较高水平饱和且波动较大，表明时间距离结构没有得到充分精炼。较小的学习率（3×10^{−6} 和 10^{−6}）带来更平滑的收敛和更好的性能，在大约 100M 步后达到稳定区间，但仍属次优。最佳性能出现在 lr=8×10^{−7}，在标准训练预算内取得最低 EMD。进一步降至 10^{−9} 可缓慢改善性能，但显著拖慢收敛。总体而言，收敛速度与最终性能之间存在明显权衡，我们在所有实验中采用 lr=8×10^{−7} 作为默认设置。\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-10-encoder-learning-rate-ablation.png\"\nalt: \"TeCH 编码器学习率消融曲线\"\ncaption: \"图 10. TeCH 编码器学习率消融：不同 lr 值下训练步数对应的 EMD。\"\n---\n::\n\n### 在线隐目标更新频率\n\n除编码器学习率外，TeCH 还依赖于在线交互期间周期性刷新隐目标 z（对应 update-z-every 超参数）。该间隔决定一个被采样的隐变量在重新采样之前保留多少环境步。过短的间隔可能削弱长时序一致性，而过长的间隔可能降低技能多样性并使 replay 数据在隐空间过度集中。我们固定 D=256 和上述最优学习率，扫描 update-z-every∈{5, 10, 50, 100, 200}。\n\n如图 11 所示，所有曲线在早期迅速下降，并在约 40M 步后开始分化。每 5 或 10 步更新一次 z 带来最快的收敛和最平滑的曲线，在 110M 步时取得最低 EMD（约 0.70）。update-z-every=50 略差，最终 EMD 约 0.78；而 100 和 200 分别为 0.81 与 0.83–0.85。总体而言，更频繁的 z 重采样通过在并行环境中增加行为覆盖同时保持稳定的时间表征，改善了动作分布拟合。然而，当间隔从 50 增加到 100 和 200 时，性能增益饱和。\n\nupdate-z-every=5 与 =10 之间的差异在 80M 步后变得可忽略。综合所有消融结果，我们在主实验中采用 lr=8×10^{−7} 与 update-z-every=10 作为默认设置。\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-11-latent-goal-update-frequency.png\"\nalt: \"TeCH 在线隐目标更新频率消融曲线\"\ncaption: \"图 11. TeCH 在线隐目标更新频率消融：不同更新间隔下训练步数对应的 EMD。\"\n---\n::\n\n## 结论\n\n本文提出了一种基于对比学习 TLDR 框架构建的新型无监督人形机器人控制方法。与依赖严格线性 MDP 假设的现有 FB 表征不同，TeCH 直接在隐空间中建模时间可达性，产生稳定、密集的奖励信号并支持有效的 off-policy 训练。结合一系列实用的训练策略（包括非对称训练、域随机化、辅助目标与风格正则化），本方法很好地解决了部分可观测性、sim-to-real 鸿沟以及不稳定或异常运动等问题。大量仿真和物理机器人实验验证了本方法以高数据效率和强泛化能力实现高精度零样本跟踪与目标到达，展现出在真实人形机器人部署方面的巨大潜力。\n\n## 参考文献\n\n::paper-references\n\n1. Junik Bae, Kwanyoung Park, and Youngwoon Lee. Tldr: Unsupervised goal-conditioned rl via temporal distanceaware representations, 2024. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F 2407.08464.\n2. Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation, 2018. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F1810.12894.\n3. Zixuan Chen, Xialin He, Yen-Jen Wang, Qiayuan Liao, Yanjie Ze, Zhongyu Li, S. Shankar Sastry, Jiajun Wu, Koushil Sreenath, Saurabh Gupta, and Xue Bin Peng. Learning smooth humanoid locomotion through lipschitz-constrained policies.CoRR, abs\u002F2410.11825, 2024.\n4. Zixuan Chen, Mazeyu Ji, Xuxin Cheng, Xuanbin Peng, Xue Bin Peng, and Xiaolong Wang. GMT: general motion tracking for humanoid whole-body control.CoRR, abs\u002F2506.14770, 2025.\n5. Xuxin Cheng, Yandong Ji, Junming Chen, Ruihan Yang, Ge Yang, and Xiaolong Wang. Expressive wholebody control for humanoid robots.arXiv preprint arXiv:2402.16796, 2024.\n6. Pranay Dugar, Aayam Shrestha, Fangzhou Yu, Bart van Marum, and Alan Fern. Learning multi-modal wholebody control for real-world humanoid robots, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2408.07295.\n7. Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function, 2018. URL https:\u002F\u002Farxiv.org\u002F abs\u002F1802.06070.\n8. Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, and Chelsea Finn. Humanplus: Humanoid shadowing and imitation from humans, 2024. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F 2406.10454.\n9. Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational intrinsic control, 2016. URL https: \u002F\u002Farxiv.org\u002Fabs\u002F1611.07507.\n10. F ´elix G. Harvey, Mike Yurick, Derek Nowrouzezahrai, and Christopher J. Pal. Robust motion in-betweening. ACM Trans. Graph., 39(4):60, 2020.\n11. Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Learning humanto-humanoid real-time whole-body teleoperation.arXiv preprint arXiv:2403.04436, 2024.\n12. Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, and Yuke Zhu. HOVER: versatile neural whole-body controller for humanoid robots. CoRR, abs\u002F2410.21229, 2024.\n13. Tairan He, Jiawei Gao, Wenli Xiao, Yuanhang Zhang, Zi Wang, Jiashun Wang, Zhengyi Luo, Guanqi He, Nikhil Sobanbab, Chaoyi Pan, Zeji Yi, Guannan Qu, Kris Kitani, Jessica K. Hodgins, Linxi Fan, Yuke Zhu, Changliu Liu, and Guanya Shi. ASAP: aligning simulation and realworld physics for learning agile humanoid whole-body skills.CoRR, abs\u002F2502.01143, 2025.\n14. Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris M Kitani, Changliu Liu, and Guanya Shi. Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning. InConference on Robot Learning, pages 1516–1540. PMLR, 2025.\n15. Xialin He, Runpei Dong, Zixuan Chen, and Saurabh Gupta. Learning getting-up policies for real-world humanoid robots, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2502. 12152.\n16. Jongmin Kim, Youngwoon Lee, Pieter Abbeel, and Guanya Shi. Variational curriculum reinforcement learning for unsupervised discovery of skills, 2023. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.16342.\n17. Lisa Lee, Benjamin Eysenbach, Emilio Parisotto, Eric Xing, Sergey Levine, and Ruslan Salakhutdinov. Efficient exploration via state marginal matching, 2020. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F1906.05274.\n18. Yitang Li, Zhengyi Luo, Tonghe Zhang, Cunxi Dai, Anssi Kanervisto, Andrea Tirinzoni, Haoyang Weng, Kris Kitani, Mateusz Guzek, Ahmed Touati, Alessandro Lazaric, Matteo Pirotta, and Guanya Shi. Bfm-zero: A promptable behavioral foundation model for humanoid control using unsupervised reinforcement learning, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2511.04131.\n19. Qiayuan Liao, Takara E. Truong, Xiaoyu Huang, Guy Tevet, Koushil Sreenath, and C. Karen Liu. Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion, 2025. URL https:\u002F\u002Farxiv. org\u002Fabs\u002F2508.08241.\n20. Hao Liu and Pieter Abbeel. Cic: Contrastive intrinsic control for unsupervised skill discovery, 2022. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2202.00161.\n21. Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, and Weipeng Xu. Perpetual humanoid control for real-time simulated avatars. InICCV, pages 10861– 10870. IEEE, 2023.\n22. Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris M. Kitani, and Weipeng Xu. Universal humanoid motion representations for physicsbased control. InICLR. OpenReview.net, 2024.\n23. Zhengyi Luo, Ye Yuan, Tingwu Wang, Chenran Li, Sirui Chen, Fernando Casta ˜neda, Zi-Ang Cao, Jiefeng Li, David Minor, Qingwei Ben, Xingye Da, Runyu Ding, Cyrus Hogg, Lina Song, Edy Lim, Eugene Jeong, Tairan He, Haoru Xue, Wenli Xiao, Zi Wang, Simon Yuen, Jan Kautz, Yan Chang, Umar Iqbal, Linxi ”Jim” Fan, and Yuke Zhu. Sonic: Supersizing motion tracking for natural humanoid whole-body control, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2511.07820.\n24. Ian Mason. 100STYLE: A motion capture dataset of 100 locomotion styles, 2023. URL https:\u002F\u002Fwww.ianxmason. com\u002F100style\u002F.\n25. Mayank Mittal, Calvin Yu, Qinxi Yu, Jingzhou Liu, Nikita Rudin, David Hoeller, Jia Lin Yuan, Ritvik Singh, Yunrong Guo, Hammad Mazhar, Ajay Mandlekar, Buck Babich, Gavriel State, Marco Hutter, and Animesh Garg. Orbit: A unified simulation framework for interactive robot learning environments.IEEE Robotics Autom. Lett., 8(6):3740–3747, 2023.\n26. Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. Curiosity-driven exploration by selfsupervised prediction, 2017. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F 1705.05363.\n27. Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Sim-to-real transfer of robotic control with dynamics randomization.CoRR, abs\u002F1710.06537, 2017. URL http:\u002F\u002Farxiv.org\u002Fabs\u002F1710. 06537.\n28. Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. Deepmimic: example-guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph., 37(4):143, 2018.\n29. Ilija Radosavovic, Tete Xiao, Bike Zhang, Trevor Darrell, Jitendra Malik, and Koushil Sreenath. Real-world humanoid locomotion with reinforcement learning.Sci. Robotics, 9(89), 2024.\n30. Ilija Radosavovic, Bike Zhang, Baifeng Shi, Jathushan Rajasegaran, Sarthak Kamat, Trevor Darrell, Koushil Sreenath, and Jitendra Malik. Humanoid locomotion as next token prediction.CoRR, abs\u002F2402.19469, 2024.\n31. Yossi Rubner, Carlo Tomasi, and Leonidas J. Guibas. The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision, 40(2):99–121, 2000.\n32. Younggyo Seo, Carmelo Sferrazza, Haoran Geng, Michal Nauman, Zhao-Heng Yin, and Pieter Abbeel. Fasttd3: Simple, fast, and capable reinforcement learning for humanoid control.CoRR, abs\u002F2505.22642, 2025.\n33. Agon Serifi et al. Vmp: Versatile motion priors for robustly tracking motion on physical characters.Computer Graphics F orum, 43(8), 2024. doi: 10.1111\u002Fcgf.15175. URL https:\u002F\u002Fdoi.org\u002F10.1111\u002Fcgf.15175.\n34. Chen Tessler, Yunrong Guo, Ofir Nabati, Gal Chechik, and Xue Bin Peng. Maskedmimic: Unified physics-based character control through masked motion inpainting. ACM Trans. Graph., 43(6):209:1–209:21, 2024.\n35. Ahmed Touati and Yann Ollivier. Learning one representation to optimize all rewards. InNeurIPS, pages 13–23, 2021.\n36. Ahmed Touati and Yann Ollivier. Learning one representation to optimize all rewards, 2021. URL https: \u002F\u002Farxiv.org\u002Fabs\u002F2103.07945.\n37. Denis Yarats, Rob Fergus, Alessandro Lazaric, and Lerrel Pinto. Reinforcement learning with prototypical representations, 2021. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2102.11271.\n38. Kangning Yin, Weishuai Zeng, Ke Fan, Zirui Wang, Qiang Zhang, Zheng Tian, Jingbo Wang, Jiangmiao Pang, and Weinan Zhang. Unitracker: Learning universal whole-body motion tracker for humanoid robots.CoRR, abs\u002F2507.07356, 2025.\n39. Kevin Zakka, Baruch Tabanpour, Qiayuan Liao, Mustafa Haiderbhai, Samuel Holt, Jing Yuan Luo, Arthur Allshire, Erik Frey, Koushil Sreenath, Lueder A. Kahrs, Carmelo Sferrazza, Yuval Tassa, and Pieter Abbeel. Mujoco playground.CoRR, abs\u002F2502.08844, 2025.\n40. Weishuai Zeng, Shunlin Lu, Kangning Yin, Xiaojie Niu, Minyue Dai, Jingbo Wang, and Jiangmiao Pang. Behavior foundation model for humanoid robots, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.13780.\n41. Zhikai Zhang, Chao Chen, Han Xue, Jilong Wang, Sikai Liang, Yun Liu, Zongzhang Zhang, He Wang, and Li Yi. Unleashing humanoid reaching potential via real-worldready skill space.CoRR, abs\u002F2505.10918, 2025.\n42. Zhikai Zhang, Jun Guo, Chao Chen, Jilong Wang, Chenghuai Lin, Yunrui Lian, Han Xue, Zhenrong Wang, Maoqi Liu, Jiangran Lyu, Huaping Liu, He Wang, and Li Yi. Track any motions under any disturbances, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.13833.\n43. . G1, 2025. URL https:\u002F\u002Fwww.unitree.com\u002Fcn\u002Fg1\u002F.\n\n::\n\n## 相关资源\n\n::paper-download\n---\ntitle: \"TECH.pdf\"\nhref: \"\u002Fdocuments\u002Ftech\u002FTECH.pdf\"\nsize: \"6.0 MB\"\nformat: \"PDF\"\nbuttonLabel: \"下载文档\"\n---\n::","::article-video-grid\n---\nvideos:\n  - src: \"\u002Fvideos\u002Foutputs\u002Ftech\u002Ftech-under-disturbance.mp4\"\n    label: \"TeCH · Under Disturbance\"\n  - src: \"\u002Fvideos\u002Foutputs\u002Ftech\u002Ftech-motion-tracking.mp4\"\n    label: \"TeCH · Motion Tracking\"\n---\n::\n\n## Abstract\n\nAbstract—Humanoid robots are ideal general AI platforms owing to their environmental adaptability, while their high degrees of freedom complicate scalable motor learning. Traditional on-policy reinforcement learning methods such as PPO rely on handcrafted rewards, suffering from prohibitive training costs, limited state coverage, and poor generalization. To address these drawbacks, unsupervised representation learning has been widely adopted for humanoid control. Existing solutions mainly include Forward-Backward (FB) representation learning and contrastive metric learning-based TemporaL Distance Representation (TLDR). Limited by the linear MDP assumption, FB fails in complex contact-rich scenarios and yields unstable representations. This paper presents a novel unsupervised humanoid control method TeCH built upon the contrastive learning framework of TLDR. Our method constructs dense rewards via latent temporal distance modeling to guide structured exploration, delivering a concise, stable training pipeline and well-organized latent space. It enables high-precision zero-shot motion tracking and goal reaching with prominent data efficiency, and can be seamlessly deployed on physical robots. Extensive simulated and real-world experiments validate the superior tracking accuracy and generalization of our method, demonstrating its great potential for practical humanoid control.\n\n## Introduction\n\nHumanoid robots serve as ideal physical platforms for general AI due to their natural compatibility with human environments. Their high degrees of freedom support flexible whole-body loco-manipulation, yet complicate scalable motor learning. Traditional methods using handcrafted rewards perform well only on limited tasks and lack robustness and generalization in complex scenarios. To address this, researchers shift toward learning general behavioral priors to obtain reusable low-level motor skills, instead of optimizing for separate tasks. In recent years, reinforcement learning has advanced sim-to-real motor skill transfer for humanoid robots. Centered on whole-body motion tracking, these methods enable robots to learn dynamic, reusable skills for locomotion and loco-manipulation tasks. However, most prevailing approaches adopt on-policy optimizers such as PPO with manually designed tracking or task-specific rewards, yielding two key drawbacks. First, they demand massive high-dimensional motion data and large-scale parallel environments, leading to prohibitive training costs. For example, SONIC [23] requires 128 GPUs for three consecutive days of training. Second, constrained dataset coverage fails to span the full control space of high-degree-of-freedom humanoids. Lacking structured and smooth representations, existing methods limit skill generalization and reuse, and cannot efficiently generate robust responses against external disturbances. To tackle these limitations, representation learning-based unsupervised reinforcement learning methods have emerged for humanoid control. A representative branch adopts successor feature-based value decomposition, exemplified by Forward-Backward (FB) representation learning [35], which learns composable latent representations for unsupervised skill discovery and behavior reuse. With high sample efficiency, strong reusability, and compatibility with large-scale replay buffers, off-policy unsupervised reinforcement learning is promising for constructing humanoid behavioral foundation models. Introduced to humanoid control by BFM-Zero [18], FB learning acquires smooth, composable motion primitives via structured exploration, delivering favorable skill interpolation, robustness, and generalization by modeling behavioral manifolds. Nevertheless, FB suffers from critical limitations in high-dimensional humanoid systems. It builds on the linear Markov Decision Process (linear MDP) assumption in latent embedding space, which fails under complex contact dynamics, causing representation instability and degradation. Overall, existing unsupervised skill learning methods inevitably trade off representation expressiveness, training stability, and task scalability. Another branch of unsupervised reinforcement learning leverages contrastive metric learning to learn state representations. It models temporal correlations and behavioral distances to produce dense, stable optimization signals. As a representative paradigm learned via contrastive learning, TemporaL Distance Representation (TLDR) [1] differs from FB methods:\n\nit abandons value function decomposition and directly learns temporal reachability and system dynamics using temporal distance objectives. On this basis, this paper proposesTeCH(as shown in Figure 1), an unsupervised control framework for humanoid robots built upon the temporal distance representation [1]. This method constructs dense progress rewards using distance variations in the latent space, which endows autonomous exploration with clearer directionality and naturally establishes correspondences between states and goals. Benefiting from more straightforward optimization objectives and a concise, stable training pipeline,TeCHdelivers better scalability and optimization stability in global goal control tasks. We also observe thatTeCHcan construct well-structured latent spaces to support zero-shot motion tracking and goal reaching with enhanced precision. It achieves exceptional data efficiency during training and demonstrates promising performance when deployed on physical humanoid robots.\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-01-overview-tech.png\"\nalt: \"Overview of TeCH latent manifold and downstream humanoid-control tasks\"\ncaption: \"Fig. 1. Overview of TeCH: Self-explored robot states are embedded via a state encoder into a latent manifold, with latent distance serving as a proxy for transition step count; the learned representation enables both continuous motion tracking and discontinuous goal reaching.\"\nwide: true\n---\n::\n\nAs the two mainstream unsupervised reinforcement learning paradigms, both FB and TLDR frameworks have been proven capable of transferring learned capabilities to real-world humanoid robots. The main contributions of this paper are summarized as follows: •Methodological Innovation: We present a novel method TeCHfor humanoid control based on temporal distance learning. It produces dense latent rewards for structured exploration and improves tracking accuracy in unsupervised reinforcement learning. •Comparative Benchmark: We conduct the first systematic comparison of unsupervised RL on humanoid robots, which provides practical insights for future design. •Experimental Validation: Extensive simulated and real-robot experiments validate thatTeCHachieves high-precision whole-body tracking and goal reaching for practical deployment.\n\n## Related Works\n\n### Humanoid Whole-body Control\n\nIn recent years, learning-based methods have achieved remarkable progress in humanoid whole-body control. Though reinforcement learning can generate complex dynamic behaviors effectively in simulation [28, 21, 22, 34, 33], sim-to-real transfer remains a critical barrier to real-robot deployment. Various strategies including domain randomization and system identification have been proposed, yet most existing works focus on single-task learning for specific behaviors such as walking, running and standing up [29, 30, 3, 32, 39, 15, 27]. Since 2025, multi-task and general-purpose humanoid control have attracted growing attention, aiming to build a single policy for diverse tasks [12, 13, 41, 40, 38, 4, 11, 14, 5, 6, 8]. These methods typically train tracking policies in simulation and distill them into unified multi-task policies via CV AE-based latent learning or diffusion models [12, 38, 40, 4, 41, 19]. Nevertheless, they are inherently limited by motion dataset quality and generally adopt on-policy reinforcement learning. Distinct from such paradigms, our method leverages off-policy reinforcement learning to directly learn general policies. It yields a richer, more structured skill space and is free from constraints brought by motion datasets.\n\n### Unsupervised Reinforcement Learning\n\nUnsupervised reinforcement learning enables learning of diverse, controllable behaviors without task-specific rewards. Mutual-information-based skill discovery methods, including DIAYN [7], VIC [9], and V ALOR [16], improve skill distinguishability by maximizing mutual information between latent variables and state trajectories. However, these methods are primarily validated in simple locomotion settings and typically adopt on-policy training, limiting their scalability to high-dimensional humanoid systems. Common contrastive and representation learning approaches, such as CIC [20] and Proto-RL [37], learn structured state and skill embeddings via InfoNCE and prototype matching for more stable representations. Meanwhile, intrinsic motivation methods including RND [2], ICM [26], and SMM [17] promote exploration by rewarding state novelty and coverage. Nevertheless, these paradigms lack explicit goal-progress modeling and deliver weak supervision for goal-reaching behaviors. Forward–Backward (FB) representation learning [36] offers structured behavior modeling by decomposing value functions into forward state embeddings and backward skill embeddings, supporting composable skill representation and zero-shot generalization. Despite these merits, FB relies on strict successor-feature decomposition assumptions and requires joint optimization of dual mappings, with insufficient validation on high-dimensional, contact-rich humanoid control. As another effective contrastive learning paradigm, TemporaL Distance Representation (TLDR) [1] abandons forward-backward factorization. It explicitly models temporal reachability via a temporal-distance-aware encoderϕ ψ, which embeds temporally adjacent states closely and produces dense progress-aligned rewardsr tldr. This design yields a simpler, more stable off-policy optimization objective. In this work, we extend TLDR to real humanoid robots and conduct systematic evaluations on general motion tracking and goal-reaching tasks.\n\n## Pre-Training of TeCH\n\nTeCHcomprise two core components: an unsupervised pre-training phase and a zero-shot inference pipeline (Section IV). In this chapter, we outline their unsupervised pre-training workflow, covering the training procedure in simulation environments and the design specifics for transfer to real humanoid robots. We provide the problem formulation in Section III-A, and introduce the unsupervised reinforcement learning framework of theTeCHin Section III-B and Section III-C which is based on the temporal distance representation framework. ForTeCH, the learning objectives are twofold: 1) a latent representation space that captures the temporal structure of behavior trajectories. In this space, states or trajectories with temporal proximity and similar behavior patterns are mapped\n\nto adjacent positions, empowering the policy to generalize based on trajectory-level behavior semantics (Section III-B); 2) a policy conditioned on latent task representations. We can map macroscopic robot poses into this structured space to achieve unified control over diverse motion targets and behavior patterns (Section III-C). To achieve zero-shot sim-to-real transfer of the policy, we follow [18] introduce an adversarial objective during training to align humanoid robot behaviors with the human data distribution (Section III-D); we also add auxiliary training objectives to enforce critical constraints for sim-to-real transfer, such as joint limit compliance and bounded action change rates. The full pre-training pipeline and pseudocode ofTeCH are further elaborated in Section III-E.\n\n### Problem Formulation\n\nWe formulate the real-world humanoid robot control problem as a Partially Observable Markov Decision Process (POMDP), defined as a five-tuple(S, O, A, P, γ), whereS denotes the full state space,Othe observation space,Athe action space,P(s t+1 |s t, at)the state transition dynamics, andγ∈(0,1)the discount factor. For a humanoid robot with 29 degrees of freedom, the actiona∈A⊂R 29 consists of target values for a proportional-derivative (PD) controller over all degrees of freedom. The privileged state informations∈R 463 includes root height, body pose, body orientation, linear velocity, and angular velocity. The observable state is defined aso t = {qt −¯q,˙qt, ωroot t \u002F4, gt} ∈R 64,whereq t ∈R 29 denotes joint positions normalized with respect to the nominal pose ¯q,˙qt ∈R 29 denotes joint velocities,ω root t ∈R 3 denotes root angular velocity, andg t ∈R 3 denotes the projected gravity vector in the root frame. We further define the observation history aso t,H ={o t−H , at−H , . . . , ot} ∈R 93·H+64, which consists of proprioceptive states and actions. Except for root height, all state components are normalized with respect to the current heading direction and root position. During the pretraining stage, the agent has access to an unlabeled motion datasetM={τ}, where each trajectory contains both observation and privileged state sequences, i.e.,τ= (o 1, s1, . . . , ol(τ) , sl(τ)), wherel(τ)denotes the length of trajectoryτ.\n\n### Temporal Distance Representation Learning\n\nFormally, given a state transition(s t, st+1), we construct pseudo-goals via temporal rolling: gt = roll(st+1),(1) This temporal rolling operation extends future states beyond single-step transitions, moving beyond immediate adjacency and modeling longer-range temporal dependencies. As a result, the learned representation is optimized under a contrastive learning paradigm to capture multi-step environmental dynamics, rather than merely fitting instantaneous state transitions. We then map all observed states and constructed pseudo-goals into a unified latent space via a shared encoder: ϕx =ϕ ψ(st), ϕ y =ϕ ψ(st+1), ϕ g =ϕ ψ(gt).(2) The encoder is trained with a dual-objective contrastive loss to regularize latent space geometry: LTE(ψ, λ) =−J dist(ϕx, ϕg) +λC step(ϕx, ϕy),(3) where the core termJ dist serves as the contrastive objective that enforces explicit feature separation between temporally distant state pairs in the latent space, distinguishing reachable and unreachable temporal state relationships. Complementarily, the step-wise smoothness termC step acts as a regularization constraint between consecutive states, preventing abrupt feature shifts and maintaining continuous latent evolution. Jointly optimized, this dual contrastive objective shapes a physically consistent latent geometry, where latent-space distances explicitly reflect the temporal reachability and multi-step transition logic of real robot trajectories.\n\n### Goal Progress Reward with Temporal Distance\n\nFor the policyπ θ, the key objective is to measure progress toward a latent goalz. Let¯z(·)denote the projected latent representation. The reward is defined as: rtldr(st, st+1, z) =∥z−¯z(s t)∥2 − ∥z−¯z(st+1)∥2.(4) This formulation has a clear intuition: if the next state is closer to the goal in the latent space, the reward ispositive; otherwise, it is negative. Consequently, the policy is continuously driven to optimize toward the direction of goal proximity, resulting in a dense and directionally informative learning signal.\n\n### Transfer TeCH to Real Robots\n\nDeploying unsupervised reinforcement learning on humanoid robots faces issues such as limited real-world observations, sim-to-real dynamics gaps, unstable motions and unnatural behaviors from unregulated exploration. To resolve these problems and enhance sim-to-real transfer, we adopt four training components. We apply asymmetric training to handle partial observability, and leverage domain randomization to reduce simulation overfitting. Auxiliary objectives are introduced to ensure motion stability and safety, while a latent-conditioned style discriminator regularizes policies to generate human-like behaviors. The detailed implementation of all above modules follows BFM-Zero [18].\n\n### Pre-training Pipeline\n\nAs illustrated in Figure 2, the pretraining pipeline consists of three iterative modules:online interaction,experience replay, andmulti-objective optimization. These components form a closed training loop that continuously operates across parallel simulation environments.\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-02-pre-training-pipeline.png\"\nalt: \"TeCH pre-training pipeline\"\ncaption: \"Fig. 2. This pre-training pipeline employs three parallel critics to jointly optimize the policy: a discriminative critic for humanoid motion regularization, an auxiliary critic providing sim-to-real transfer reward, and a representation critic that regularizes state embeddings to align with latent goal.\"\nwide: true\n---\n::\n\nDuring online interaction, the policy executes actions conditioned on a latent variablezand collects transitions of the form(s t, ot−k:t, at, st+1, zt), which are stored in an online replay buffer. Importantly, the latent variablezis not drawn from a single source but is instead constructed from a mixture of three complementary signals, jointly balancing exploration diversity, goal reachability, and behavioral priors: 1) Randomly sampled latent variables (e.g., from Gaussian or hyperspherical priors), which expand the coverage of the latent task space and encourage behavioral diversity; 2) Latent representations associated with online samples, which align learning with goals that are currently reachable by the policy; 3) Latent representations derived from expert trajectories, which inject human-motion priors and provide stable anchors in the latent space. Notably,TeCHenables a more flexible exploration strategy by leveraging its learned temporal distance metric. In addition to the mixed-goal sampling scheme adopted by [18]—including random goals, replayed high-value goals, and expert goals—TeCHfurther estimates the temporal reachability between the current state and candidate goal states. This reachability estimate is then used to dynamically adjust goal-sampling probabilities throughout training. Specifically, during the early stages of training,TeCH assigns higher sampling probabilities to temporally distant and infrequently visited goals, thereby encouraging broader exploration. As the policy gradually stabilizes, the sampling distribution shifts toward regions associated with higher expected returns, allowing learning to focus on refining task-relevant skills and improving overall policy performance. During theexperience replaystage, the learner simultaneously samples transition batches from the online replay buffer and trajectory segments from unlabeled expert demonstrations. These samples are used to construct the discriminator learning signal and the representation learning objective, respectively. The training then proceeds to themulti-objective optimizationstage, where parameters are updated sequentially according to Discriminator→Encoder(ϕ ψ)→Main Critic and Auxiliary Critic→Actor→Target Network Soft Update. This closed-loop procedure is repeated until convergence. From an optimization perspective, the Actor is driven by three complementary learning signals: the representation objectiveQ REP (induced byr tldr), the style objectiveQ D, and the auxiliary objectiveQ R. Combining these terms yields the final Actor objective: L(π) =−E (ot,H ,st)∼D at=π(ot,H ,z), z∼ν h λREP QREP +λDQD+λRQR i . (5) The complete pretraining procedure ofTeCHis summarized in Algorithm??. For clarity, implementation details related to parallel environment simulation and distributed training are omitted.\n\n## Zero-Shot Inference in TeCH\n\nDuring the testing stage,TeCHcan solve a variety of tasks in azero-shotmanner, without any additional task-specific learning, planning, or fine-tuning. We first embed different downstream tasks into the latent space, and consider two representative types of tasks. •Goal Reaching: Given discontinuous goal states, we map them into multiple policy conditionsz, enabling the robot to switch between different target poses. •Trajectory Tracking: We aggregate future states within a reference trajectory window and map them into a sequence of latent conditions{z t}, allowing the policy to progressively track the target motion. TeCHdoes not belong to the successor-feature-based unsupervised reinforcement learning paradigm and therefore does not provide a unified value-based inference interface. As a result, it typically cannot directly support reward optimization tasks such as those in [18]. In contrast,TeCHintroduces temporal-distance-aware representations that explicitly encode the temporal reachability relationship with respect to the current state. Consequently, in zero-shot tracking scenarios, TeCHoften exhibits faster and tighter trajectory tracking behavior. Goal Reaching.For the goal reaching task, the corresponding latent vector is formulated as: zg =ϕ(s g),(6) whereϕ(·)denotes the temporal distance encoder ofTeCH. Tracking.For motion trajectory tracking, given a trajectory τ={s 1, . . . , sn}, the sequence of latent variables{z t}for the policy is defined as: zt = t+HX t′=t wt′ ϕ(st′),(7) whereHrepresents the look-ahead time window, andw t′ are discount weights satisfyingw t′ ≥0and Pt+H t′=t wt′ = 1.\n\n## Experiments\n\n### Experimental Setup\n\nIn this section, we evaluateTeCHcomprehensively in both simulation and real-world robotic platforms. We train both methods in the Unitree G1 humanoid simulation environment [43] based on IsaacLab [25]. The simulation frequency is set to 200 Hz, while the control frequency is 50 Hz. For the motion dataset, we use the LAFAN1 dataset [10], which has been retargeted to the Unitree G1 robot and contains approximately 40 motion clips of several minutes each.\n\n### Simulation Results\n\n#### Motion Tracking\n\nThis evaluation measures the imitation\naccuracy of motion sequences. We evaluate the policies on\nboth the training dataset (LAFAN1 [10], 40 clips, approximately 2–3 hours in total) and the test dataset (100-Style [24],\n3–4k clips, approximately 18–19 hours in total), where all\nmotions are retargeted to the G1 robot. We use joint position\nerror as the evaluation metric:\nEmae(e, m) = 1\n|e|\n|e|X\nt=1\n∥qt(e)−q t(m)∥,(8)\nwheremdenotes the target motion trajectory. The final results\nare reported as the average error over all motion clips.\nUnlike prior frameworks such as Sonic that rely on task-specific motion tracking rewards and imitation learning, BFM-Zero andTeCHexplore unsupervised reinforcement learning for humanoid control, learning a unified latent space of\nreusable skills that enables discontinuous goal reaching and\nhierarchical planning.\nNevertheless, we compare against the state-of-the-art\ngeneral-purpose motion tracking controller SONIC [23]. We\nevaluate the publicly released model under the same environment and test sets, reportingE mae and the standard\ndeviation across motion clips. For fairness, BFM-Zero,TeCH,\nand SONIC [23] are evaluated without early termination, i.e.,\nmetrics are computed over full trajectories. Since SONIC\ntypically struggles to recover quickly after severe failures or\nfalls, we additionally report SONIC (TER.), where metrics are\ncomputed only over segments before termination.\n**TABLE I. Motion Tracking Error (Emae) (↓) on training and test datasets for TeCH and baselines (mean ± std).**\n\n| Method | LAFAN1 | 100-Style |\n| --- | ---: | ---: |\n| SONIC\u003Csup>b\u003C\u002Fsup> | 0.2916 ± 0.1887 | 0.1522 ± 0.1049 |\n| SONIC (TER.)\u003Csup>a\u003C\u002Fsup> | 0.1081 ± 0.0213 | 0.1355 ± 0.0351 |\n| BFM-Zero\u003Csup>b\u003C\u002Fsup> | 0.1510 ± 0.0255 | 0.1674 ± 0.0459 |\n| TeCH\u003Csup>b\u003C\u002Fsup> | 0.1318 ± 0.0329 | 0.1474 ± 0.0405 |\n\n\u003Csmall>\u003Csup>a\u003C\u002Fsup> SONIC (TER.) is evaluated only on segments before termination. \u003Csup>b\u003C\u002Fsup> SONIC, BFM-Zero, and TeCH are evaluated on full trajectories.\u003C\u002Fsmall>\n\nFrom the tracking results, we observe that SONIC [23] performs poorly in recovery after falls and during ground transitions, which significantly degrades its overall tracking performance on the test set. Under fully out-of-distribution conditions (e.g., external perturbations, falls, or random initializations), BFM-Zero andTeCHare generally able to recover balance more naturally and continue stable tracking, whereas SONIC often fails to return to a stable motion regime. We attribute this advantage primarily to off-policy unsupervised exploration. Compared to methods relying on large-scale motion imitation supervision, unsupervised RL frameworks like TeCHcover a much broader range of non-standard states and dynamic transitions during training, leading to a smoother and more continuous skill space. Even in terms of tracking accuracy alone,TeCHachieve performance comparable to SONIC (TER.), as shown in Table I. Notably, SONIC already outperforms GMT [4], Any2Track [42], and BeyondMimic [19] in its original evaluation. We also emphasize the significant difference in training efficiency: SONIC [23] uses 128 GPUs, large-scale motion datasets, and massive parallel environments for training. In contrast, our method is trained with a single GPU and only several hours of LAFAN1 data, reducing GPU-hours and environment samples by nearly two orders of magnitude. For a more granular comparison of the two unsupervised RL methods, we present qualitative motion tracking results in Figure 3.TeCHachieves highly accurate motion tracking, with generated trajectories closely aligned with reference motions at the joint level, indicating that the learned representation effectively captures local pose structure. Moreover,TeCH consistently outperforms BFM-Zero, particularly in mitigating global root rotation drift, as further supported by the quantitative results in Figure 4, especially in fast-rotation scenarios. This performance gap stems from limitations of BFM-Zero, which learns representations via indirect forward-backward factorization, introducing approximation errors in the successor measureMand constraining its ability to model global orientation, long-horizon phase information, and cumulative root drift. In contrast,TeCHformulates tracking objectives more directly, resulting in improved global pose consistency while preserving precise local joint accuracy.\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-03-motion-tracking-simulation.png\"\nalt: \"Qualitative simulation motion tracking comparison between BFM-Zero and TeCH\"\ncaption: \"Fig. 3. Qualitative comparison of motion tracking between BFM-Zero and TeCH in simulation: We can see both methods can effectively track in domain motions, and TeCH is slightly better.\"\nwide: true\n---\n::\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-04-root-global-rotation-error.png\"\nalt: \"Root global rotation error comparison under out-of-distribution fast 360-degree rotation\"\ncaption: \"Fig. 4. Comparison of Root Global Rotation Error between BFM-Zero and TeCH under Out-of-Distribution Fast 360-Degree Rotation Task.\"\nwide: true\n---\n::\n\n#### Goal Reaching\n\nWe manually extract21“stable” poses\nfrom the training dataset (i.e., LAFAN1), defined as states with\nzero velocity. To evaluate the proximity between the agent and\nthe target pose, we define the joint error as:\nEmae(e, g) = 1\n|e|\n|e|X\nt=1\n∥qt(e)−q(g)∥,(9)\nwhereedenotes an episode andq(·)represents the joint\nconfiguration (i.e., a 29-dimensional vector). The final results\nare reported as the average over all target poses. Each episode\nhas a fixed horizon ofH= 500.\nFigure 5(a) compares the joint-level mean absolute error\nbetween BFM-Zero andTeCHon the goal-reaching task. The\nresults show that both methods achieve comparable accuracy,\nwhileTeCHreaches the target more efficiently with a slight\ndegradation in precision. Figure 5(b) shows three randomly selected target poses, illustrating their diversity and complexity. Figure 5(c) presents qualitative results, where the robot\nsuccessfully transitions from lying down to sitting and then to\nstanding, demonstrating robust recovery behaviors.\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-05-goal-reaching-comparison.png\"\nalt: \"Goal reaching joint error and qualitative examples\"\ncaption: \"Fig. 5. Comparison of joint error and qualitative results for goal reaching: (a) average joint MAE over 21 target poses; (b) example target poses; (c) qualitative results of TeCH on the goal reaching task.\"\nwide: true\n---\n::\n\nNotably,TeCHreaches target goals produces coherent trajectories even in challenging recovery scenarios. In contrast, tracking-based methods such as SONIC [23] struggle with discontinuous goal transitions, highlighting the limitation of purely tracking-driven objectives for discontinuous goal-reaching.\n\n### Real-World Results of TeCH\n\nAs shown in Figure 6 and Figure 7,TeCHalso demonstrates strong performance in both motion tracking and goal reaching when deployed on real hardware. When large external pushes perturb the robot,TeCHcan autonomously stand back up after a fall just like BFM-Zero, whereas the baseline SONIC [23] cannot recover from falling states. This is due to our unified latent goal space encoding full-body recovery motions during pre-training, allowing the policy to generate self-righting trajectories without separate fall-recovery fine-tuning. However, the two URL methods exhibit distinct behavioral characteristics in real-world tests. Compared to BFM-Zero, TeCHtends to execute target motions more directly and responds more aggressively, sometimes resulting in slightly less smooth behaviors. This is due to our representation loss optimizing for minimal transition steps between latent goals, which prioritizes fast state transfer over motion smoothness; in contrast, BFM-Zero’s motion-imitation discriminator loss biases the policy toward slower, gradual movements extracted from mocap datasets.\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-06-real-robot-motion-tracking.png\"\nalt: \"TeCH real robot motion tracking examples\"\ncaption: \"Fig. 6. TeCH performs robust motion tracking on high-dynamic motions in real robot tests.\"\nwide: true\n---\n::\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-07-real-world-goal-reaching.png\"\nalt: \"TeCH real-world goal reaching examples\"\ncaption: \"Fig. 7. TeCH completes real-world goal reaching tasks with stable movements and smooth behavioral transitions.\"\nwide: true\n---\n::\n\n### Random Rollout\n\nWe further investigate the quality of unconditioned random rollouts sampled from our latent action spaceZ, as visualized in Figure 8. We observe that fully random latent samples yield coherent, human-like humanoid motion sequences without any task guidance or motion priors. The humanoid avatar smoothly transitions across diverse balanced poses, ranging from deep squatting to wide-stance weight shifting and outstretched limb movements, exhibiting natural human-like fluidity and avoiding the jerky joint spasms or floating artifacts prevalent in poorly regularized latent representations.\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-08-random-latent-samples.png\"\nalt: \"Random latent samples produce natural humanoid motion sequences\"\ncaption: \"Fig. 8. Random latent samples produce natural, artifact-free humanoid motion sequences without task supervision or motion prior.\"\nwide: true\n---\n::\n\n### Latent Interpolation\n\nAs illustrated by the green dotted trajectory on the right side of Figure 1, linear latent-space interpolation between two target latent embeddings produces smooth, physically consistent humanoid motion sequences. The intermediate robot poses along the interpolation path evolve continuously and gradually without distorted or fragmented unnatural motions. This verifies that our learned latent goal space owns valid continuous structure: straightforward linear interpolation in the embedding space directly maps to smooth sequential state transitions of the robot, supporting both motion tracking and goal-reaching planning.\n\n## Ablation Studies\n\n### Latent Space Dimensionality\n\nThe performance of unsupervised behavior foundation models largely depends on the representational capacity of the latent spaceZ⊆R D: whenDis too small, task embeddings and successor features may fail to distinguish different skills; whenDis too large, optimization becomes more difficult and sample-inefficient, and training instability may arise due to redundant representations and increased estimation variance. To analyze the effect of this hyperparameter, we keep all other training settings fixed (network architecture, learning rate, domain randomization, and discriminator reward weights) and evaluate both BFM-Zero andTeCHin the Unitree G1 simulation environment. We sweep the latent dimension D∈ {32,64,128,256,384,512,1024}, and use Earth Mover’s Distance (EMD) [31] on the LAFAN1 dataset as the metric for action distribution matching (lower is better, indicating closer alignment to the reference motion distribution). As shown in Figure 9, the EMD of both methods decreases asDincreases, indicating improved action distribution fitting, but with diminishing returns. The performance largely saturates whenD≥256. Across all settings,TeCHconsistently achieves lower EMD than BFM-Zero, suggesting that it can more accurately capture the target motion distribution under the same capacity. WhenDis sufficiently large, BFM-Zero with successor-feature factorization also achieves comparable performance. Considering the trade-off between performance, computational cost, and training stability, we useD= 256as the default latent dimensionality in all main experiments.\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-09-latent-dimension-ablation.png\"\nalt: \"EMD comparison under different latent dimensions\"\ncaption: \"Fig. 9. EMD comparison of BFM-Zero and TeCH under different latent dimensions D.\"\n---\n::\n\n### TeCH Encoder Learning Rate\n\nFor encoder learning inTeCH, a large learning rate may cause instability in the latent geometry, while a small learning rate may lead to sparse progress signals and delayed policy updates. To study its effect on motion distribution fitting, we fixD= 256and all other hyperparameters, and sweep the encoder learning ratelr∈ {5×10−5,3×10 −6,10 −5,3×10 −6,10 −6,8×10 −7,10 −9}. We report EMD evaluated during training (lower is better). As shown in Figure 10, a large learning rate (lr≈3×10 −5) leads to rapid early decrease in EMD but quickly saturates at a higher level with large fluctuations, indicating insufficient refinement of the temporal distance structure. Smaller learning rates (3×10−6 and10 −6) yield smoother convergence and better performance, reaching a stable range after around 100,Msteps, but remain suboptimal. The best performance is achieved atlr= 8×10 −7, which attains the lowest EMD within the standard training budget. A further reduction to 10−9 improves performance slowly but significantly slows convergence. Overall, there is a clear trade-off between convergence speed and final performance, and we uselr= 8×10 −7 as the default setting in all experiments.\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-10-encoder-learning-rate-ablation.png\"\nalt: \"TeCH encoder learning rate ablation curves\"\ncaption: \"Fig. 10. TeCH encoder learning rate ablation: EMD over training steps under different lr values.\"\n---\n::\n\n### Online Latent Goal Update Frequency\n\nIn addition to the encoder learning rate,TeCHalso relies on periodically refreshing the latent goalzduring online interaction (corresponding to theupdate-z-everyhyperparameter). This interval determines how many environment steps a sampled latent variable is kept before resampling. A short interval may weaken long-horizon consistency, while a long interval may reduce skill diversity and overly concentrate replay data in latent space. We sweepupdate-z-every∈ {5,10,50,100,200}while fixingD= 256and the optimal learning rate above. As shown in Figure 11, all curves decrease rapidly in the early stage and begin to diverge after approximately 40 Msteps. Updatingzevery 5 or 10 steps leads to the fastest convergence and the smoothest curves, achieving the lowest EMD (around 0.70) at110 Msteps. The setting update-z-every= 50performs slightly worse, with a final EMD of approximately 0.78, while100and200result in 0.81 and 0.83–0.85, respectively. Overall, more frequentz resampling improves action distribution fitting by increasing behavioral coverage in parallel environments while maintaining stable temporal representations. However, the performance gain saturates as the interval increases from 50 to 100 and 200. The difference betweenupdate-z-every= 5and= 10 becomes negligible after80 Msteps. Combining all ablation results, we uselr= 8×10 −7 andupdate-z-every= 10 as default settings in the main experiments.\n\n::paper-figure\n---\nsrc: \"\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-11-latent-goal-update-frequency.png\"\nalt: \"TeCH online latent goal update frequency ablation curves\"\ncaption: \"Fig. 11. TeCH ablation on online latent goal update frequency: EMD over training steps under different update intervals.\"\n---\n::\n\n## Conclusion\n\nIn conclusion, this paper presents a novel unsupervised humanoid control method built upon the contrastive learning based TLDR framework. Different from existing FB representation that relies on strict linear MDP assumptions,TeCHdirectly models temporal reachability in latent space to produce stable, dense reward signals and supports effective off-policy training. Combined with a series of practical training strategies including asymmetric training, domain randomization, auxiliary objectives and style regularization, our method well addresses partial observability, sim-to-real gaps and unstable or unnatural motions. Extensive experiments across simulation and physical robots verify that our approach achieves high-precision zero-shot tracking and goal reaching with strong data efficiency and generalization ability, demonstrating promising potential for real-world humanoid robot deployment.\n\n## References\n\n::paper-references\n\n1. Junik Bae, Kwanyoung Park, and Youngwoon Lee. Tldr: Unsupervised goal-conditioned rl via temporal distanceaware representations, 2024. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F 2407.08464.\n2. Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation, 2018. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F1810.12894.\n3. Zixuan Chen, Xialin He, Yen-Jen Wang, Qiayuan Liao, Yanjie Ze, Zhongyu Li, S. Shankar Sastry, Jiajun Wu, Koushil Sreenath, Saurabh Gupta, and Xue Bin Peng. Learning smooth humanoid locomotion through lipschitz-constrained policies.CoRR, abs\u002F2410.11825, 2024.\n4. Zixuan Chen, Mazeyu Ji, Xuxin Cheng, Xuanbin Peng, Xue Bin Peng, and Xiaolong Wang. GMT: general motion tracking for humanoid whole-body control.CoRR, abs\u002F2506.14770, 2025.\n5. Xuxin Cheng, Yandong Ji, Junming Chen, Ruihan Yang, Ge Yang, and Xiaolong Wang. Expressive wholebody control for humanoid robots.arXiv preprint arXiv:2402.16796, 2024.\n6. Pranay Dugar, Aayam Shrestha, Fangzhou Yu, Bart van Marum, and Alan Fern. Learning multi-modal wholebody control for real-world humanoid robots, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2408.07295.\n7. Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function, 2018. URL https:\u002F\u002Farxiv.org\u002F abs\u002F1802.06070.\n8. Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, and Chelsea Finn. Humanplus: Humanoid shadowing and imitation from humans, 2024. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F 2406.10454.\n9. Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational intrinsic control, 2016. URL https: \u002F\u002Farxiv.org\u002Fabs\u002F1611.07507.\n10. F ´elix G. Harvey, Mike Yurick, Derek Nowrouzezahrai, and Christopher J. Pal. Robust motion in-betweening. ACM Trans. Graph., 39(4):60, 2020.\n11. Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Learning humanto-humanoid real-time whole-body teleoperation.arXiv preprint arXiv:2403.04436, 2024.\n12. Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, and Yuke Zhu. HOVER: versatile neural whole-body controller for humanoid robots. CoRR, abs\u002F2410.21229, 2024.\n13. Tairan He, Jiawei Gao, Wenli Xiao, Yuanhang Zhang, Zi Wang, Jiashun Wang, Zhengyi Luo, Guanqi He, Nikhil Sobanbab, Chaoyi Pan, Zeji Yi, Guannan Qu, Kris Kitani, Jessica K. Hodgins, Linxi Fan, Yuke Zhu, Changliu Liu, and Guanya Shi. ASAP: aligning simulation and realworld physics for learning agile humanoid whole-body skills.CoRR, abs\u002F2502.01143, 2025.\n14. Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris M Kitani, Changliu Liu, and Guanya Shi. Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning. InConference on Robot Learning, pages 1516–1540. PMLR, 2025.\n15. Xialin He, Runpei Dong, Zixuan Chen, and Saurabh Gupta. Learning getting-up policies for real-world humanoid robots, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2502. 12152.\n16. Jongmin Kim, Youngwoon Lee, Pieter Abbeel, and Guanya Shi. Variational curriculum reinforcement learning for unsupervised discovery of skills, 2023. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.16342.\n17. Lisa Lee, Benjamin Eysenbach, Emilio Parisotto, Eric Xing, Sergey Levine, and Ruslan Salakhutdinov. Efficient exploration via state marginal matching, 2020. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F1906.05274.\n18. Yitang Li, Zhengyi Luo, Tonghe Zhang, Cunxi Dai, Anssi Kanervisto, Andrea Tirinzoni, Haoyang Weng, Kris Kitani, Mateusz Guzek, Ahmed Touati, Alessandro Lazaric, Matteo Pirotta, and Guanya Shi. Bfm-zero: A promptable behavioral foundation model for humanoid control using unsupervised reinforcement learning, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2511.04131.\n19. Qiayuan Liao, Takara E. Truong, Xiaoyu Huang, Guy Tevet, Koushil Sreenath, and C. Karen Liu. Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion, 2025. URL https:\u002F\u002Farxiv. org\u002Fabs\u002F2508.08241.\n20. Hao Liu and Pieter Abbeel. Cic: Contrastive intrinsic control for unsupervised skill discovery, 2022. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2202.00161.\n21. Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, and Weipeng Xu. Perpetual humanoid control for real-time simulated avatars. InICCV, pages 10861– 10870. IEEE, 2023.\n22. Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris M. Kitani, and Weipeng Xu. Universal humanoid motion representations for physicsbased control. InICLR. OpenReview.net, 2024.\n23. Zhengyi Luo, Ye Yuan, Tingwu Wang, Chenran Li, Sirui Chen, Fernando Casta ˜neda, Zi-Ang Cao, Jiefeng Li, David Minor, Qingwei Ben, Xingye Da, Runyu Ding, Cyrus Hogg, Lina Song, Edy Lim, Eugene Jeong, Tairan He, Haoru Xue, Wenli Xiao, Zi Wang, Simon Yuen, Jan Kautz, Yan Chang, Umar Iqbal, Linxi ”Jim” Fan, and Yuke Zhu. Sonic: Supersizing motion tracking for natural humanoid whole-body control, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2511.07820.\n24. Ian Mason. 100STYLE: A motion capture dataset of 100 locomotion styles, 2023. URL https:\u002F\u002Fwww.ianxmason. com\u002F100style\u002F.\n25. Mayank Mittal, Calvin Yu, Qinxi Yu, Jingzhou Liu, Nikita Rudin, David Hoeller, Jia Lin Yuan, Ritvik Singh, Yunrong Guo, Hammad Mazhar, Ajay Mandlekar, Buck Babich, Gavriel State, Marco Hutter, and Animesh Garg. Orbit: A unified simulation framework for interactive robot learning environments.IEEE Robotics Autom. Lett., 8(6):3740–3747, 2023.\n26. Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. Curiosity-driven exploration by selfsupervised prediction, 2017. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F 1705.05363.\n27. Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Sim-to-real transfer of robotic control with dynamics randomization.CoRR, abs\u002F1710.06537, 2017. URL http:\u002F\u002Farxiv.org\u002Fabs\u002F1710. 06537.\n28. Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. Deepmimic: example-guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph., 37(4):143, 2018.\n29. Ilija Radosavovic, Tete Xiao, Bike Zhang, Trevor Darrell, Jitendra Malik, and Koushil Sreenath. Real-world humanoid locomotion with reinforcement learning.Sci. Robotics, 9(89), 2024.\n30. Ilija Radosavovic, Bike Zhang, Baifeng Shi, Jathushan Rajasegaran, Sarthak Kamat, Trevor Darrell, Koushil Sreenath, and Jitendra Malik. Humanoid locomotion as next token prediction.CoRR, abs\u002F2402.19469, 2024.\n31. Yossi Rubner, Carlo Tomasi, and Leonidas J. Guibas. The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision, 40(2):99–121, 2000.\n32. Younggyo Seo, Carmelo Sferrazza, Haoran Geng, Michal Nauman, Zhao-Heng Yin, and Pieter Abbeel. Fasttd3: Simple, fast, and capable reinforcement learning for humanoid control.CoRR, abs\u002F2505.22642, 2025.\n33. Agon Serifi et al. Vmp: Versatile motion priors for robustly tracking motion on physical characters.Computer Graphics F orum, 43(8), 2024. doi: 10.1111\u002Fcgf.15175. URL https:\u002F\u002Fdoi.org\u002F10.1111\u002Fcgf.15175.\n34. Chen Tessler, Yunrong Guo, Ofir Nabati, Gal Chechik, and Xue Bin Peng. Maskedmimic: Unified physics-based character control through masked motion inpainting. ACM Trans. Graph., 43(6):209:1–209:21, 2024.\n35. Ahmed Touati and Yann Ollivier. Learning one representation to optimize all rewards. InNeurIPS, pages 13–23, 2021.\n36. Ahmed Touati and Yann Ollivier. Learning one representation to optimize all rewards, 2021. URL https: \u002F\u002Farxiv.org\u002Fabs\u002F2103.07945.\n37. Denis Yarats, Rob Fergus, Alessandro Lazaric, and Lerrel Pinto. Reinforcement learning with prototypical representations, 2021. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2102.11271.\n38. Kangning Yin, Weishuai Zeng, Ke Fan, Zirui Wang, Qiang Zhang, Zheng Tian, Jingbo Wang, Jiangmiao Pang, and Weinan Zhang. Unitracker: Learning universal whole-body motion tracker for humanoid robots.CoRR, abs\u002F2507.07356, 2025.\n39. Kevin Zakka, Baruch Tabanpour, Qiayuan Liao, Mustafa Haiderbhai, Samuel Holt, Jing Yuan Luo, Arthur Allshire, Erik Frey, Koushil Sreenath, Lueder A. Kahrs, Carmelo Sferrazza, Yuval Tassa, and Pieter Abbeel. Mujoco playground.CoRR, abs\u002F2502.08844, 2025.\n40. Weishuai Zeng, Shunlin Lu, Kangning Yin, Xiaojie Niu, Minyue Dai, Jingbo Wang, and Jiangmiao Pang. Behavior foundation model for humanoid robots, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.13780.\n41. Zhikai Zhang, Chao Chen, Han Xue, Jilong Wang, Sikai Liang, Yun Liu, Zongzhang Zhang, He Wang, and Li Yi. Unleashing humanoid reaching potential via real-worldready skill space.CoRR, abs\u002F2505.10918, 2025.\n42. Zhikai Zhang, Jun Guo, Chao Chen, Jilong Wang, Chenghuai Lin, Yunrui Lian, Han Xue, Zhenrong Wang, Maoqi Liu, Jiangran Lyu, Huaping Liu, He Wang, and Li Yi. Track any motions under any disturbances, 2025. URL https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.13833.\n43. . G1, 2025. URL https:\u002F\u002Fwww.unitree.com\u002Fcn\u002Fg1\u002F.\n\n::\n\n## Related Resources\n\n::paper-download\n---\ntitle: \"TECH.pdf\"\nhref: \"\u002Fdocuments\u002Ftech\u002FTECH.pdf\"\nsize: \"6.0 MB\"\nformat: \"PDF\"\nbuttonLabel: \"Download\"\n---\n::",{"githubUrl":10,"paperUrl":30},"\u002Fdocuments\u002Ftech\u002FTECH.pdf",{"source":9,"releaseId":10,"releaseVersion":10,"contentRevision":11,"checksum":32},"f17a2ec57d7ad6d4582161d326fdb87844a6574a8644ce60eb92c815c57bb195",{"title":34,"description":34,"body":35},"",{"type":36,"children":37,"toc":1382},"root",[38,44,51,57,62,67,80,99,111,122,130,135,140,146,179,184,210,250,262,273,279,284,289,300,305,310,315,320,325,330,335,340,345,350,355,368,373,379,390,395,400,406,411,431,442,447,452,457,469,474,480,485,496,501,506,511,516,542,547,553,571,576,593,602,727,746,757,789,794,800,806,811,816,821,827,838,844,855,860,866,872,877,882,888,893,898,903,908,913,925,930,936,942,947,952,958,963,968,973,978,984,989,994,999,1369,1374],{"type":39,"tag":40,"props":41,"children":43},"element","article-video-grid",{":videos":42},"[{\"src\":\"\u002Fvideos\u002Foutputs\u002Ftech\u002Ftech-under-disturbance.mp4\",\"label\":\"TeCH · 抗扰动实验\"},{\"src\":\"\u002Fvideos\u002Foutputs\u002Ftech\u002Ftech-motion-tracking.mp4\",\"label\":\"TeCH · 运动跟踪实验\"}]",[],{"type":39,"tag":45,"props":46,"children":48},"h2",{"id":47},"摘要",[49],{"type":50,"value":47},"text",{"type":39,"tag":52,"props":53,"children":54},"p",{},[55],{"type":50,"value":56},"人形机器人凭借其环境适应能力，成为理想的通用 AI 平台，但其高自由度使得可扩展的运动学习变得复杂。传统的 on-policy 强化学习方法（如 PPO）依赖手工设计的奖励函数，存在训练成本高昂、状态覆盖有限、泛化能力差等问题。为解决这些缺陷，无监督表征学习已被广泛应用于人形机器人控制。现有方案主要包括 Forward-Backward（FB）表征学习与基于对比度量学习的 TemporaL Distance Representation（TLDR）。FB 受线性 MDP 假设的限制，在富接触复杂场景下表现不稳定且表征退化。本文提出一种新的无监督人形机器人控制方法 TeCH，基于 TLDR 的对比学习框架构建。该方法通过隐空间时间距离建模构造密集奖励，引导结构化探索，提供简洁稳定的训练流程与组织良好的隐空间。它能够实现高精度的零样本运动跟踪与目标到达，具备出色的样本效率，并可无缝部署到物理机器人上。大量仿真与真机实验验证了本方法在跟踪精度和泛化能力上的优越性，展现出其在实际人形机器人控制中的巨大潜力。",{"type":39,"tag":45,"props":58,"children":60},{"id":59},"引言",[61],{"type":50,"value":59},{"type":39,"tag":52,"props":63,"children":64},{},[65],{"type":50,"value":66},"人形机器人因与人类环境的天然兼容性，成为通用 AI 的理想物理平台。其高自由度支持灵活的全身 loco-manipulation，但也使可扩展的运动学习复杂化。使用手工设计奖励的传统方法仅在有限任务上表现良好，缺乏在复杂场景中的鲁棒性与泛化能力。为此，研究者转向学习通用的行为先验以获取可复用的底层运动技能，而非为单独任务进行优化。",{"type":39,"tag":52,"props":68,"children":69},{},[70,72,78],{"type":50,"value":71},"近年来，强化学习推动了人形机器人 sim-to-real 的运动技能迁移。以全身运动跟踪为核心，这些方法使机器人能够学习用于 locomotion 和 loco-manipulation 任务的动态、可复用技能。然而，主流方法大多采用 PPO 等 on-policy 优化器，并结合手工设计的跟踪奖励或任务特定奖励，存在两个关键缺陷。其一，它们需要海量高维运动数据和大规模并行环境，导致训练成本难以承受。例如，SONIC ",{"type":39,"tag":73,"props":74,"children":75},"span",{},[76],{"type":50,"value":77},"23",{"type":50,"value":79}," 需要 128 块 GPU 连续训练三天。其二，受限的数据集覆盖无法涵盖高自由度人形机器人的完整控制空间。缺乏结构化且平滑的表征，现有方法限制了技能的泛化与复用，难以针对外部扰动高效生成鲁棒响应。",{"type":39,"tag":52,"props":81,"children":82},{},[83,85,90,92,97],{"type":50,"value":84},"为应对上述局限，基于表征学习的无监督强化学习方法开始应用于人形机器人控制。其中一个代表性分支采用基于 successor feature 的值分解，例如 Forward-Backward（FB）表征学习 ",{"type":39,"tag":73,"props":86,"children":87},{},[88],{"type":50,"value":89},"35",{"type":50,"value":91},"，通过学习可组合的隐表征实现无监督技能发现和行为复用。off-policy 无监督强化学习具有高样本效率、强复用性以及与大规模 replay buffer 的兼容性，在构建人形机器人行为基础模型方面极具前景。BFM-Zero ",{"type":39,"tag":73,"props":93,"children":94},{},[95],{"type":50,"value":96},"18",{"type":50,"value":98}," 将 FB 学习引入人形机器人控制，通过结构化探索获取平滑、可组合的运动基元，借助行为流形建模提供良好的技能插值、鲁棒性与泛化能力。然而，FB 在高维人形机器人系统中存在关键局限：它建立在隐嵌入空间的线性 Markov Decision Process（linear MDP）假设之上，在复杂接触动力学下失效，导致表征不稳定与退化。总体而言，现有的无监督技能学习方法不可避免地要在表征表达力、训练稳定性与任务可扩展性之间权衡。",{"type":39,"tag":52,"props":100,"children":101},{},[102,104,109],{"type":50,"value":103},"另一类无监督强化学习分支利用对比度量学习来学习状态表征，通过对时序相关性和行为距离建模，产生密集、稳定的优化信号。作为通过对比学习习得的代表性范式，TemporaL Distance Representation（TLDR）",{"type":39,"tag":73,"props":105,"children":106},{},[107],{"type":50,"value":108},"1",{"type":50,"value":110}," 与 FB 方法不同：它放弃值函数分解，直接使用时间距离目标学习时间可达性与系统动力学。",{"type":39,"tag":52,"props":112,"children":113},{},[114,116,120],{"type":50,"value":115},"在此基础上，本文提出 TeCH（如图 1 所示），一种基于时间距离表征 ",{"type":39,"tag":73,"props":117,"children":118},{},[119],{"type":50,"value":108},{"type":50,"value":121}," 构建的人形机器人无监督控制框架。该方法利用隐空间中的距离变化构造密集的过程奖励，赋予自主探索更清晰的方向性，并自然建立状态与目标之间的对应关系。得益于更直接的优化目标和简洁稳定的训练流程，TeCH 在全局目标控制任务中展现出更好的可扩展性和优化稳定性。我们还观察到，TeCH 能够构造结构良好的隐空间，以更高精度支持零样本运动跟踪和目标到达。它在训练阶段表现出卓越的数据效率，并在部署到物理人形机器人时展现出有前景的性能。",{"type":39,"tag":123,"props":124,"children":129},"paper-figure",{"alt":125,"caption":126,"src":127,":wide":128},"TeCH 隐流形与下游人形机器人控制任务概览","图 1. TeCH 概览：机器人自探索状态经状态编码器嵌入到隐流形中，隐空间距离作为转移步数的代理度量；所学表征同时支持连续运动跟踪与不连续目标到达。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-01-overview-tech.png","true",[],{"type":39,"tag":52,"props":131,"children":132},{},[133],{"type":50,"value":134},"作为两大主流无监督强化学习范式，FB 与 TLDR 框架都已被证明能够将习得的能力迁移到真实人形机器人。本文的主要贡献概括如下：\n• 方法创新：我们提出一种基于时间距离学习的新型人形机器人控制方法 TeCH，通过产生密集的隐奖励进行结构化探索，提升无监督强化学习的跟踪精度。\n• 对比基准：我们首次在人形机器人上开展无监督强化学习的系统性对比，为未来设计提供实用洞见。\n• 实验验证：大量仿真与真机实验验证了 TeCH 在实际部署中实现高精度全身跟踪与目标到达的能力。",{"type":39,"tag":45,"props":136,"children":138},{"id":137},"相关工作",[139],{"type":50,"value":137},{"type":39,"tag":141,"props":142,"children":144},"h3",{"id":143},"人形机器人全身控制",[145],{"type":50,"value":143},{"type":39,"tag":52,"props":147,"children":148},{},[149,151,156,158,163,165,170,172,177],{"type":50,"value":150},"近年来，基于学习的方法在人形机器人全身控制方面取得显著进展。尽管强化学习可以在仿真中有效生成复杂动态行为 ",{"type":39,"tag":73,"props":152,"children":153},{},[154],{"type":50,"value":155},"28, 21, 22, 34, 33",{"type":50,"value":157},"，sim-to-real 迁移仍是真机部署的关键瓶颈。研究者提出了包括域随机化和系统辨识在内的多种策略，但多数现有工作聚焦于行走、奔跑、起立等特定行为的单任务学习 ",{"type":39,"tag":73,"props":159,"children":160},{},[161],{"type":50,"value":162},"29, 30, 3, 32, 39, 15, 27",{"type":50,"value":164},"。自 2025 年起，多任务与通用人形机器人控制受到越来越多关注，旨在用单一策略处理多样任务 ",{"type":39,"tag":73,"props":166,"children":167},{},[168],{"type":50,"value":169},"12, 13, 41, 40, 38, 4, 11, 14, 5, 6, 8",{"type":50,"value":171},"。这些方法通常在仿真中训练跟踪策略，再通过基于 VAE 的隐学习或扩散模型 ",{"type":39,"tag":73,"props":173,"children":174},{},[175],{"type":50,"value":176},"12, 38, 40, 4, 41, 19",{"type":50,"value":178}," 蒸馏为统一的多任务策略。然而，它们本质上受限于运动数据集质量，且普遍采用 on-policy 强化学习。与此不同，本方法利用 off-policy 强化学习直接学习通用策略，得到更丰富、更结构化的技能空间，且不受运动数据集约束。",{"type":39,"tag":141,"props":180,"children":182},{"id":181},"无监督强化学习",[183],{"type":50,"value":181},{"type":39,"tag":52,"props":185,"children":186},{},[187,189,194,196,201,203,208],{"type":50,"value":188},"无监督强化学习使智能体能在没有任务特定奖励的情况下学习多样、可控的行为。基于互信息的技能发现方法（包括 DIAYN ",{"type":39,"tag":73,"props":190,"children":191},{},[192],{"type":50,"value":193},"7",{"type":50,"value":195},"、VIC ",{"type":39,"tag":73,"props":197,"children":198},{},[199],{"type":50,"value":200},"9",{"type":50,"value":202}," 和 VALOR ",{"type":39,"tag":73,"props":204,"children":205},{},[206],{"type":50,"value":207},"16",{"type":50,"value":209},"）通过最大化隐变量与状态轨迹之间的互信息来提升技能可区分性。但这些方法主要在简单的 locomotion 场景下验证，且通常采用 on-policy 训练，限制了它们在高维人形机器人系统中的可扩展性。",{"type":39,"tag":52,"props":211,"children":212},{},[213,215,220,222,227,229,234,236,241,243,248],{"type":50,"value":214},"常见的对比与表征学习方法（如 CIC ",{"type":39,"tag":73,"props":216,"children":217},{},[218],{"type":50,"value":219},"20",{"type":50,"value":221}," 和 Proto-RL ",{"type":39,"tag":73,"props":223,"children":224},{},[225],{"type":50,"value":226},"37",{"type":50,"value":228},"）通过 InfoNCE 与原型匹配学习结构化的状态与技能嵌入，获得更稳定的表征。与此同时，内在动机类方法（包括 RND ",{"type":39,"tag":73,"props":230,"children":231},{},[232],{"type":50,"value":233},"2",{"type":50,"value":235},"、ICM ",{"type":39,"tag":73,"props":237,"children":238},{},[239],{"type":50,"value":240},"26",{"type":50,"value":242}," 和 SMM ",{"type":39,"tag":73,"props":244,"children":245},{},[246],{"type":50,"value":247},"17",{"type":50,"value":249},"）通过奖励状态新颖性和覆盖率来促进探索。然而，这些范式缺乏显式的目标进度建模，对目标到达行为的监督较弱。",{"type":39,"tag":52,"props":251,"children":252},{},[253,255,260],{"type":50,"value":254},"Forward-Backward（FB）表征学习 ",{"type":39,"tag":73,"props":256,"children":257},{},[258],{"type":50,"value":259},"36",{"type":50,"value":261}," 通过将值函数分解为前向状态嵌入和后向技能嵌入，提供结构化的行为建模，支持可组合的技能表征与零样本泛化。尽管有这些优势，FB 依赖严格的 successor feature 分解假设，并需要对偶映射的联合优化，且在高维、富接触的人形机器人控制场景中验证不足。",{"type":39,"tag":52,"props":263,"children":264},{},[265,267,271],{"type":50,"value":266},"作为另一种有效的对比学习范式，TemporaL Distance Representation（TLDR）",{"type":39,"tag":73,"props":268,"children":269},{},[270],{"type":50,"value":108},{"type":50,"value":272}," 放弃了 forward-backward 分解，通过时间距离感知编码器 ϕ_ψ 显式建模时间可达性，将时间相邻的状态嵌入到相近位置，并产生与过程对齐的密集奖励 r_tldr。这一设计带来更简单、更稳定的 off-policy 优化目标。在本工作中，我们将 TLDR 扩展到真实人形机器人，并对通用运动跟踪与目标到达任务进行系统评估。",{"type":39,"tag":45,"props":274,"children":276},{"id":275},"tech-预训练",[277],{"type":50,"value":278},"TeCH 预训练",{"type":39,"tag":52,"props":280,"children":281},{},[282],{"type":50,"value":283},"TeCH 由两个核心组件构成：无监督预训练阶段和零样本推理流程（第 IV 节）。本章概述其无监督预训练流程，包括仿真环境中的训练过程以及面向真机迁移的设计细节。在第 III-A 节给出问题建模，并在第 III-B 与 III-C 节介绍基于时间距离表征框架的 TeCH 无监督强化学习框架。",{"type":39,"tag":52,"props":285,"children":286},{},[287],{"type":50,"value":288},"对 TeCH 而言，学习目标有两个方面：1) 一个能够捕捉行为轨迹时间结构的隐表征空间。在该空间中，具有时间邻近性和相似行为模式的状态或轨迹被映射到相邻位置，使策略能够基于轨迹级别的行为语义进行泛化（第 III-B 节）；2) 一个以隐任务表征为条件的策略。我们可将宏观机器人姿态映射到该结构化空间，实现对多样运动目标和行为模式的统一控制（第 III-C 节）。",{"type":39,"tag":52,"props":290,"children":291},{},[292,294,298],{"type":50,"value":293},"为实现策略的零样本 sim-to-real 迁移，我们遵循 ",{"type":39,"tag":73,"props":295,"children":296},{},[297],{"type":50,"value":96},{"type":50,"value":299}," 在训练期间引入对抗目标，使人形机器人行为与人类数据分布对齐（第 III-D 节）；我们还加入辅助训练目标以强制施加 sim-to-real 迁移所需的关键约束，例如关节限位合规性和有界的动作变化率。TeCH 的完整预训练流程与伪代码在第 III-E 节进一步展开。",{"type":39,"tag":141,"props":301,"children":303},{"id":302},"问题建模",[304],{"type":50,"value":302},{"type":39,"tag":52,"props":306,"children":307},{},[308],{"type":50,"value":309},"我们将真实世界的人形机器人控制问题形式化为部分可观测 Markov Decision Process（POMDP），定义为五元组 (S, O, A, P, γ)，其中 S 表示完整状态空间，O 为观测空间，A 为动作空间，P(s_{t+1} | s_t, a_t) 为状态转移动力学，γ∈(0,1) 为折扣因子。",{"type":39,"tag":52,"props":311,"children":312},{},[313],{"type":50,"value":314},"对于一个具有 29 个自由度的人形机器人，动作 a∈A⊂R^29 由所有自由度的 PD 控制器目标值组成。特权状态信息 s∈R^463 包括根部高度、本体位姿、本体朝向、线速度与角速度。可观测量定义为 o_t = {q_t − q̄, q̇_t, ω^root_t \u002F 4, g_t} ∈ R^64，其中 q_t ∈ R^29 表示相对于标称位姿 q̄ 归一化的关节位置，q̇_t ∈ R^29 表示关节速度，ω^root_t ∈ R^3 表示根部角速度，g_t ∈ R^3 表示根部坐标系下的投影重力向量。我们进一步定义观测历史 o_{t,H} = {o_{t−H}, a_{t−H}, …, o_t} ∈ R^{93·H+64}，由本体感知状态和动作组成。除根部高度外，所有状态分量都相对于当前朝向和根部位置进行归一化。",{"type":39,"tag":52,"props":316,"children":317},{},[318],{"type":50,"value":319},"在预训练阶段，智能体可访问无标签运动数据集 M={τ}，其中每条轨迹同时包含观测序列和特权状态序列，即 τ = (o_1, s_1, …, o_{l(τ)}, s_{l(τ)})，l(τ) 表示轨迹 τ 的长度。",{"type":39,"tag":141,"props":321,"children":323},{"id":322},"时间距离表征学习",[324],{"type":50,"value":322},{"type":39,"tag":52,"props":326,"children":327},{},[328],{"type":50,"value":329},"形式上，给定状态转移 (s_t, s_{t+1})，我们通过时间滚动构造伪目标：\ng_t = roll(s_{t+1}),(1)",{"type":39,"tag":52,"props":331,"children":332},{},[333],{"type":50,"value":334},"该时间滚动操作将未来状态扩展到单步转移之外，超越瞬时相邻性，建模更长距离的时间依赖。因此，所学表征在对比学习范式下被优化以捕捉多步环境动力学，而非仅仅拟合瞬时状态转移。",{"type":39,"tag":52,"props":336,"children":337},{},[338],{"type":50,"value":339},"随后，我们通过共享编码器将所有观测状态和构造的伪目标映射到统一的隐空间：\nϕ_x = ϕ_ψ(s_t), ϕ_y = ϕ_ψ(s_{t+1}), ϕ_g = ϕ_ψ(g_t).(2)",{"type":39,"tag":52,"props":341,"children":342},{},[343],{"type":50,"value":344},"编码器使用双目标对比损失进行训练，以正则化隐空间几何：\nL_TE(ψ, λ) = −J_dist(ϕ_x, ϕ_g) + λ C_step(ϕ_x, ϕ_y),(3)",{"type":39,"tag":52,"props":346,"children":347},{},[348],{"type":50,"value":349},"其中核心项 J_dist 作为对比目标，强制在隐空间中时间相距较远的状态对之间进行显式特征分离，区分可达与不可达的时间状态关系；作为补充，步级平滑项 C_step 作为连续状态之间的正则约束，防止特征突变并保持隐演化的连续性。两者联合优化塑造了物理上一致的隐几何，使隐空间距离显式反映真实机器人轨迹的时间可达性与多步转移逻辑。",{"type":39,"tag":141,"props":351,"children":353},{"id":352},"基于时间距离的目标进度奖励",[354],{"type":50,"value":352},{"type":39,"tag":52,"props":356,"children":357},{},[358,360,366],{"type":50,"value":359},"对策略 π_θ 而言，关键目标是衡量朝向隐目标 z 的进度。令 z̄(·) 表示投影后的隐表征，奖励定义为：\nr_tldr(s_t, s_{t+1}, z) = ‖z − z̄(s_t)‖",{"type":39,"tag":361,"props":362,"children":363},"em",{},[364],{"type":50,"value":365},"2 − ‖z − z̄(s",{"type":50,"value":367},"{t+1})‖_2.(4)",{"type":39,"tag":52,"props":369,"children":370},{},[371],{"type":50,"value":372},"该形式具有清晰的直观含义：若下一状态在隐空间中更接近目标，则奖励为正；否则为负。因此，策略被持续驱动朝目标接近的方向优化，得到密集且具方向信息的学习信号。",{"type":39,"tag":141,"props":374,"children":376},{"id":375},"tech-真机迁移",[377],{"type":50,"value":378},"TeCH 真机迁移",{"type":39,"tag":52,"props":380,"children":381},{},[382,384,388],{"type":50,"value":383},"将无监督强化学习部署到人形机器人面临真实观测有限、sim-to-real 动力学鸿沟、运动不稳定以及无约束探索导致的异常行为等问题。为解决这些问题并增强 sim-to-real 迁移，我们采用四个训练组件：使用非对称训练处理部分可观测性；利用域随机化降低仿真过拟合；引入辅助目标保证运动稳定性与安全性；使用以隐变量为条件的风格判别器，正则化策略生成类人行为。上述模块的详细实现均遵循 BFM-Zero ",{"type":39,"tag":73,"props":385,"children":386},{},[387],{"type":50,"value":96},{"type":50,"value":389},"。",{"type":39,"tag":141,"props":391,"children":393},{"id":392},"预训练流程",[394],{"type":50,"value":392},{"type":39,"tag":52,"props":396,"children":397},{},[398],{"type":50,"value":399},"如图 2 所示，预训练流程由三个迭代模块组成：在线交互（online interaction）、经验回放（experience replay）和 multi-objective optimization。这些组件构成一个闭环训练回路，在并行仿真环境中持续运行。",{"type":39,"tag":123,"props":401,"children":405},{"alt":402,"caption":403,"src":404,":wide":128},"TeCH 预训练流程","图 2. 该预训练流程使用三个并行 critic 联合优化策略：一个判别性 critic 用于人形机器人运动正则化，一个辅助 critic 提供 sim-to-real 迁移奖励，以及一个表征 critic 正则化状态嵌入以对齐隐目标。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-02-pre-training-pipeline.png",[],{"type":39,"tag":52,"props":407,"children":408},{},[409],{"type":50,"value":410},"在在线交互阶段，策略以隐变量 z 为条件执行动作，并收集形如 (s_t, o_{t−k:t}, a_t, s_{t+1}, z_t) 的转移，存入在线 replay buffer。重要的是，隐变量 z 并非来自单一来源，而是由三种互补信号混合构造，共同平衡探索多样性、目标可达性和行为先验：",{"type":39,"tag":412,"props":413,"children":414},"ol",{},[415,421,426],{"type":39,"tag":416,"props":417,"children":418},"li",{},[419],{"type":50,"value":420},"随机采样的隐变量（例如来自高斯或超球面先验），扩大隐任务空间覆盖并鼓励行为多样性；",{"type":39,"tag":416,"props":422,"children":423},{},[424],{"type":50,"value":425},"与在线样本相关联的隐表征，将学习对齐到当前策略可达的目标；",{"type":39,"tag":416,"props":427,"children":428},{},[429],{"type":50,"value":430},"来自专家轨迹的隐表征，注入人类运动先验并在隐空间中提供稳定锚点。",{"type":39,"tag":52,"props":432,"children":433},{},[434,436,440],{"type":50,"value":435},"值得注意的是，TeCH 通过利用所学的时间距离度量实现更灵活的探索策略。除 ",{"type":39,"tag":73,"props":437,"children":438},{},[439],{"type":50,"value":96},{"type":50,"value":441}," 采用的混合目标采样方案（包括随机目标、回放高价值目标和专家目标）外，TeCH 还估计当前状态与候选目标状态之间的时间可达性，并在整个训练过程中根据该可达性估计动态调整目标采样概率。",{"type":39,"tag":52,"props":443,"children":444},{},[445],{"type":50,"value":446},"具体而言，在训练早期阶段，TeCH 为时间相距较远且不常访问的目标分配更高的采样概率，从而鼓励更广泛的探索。随着策略逐步稳定，采样分布逐渐转向与更高期望回报相关联的区域，使学习聚焦于精炼与任务相关的技能，提升整体策略性能。",{"type":39,"tag":52,"props":448,"children":449},{},[450],{"type":50,"value":451},"在经验回放阶段，学习器同时从在线 replay buffer 中采样转移批次，并从无标签专家演示中采样轨迹片段，分别用于构造判别器学习信号和表征学习目标。",{"type":39,"tag":52,"props":453,"children":454},{},[455],{"type":50,"value":456},"随后训练进入 multi-objective optimization 阶段，参数按 判别器 → 编码器(ϕ_ψ) → 主 Critic 与辅助 Critic → Actor → 目标网络软更新 的顺序依次更新。该闭环过程重复直至收敛。",{"type":39,"tag":52,"props":458,"children":459},{},[460,462,467],{"type":50,"value":461},"从优化角度看，Actor 由三个互补学习信号驱动：由 r_tldr 诱导的表征目标 Q_REP、风格目标 Q_D，以及辅助目标 Q_R。将这些项组合得到最终 Actor 目标：\nL(π) = − E_{(o_{t,H}, s_t)∼D, a_t=π(o_{t,H}, z), z∼ν} ",{"type":39,"tag":73,"props":463,"children":464},{},[465],{"type":50,"value":466},"λ_REP Q_REP + λ_D Q_D + λ_R Q_R",{"type":50,"value":468},".(5)",{"type":39,"tag":52,"props":470,"children":471},{},[472],{"type":50,"value":473},"TeCH 的完整预训练过程见算法 ??。为简洁起见，与并行环境仿真和分布式训练相关的实现细节被省略。",{"type":39,"tag":45,"props":475,"children":477},{"id":476},"tech-零样本推理",[478],{"type":50,"value":479},"TeCH 零样本推理",{"type":39,"tag":52,"props":481,"children":482},{},[483],{"type":50,"value":484},"在测试阶段，TeCH 可以 zero-shot 方式求解多种任务，无需任何额外的任务特定学习、规划或微调。我们首先将不同的下游任务嵌入到隐空间，并考虑两类代表性任务。\n• 目标到达（Goal Reaching）：给定不连续的目标状态，将其映射为多个策略条件 z，使机器人能够在不同目标姿态之间切换。\n• 轨迹跟踪（Trajectory Tracking）：在参考轨迹窗口内聚合未来状态，并将其映射为隐条件序列 {z_t}，使策略能够逐步跟踪目标运动。",{"type":39,"tag":52,"props":486,"children":487},{},[488,490,494],{"type":50,"value":489},"TeCH 不属于基于 successor feature 的无监督强化学习范式，因此不提供统一的基于值的推理接口。这导致它通常无法直接支持如 ",{"type":39,"tag":73,"props":491,"children":492},{},[493],{"type":50,"value":96},{"type":50,"value":495}," 中那样的奖励优化任务。相比之下，TeCH 引入了时间距离感知表征，显式编码相对于当前状态的时间可达性关系。因此，在零样本跟踪场景下，TeCH 常常表现出更快、更紧致的轨迹跟踪行为。",{"type":39,"tag":52,"props":497,"children":498},{},[499],{"type":50,"value":500},"目标到达。对于目标到达任务，对应的隐向量形式化为：\nz_g = ϕ(s_g),(6)\n其中 ϕ(·) 表示 TeCH 的时间距离编码器。",{"type":39,"tag":52,"props":502,"children":503},{},[504],{"type":50,"value":505},"跟踪。对于运动轨迹跟踪，给定轨迹 τ = {s_1, …, s_n}，策略的隐变量序列 {z_t} 定义为：\nz_t = Σ_{t′=t}^{t+H} w_{t′} ϕ(s_{t′}),(7)\n其中 H 表示前瞻时间窗口，w_{t′} 为满足 w_{t′} ≥ 0 且 Σ_{t′=t}^{t+H} w_{t′} = 1 的折扣权重。",{"type":39,"tag":45,"props":507,"children":509},{"id":508},"实验",[510],{"type":50,"value":508},{"type":39,"tag":141,"props":512,"children":514},{"id":513},"实验设置",[515],{"type":50,"value":513},{"type":39,"tag":52,"props":517,"children":518},{},[519,521,526,528,533,535,540],{"type":50,"value":520},"本节在仿真和真机平台上全面评估 TeCH。我们基于 IsaacLab ",{"type":39,"tag":73,"props":522,"children":523},{},[524],{"type":50,"value":525},"25",{"type":50,"value":527}," 在 Unitree G1 人形机器人仿真环境 ",{"type":39,"tag":73,"props":529,"children":530},{},[531],{"type":50,"value":532},"43",{"type":50,"value":534}," 中训练两种方法。仿真频率设为 200 Hz，控制频率为 50 Hz。运动数据集采用 LAFAN1 ",{"type":39,"tag":73,"props":536,"children":537},{},[538],{"type":50,"value":539},"10",{"type":50,"value":541},"，已重定向到 Unitree G1 机器人，包含约 40 段各数分钟的运动片段。",{"type":39,"tag":141,"props":543,"children":545},{"id":544},"仿真结果",[546],{"type":50,"value":544},{"type":39,"tag":548,"props":549,"children":551},"h4",{"id":550},"运动跟踪",[552],{"type":50,"value":550},{"type":39,"tag":52,"props":554,"children":555},{},[556,558,562,564,569],{"type":50,"value":557},"该评测衡量运动序列的模仿精度。我们在训练数据集（LAFAN1 ",{"type":39,"tag":73,"props":559,"children":560},{},[561],{"type":50,"value":539},{"type":50,"value":563},"，40 段片段，总计约 2–3 小时）和测试数据集（100-Style ",{"type":39,"tag":73,"props":565,"children":566},{},[567],{"type":50,"value":568},"24",{"type":50,"value":570},"，3–4k 段片段，总计约 18–19 小时）上评估策略，所有运动均重定向到 G1 机器人。我们使用关节位置误差作为评估指标：\nE_mae(e, m) = (1\u002F|e|) Σ_^{|e|} ‖q_t(e) − q_t(m)‖,(8)\n其中 m 表示目标运动轨迹。最终结果报告为所有运动片段的平均误差。",{"type":39,"tag":52,"props":572,"children":573},{},[574],{"type":50,"value":575},"不同于 SONIC 等依赖任务特定运动跟踪奖励和模仿学习的框架，BFM-Zero 和 TeCH 探索无监督强化学习进行人形机器人控制，学习统一的可复用技能隐空间，从而支持不连续的目标到达和分层规划。",{"type":39,"tag":52,"props":577,"children":578},{},[579,581,585,587,591],{"type":50,"value":580},"尽管如此，我们仍与最先进的通用运动跟踪控制器 SONIC ",{"type":39,"tag":73,"props":582,"children":583},{},[584],{"type":50,"value":77},{"type":50,"value":586}," 进行对比。我们在相同环境和测试集上评估公开发布的模型，报告 E_mae 及其在运动片段上的标准差。为公平起见，BFM-Zero、TeCH 和 SONIC ",{"type":39,"tag":73,"props":588,"children":589},{},[590],{"type":50,"value":77},{"type":50,"value":592}," 均不启用 early termination，即指标在完整轨迹上计算。由于 SONIC 在严重失败或跌倒后通常难以快速恢复，我们额外报告 SONIC (TER.)，其指标仅在终止前的片段上计算。",{"type":39,"tag":52,"props":594,"children":595},{},[596],{"type":39,"tag":597,"props":598,"children":599},"strong",{},[600],{"type":50,"value":601},"表 I. TeCH 与基线方法在训练集和测试集上的运动跟踪误差（E_mae）（↓）（均值 ± 标准差）。",{"type":39,"tag":603,"props":604,"children":605},"table",{},[606,631],{"type":39,"tag":607,"props":608,"children":609},"thead",{},[610],{"type":39,"tag":611,"props":612,"children":613},"tr",{},[614,620,626],{"type":39,"tag":615,"props":616,"children":617},"th",{},[618],{"type":50,"value":619},"Method",{"type":39,"tag":615,"props":621,"children":623},{"align":622},"right",[624],{"type":50,"value":625},"LAFAN1",{"type":39,"tag":615,"props":627,"children":628},{"align":622},[629],{"type":50,"value":630},"100-Style",{"type":39,"tag":632,"props":633,"children":634},"tbody",{},[635,660,683,705],{"type":39,"tag":611,"props":636,"children":637},{},[638,650,655],{"type":39,"tag":639,"props":640,"children":641},"td",{},[642,644],{"type":50,"value":643},"SONIC",{"type":39,"tag":645,"props":646,"children":647},"sup",{},[648],{"type":50,"value":649},"b",{"type":39,"tag":639,"props":651,"children":652},{"align":622},[653],{"type":50,"value":654},"0.2916 ± 0.1887",{"type":39,"tag":639,"props":656,"children":657},{"align":622},[658],{"type":50,"value":659},"0.1522 ± 0.1049",{"type":39,"tag":611,"props":661,"children":662},{},[663,673,678],{"type":39,"tag":639,"props":664,"children":665},{},[666,668],{"type":50,"value":667},"SONIC (TER.)",{"type":39,"tag":645,"props":669,"children":670},{},[671],{"type":50,"value":672},"a",{"type":39,"tag":639,"props":674,"children":675},{"align":622},[676],{"type":50,"value":677},"0.1081 ± 0.0213",{"type":39,"tag":639,"props":679,"children":680},{"align":622},[681],{"type":50,"value":682},"0.1355 ± 0.0351",{"type":39,"tag":611,"props":684,"children":685},{},[686,695,700],{"type":39,"tag":639,"props":687,"children":688},{},[689,691],{"type":50,"value":690},"BFM-Zero",{"type":39,"tag":645,"props":692,"children":693},{},[694],{"type":50,"value":649},{"type":39,"tag":639,"props":696,"children":697},{"align":622},[698],{"type":50,"value":699},"0.1510 ± 0.0255",{"type":39,"tag":639,"props":701,"children":702},{"align":622},[703],{"type":50,"value":704},"0.1674 ± 0.0459",{"type":39,"tag":611,"props":706,"children":707},{},[708,717,722],{"type":39,"tag":639,"props":709,"children":710},{},[711,713],{"type":50,"value":712},"TeCH",{"type":39,"tag":645,"props":714,"children":715},{},[716],{"type":50,"value":649},{"type":39,"tag":639,"props":718,"children":719},{"align":622},[720],{"type":50,"value":721},"0.1318 ± 0.0329",{"type":39,"tag":639,"props":723,"children":724},{"align":622},[725],{"type":50,"value":726},"0.1474 ± 0.0405",{"type":39,"tag":52,"props":728,"children":729},{},[730],{"type":39,"tag":731,"props":732,"children":733},"small",{},[734,738,740,744],{"type":39,"tag":645,"props":735,"children":736},{},[737],{"type":50,"value":672},{"type":50,"value":739}," SONIC (TER.) 仅在终止前的片段上评估。 ",{"type":39,"tag":645,"props":741,"children":742},{},[743],{"type":50,"value":649},{"type":50,"value":745}," SONIC、BFM-Zero 与 TeCH 均在完整轨迹上评估。",{"type":39,"tag":52,"props":747,"children":748},{},[749,751,755],{"type":50,"value":750},"从跟踪结果可以看出，SONIC ",{"type":39,"tag":73,"props":752,"children":753},{},[754],{"type":50,"value":77},{"type":50,"value":756}," 在跌倒恢复和地面过渡期间表现较差，这显著降低了其在测试集上的整体跟踪表现。在完全分布外条件下（例如外部扰动、跌倒或随机初始化），BFM-Zero 和 TeCH 通常能够更自然地恢复平衡并继续稳定跟踪，而 SONIC 往往无法回到稳定的运动状态。我们将这一优势主要归因于 off-policy 的无监督探索。与依赖大规模运动模仿监督的方法相比，像 TeCH 这样的无监督 RL 框架在训练期间覆盖了更广泛的非标准状态和动态转移，从而得到更平滑、更连续的技能空间。",{"type":39,"tag":52,"props":758,"children":759},{},[760,762,767,769,774,776,781,783,787],{"type":50,"value":761},"即便仅就跟踪精度而言，TeCH 也达到了与 SONIC (TER.) 相当的水平，如表 I 所示。值得注意的是，SONIC 在其原始评估中已经优于 GMT ",{"type":39,"tag":73,"props":763,"children":764},{},[765],{"type":50,"value":766},"4",{"type":50,"value":768},"、Any2Track ",{"type":39,"tag":73,"props":770,"children":771},{},[772],{"type":50,"value":773},"42",{"type":50,"value":775}," 和 BeyondMimic ",{"type":39,"tag":73,"props":777,"children":778},{},[779],{"type":50,"value":780},"19",{"type":50,"value":782},"。我们还要强调训练效率上的显著差异：SONIC ",{"type":39,"tag":73,"props":784,"children":785},{},[786],{"type":50,"value":77},{"type":50,"value":788}," 使用 128 块 GPU、大规模运动数据集和海量并行环境进行训练；而本方法仅用单卡 GPU 和数小时的 LAFAN1 数据完成训练，将 GPU 小时和环境样本数降低了近两个数量级。",{"type":39,"tag":52,"props":790,"children":791},{},[792],{"type":50,"value":793},"为更细致地比较这两种无监督 RL 方法，我们在图 3 中给出定性的运动跟踪结果。TeCH 实现了高精度的运动跟踪，生成的轨迹在关节层面与参考运动紧密对齐，表明所学表征有效捕捉了局部姿态结构。此外，TeCH 始终优于 BFM-Zero，尤其是在缓解全局根部旋转漂移方面，如图 4 的定量结果所进一步支持，特别是在快速旋转场景下。这一性能差距源于 BFM-Zero 的局限：它通过间接的 forward-backward 分解学习表征，在 successor measure M 中引入近似误差，限制了其对全局朝向、长时序相位信息和累积根部漂移的建模能力。相比之下，TeCH 更直接地构造跟踪目标，在保持精确局部关节精度的同时获得更优的全局姿态一致性。",{"type":39,"tag":123,"props":795,"children":799},{"alt":796,"caption":797,"src":798,":wide":128},"BFM-Zero 与 TeCH 在仿真中的定性运动跟踪对比","图 3. BFM-Zero 与 TeCH 在仿真中运动跟踪的定性对比：可见两种方法都能有效跟踪域内运动，且 TeCH 略胜一筹。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-03-motion-tracking-simulation.png",[],{"type":39,"tag":123,"props":801,"children":805},{"alt":802,"caption":803,"src":804,":wide":128},"分布外快速 360 度旋转任务下的根部全局旋转误差对比","图 4. 分布外快速 360 度旋转任务下 BFM-Zero 与 TeCH 的根部全局旋转误差对比。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-04-root-global-rotation-error.png",[],{"type":39,"tag":548,"props":807,"children":809},{"id":808},"目标到达",[810],{"type":50,"value":808},{"type":39,"tag":52,"props":812,"children":813},{},[814],{"type":50,"value":815},"我们从训练数据集（即 LAFAN1）中人工提取 21 个“稳定”姿态，定义为速度为零的状态。为衡量智能体与目标姿态之间的接近程度，我们将关节误差定义为：\nE_mae(e, g) = (1\u002F|e|) Σ_^{|e|} ‖q_t(e) − q(g)‖,(9)\n其中 e 表示一个 episode，q(·) 表示关节配置（即 29 维向量）。最终结果报告为所有目标姿态的平均值。每个 episode 的固定 horizon 为 H = 500。",{"type":39,"tag":52,"props":817,"children":818},{},[819],{"type":50,"value":820},"图 5(a) 比较了 BFM-Zero 与 TeCH 在目标到达任务上的关节级平均绝对误差。结果显示两种方法精度相当，而 TeCH 在精度略有下降的情况下能更高效地到达目标。图 5(b) 展示了三个随机选取的目标姿态，体现其多样性与复杂度。图 5(c) 给出定性结果，机器人成功从躺下过渡到坐立再到站立，展现出鲁棒的恢复行为。",{"type":39,"tag":123,"props":822,"children":826},{"alt":823,"caption":824,"src":825,":wide":128},"目标到达的关节误差与定性示例","图 5. 目标到达的关节误差与定性结果对比：(a) 21 个目标姿态上的平均关节 MAE；(b) 示例目标姿态；(c) TeCH 在目标到达任务上的定性结果。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-05-goal-reaching-comparison.png",[],{"type":39,"tag":52,"props":828,"children":829},{},[830,832,836],{"type":50,"value":831},"值得注意的是，即便在具有挑战性的恢复场景中，TeCH 也能产生连贯的轨迹到达目标。相比之下，如 SONIC ",{"type":39,"tag":73,"props":833,"children":834},{},[835],{"type":50,"value":77},{"type":50,"value":837}," 等基于跟踪的方法在不连续目标过渡上表现挣扎，凸显出纯跟踪驱动目标在不连续目标到达上的局限。",{"type":39,"tag":141,"props":839,"children":841},{"id":840},"tech-真机实验结果",[842],{"type":50,"value":843},"TeCH 真机实验结果",{"type":39,"tag":52,"props":845,"children":846},{},[847,849,853],{"type":50,"value":848},"如图 6 与图 7 所示，TeCH 部署到真实硬件后在运动跟踪和目标到达上同样表现出色。当大型外力扰动推搡机器人时，TeCH 能像 BFM-Zero 一样在跌倒后自主站起，而基线 SONIC ",{"type":39,"tag":73,"props":850,"children":851},{},[852],{"type":50,"value":77},{"type":50,"value":854}," 则无法从跌倒状态恢复。这是因为我们的统一隐目标空间在预训练中编码了全身恢复运动，使策略无需单独的跌倒恢复微调即可生成自扶正轨迹。",{"type":39,"tag":52,"props":856,"children":857},{},[858],{"type":50,"value":859},"然而，两种无监督 RL 方法在真机测试中表现出不同的行为特性。与 BFM-Zero 相比，TeCH 倾向于更直接地执行目标运动，响应更激进，有时会导致行为略微不平滑。这是因为我们的表征损失以最小化隐目标之间的转移步数为优化目标，将快速状态切换优先于运动平滑性；而 BFM-Zero 的运动模仿判别器损失则使策略偏向从 mocap 数据集中提取的更缓慢、渐进的运动。",{"type":39,"tag":123,"props":861,"children":865},{"alt":862,"caption":863,"src":864,":wide":128},"TeCH 真机运动跟踪示例","图 6. TeCH 在真机测试中对高动态运动实现鲁棒的运动跟踪。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-06-real-robot-motion-tracking.png",[],{"type":39,"tag":123,"props":867,"children":871},{"alt":868,"caption":869,"src":870,":wide":128},"TeCH 真机目标达示例","图 7. TeCH 以稳定的运动和流畅的行为过渡完成真机目标到达任务。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-07-real-world-goal-reaching.png",[],{"type":39,"tag":141,"props":873,"children":875},{"id":874},"随机采样运动",[876],{"type":50,"value":874},{"type":39,"tag":52,"props":878,"children":879},{},[880],{"type":50,"value":881},"我们进一步研究从隐动作空间 Z 中采样的无条件随机 rollout 的质量，如图 8 所示。我们观察到，完全随机的隐样本即可在没有任何任务引导或运动先验的情况下产生连贯、类人的人形机器人运动序列。人形 avatar 在多样平衡姿态之间平滑过渡，从深蹲到大开站位重心转移以及四肢伸展，展现出自然的人类般流畅性，避免了在正则化不足的隐表征中常见的关节痉挛或漂浮伪影。",{"type":39,"tag":123,"props":883,"children":887},{"alt":884,"caption":885,"src":886,":wide":128},"随机隐样本产生自然的人形机器人运动序列","图 8. 在没有任务监督或运动先验的情况下，随机隐样本产生自然、无伪影的人形机器人运动序列。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-08-random-latent-samples.png",[],{"type":39,"tag":141,"props":889,"children":891},{"id":890},"隐空间插值",[892],{"type":50,"value":890},{"type":39,"tag":52,"props":894,"children":895},{},[896],{"type":50,"value":897},"如图 1 右侧绿色虚线轨迹所示，在两个目标隐嵌入之间进行线性隐空间插值，可产生平滑、物理一致的人形机器人运动序列。沿插值路径的中间机器人姿态连续且渐进地演化，没有扭曲或破碎的异常运动。这验证了我们学到的隐目标空间具有有效的连续结构：嵌入空间中直接的线性插值即可映射到机器人平滑的顺序状态转移，同时支持运动跟踪和目标到达规划。",{"type":39,"tag":45,"props":899,"children":901},{"id":900},"消融实验",[902],{"type":50,"value":900},{"type":39,"tag":141,"props":904,"children":906},{"id":905},"隐空间维度",[907],{"type":50,"value":905},{"type":39,"tag":52,"props":909,"children":910},{},[911],{"type":50,"value":912},"无监督行为基础模型的性能在很大程度上取决于隐空间 Z⊆R^D 的表征能力：当 D 过小时，任务嵌入和 successor feature 可能无法区分不同技能；当 D 过大时，优化变得更困难、样本效率下降，并可能因冗余表征和估计方差增大而出现训练不稳定。",{"type":39,"tag":52,"props":914,"children":915},{},[916,918,923],{"type":50,"value":917},"为分析该超参数的影响，我们固定其他所有训练设置（网络架构、学习率、域随机化和判别器奖励权重），在 Unitree G1 仿真环境中评估 BFM-Zero 和 TeCH。我们扫描隐维度 D∈{32, 64, 128, 256, 384, 512, 1024}，并以 LAFAN1 数据集上的 Earth Mover's Distance（EMD）",{"type":39,"tag":73,"props":919,"children":920},{},[921],{"type":50,"value":922},"31",{"type":50,"value":924}," 作为动作分布匹配的指标（越低越好，表示与参考运动分布更接近）。",{"type":39,"tag":52,"props":926,"children":927},{},[928],{"type":50,"value":929},"如图 9 所示，两种方法的 EMD 都随 D 增大而下降，表明动作分布拟合改善，但收益递减。性能在 D≥256 时基本饱和。在所有设置下，TeCH 始终取得低于 BFM-Zero 的 EMD，表明在相同容量下它能更准确地捕捉目标运动分布。当 D 足够大时，采用 successor feature 分解的 BFM-Zero 也能达到相当的性能。综合考虑性能、计算成本和训练稳定性之间的权衡，我们在所有主实验中采用 D=256 作为默认隐维度。",{"type":39,"tag":123,"props":931,"children":935},{"alt":932,"caption":933,"src":934},"不同隐维度下的 EMD 对比","图 9. BFM-Zero 与 TeCH 在不同隐维度 D 下的 EMD 对比。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-09-latent-dimension-ablation.png",[],{"type":39,"tag":141,"props":937,"children":939},{"id":938},"tech-编码器学习率",[940],{"type":50,"value":941},"TeCH 编码器学习率",{"type":39,"tag":52,"props":943,"children":944},{},[945],{"type":50,"value":946},"对 TeCH 中的编码器学习而言，过大的学习率可能导致隐几何不稳定，而过小的学习率可能导致进度信号稀疏和策略更新滞后。为研究其对运动分布拟合的影响，我们固定 D=256 及其他所有超参数，扫描编码器学习率 lr∈{5×10^{−5}, 3×10^{−6}, 10^{−5}, 3×10^{−6}, 10^{−6}, 8×10^{−7}, 10^{−9}}。我们报告训练过程中的 EMD（越低越好）。",{"type":39,"tag":52,"props":948,"children":949},{},[950],{"type":50,"value":951},"如图 10 所示，较大的学习率（lr≈3×10^{−5}）使 EMD 在早期迅速下降，但很快在较高水平饱和且波动较大，表明时间距离结构没有得到充分精炼。较小的学习率（3×10^{−6} 和 10^{−6}）带来更平滑的收敛和更好的性能，在大约 100M 步后达到稳定区间，但仍属次优。最佳性能出现在 lr=8×10^{−7}，在标准训练预算内取得最低 EMD。进一步降至 10^{−9} 可缓慢改善性能，但显著拖慢收敛。总体而言，收敛速度与最终性能之间存在明显权衡，我们在所有实验中采用 lr=8×10^{−7} 作为默认设置。",{"type":39,"tag":123,"props":953,"children":957},{"alt":954,"caption":955,"src":956},"TeCH 编码器学习率消融曲线","图 10. TeCH 编码器学习率消融：不同 lr 值下训练步数对应的 EMD。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-10-encoder-learning-rate-ablation.png",[],{"type":39,"tag":141,"props":959,"children":961},{"id":960},"在线隐目标更新频率",[962],{"type":50,"value":960},{"type":39,"tag":52,"props":964,"children":965},{},[966],{"type":50,"value":967},"除编码器学习率外，TeCH 还依赖于在线交互期间周期性刷新隐目标 z（对应 update-z-every 超参数）。该间隔决定一个被采样的隐变量在重新采样之前保留多少环境步。过短的间隔可能削弱长时序一致性，而过长的间隔可能降低技能多样性并使 replay 数据在隐空间过度集中。我们固定 D=256 和上述最优学习率，扫描 update-z-every∈{5, 10, 50, 100, 200}。",{"type":39,"tag":52,"props":969,"children":970},{},[971],{"type":50,"value":972},"如图 11 所示，所有曲线在早期迅速下降，并在约 40M 步后开始分化。每 5 或 10 步更新一次 z 带来最快的收敛和最平滑的曲线，在 110M 步时取得最低 EMD（约 0.70）。update-z-every=50 略差，最终 EMD 约 0.78；而 100 和 200 分别为 0.81 与 0.83–0.85。总体而言，更频繁的 z 重采样通过在并行环境中增加行为覆盖同时保持稳定的时间表征，改善了动作分布拟合。然而，当间隔从 50 增加到 100 和 200 时，性能增益饱和。",{"type":39,"tag":52,"props":974,"children":975},{},[976],{"type":50,"value":977},"update-z-every=5 与 =10 之间的差异在 80M 步后变得可忽略。综合所有消融结果，我们在主实验中采用 lr=8×10^{−7} 与 update-z-every=10 作为默认设置。",{"type":39,"tag":123,"props":979,"children":983},{"alt":980,"caption":981,"src":982},"TeCH 在线隐目标更新频率消融曲线","图 11. TeCH 在线隐目标更新频率消融：不同更新间隔下训练步数对应的 EMD。","\u002Fimages\u002Foutputs\u002Ftech\u002Ffig-11-latent-goal-update-frequency.png",[],{"type":39,"tag":45,"props":985,"children":987},{"id":986},"结论",[988],{"type":50,"value":986},{"type":39,"tag":52,"props":990,"children":991},{},[992],{"type":50,"value":993},"本文提出了一种基于对比学习 TLDR 框架构建的新型无监督人形机器人控制方法。与依赖严格线性 MDP 假设的现有 FB 表征不同，TeCH 直接在隐空间中建模时间可达性，产生稳定、密集的奖励信号并支持有效的 off-policy 训练。结合一系列实用的训练策略（包括非对称训练、域随机化、辅助目标与风格正则化），本方法很好地解决了部分可观测性、sim-to-real 鸿沟以及不稳定或异常运动等问题。大量仿真和物理机器人实验验证了本方法以高数据效率和强泛化能力实现高精度零样本跟踪与目标到达，展现出在真实人形机器人部署方面的巨大潜力。",{"type":39,"tag":45,"props":995,"children":997},{"id":996},"参考文献",[998],{"type":50,"value":996},{"type":39,"tag":1000,"props":1001,"children":1002},"paper-references",{},[1003],{"type":39,"tag":412,"props":1004,"children":1005},{},[1006,1020,1033,1038,1043,1048,1060,1073,1085,1090,1095,1100,1105,1110,1115,1128,1140,1152,1164,1177,1189,1194,1199,1211,1224,1229,1241,1254,1259,1264,1269,1274,1279,1291,1296,1301,1306,1318,1323,1328,1340,1345,1357],{"type":39,"tag":416,"props":1007,"children":1008},{},[1009,1011,1018],{"type":50,"value":1010},"Junik Bae, Kwanyoung Park, and Youngwoon Lee. Tldr: Unsupervised goal-conditioned rl via temporal distanceaware representations, 2024. URL ",{"type":39,"tag":672,"props":1012,"children":1016},{"href":1013,"rel":1014},"https:\u002F\u002Farxiv.org\u002Fabs\u002F",[1015],"nofollow",[1017],{"type":50,"value":1013},{"type":50,"value":1019}," 2407.08464.",{"type":39,"tag":416,"props":1021,"children":1022},{},[1023,1025,1031],{"type":50,"value":1024},"Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation, 2018. URL ",{"type":39,"tag":672,"props":1026,"children":1029},{"href":1027,"rel":1028},"https:\u002F\u002Farxiv.org\u002Fabs\u002F1810.12894",[1015],[1030],{"type":50,"value":1027},{"type":50,"value":1032},".",{"type":39,"tag":416,"props":1034,"children":1035},{},[1036],{"type":50,"value":1037},"Zixuan Chen, Xialin He, Yen-Jen Wang, Qiayuan Liao, Yanjie Ze, Zhongyu Li, S. Shankar Sastry, Jiajun Wu, Koushil Sreenath, Saurabh Gupta, and Xue Bin Peng. Learning smooth humanoid locomotion through lipschitz-constrained policies.CoRR, abs\u002F2410.11825, 2024.",{"type":39,"tag":416,"props":1039,"children":1040},{},[1041],{"type":50,"value":1042},"Zixuan Chen, Mazeyu Ji, Xuxin Cheng, Xuanbin Peng, Xue Bin Peng, and Xiaolong Wang. GMT: general motion tracking for humanoid whole-body control.CoRR, abs\u002F2506.14770, 2025.",{"type":39,"tag":416,"props":1044,"children":1045},{},[1046],{"type":50,"value":1047},"Xuxin Cheng, Yandong Ji, Junming Chen, Ruihan Yang, Ge Yang, and Xiaolong Wang. Expressive wholebody control for humanoid robots.arXiv preprint arXiv:2402.16796, 2024.",{"type":39,"tag":416,"props":1049,"children":1050},{},[1051,1053,1059],{"type":50,"value":1052},"Pranay Dugar, Aayam Shrestha, Fangzhou Yu, Bart van Marum, and Alan Fern. Learning multi-modal wholebody control for real-world humanoid robots, 2025. URL ",{"type":39,"tag":672,"props":1054,"children":1057},{"href":1055,"rel":1056},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2408.07295",[1015],[1058],{"type":50,"value":1055},{"type":50,"value":1032},{"type":39,"tag":416,"props":1061,"children":1062},{},[1063,1065,1071],{"type":50,"value":1064},"Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function, 2018. URL ",{"type":39,"tag":672,"props":1066,"children":1069},{"href":1067,"rel":1068},"https:\u002F\u002Farxiv.org\u002F",[1015],[1070],{"type":50,"value":1067},{"type":50,"value":1072}," abs\u002F1802.06070.",{"type":39,"tag":416,"props":1074,"children":1075},{},[1076,1078,1083],{"type":50,"value":1077},"Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, and Chelsea Finn. Humanplus: Humanoid shadowing and imitation from humans, 2024. URL ",{"type":39,"tag":672,"props":1079,"children":1081},{"href":1013,"rel":1080},[1015],[1082],{"type":50,"value":1013},{"type":50,"value":1084}," 2406.10454.",{"type":39,"tag":416,"props":1086,"children":1087},{},[1088],{"type":50,"value":1089},"Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational intrinsic control, 2016. URL https: \u002F\u002Farxiv.org\u002Fabs\u002F1611.07507.",{"type":39,"tag":416,"props":1091,"children":1092},{},[1093],{"type":50,"value":1094},"F ´elix G. Harvey, Mike Yurick, Derek Nowrouzezahrai, and Christopher J. Pal. Robust motion in-betweening. ACM Trans. Graph., 39(4):60, 2020.",{"type":39,"tag":416,"props":1096,"children":1097},{},[1098],{"type":50,"value":1099},"Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Learning humanto-humanoid real-time whole-body teleoperation.arXiv preprint arXiv:2403.04436, 2024.",{"type":39,"tag":416,"props":1101,"children":1102},{},[1103],{"type":50,"value":1104},"Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, and Yuke Zhu. HOVER: versatile neural whole-body controller for humanoid robots. CoRR, abs\u002F2410.21229, 2024.",{"type":39,"tag":416,"props":1106,"children":1107},{},[1108],{"type":50,"value":1109},"Tairan He, Jiawei Gao, Wenli Xiao, Yuanhang Zhang, Zi Wang, Jiashun Wang, Zhengyi Luo, Guanqi He, Nikhil Sobanbab, Chaoyi Pan, Zeji Yi, Guannan Qu, Kris Kitani, Jessica K. Hodgins, Linxi Fan, Yuke Zhu, Changliu Liu, and Guanya Shi. ASAP: aligning simulation and realworld physics for learning agile humanoid whole-body skills.CoRR, abs\u002F2502.01143, 2025.",{"type":39,"tag":416,"props":1111,"children":1112},{},[1113],{"type":50,"value":1114},"Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris M Kitani, Changliu Liu, and Guanya Shi. Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning. InConference on Robot Learning, pages 1516–1540. PMLR, 2025.",{"type":39,"tag":416,"props":1116,"children":1117},{},[1118,1120,1126],{"type":50,"value":1119},"Xialin He, Runpei Dong, Zixuan Chen, and Saurabh Gupta. Learning getting-up policies for real-world humanoid robots, 2025. URL ",{"type":39,"tag":672,"props":1121,"children":1124},{"href":1122,"rel":1123},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2502",[1015],[1125],{"type":50,"value":1122},{"type":50,"value":1127},". 12152.",{"type":39,"tag":416,"props":1129,"children":1130},{},[1131,1133,1139],{"type":50,"value":1132},"Jongmin Kim, Youngwoon Lee, Pieter Abbeel, and Guanya Shi. Variational curriculum reinforcement learning for unsupervised discovery of skills, 2023. URL ",{"type":39,"tag":672,"props":1134,"children":1137},{"href":1135,"rel":1136},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.16342",[1015],[1138],{"type":50,"value":1135},{"type":50,"value":1032},{"type":39,"tag":416,"props":1141,"children":1142},{},[1143,1145,1151],{"type":50,"value":1144},"Lisa Lee, Benjamin Eysenbach, Emilio Parisotto, Eric Xing, Sergey Levine, and Ruslan Salakhutdinov. Efficient exploration via state marginal matching, 2020. URL ",{"type":39,"tag":672,"props":1146,"children":1149},{"href":1147,"rel":1148},"https:\u002F\u002Farxiv.org\u002Fabs\u002F1906.05274",[1015],[1150],{"type":50,"value":1147},{"type":50,"value":1032},{"type":39,"tag":416,"props":1153,"children":1154},{},[1155,1157,1163],{"type":50,"value":1156},"Yitang Li, Zhengyi Luo, Tonghe Zhang, Cunxi Dai, Anssi Kanervisto, Andrea Tirinzoni, Haoyang Weng, Kris Kitani, Mateusz Guzek, Ahmed Touati, Alessandro Lazaric, Matteo Pirotta, and Guanya Shi. Bfm-zero: A promptable behavioral foundation model for humanoid control using unsupervised reinforcement learning, 2025. URL ",{"type":39,"tag":672,"props":1158,"children":1161},{"href":1159,"rel":1160},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2511.04131",[1015],[1162],{"type":50,"value":1159},{"type":50,"value":1032},{"type":39,"tag":416,"props":1165,"children":1166},{},[1167,1169,1175],{"type":50,"value":1168},"Qiayuan Liao, Takara E. Truong, Xiaoyu Huang, Guy Tevet, Koushil Sreenath, and C. Karen Liu. Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion, 2025. URL ",{"type":39,"tag":672,"props":1170,"children":1173},{"href":1171,"rel":1172},"https:\u002F\u002Farxiv",[1015],[1174],{"type":50,"value":1171},{"type":50,"value":1176},". org\u002Fabs\u002F2508.08241.",{"type":39,"tag":416,"props":1178,"children":1179},{},[1180,1182,1188],{"type":50,"value":1181},"Hao Liu and Pieter Abbeel. Cic: Contrastive intrinsic control for unsupervised skill discovery, 2022. URL ",{"type":39,"tag":672,"props":1183,"children":1186},{"href":1184,"rel":1185},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2202.00161",[1015],[1187],{"type":50,"value":1184},{"type":50,"value":1032},{"type":39,"tag":416,"props":1190,"children":1191},{},[1192],{"type":50,"value":1193},"Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, and Weipeng Xu. Perpetual humanoid control for real-time simulated avatars. InICCV, pages 10861– 10870. IEEE, 2023.",{"type":39,"tag":416,"props":1195,"children":1196},{},[1197],{"type":50,"value":1198},"Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris M. Kitani, and Weipeng Xu. Universal humanoid motion representations for physicsbased control. InICLR. OpenReview.net, 2024.",{"type":39,"tag":416,"props":1200,"children":1201},{},[1202,1204,1210],{"type":50,"value":1203},"Zhengyi Luo, Ye Yuan, Tingwu Wang, Chenran Li, Sirui Chen, Fernando Casta ˜neda, Zi-Ang Cao, Jiefeng Li, David Minor, Qingwei Ben, Xingye Da, Runyu Ding, Cyrus Hogg, Lina Song, Edy Lim, Eugene Jeong, Tairan He, Haoru Xue, Wenli Xiao, Zi Wang, Simon Yuen, Jan Kautz, Yan Chang, Umar Iqbal, Linxi ”Jim” Fan, and Yuke Zhu. Sonic: Supersizing motion tracking for natural humanoid whole-body control, 2025. URL ",{"type":39,"tag":672,"props":1205,"children":1208},{"href":1206,"rel":1207},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2511.07820",[1015],[1209],{"type":50,"value":1206},{"type":50,"value":1032},{"type":39,"tag":416,"props":1212,"children":1213},{},[1214,1216,1222],{"type":50,"value":1215},"Ian Mason. 100STYLE: A motion capture dataset of 100 locomotion styles, 2023. URL ",{"type":39,"tag":672,"props":1217,"children":1220},{"href":1218,"rel":1219},"https:\u002F\u002Fwww.ianxmason",[1015],[1221],{"type":50,"value":1218},{"type":50,"value":1223},". com\u002F100style\u002F.",{"type":39,"tag":416,"props":1225,"children":1226},{},[1227],{"type":50,"value":1228},"Mayank Mittal, Calvin Yu, Qinxi Yu, Jingzhou Liu, Nikita Rudin, David Hoeller, Jia Lin Yuan, Ritvik Singh, Yunrong Guo, Hammad Mazhar, Ajay Mandlekar, Buck Babich, Gavriel State, Marco Hutter, and Animesh Garg. Orbit: A unified simulation framework for interactive robot learning environments.IEEE Robotics Autom. Lett., 8(6):3740–3747, 2023.",{"type":39,"tag":416,"props":1230,"children":1231},{},[1232,1234,1239],{"type":50,"value":1233},"Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. Curiosity-driven exploration by selfsupervised prediction, 2017. URL ",{"type":39,"tag":672,"props":1235,"children":1237},{"href":1013,"rel":1236},[1015],[1238],{"type":50,"value":1013},{"type":50,"value":1240}," 1705.05363.",{"type":39,"tag":416,"props":1242,"children":1243},{},[1244,1246,1252],{"type":50,"value":1245},"Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Sim-to-real transfer of robotic control with dynamics randomization.CoRR, abs\u002F1710.06537, 2017. URL ",{"type":39,"tag":672,"props":1247,"children":1250},{"href":1248,"rel":1249},"http:\u002F\u002Farxiv.org\u002Fabs\u002F1710",[1015],[1251],{"type":50,"value":1248},{"type":50,"value":1253},". 06537.",{"type":39,"tag":416,"props":1255,"children":1256},{},[1257],{"type":50,"value":1258},"Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. Deepmimic: example-guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph., 37(4):143, 2018.",{"type":39,"tag":416,"props":1260,"children":1261},{},[1262],{"type":50,"value":1263},"Ilija Radosavovic, Tete Xiao, Bike Zhang, Trevor Darrell, Jitendra Malik, and Koushil Sreenath. Real-world humanoid locomotion with reinforcement learning.Sci. Robotics, 9(89), 2024.",{"type":39,"tag":416,"props":1265,"children":1266},{},[1267],{"type":50,"value":1268},"Ilija Radosavovic, Bike Zhang, Baifeng Shi, Jathushan Rajasegaran, Sarthak Kamat, Trevor Darrell, Koushil Sreenath, and Jitendra Malik. Humanoid locomotion as next token prediction.CoRR, abs\u002F2402.19469, 2024.",{"type":39,"tag":416,"props":1270,"children":1271},{},[1272],{"type":50,"value":1273},"Yossi Rubner, Carlo Tomasi, and Leonidas J. Guibas. The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision, 40(2):99–121, 2000.",{"type":39,"tag":416,"props":1275,"children":1276},{},[1277],{"type":50,"value":1278},"Younggyo Seo, Carmelo Sferrazza, Haoran Geng, Michal Nauman, Zhao-Heng Yin, and Pieter Abbeel. Fasttd3: Simple, fast, and capable reinforcement learning for humanoid control.CoRR, abs\u002F2505.22642, 2025.",{"type":39,"tag":416,"props":1280,"children":1281},{},[1282,1284,1290],{"type":50,"value":1283},"Agon Serifi et al. Vmp: Versatile motion priors for robustly tracking motion on physical characters.Computer Graphics F orum, 43(8), 2024. doi: 10.1111\u002Fcgf.15175. URL ",{"type":39,"tag":672,"props":1285,"children":1288},{"href":1286,"rel":1287},"https:\u002F\u002Fdoi.org\u002F10.1111\u002Fcgf.15175",[1015],[1289],{"type":50,"value":1286},{"type":50,"value":1032},{"type":39,"tag":416,"props":1292,"children":1293},{},[1294],{"type":50,"value":1295},"Chen Tessler, Yunrong Guo, Ofir Nabati, Gal Chechik, and Xue Bin Peng. Maskedmimic: Unified physics-based character control through masked motion inpainting. ACM Trans. Graph., 43(6):209:1–209:21, 2024.",{"type":39,"tag":416,"props":1297,"children":1298},{},[1299],{"type":50,"value":1300},"Ahmed Touati and Yann Ollivier. Learning one representation to optimize all rewards. InNeurIPS, pages 13–23, 2021.",{"type":39,"tag":416,"props":1302,"children":1303},{},[1304],{"type":50,"value":1305},"Ahmed Touati and Yann Ollivier. Learning one representation to optimize all rewards, 2021. URL https: \u002F\u002Farxiv.org\u002Fabs\u002F2103.07945.",{"type":39,"tag":416,"props":1307,"children":1308},{},[1309,1311,1317],{"type":50,"value":1310},"Denis Yarats, Rob Fergus, Alessandro Lazaric, and Lerrel Pinto. Reinforcement learning with prototypical representations, 2021. URL ",{"type":39,"tag":672,"props":1312,"children":1315},{"href":1313,"rel":1314},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2102.11271",[1015],[1316],{"type":50,"value":1313},{"type":50,"value":1032},{"type":39,"tag":416,"props":1319,"children":1320},{},[1321],{"type":50,"value":1322},"Kangning Yin, Weishuai Zeng, Ke Fan, Zirui Wang, Qiang Zhang, Zheng Tian, Jingbo Wang, Jiangmiao Pang, and Weinan Zhang. Unitracker: Learning universal whole-body motion tracker for humanoid robots.CoRR, abs\u002F2507.07356, 2025.",{"type":39,"tag":416,"props":1324,"children":1325},{},[1326],{"type":50,"value":1327},"Kevin Zakka, Baruch Tabanpour, Qiayuan Liao, Mustafa Haiderbhai, Samuel Holt, Jing Yuan Luo, Arthur Allshire, Erik Frey, Koushil Sreenath, Lueder A. Kahrs, Carmelo Sferrazza, Yuval Tassa, and Pieter Abbeel. Mujoco playground.CoRR, abs\u002F2502.08844, 2025.",{"type":39,"tag":416,"props":1329,"children":1330},{},[1331,1333,1339],{"type":50,"value":1332},"Weishuai Zeng, Shunlin Lu, Kangning Yin, Xiaojie Niu, Minyue Dai, Jingbo Wang, and Jiangmiao Pang. Behavior foundation model for humanoid robots, 2025. URL ",{"type":39,"tag":672,"props":1334,"children":1337},{"href":1335,"rel":1336},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.13780",[1015],[1338],{"type":50,"value":1335},{"type":50,"value":1032},{"type":39,"tag":416,"props":1341,"children":1342},{},[1343],{"type":50,"value":1344},"Zhikai Zhang, Chao Chen, Han Xue, Jilong Wang, Sikai Liang, Yun Liu, Zongzhang Zhang, He Wang, and Li Yi. Unleashing humanoid reaching potential via real-worldready skill space.CoRR, abs\u002F2505.10918, 2025.",{"type":39,"tag":416,"props":1346,"children":1347},{},[1348,1350,1356],{"type":50,"value":1349},"Zhikai Zhang, Jun Guo, Chao Chen, Jilong Wang, Chenghuai Lin, Yunrui Lian, Han Xue, Zhenrong Wang, Maoqi Liu, Jiangran Lyu, Huaping Liu, He Wang, and Li Yi. Track any motions under any disturbances, 2025. URL ",{"type":39,"tag":672,"props":1351,"children":1354},{"href":1352,"rel":1353},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.13833",[1015],[1355],{"type":50,"value":1352},{"type":50,"value":1032},{"type":39,"tag":416,"props":1358,"children":1359},{},[1360,1362,1368],{"type":50,"value":1361},". G1, 2025. URL ",{"type":39,"tag":672,"props":1363,"children":1366},{"href":1364,"rel":1365},"https:\u002F\u002Fwww.unitree.com\u002Fcn\u002Fg1\u002F",[1015],[1367],{"type":50,"value":1364},{"type":50,"value":1032},{"type":39,"tag":45,"props":1370,"children":1372},{"id":1371},"相关资源",[1373],{"type":50,"value":1371},{"type":39,"tag":1375,"props":1376,"children":1381},"paper-download",{"buttonLabel":1377,"format":1378,"href":30,"size":1379,"title":1380},"下载文档","PDF","6.0 MB","TECH.pdf",[],{"title":34,"searchDepth":1383,"depth":1383,"links":1384},2,[1385,1386,1387,1392,1399,1400,1407,1412,1413,1414],{"id":47,"depth":1383,"text":47},{"id":59,"depth":1383,"text":59},{"id":137,"depth":1383,"text":137,"children":1388},[1389,1391],{"id":143,"depth":1390,"text":143},3,{"id":181,"depth":1390,"text":181},{"id":275,"depth":1383,"text":278,"children":1393},[1394,1395,1396,1397,1398],{"id":302,"depth":1390,"text":302},{"id":322,"depth":1390,"text":322},{"id":352,"depth":1390,"text":352},{"id":375,"depth":1390,"text":378},{"id":392,"depth":1390,"text":392},{"id":476,"depth":1383,"text":479},{"id":508,"depth":1383,"text":508,"children":1401},[1402,1403,1404,1405,1406],{"id":513,"depth":1390,"text":513},{"id":544,"depth":1390,"text":544},{"id":840,"depth":1390,"text":843},{"id":874,"depth":1390,"text":874},{"id":890,"depth":1390,"text":890},{"id":900,"depth":1383,"text":900,"children":1408},[1409,1410,1411],{"id":905,"depth":1390,"text":905},{"id":938,"depth":1390,"text":941},{"id":960,"depth":1390,"text":960},{"id":986,"depth":1383,"text":986},{"id":996,"depth":1383,"text":996},{"id":1371,"depth":1383,"text":1371}]