Dexterous manipulation: research into runnable tasks#

The updated evidence ledger supersedes the source-entry snapshot below:38 paper identities and19 repository entrypoints, with review depth, reuse/defer decisions and explicit unknowns. Optional execution issues: #52, #53.

The gap is task and mechanism coverage, not only engine count. Apple-stem holding cannot stand in for in-hand rotation, finger gaiting, handover, tool contact or deformable manipulation.

This page turns the six-layer framework into tasks. The refreshed evidence ledger covers38 paper identities and19 repository entrypoints, distinguishing targeted source review from abstract-only screening. Deferred implementations retain unresolved solver, input and asset terms; none is presented as newly reproduced.

Six experimental contracts#

Layer	Question	Required evidence
Task/contact	Hold, slide, roll, change fingers or release?	Contact phases, first success, sustained success and failure time
Embodiment/actuation	Did measured state execute the command?	Coupling, actual/target joints, saturation, rates, delay, wrist constraints
Physics	Which error changes behavior?	Intrusion, supporting force, slip onset, temporal/spatial sensitivity
Sensing	Which inputs exist at deployment?	Privilege audit, occlusion, noise, calibration, tactile history/reset
Data/policy	Does added data cover new contact strategies?	Generation/execution/validity rates, filtered failures, splits, total cost
Calibration/transfer	Does simulation predict real failure?	Independent measurements, fitting error, held-out objects and rankings

First reuse decisions#

These are proposed DexLab experiments, not upstream matched comparisons or claims of implemented support.

Work	Reuse	Falsifiable minimal experiment	Boundary	Task
ManiSkill 1/2/3	Task cards, generalization splits, native execution and action replay	PushCube integration → PickCube hold/release → in-hand rotation; verify reset and failure separately	MS2 MPM is not MS3 thin cloth; start with upstream robots	#47 / #48
BODex	Grasp feasibility and loading protocol	Apply fixed directional loads to the same grasp and record first loss	Separate noncommercial and cuRobo terms; no mass-generation pipeline copy	#10
DexGraspBench	Grasp evaluation and control sensitivity	Freeze objects/grasps; compare retention under mass/gain changes	Do not mix historical baseline and main; not in-hand skill evaluation	#3 / #10
DexMachina	Embodiment and functional-retargeting design	Hold object goal fixed; vary wrist constraints and drive limits separately	Audit customized dependencies; floating-wrist results are not arm performance	#48
Taccel	Rigid-soft contact and tactile cost breakdown	Indent, shear and unload; compare net force, deformation and stage costs	Validate gel and rigid solver separately; defer primary ranking	#53
TacEx	Tactile interfaces and observation fidelity	Compare net force, geometric depth and tactile images for one trajectory	README pins older Isaac versions and preview status; reference only under stable policy	暂缓 / deferred
HydroShear	History-dependent tactile observation and reset	Compare direct arrival, slide-and-return, and detach/recontact at one final pose	Qualify observation model first; do not claim replacement rigid dynamics	#53
DexMimicGen	Demonstration augmentation and filtering audit	Small demo batch: generation, execution, physical validity and contact coverage	Separate code/data terms; more trajectories do not prove new contact strategies	#52
DexGarmentLab	Garment task decomposition and deformable-contact failure	Qualify single-layer grasp/release and self-contact before folding	Older Isaac dependencies are not a ready port; appearance is not nonpenetration	#28 / #32
DexScrew	Skill versus real feedback boundary	Declare pre-inserted start; record rotation progress, load and termination	Historical IsaacGym reference; progress ratio is not success rate	#6 / future tool task

Task support matrix#

Planned/reference entries are not supported capabilities. Every implementation separately qualifies the latest stable release and actual native solver.

Family	Current status	Next acceptance
Apple-stem holding	Historical native MuJoCo/SuperDex evidence	Keep candidate and historical profiles separate
Cloth grasp/folding	Diagnostics exist; overall physical validity unresolved	#32 collision repair; animation is not acceptance
ManiSkill cube push/pick	Planned; no runnable support yet	#47 native execution, recording, reset isolation and independent scoring
Multifinger in-hand rotation	Planned; no runnable support yet	#48 rotation, drops, contact switching and hand-only-rotation negatives
Insertion/tools/handover	Research candidates	Define starts, freedoms, loads and failures before implementation
Tactile history/data/garments	References and candidates	Qualify small probes before training or real-world claims

Bringing ManiSkill into code#

ManiSkill 1 object generalization, ManiSkill2 task diversity and ManiSkill3 parallel task management offer different reference dimensions. Start with upstream-supported robots and native tasks; record ManiSkill, SAPIEN and actual PhysX identities. Reuse current UniLab experiment management instead of adding another scheduler.

The official task documentation places soft-body tasks in MS2, not fully integrated in MS3. Volumetric MPM is also not the current triangle-based thin cloth.

M1 / #47: PushCube checks integration/reset, then PickCube checks grasp, support, hold and release. Record upstream reward/success separately from independent physical scoring; unavailable contacts remain missing data.
M2 / #48: Verify the actual registered rotation environment and hand. Reproduce an available controller/trajectory first; unavailable policy is a blocker, not permission to label random actions successful. A Wuji port needs separate drive/asset qualification.
Frozen evaluation: Separate development from ten preregistered held-out cases. Retain every failure. Action replay differs from state injection; equal seeds do not prove matched initial states. Record continuous contact close-ups.

Experiments prefer a qualified local machine; remote execution remains optional. Verify enforced memory, CPU, process and runtime limits with experiment swap disabled, preserve desktop headroom, and store bulk outputs on the configured data volume. Qualify isolation on one GPU before scaling. Report preparation, stepping, rendering and end-to-end costs separately; stop transfers during timing. Missing real measurements must not block simulation integration but preclude hardware-accuracy claims.

Candidate decisions and runtime admission#

The ledger records a reuse or defer decision and a falsifiable probe for every candidate. Detailed runtime source and asset checks remain admission requirements. Abstract-only screening is sufficient to defer adoption, never to claim implementation equivalence.

Optional action-replay (#52) and tactile-history (#53) issues now declare dependencies, at most10 cases/30min per batch and at most3 batches, with no training or bulk downloads. Core sequence remains #46 → #47 → #48; contact and cloth repairs keep their existing priorities.

Historical validation of published v0.17.1#

The strict bilingual site build passed. In the base setup.sh environment, the 309-test suite reported a PhysX contact-details import error because the optional UniSim adapter patch was absent; that failure log is retained. All 309 tests passed after installing adapter source verified byte-for-byte against the disclosed scripts/patches/unisim-1.7.10-physx-adapter.patch. This changes the adapter, not official physics engines. Patched-environment results are not a claim that the base installation passes the full suite. This page contains no ManiSkill runtime result.