Tactile × Force · Research Project

TaF-Foundation

Wrench-Grounded Cross-Sensor Tactile Representations

arXiv ↗GitHub ↗◈  ModelScope ↗
A large-scale tactile and force dataset across heterogeneous tactile sensors

A shared physical reference connects tactile observations across diverse sensors.

100M+paired tactile frames
24tactile sensors
6Dforce and torque

Abstract

Tactile sensing is essential for robotic manipulation, yet heterogeneous sensor hardware fragments datasets and algorithms, limiting the reuse of tactile knowledge across platforms. We use six-dimensional interaction wrenches, comprising force and torque, as a common physical reference for connecting heterogeneous tactile observations.

We design a dedicated acquisition platform to record synchronized tactile observations and six-axis wrench measurements, collecting over 100 million paired frames from 24 tactile sensors. Building on this dataset, TaF-Foundation jointly learns tactile and wrench representations through wrench reconstruction and representation alignment. We evaluate the learned representations on wrench estimation, cross-sensor retrieval, adaptation to unseen sensors, and robotic manipulation.

Touch at scale, paired with wrench.

A synchronized acquisition platform brings diverse tactile sensors into a common physical space.

The dataset will be released upon acceptance of the paper.

Overview of sensor coverage in the tactile dataset

Sensor coverage across heterogeneous tactile sensing mechanisms.

One physical reference, many sensors.

Sensor-specific input adaptation and a shared tactile encoder learn representations grounded in measured interaction forces and torques. During inference, the tactile encoder operates without external force measurements.

Overview of data acquisition, the shared tactile encoder, and learning and evaluation

From synchronized data acquisition to a shared tactile representation and downstream evaluation.

From data to representation.

Explore the acquisition device, diverse tactile observations, and the representation learning pipeline.

Data acquisition device

Heterogeneous tactile observations

Learning pipeline