PokeFlex is a dataset containing real-world paired and annotated multimodal data of deformable objects
that includes 3D textured meshes, point clouds, RGB images, and depth maps.
+ Website might take some time to load for the first time.
Data-driven methods have shown great potential in solving challenging manipulation tasks; however, their application in the domain of deformable objects has been constrained, in part, by the lack of data. To address this lack, we propose PokeFlex, a dataset featuring real-world multimodal data that is paired and annotated. The modalities include 3D textured meshes, point clouds, RGB images, and depth maps. Such data can be leveraged for several downstream tasks, such as online 3D mesh reconstruction, and it can potentially enable underexplored applications such as the real-world deployment of traditional control methods based on mesh simulations. To deal with the challenges posed by real-world 3D mesh reconstruction, we leverage a professional volumetric capture system that allows complete 360° reconstruction. PokeFlex consists of 18 deformable objects with varying stiffness and shapes. Deformations are generated by dropping objects onto a flat surface or by poking the objects with a robot arm. Interaction wrenches and contact locations are also reported for the latter case. Using different data modalities, we demonstrated a use case for our dataset training models that, given the novelty of the multimodal nature of Pokeflex, constitute the state-of-the-art in multi-object online template-based mesh reconstruction from multimodal data, to the best of our knowledge.
Top: Mesh reconstructions of foam dice for a poking sequence shown in every third frame.
Bottom: Mesh reconstructions of plush octopus for a dropping sequence.
Superimposed representation of the proposed network architectures for ingesting the multi-modal PokeFlex data to predict deformed mesh reconstruction. Inference rates across different data modalities range from 33 Hz to 185 Hz for dense point clouds and forces, respectively. (See paper for more details).
Physical properties of objects featured in the PokeFlex dataset. Dimensions of sphere-like objects are described by their diameter (D). Cylinder-like objects are characterized by their diameter (D) and height (H). For objects with irregular or complex shapes, dimensions are provided using a bounding box defined by length (L), width (W), and height (H). Stiffness of the objects is estimated using Hooke's law and applying RANSAC for linear regression to avoid outliers. \begin{array}{lllll} \hline \textbf{Object} & \textbf{Weight[g]} & \textbf{Dimensions [cm]} & \textbf{Est. stiffness [N/m]} & \textbf{Total Frames} & \textbf{Frames in contact} \\ \hline \text{Beanbag} & 184 & \text{DxH: } 26\text{x}9 & 523 & 1084 & 363 \\ \text{Foam cylinder} & 153 & \text{DxH: } 12\text{x}29 & 250 & 990 & 407 \\ \text{Foam dice} & 140 & \text{L: } 15.5 & 748 & 1220 & 738 \\ \text{Foam half sphere} & 41 & \text{D: } 15 & 1252 & 939 & 546 \\ \text{Memory foam} & 213 & \text{LxWxH: } 17.5\text{x}8.5\text{x}7 & 395 & 958 & 282 \\ \text{Pillow} & 975 & \text{LxWxH: } 58\text{x}50\text{x}10 & 474 & 1085 & 565 \\ \text{Plush dice} & 340 & \text{L: } 22 & 149 & 1259 & 567 \\ \text{Plush moon} & 151 & \text{D: } 17 & 366 & 959 & 517 \\ \text{Plush octopus} & 130 & \text{LxWxH: } 22\text{x}22\text{x}11 & 325 & 1085 & 525 \\ \text{Plush turtle} & 194 & \text{LxWxH: } 35\text{x}30\text{x}10 & 1035 & 930 & 427 \\ \text{Plush volleyball} & 303 & \text{D: } 22 & 323 & 939 & 488 \\ \text{Sponge} & 28 & \text{LxWxH: } 22\text{x}12\text{x}6.1 & 1045 & 775 & 490 \\ \text{Toilet paper roll} & 134 & \text{DxH: } 10.5\text{x}9.5 & 2156 & 920 & 411 \\ \text{3D printed bunny} & 105 & \text{LxWxH: } 13\text{x}9\text{x}15 & 950 & 1117 & 520 \\ \text{3D printed cylinder} & 223 & \text{DxH: } 10\text{x}20 & 585 & 1020 & 574 \\ \text{3D printed heart} & 100 & \text{LxWxH: } 16\text{x}9\text{x}10 & 1198 & 940 & 444 \\ \text{3D printed pizza} & 68 & \text{LxWxH: } 18\text{x}15\text{x}3 & 884 & 958 & 360 \\ \text{3D printed pyramid} & 48 & \text{LxWxH: } 14.5\text{x}14.5\text{x}7 & 861 & 899 & 386 \\ \hline \end{array}
All 3D printed objects were printed using thermoplastic polyurethane (TPU) Filaflex Shore 60A Pro White filament on Prusa MK3S+ and Prusa XL 3D printers equipped with 0.4mm nozzles. The mechanical properties of the filament are presented in the table below. \begin{array}{lccc} \hline \textbf{Mechanical properties} & \textbf{Value} & \textbf{Unit} & \textbf{Test method according to} \\ \hline \text{Tensile strenght} & 26 & \text{MPa} & \text{DIN 53504-S2} \\ \text{Stress at 20% elongation} & 1 & \text{MPa} & \text{DIN 53504-S2} \\ \hline \end{array} The printing parameters of the 3D printed objects are summarized in the table below, where the infill used for all objects is the isotropic gyroid pattern with uniform properties and behavior in all directions. \begin{array}{lcccc} \hline \textbf{Object} & \textbf{Infill density} & \textbf{Layer thickness [mm]} & \textbf{Perimeters} & \textbf{Bottom layers} & \textbf{Top layers} \\ \hline \text{Bunny} & 10 & 0.2 & 3 & 3 & 3 \\ \text{Cylinder} & 10 & 0.15 & 2 & 3 & 3 \\ \text{Heart} & 10 & 0.2 & 3 & 3 & 3 \\ \text{Pizza} & 10 & 0.2 & 3 & 3 & 3 \\ \text{Pyramid} & 8 & 0.2 & 3 & 3 & 3 \\ \hline \end{array}
Top (Left) and bottom (Right) view of the 3D printed pyramid, with a close-up view of the gyroid infill pattern.
@article{obrist2025pokeflex,
author = {Obrist, Jan and Zamora, Miguel and Zheng, Hehui and Hinchet, Ronan and Ozdemir, Firat and Zarate, Juan and Katzschmann, Robert K. and Coros, Stelian},
title = {PokeFlex: A Real-World Dataset of Volumetirc Deformable Objects for Robotics},
journal = {Under review},
year = {2025}
url = {https://arxiv.org/pdf/2410.07688}
}