Abstract
Force-aware manipulation typically relies on specialized force or tactile sensors. We show that force-aware manipulation can instead be achieved through visual force prediction from the deformation of a compliant Fin Ray gripper. Our approach trains two models. First, we train a visual force estimator on calibration data and use it to annotate task demonstrations with force estimates. Second, we train an action--force proposal policy on these force-augmented demonstrations to jointly generate candidate robot actions and their associated forces. At test time, we sample candidate actions and the forces they are expected to produce, then execute the action whose predicted force is closest to a target from the demonstrations. We evaluate our approach on berry picking, empty-can grasping, in-hand reorientation, and plug insertion. Our results show that visual force prediction can guide inference-time action selection for contact-rich manipulation without requiring force or tactile sensors at deployment.
Keywords
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Chen, H., Wu, F., Ma, Y., Mete, M., Ye, P., Shen, J., Zhu, C., Ruggeri, A., Cheung, K., Mao, J., Adelson, E., Wu, J., Howe, R. D., & Du, Y. (2026). Robot Learning with Visual Predicted Force. https://omanscience.com/en/articles/robot-learning-with-visual-predicted-force
MLA 9
Chen, Haonan, et al. "Robot Learning with Visual Predicted Force." https://omanscience.com/en/articles/robot-learning-with-visual-predicted-force.
Chicago (author–date)
Chen, Haonan, Feiyang Wu, Yuxiang Ma, Mustafa Mete, Pengfei Ye, Junxuan Shen, Cheng Zhu, Aurora Ruggeri, Kelvin Cheung, Jiayuan Mao, Edward Adelson, Jiajun Wu, Robert D. Howe, and Yilun Du. 2026. "Robot Learning with Visual Predicted Force." https://omanscience.com/en/articles/robot-learning-with-visual-predicted-force.
Harvard
Chen, H., Wu, F., Ma, Y., Mete, M., Ye, P., Shen, J., Zhu, C., Ruggeri, A., Cheung, K., Mao, J., Adelson, E., Wu, J., Howe, R. D. and Du, Y. (2026) 'Robot Learning with Visual Predicted Force', Available at: https://omanscience.com/en/articles/robot-learning-with-visual-predicted-force.
Vancouver
Chen H, Wu F, Ma Y, Mete M, Ye P, Shen J, et al. Robot Learning with Visual Predicted Force. https://omanscience.com/en/articles/robot-learning-with-visual-predicted-force
IEEE
H. Chen, F. Wu, Y. Ma, M. Mete, P. Ye, J. Shen, C. Zhu, A. Ruggeri, K. Cheung, J. Mao, E. Adelson, J. Wu, R. D. Howe, and Y. Du, "Robot Learning with Visual Predicted Force," https://omanscience.com/en/articles/robot-learning-with-visual-predicted-force.