Physical modeling of a bag knot in a robot learning system

This paper presents a physical model developed to find the directions of forces and moments required to open a plastic bag-which forces will contribute toward opening the knot and which forces will lock it further. The analysis is part of the implementation of a Q(λ)-learning algorithm on a robot system. The learning task is to let a fixed-arm robot observe the position of a plastic bag located on a platform, grasp it, and learn how to shake out its contents in minimum time. The physical model proves that the learned optimal bag shaking policy is consistent with the physical model and shows that there were no subjective influences. Experimental results show that the learned policy actually converged to the best policy. Note to Practitioners-This paper was motivated by the problem of developing a physical model capable of finding the directions of force and moments required to open a knotted plastic bag-which forces will contribute toward opening the knot and which forces will lock it further. The analysis was part of the implementation of a reinforcement-learning algorithm for a fixed-arm robot system called "RoboShake." The learning task was to observe the position of a bag contains suspicious items (such as biological or chemical items), grasp it, and learn the optimal sequence of motions to shake out its contents in minimum time. An extremely interesting finding was the fact that the robot's optimal policy always converged to a one that was consistent with Newton's three laws of motion. The system learned the optimal policies by interaction with the environment without using these laws as part of the system design/implementation. From a philosophical aspect, the experimental robotic platform found solutions that were consistent with Newton's laws of motion, i.e., the system was capable of explaining Newton's laws of motion without being "aware" of them, and in a way, and to "discover" the existence of the second and the third laws. Experimental systems of this kind, especially if designed with a capability to carry out new experiments independently, may help to explain, discover, or predict laws/rules (not necessarily of machines/robots) that are not known to us yet.