Policy Deployment

On this page we'll be covering how to deploy your trained policy, specifically with respect to the Hello Robot Stretch. The first half will be general instructions for operating the Stretch robot, followed by specifics of how to load and run your policy.

Operating the Stretch Robot

  1. Turn on the robot and set charger to 12V supply mode (it should be in 12V AGM mode when turned off).
      • In general, keep the robot in charge, unless it's moving around and needs to be untethered.
  2. SSH into robot via the Tailscale IP (you will need to be invited to the Tailscale network).
      • We recommend using the Tailscale IP as it is static, and doesn't change over time like the local IP.
  3. Calibrate the joints of the robot (required every time the robot is powered on).
      • Place the clamp about one-third of the way from the bottom so that the lift doesn't collide during the calibration process.
      • Snap off the gripper during the calibration so that it doesn't collide with the arm during the calibration process.
      • Run the joint calibration script 
        stretch_robot_home.py
        • Here's a video of the calibration process (2x speed):

General Rules to Follow

  1. Don't move the robot, nor any of its joints, while not in “run-stop” mode.
      • There is a white button on the robot's head as shown below.
        • Robot run-stop button
      • If the light is not flashing, then the joints can be locked and pressing the button once will put it into "run-stop" mode where you can move the robot freely.
      • If the light is flashing, then the robot is in "run-stop" mode, and holding it for a few seconds will take it out. For the robot to operate, it must be taken out of "run-stop" mode.
  2. The "run-stop" button acts as an emergency stop, so if you ever see the robot doing anything unsafe, or if you need to stop it for any reason, just press the button.
  3. If you try to run some code but the robot is not moving, first check whether it's in "run-stop" mode.
  4. When you're done using the robot, follow the following steps:
      • Shut it down by running sudo shutdown now.
      • Wait a few seconds until the LiDAR on the base stops spinning.
      • Move the lift down to the clamp.
      • Turn off the robot

Deploying a Trained Policy

  1. Set up the iPhone on the robot
      • Slide the iPhone into the gripper's phone holder until the camera block hits the mount, then tighten the knob. Be sure not to tighten the knob too tight, but just enough such that the phone doesn't slide out.
        • iPhone slid into the phone mount
      • Connect the iPhone to the robot via the USB cable.
        • iPhone connected to the robot via USB
      • On the AnySense app, go into settings and set the "RGBD Streaming" mode to "USB"
      • Press record on the AnySense app to put it into "stream-ready" mode.
        • iPhone AnySense app is set to start streaming
  2. Enter the min-stretch repository
      • Clone it if it doesn't exist: git clone https://github.com/NYU-robot-learning/min-stretch.git.
  3. Run the deployment bash script 
    ./run_robot.sh
    This will open up a tmux window with 2 panes, with system python on the left and the mamba environment activated on the right.
  4. Start the robot server on the left-hand side 
    python3 start_server.py
    5. This will start streaming images from the iPhone and publishing them to a topic, and the robot will start listening for action commands.
  5. Run the policy on the right-hand side 
    python run.py
    6. This assumes you did not modify any hyperparameters during training. If you did, modify them on the robot side as well, accordingly.
      • This will download and run a pretrained RUM policy for "Door Opening"; however, this will policy will not work with the new gripper.
      • You may specify your own trained model by 
        python run.py model_weight_pth=YOUR_MODEL_PATH
      • You can transfer the policy to the robot by running a command like the one below (modify as it fits your needs) 
        rsync -avmP --include='*/' --include='checkpoint.pt' --exclude='*' imitation-in-homes/checkpoints/2025-02-20/bag_pick_up_checkpoints_folder hello-robot@ROBOT_IP:/home/hello-robot/min-stretch/imitation-in-homes/2025-02-20
        • This command assumes you are in the min-stretch repo of where your policy was trained (likely Greene). You will also need to replace the ROBOT_IP with the robot's local IP address as Tailscale is not supported on Greene.
        Note: the rsync command will copy only the latest checkpoint to the robot.
  6. Enter the deployment UI on browser as instructed. It should be at http://ROBOT_IP:7860. The rest of the instructions for running the policy should be written at the top of the UI.