From 2de0d3d18c3eef6328eb351839bb07d289b03f23 Mon Sep 17 00:00:00 2001 From: Oleg Alexandrov Date: Fri, 3 Dec 2021 10:58:25 -0800 Subject: [PATCH] Set up all needed paths before running the mapping tools --- README.md | 2 +- localization/sparse_mapping/build_map.md | 113 +++++++++++++---------- localization/sparse_mapping/readme.md | 27 ++++-- localization/sparse_mapping/theia_map.md | 5 +- 4 files changed, 90 insertions(+), 57 deletions(-) diff --git a/README.md b/README.md index 1fab0582c5..747c431e9b 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ signaling, and sound. The flight software is hosted on each Astrobee's three int uses the open-source [Robot Operating System (ROS)](https://www.ros.org/) framework as message-passing middleware. It provides a high-level [Astrobee Command API](https://nasa.github.io/astrobee/html/command_dictionary.html) for controlling the robot and has multiple operating modes. It can execute a plan (command sequence), individual operator commands (teleoperation), or commands from guest science code running onboard Astrobee. -The Astrobee Robot Software simulator, built using [ROS](https://www.ros.org/) and [Gazebo](http://gazebosim.org/), enables the flight software to be evaluated without the need for robot hardware. The supporting tools include a tool that processes ISS imagery to [build maps for Astrobee localization](https://github.com/nasa/astrobee/tree/master/localization/sparse_mapping), along with many others. +The Astrobee Robot Software simulator, built using [ROS](https://www.ros.org/) and [Gazebo](http://gazebosim.org/), enables the flight software to be evaluated without the need for robot hardware. The supporting tools include a tool that processes ISS imagery to [build maps for Astrobee localization](https://nasa.github.io/astrobee/html/sparsemapping.html), along with many others. Released separately, the Astrobee ground data system (GDS) includes Astrobee control station software that communicates with Astrobee flight software via the Data Distribution Service (DDS) network protocol over the ISS Ku-IP space-to-ground link. The control station is written primarily in Java using the Eclipse RCP framework. Source code is in the [`astrobee_gds`](https://github.com/nasa/astrobee_gds) repository, or you can download the [binary release](https://software.nasa.gov/software/ARC-17994-1B). diff --git a/localization/sparse_mapping/build_map.md b/localization/sparse_mapping/build_map.md index 8f85d0ad5c..8a774e8fa3 100644 --- a/localization/sparse_mapping/build_map.md +++ b/localization/sparse_mapping/build_map.md @@ -4,9 +4,9 @@ Here we describe how to build a map. ## Summary -1. Reduce the number of images. +1. Set up the environment. -2. Set up the environment. +2. Reduce the number of images. 3. Build the map. @@ -16,6 +16,48 @@ Here we describe how to build a map. # Detailed explanation +## Setup the environment + +In the first step, one needs to set some environmental variables, as +follows: + + export ASTROBEE_SOURCE_PATH=$HOME/astrobee/src + export ASTROBEE_BUILD_PATH=$HOME/astrobee + export ASTROBEE_RESOURCE_DIR=$ASTROBEE_SOURCE_PATH/astrobee/resources + export ASTROBEE_CONFIG_DIR=$ASTROBEE_SOURCE_PATH/astrobee/config + export ASTROBEE_ROBOT=p4d + export ASTROBEE_WORLD=granite + +The source and build paths need to be adjusted for your particular +setup. + +Also consider setting: + + export PATH=$ASTROBEE_BUILD_PATH/devel/lib/sparse_mapping:$PATH + +to have the ``build_map`` and other related tools in your path. + +Above, ``p4d`` is the robot being used to take pictures, and the world +is the granite table. These may need to change, depending on your +goals. + +Under the hood, the following configuration files will be read: + + $ASTROBEE_CONFIG_DIR/cameras.config + +which contains the image width and height (the camera we use is +the nav cam) and + + $ASTROBEE_CONFIG_DIR/robots/$ASTROBEE_ROBOT.config + +having nav cam's intrinsics. If your camera is not the nav cam on p4d, +and none of the other available config files apply, you can just +temporarily modify the above files to reflect your camera's parameters +(without checking in your changes). + +More details on these and other environmental variables can be found +in the \subpage astrobee configuration documentation. + ## Reduce the number of images Here, we delete the images that overlap highly. (This tool is, like all @@ -55,34 +97,6 @@ images, as then the map could be of poor quality. Hence, the robot should have some translation motion (in addition to any rotation) when the data is acquired. -## Setup the environment - -In the first step, one needs to set some environmental variables, as -follows: - - export ASTROBEE_RESOURCE_DIR=$SOURCE_PATH/astrobee/resources - export ASTROBEE_CONFIG_DIR=$SOURCE_PATH/astrobee/config - export ASTROBEE_ROBOT=p4d - export ASTROBEE_WORLD=granite - -Here, `p4d` is the robot being used to take pictures, and the world is -the granite table. These may need to change, depending on your -goals. Under the hood, the following configuration files will be read: - - $ASTROBEE_CONFIG_DIR/cameras.config - -which contains the image width and height (the camera we use is -the nav cam) and - - $ASTROBEE_CONFIG_DIR/robots/$ASTROBEE_ROBOT.config - -having nav cam's intrinsics. If your camera is not the nav cam on p4d, -and none of the other available config files apply, you can just -temporarily modify the above files to reflect your camera's parameters -(without checking in your changes). - -More details on these and other environmental variables can be found -in the \subpage astrobee configuration documentation. ## Building a map @@ -156,11 +170,12 @@ consistently across multiple frames. build_map -bundle_adjustment -histogram_equalization - Adjust the initial transformations to minimize error with bundle adjustment. +Adjust the initial transformations to minimize error with bundle +adjustment. -If the options: +If the options: - -first_ba_index and -last_ba_index + -first_ba_index and -last_ba_index are specified, only cameras with indices between these (including both endpoints) will be optimized during bundle adjustment. @@ -362,8 +377,8 @@ be visualized in the ISS as follows: Open two terminals, and in each one type: - export BUILD_PATH=$HOME/astrobee - source $BUILD_PATH/devel/setup.bash + export ASTROBEE_BUILD_PATH=$HOME/astrobee + source $ASTROBEE_BUILD_PATH/devel/setup.bash The Astrobee directory above must have ``src`` and ``devel`` subdirectories, and needs to be adjusted given its location on your @@ -375,8 +390,8 @@ In the first terminal start the simulator: In the second, run: - python $SOURCE_PATH/localization/sparse_mapping/tools/view_control_points.py \ - $SOURCE_PATH/localization/sparse_mapping/iss_registration.txt + python $ASTROBEE_SOURCE_PATH/localization/sparse_mapping/tools/view_control_points.py \ + $ASTROBEE_SOURCE_PATH/localization/sparse_mapping/iss_registration.txt Go back to the simulated ISS and examine the registration points. If the Rviz display looks too cluttered, most topics can be turned off. @@ -412,9 +427,9 @@ Python command will refresh them. ### Registration in the MGTF A set of 10 registration points were measured in the MGTF with the -Total Station. They are in the file +\subpage total_station. They are in the file: - $SOURCE_PATH/localization/sparse_mapping/mgtf_registration.txt + $ASTROBEE_SOURCE_PATH/localization/sparse_mapping/mgtf_registration.txt Two of these are on the back wall, and the rest are on the metal columns on the side walls, with four on each wall. Half of the points @@ -430,7 +445,8 @@ be needed to identify them. A registered and bundle-adjusted map can be used to study how well it predicts the computed 3D locations for an independently acquired set of control points and 3D measurements. These are in the same format as -for registration. The map is not modified in any way during this step. +for registration. The map is not modified in any way during this step, +The command is: build_map -verification @@ -462,7 +478,7 @@ To test how the map may perform on the robot, do the following: ### Stage the feature counter utility (should be added to the install at one point): - scp $SOURCE_PATH/localization/marker_tracking/ros/tools/features_counter.py mlp: + scp $ASTROBEE_SOURCE_PATH/localization/marker_tracking/ros/tools/features_counter.py mlp: ### Launch the localization node on LLP @@ -477,7 +493,8 @@ this robot will give wrong results for other users. Then launch localization: ssh llp - roslaunch astrobee astrobee.launch llp:=disabled mlp:=mlp nodes:=framestore,dds_ros_bridge,localization_node + roslaunch astrobee astrobee.launch llp:=disabled mlp:=mlp \ + nodes:=framestore,dds_ros_bridge,localization_node ### Enable localization and the mapped landmark production (on MLP) @@ -532,9 +549,9 @@ computations may differ. Set up the environment in every terminal that is used. Ensure that you use the correct robot name below. - source $BUILD_PATH/devel/setup.bash - export ASTROBEE_RESOURCE_DIR=$SOURCE_PATH/astrobee/resources - export ASTROBEE_CONFIG_DIR=$SOURCE_PATH/astrobee/config + source $ASTROBEE_BUILD_PATH/devel/setup.bash + export ASTROBEE_RESOURCE_DIR=$ASTROBEE_SOURCE_PATH/astrobee/resources + export ASTROBEE_CONFIG_DIR=$ASTROBEE_SOURCE_PATH/astrobee/config export ASTROBEE_WORLD=iss export ASTROBEE_ROBOT=bumble # your robot's name may be different export ROS_MASTER_URI=http://127.0.0.1:11311/ @@ -545,9 +562,9 @@ Examine the localization configuration file: Sym link the map to test: - mkdir -p $SOURCE_PATH/astrobee/resources/maps - rm -fv $SOURCE_PATH/astrobee/resources/maps/iss.map - ln -s $(pwd)/mymap.map $SOURCE_PATH/astrobee/resources/maps/iss.map + mkdir -p $ASTROBEE_SOURCE_PATH/astrobee/resources/maps + rm -fv $ASTROBEE_SOURCE_PATH/astrobee/resources/maps/iss.map + ln -s $(pwd)/mymap.map $ASTROBEE_SOURCE_PATH/astrobee/resources/maps/iss.map Start the localization node: diff --git a/localization/sparse_mapping/readme.md b/localization/sparse_mapping/readme.md index 82af55f241..10164f5c98 100644 --- a/localization/sparse_mapping/readme.md +++ b/localization/sparse_mapping/readme.md @@ -14,8 +14,8 @@ Maps are stored as protobuf files. ## ROS node -The ROS node takes images and a map as input, and outputs visual features -detected in the image and their 3D coordinates. +The ROS node takes images and a map as input, and outputs visual +features detected in the image and their 3D coordinates. ### Inputs @@ -28,6 +28,21 @@ detected in the image and their 3D coordinates. * `/localization/mapped_landmarks/features` * `/localization/mapped_landmarks/registration` +## The environment + +It will be convenient in this document to set these environmental +variables, pointing to the source and build directories. They may need +to be adjusted given your setup. + + export ASTROBEE_SOURCE_PATH=$HOME/astrobee/src + export ASTROBEE_BUILD_PATH=$HOME/astrobee + +To access many of the localization tools, such as ``build_map``, +``merge_maps``, etc., without specifying a full path, also consider +setting: + + export PATH=$ASTROBEE_BUILD_PATH/devel/lib/sparse_mapping:$PATH + ## Tools and procedures ### Record a bag @@ -73,15 +88,15 @@ The bags created on the ISS are likely split into many smaller bags, for easy and reliability of transfer. Those can be merged into one bag as follows: - astrobee/devel/lib/localization_node/merge_bags \ + $ASTROBEE_BUILD_PATH/devel/lib/localization_node/merge_bags \ -output_bag ### Extracting images To extract images from a bag file: - astrobee/devel/lib/localization_node/extract_image_bag \ - -use_timestamp_as_image_name \ + $ASTROBEE_BUILD_PATH/devel/lib/localization_node/extract_image_bag \ + -use_timestamp_as_image_name \ -image_topic /hw/cam_nav -output_directory The above assumes that the software was built with ROS on. @@ -190,7 +205,7 @@ This functionality is implemented in the localize_cams tool. Usage: Here we use values that are different from - astrobee/config/localization.config + $ASTROBEE_SOURCE_PATH/astrobee/config/localization.config which are used for localization on the robot, since those are optimized for speed and here we want more accuracy. diff --git a/localization/sparse_mapping/theia_map.md b/localization/sparse_mapping/theia_map.md index 0817d4df25..181b16f308 100644 --- a/localization/sparse_mapping/theia_map.md +++ b/localization/sparse_mapping/theia_map.md @@ -110,7 +110,7 @@ This tool has the following command-line options: This tool is used to import a map from the NVM format, which Theia exports to. These operations are done automatically by the -``build_theia_map.py'' tool. This documentation is provided for +``build_theia_map.py`` tool. This documentation is provided for reference only. An NVM map exported by Theia (or some other SfM tool) can be saved as @@ -124,7 +124,8 @@ an Astrobee sparse map with the command:: This assumes that the images were acquired with the nav camera of the robot given by $ASTROBEE_ROBOT and undistorted with the Astrobee program ``undistort_image``. The undistorted camera parameters to use -should be as printed on the screen by ``undistort_image``. +should be as printed on the screen (and saved to disk) by +``undistort_image``. If desired to replace on importing the undistorted images with the original distorted ones, as it is usually expected of a sparse map,