Skip to content

Commit

Permalink
Set up all needed paths before running the mapping tools
Browse files Browse the repository at this point in the history
  • Loading branch information
oleg-alexandrov committed Dec 3, 2021
1 parent af17571 commit 2de0d3d
Show file tree
Hide file tree
Showing 4 changed files with 90 additions and 57 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ signaling, and sound. The flight software is hosted on each Astrobee's three int
uses the open-source [Robot Operating System (ROS)](https://www.ros.org/) framework as message-passing
middleware. It provides a high-level [Astrobee Command API](https://nasa.github.io/astrobee/html/command_dictionary.html) for controlling the robot and has multiple operating modes. It can execute a plan (command sequence), individual operator commands (teleoperation), or commands from guest science code running onboard Astrobee.

The Astrobee Robot Software simulator, built using [ROS](https://www.ros.org/) and [Gazebo](http://gazebosim.org/), enables the flight software to be evaluated without the need for robot hardware. The supporting tools include a tool that processes ISS imagery to [build maps for Astrobee localization](https://github.com/nasa/astrobee/tree/master/localization/sparse_mapping), along with many others.
The Astrobee Robot Software simulator, built using [ROS](https://www.ros.org/) and [Gazebo](http://gazebosim.org/), enables the flight software to be evaluated without the need for robot hardware. The supporting tools include a tool that processes ISS imagery to [build maps for Astrobee localization](https://nasa.github.io/astrobee/html/sparsemapping.html), along with many others.

Released separately, the Astrobee ground data system (GDS) includes Astrobee control station software that communicates with Astrobee flight software via the Data Distribution Service (DDS) network protocol over the ISS Ku-IP space-to-ground link. The control station is written primarily in Java using the Eclipse RCP framework. Source code is in the [`astrobee_gds`](https://github.com/nasa/astrobee_gds) repository, or you can download the [binary release](https://software.nasa.gov/software/ARC-17994-1B).

Expand Down
113 changes: 65 additions & 48 deletions localization/sparse_mapping/build_map.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Here we describe how to build a map.

## Summary

1. Reduce the number of images.
1. Set up the environment.

2. Set up the environment.
2. Reduce the number of images.

3. Build the map.

Expand All @@ -16,6 +16,48 @@ Here we describe how to build a map.

# Detailed explanation

## Setup the environment

In the first step, one needs to set some environmental variables, as
follows:

export ASTROBEE_SOURCE_PATH=$HOME/astrobee/src
export ASTROBEE_BUILD_PATH=$HOME/astrobee
export ASTROBEE_RESOURCE_DIR=$ASTROBEE_SOURCE_PATH/astrobee/resources
export ASTROBEE_CONFIG_DIR=$ASTROBEE_SOURCE_PATH/astrobee/config
export ASTROBEE_ROBOT=p4d
export ASTROBEE_WORLD=granite

The source and build paths need to be adjusted for your particular
setup.

Also consider setting:

export PATH=$ASTROBEE_BUILD_PATH/devel/lib/sparse_mapping:$PATH

to have the ``build_map`` and other related tools in your path.

Above, ``p4d`` is the robot being used to take pictures, and the world
is the granite table. These may need to change, depending on your
goals.

Under the hood, the following configuration files will be read:

$ASTROBEE_CONFIG_DIR/cameras.config

which contains the image width and height (the camera we use is
the nav cam) and

$ASTROBEE_CONFIG_DIR/robots/$ASTROBEE_ROBOT.config

having nav cam's intrinsics. If your camera is not the nav cam on p4d,
and none of the other available config files apply, you can just
temporarily modify the above files to reflect your camera's parameters
(without checking in your changes).

More details on these and other environmental variables can be found
in the \subpage astrobee configuration documentation.

## Reduce the number of images

Here, we delete the images that overlap highly. (This tool is, like all
Expand Down Expand Up @@ -55,34 +97,6 @@ images, as then the map could be of poor quality. Hence, the robot
should have some translation motion (in addition to any rotation) when
the data is acquired.

## Setup the environment

In the first step, one needs to set some environmental variables, as
follows:

export ASTROBEE_RESOURCE_DIR=$SOURCE_PATH/astrobee/resources
export ASTROBEE_CONFIG_DIR=$SOURCE_PATH/astrobee/config
export ASTROBEE_ROBOT=p4d
export ASTROBEE_WORLD=granite

Here, `p4d` is the robot being used to take pictures, and the world is
the granite table. These may need to change, depending on your
goals. Under the hood, the following configuration files will be read:

$ASTROBEE_CONFIG_DIR/cameras.config

which contains the image width and height (the camera we use is
the nav cam) and

$ASTROBEE_CONFIG_DIR/robots/$ASTROBEE_ROBOT.config

having nav cam's intrinsics. If your camera is not the nav cam on p4d,
and none of the other available config files apply, you can just
temporarily modify the above files to reflect your camera's parameters
(without checking in your changes).

More details on these and other environmental variables can be found
in the \subpage astrobee configuration documentation.

## Building a map

Expand Down Expand Up @@ -156,11 +170,12 @@ consistently across multiple frames.

build_map -bundle_adjustment -histogram_equalization

Adjust the initial transformations to minimize error with bundle adjustment.
Adjust the initial transformations to minimize error with bundle
adjustment.

If the options:
If the options:

-first_ba_index and -last_ba_index
-first_ba_index and -last_ba_index

are specified, only cameras with indices between these (including both
endpoints) will be optimized during bundle adjustment.
Expand Down Expand Up @@ -362,8 +377,8 @@ be visualized in the ISS as follows:

Open two terminals, and in each one type:

export BUILD_PATH=$HOME/astrobee
source $BUILD_PATH/devel/setup.bash
export ASTROBEE_BUILD_PATH=$HOME/astrobee
source $ASTROBEE_BUILD_PATH/devel/setup.bash

The Astrobee directory above must have ``src`` and ``devel``
subdirectories, and needs to be adjusted given its location on your
Expand All @@ -375,8 +390,8 @@ In the first terminal start the simulator:

In the second, run:

python $SOURCE_PATH/localization/sparse_mapping/tools/view_control_points.py \
$SOURCE_PATH/localization/sparse_mapping/iss_registration.txt
python $ASTROBEE_SOURCE_PATH/localization/sparse_mapping/tools/view_control_points.py \
$ASTROBEE_SOURCE_PATH/localization/sparse_mapping/iss_registration.txt

Go back to the simulated ISS and examine the registration points.
If the Rviz display looks too cluttered, most topics can be turned off.
Expand Down Expand Up @@ -412,9 +427,9 @@ Python command will refresh them.
### Registration in the MGTF

A set of 10 registration points were measured in the MGTF with the
Total Station. They are in the file
\subpage total_station. They are in the file:

$SOURCE_PATH/localization/sparse_mapping/mgtf_registration.txt
$ASTROBEE_SOURCE_PATH/localization/sparse_mapping/mgtf_registration.txt

Two of these are on the back wall, and the rest are on the metal
columns on the side walls, with four on each wall. Half of the points
Expand All @@ -430,7 +445,8 @@ be needed to identify them.
A registered and bundle-adjusted map can be used to study how well it
predicts the computed 3D locations for an independently acquired set
of control points and 3D measurements. These are in the same format as
for registration. The map is not modified in any way during this step.
for registration. The map is not modified in any way during this step,
The command is:

build_map -verification <hugin files> <xyz files>

Expand Down Expand Up @@ -462,7 +478,7 @@ To test how the map may perform on the robot, do the following:

### Stage the feature counter utility (should be added to the install at one point):

scp $SOURCE_PATH/localization/marker_tracking/ros/tools/features_counter.py mlp:
scp $ASTROBEE_SOURCE_PATH/localization/marker_tracking/ros/tools/features_counter.py mlp:

### Launch the localization node on LLP

Expand All @@ -477,7 +493,8 @@ this robot will give wrong results for other users.
Then launch localization:

ssh llp
roslaunch astrobee astrobee.launch llp:=disabled mlp:=mlp nodes:=framestore,dds_ros_bridge,localization_node
roslaunch astrobee astrobee.launch llp:=disabled mlp:=mlp \
nodes:=framestore,dds_ros_bridge,localization_node

### Enable localization and the mapped landmark production (on MLP)

Expand Down Expand Up @@ -532,9 +549,9 @@ computations may differ.
Set up the environment in every terminal that is used. Ensure that you
use the correct robot name below.

source $BUILD_PATH/devel/setup.bash
export ASTROBEE_RESOURCE_DIR=$SOURCE_PATH/astrobee/resources
export ASTROBEE_CONFIG_DIR=$SOURCE_PATH/astrobee/config
source $ASTROBEE_BUILD_PATH/devel/setup.bash
export ASTROBEE_RESOURCE_DIR=$ASTROBEE_SOURCE_PATH/astrobee/resources
export ASTROBEE_CONFIG_DIR=$ASTROBEE_SOURCE_PATH/astrobee/config
export ASTROBEE_WORLD=iss
export ASTROBEE_ROBOT=bumble # your robot's name may be different
export ROS_MASTER_URI=http://127.0.0.1:11311/
Expand All @@ -545,9 +562,9 @@ Examine the localization configuration file:

Sym link the map to test:

mkdir -p $SOURCE_PATH/astrobee/resources/maps
rm -fv $SOURCE_PATH/astrobee/resources/maps/iss.map
ln -s $(pwd)/mymap.map $SOURCE_PATH/astrobee/resources/maps/iss.map
mkdir -p $ASTROBEE_SOURCE_PATH/astrobee/resources/maps
rm -fv $ASTROBEE_SOURCE_PATH/astrobee/resources/maps/iss.map
ln -s $(pwd)/mymap.map $ASTROBEE_SOURCE_PATH/astrobee/resources/maps/iss.map

Start the localization node:

Expand Down
27 changes: 21 additions & 6 deletions localization/sparse_mapping/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Maps are stored as protobuf files.

## ROS node

The ROS node takes images and a map as input, and outputs visual features
detected in the image and their 3D coordinates.
The ROS node takes images and a map as input, and outputs visual
features detected in the image and their 3D coordinates.

### Inputs

Expand All @@ -28,6 +28,21 @@ detected in the image and their 3D coordinates.
* `/localization/mapped_landmarks/features`
* `/localization/mapped_landmarks/registration`

## The environment

It will be convenient in this document to set these environmental
variables, pointing to the source and build directories. They may need
to be adjusted given your setup.

export ASTROBEE_SOURCE_PATH=$HOME/astrobee/src
export ASTROBEE_BUILD_PATH=$HOME/astrobee

To access many of the localization tools, such as ``build_map``,
``merge_maps``, etc., without specifying a full path, also consider
setting:

export PATH=$ASTROBEE_BUILD_PATH/devel/lib/sparse_mapping:$PATH

## Tools and procedures

### Record a bag
Expand Down Expand Up @@ -73,15 +88,15 @@ The bags created on the ISS are likely split into many smaller bags,
for easy and reliability of transfer. Those can be merged into one bag
as follows:

astrobee/devel/lib/localization_node/merge_bags \
$ASTROBEE_BUILD_PATH/devel/lib/localization_node/merge_bags \
-output_bag <output bag> <input bags>

### Extracting images

To extract images from a bag file:

astrobee/devel/lib/localization_node/extract_image_bag \
<bagfile.bag> -use_timestamp_as_image_name \
$ASTROBEE_BUILD_PATH/devel/lib/localization_node/extract_image_bag \
<bagfile.bag> -use_timestamp_as_image_name \
-image_topic /hw/cam_nav -output_directory <output dir>

The above assumes that the software was built with ROS on.
Expand Down Expand Up @@ -190,7 +205,7 @@ This functionality is implemented in the localize_cams tool. Usage:

Here we use values that are different from

astrobee/config/localization.config
$ASTROBEE_SOURCE_PATH/astrobee/config/localization.config

which are used for localization on the robot, since those are optimized
for speed and here we want more accuracy.
Expand Down
5 changes: 3 additions & 2 deletions localization/sparse_mapping/theia_map.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ This tool has the following command-line options:

This tool is used to import a map from the NVM format, which Theia
exports to. These operations are done automatically by the
``build_theia_map.py'' tool. This documentation is provided for
``build_theia_map.py`` tool. This documentation is provided for
reference only.

An NVM map exported by Theia (or some other SfM tool) can be saved as
Expand All @@ -124,7 +124,8 @@ an Astrobee sparse map with the command::
This assumes that the images were acquired with the nav camera of the
robot given by $ASTROBEE_ROBOT and undistorted with the Astrobee
program ``undistort_image``. The undistorted camera parameters to use
should be as printed on the screen by ``undistort_image``.
should be as printed on the screen (and saved to disk) by
``undistort_image``.

If desired to replace on importing the undistorted images with the
original distorted ones, as it is usually expected of a sparse map,
Expand Down

0 comments on commit 2de0d3d

Please sign in to comment.