$$\newcommand{\vmath}[1]{\mathsf{#1}} \newcommand{\mapsfrom}{\leftarrow\!\shortmid} \newcommand{\maps}{\mapsto} \newcommand{\exc}{\vmath{\colF{exec}}} \newcommand{\eval}{\vmath{\colR{eval}}} \newcommand{\ufloor}{{\vmath{\colL floor}}} \newcommand{\uceil}{{\vmath{\colU ceil}}} \newcommand{\triv}{\text{Triv}} \newcommand{\idFunc}{\mathrm{Id}} \newcommand{\reals}{\mathbb{R}} \newcommand{\One}{\mathbf{1}} \renewcommand{\mathbbm}[1]{\mathbb{#1}} \renewcommand{\mathscr}[1]{\mathcal{#1}} \newcommand{\varocircle}{⦾} \newcommand{\varotimes}{⊗} \newcommand{\varovee}{(\vee)} \newcommand{\colR}{\color{darkred}} \newcommand{\colF}{\color{darkgreen}} \newcommand{\colH}{\color{blue}} \newcommand{\colI}{\color{orange}} \newcommand{\rtof}{{\colH{\varphi}}} \newcommand{\ftor}{{\colH{h}}} \newcommand{\ftoR}{{\colH{H}}} \newcommand{\Rcomp}{{\mathbb{R}}^{*}_{\small+}} \newcommand{\nonNegRealsComp}{\Rcomp} \newcommand{\nonNegReals}{\mathbb{R}_+} \newcommand{\Rcpu}[1]{\Rcomp{}[\textrm{#1}]} \newcommand{\funsp}{{\colF{\mathscr{F}}}} \newcommand{\impsp}{{\colI{\mathscr{I}}}} \newcommand{\ressp}{{\colR{\mathscr{R}}}} \newcommand{\funleq}{\posleq_{\funsp}} \newcommand{\fun}{\vmath{\colF{f}}} \newcommand{\res}{\vmath{\colR{r}}} \newcommand{\funtop}{\top_\funsp} \newcommand{\funbot}{\bot_\funsp} \newcommand{\imp}{\vmath{i}} \newcommand{\paramsp}{\mathscr{P}} \newcommand{\resleq}{\posleq_{\ressp}} \newcommand{\restop}{\top_\ressp} \newcommand{\resbot}{\bot_\ressp} \newcommand{\resspleq}{\resleq} \newcommand{\tressp}{\trof(\ressp)} \newcommand{\trof}{\mathscr{T}} \newcommand{\tres}{T} \newcommand{\tresleq}{\leq_{\trof}} \newcommand{\trleq}{\leq_{\trof}} \newcommand{\dpisp}{\ensuremath{\vmath{DPI}}} \newcommand{\cdpisp}{\ensuremath{\vmath{CDPI}}} \newcommand{\dprobsp}{\ensuremath{\vmath{DP}}} \newcommand{\dprob}{\vmath{dp}} \newcommand{\dpseries}{\vmath{series}} \newcommand{\dppar}{\vmath{par}} \newcommand{\dploop}{\vmath{loop}} \newcommand{\dploopb}{\vmath{loopb}} \newcommand{\cdprobsp}{\ensuremath{\vmath{CDP}}} \newcommand{\cdprob}{\vmath{cdp}} \newcommand{\dpatoms}{\vmath{atoms}} \newcommand{\resMin}{{\Min_{\resleq}}} \newcommand{\unconnectedfun}{\mathsf{UF}} \newcommand{\unconnectedres}{\mathsf{UR}} \newcommand{\Aressp}{{\mathsf{\colR A}\colR\ressp}} \newcommand{\Afunsp}{{\mathsf{\colF A}\colF\funsp}} \newcommand{\udpa}{\boldsymbol{u}_a} \newcommand{\udpb}{\boldsymbol{u}_b} \newcommand{\udpL}{\boldsymbol{\mathsf{L}}} \newcommand{\udpU}{\boldsymbol{\mathsf{U}}} \newcommand{\udpsp}{\vmath{UDP}} \newcommand{\udpleq}{\posleq_\udpsp} \newcommand{\dpsp}{\vmath{DP}} \newcommand{\dpleq}{\posleq_\dpsp} \newcommand{\terms}{\vmath{Terms}} \newcommand{\udpsem}{\Phi} \newcommand{\dpsem}{\varphi} \newcommand{\atoms}{\mathcal{A}} \newcommand{\atree}{\boldsymbol{\vmath{T}}} \newcommand{\val}{\boldsymbol{v}} \newcommand{\ops}{\vmath{ops}} \newcommand{\ftorL}{\ftor_L} \newcommand{\ftorU}{\ftor_U} \newcommand{\acprod}{\mathbin{\boldsymbol{\times}}} \newcommand{\oploop}{\dagger} \newcommand{\opseries}{\mathbin{\varocircle}} \newcommand{\oppar}{\mathbin{\varotimes}} \newcommand{\opcoprod}{\mathbin{\varovee}} \newcommand{\UId}{\vmath{UId}} \newcommand{\vdc}{\vmath{vdc}} \newcommand{\makedp}{\Gamma} \newcommand{\colU}{\color{purple}} \newcommand{\colL}{\color{orange}} \newcommand{\R}[1]{{\colR{#1}}} \newcommand{\F}[1]{{\colF{#1}}} \newcommand{\I}[1]{{\colI{#1}}} \newcommand{\cdpiN}{\mathcal{V}} \newcommand{\cdpin}{v} \newcommand{\cdpinA}{v_1} \newcommand{\cdpinB}{v_2} \newcommand{\cdpiresind}{i} \newcommand{\cdpifunind}{j} \newcommand{\cdpiresindA}{i_1} \newcommand{\cdpifunindB}{j_2} \newcommand{\dpinumf}{\vmath{nf}} \newcommand{\dpinumr}{\vmath{nr}} \newcommand{\cdpinnumf}{\dpinumf_\cdpin} \newcommand{\cdpinnumr}{\dpinumr_\cdpin} \newcommand{\cdpiE}{\mathcal{E}} \newcommand{\subto}{\text{s.t.}} \newcommand{\with}{\text{using}} \newcommand{\pset}{\mathscr{P}} \DeclareMathOperator*{\Min}{Min} \DeclareMathOperator*{\Inf}{Inf} \DeclareMathOperator*{\Sup}{Sup} \DeclareMathOperator*{\Max}{Max} \newcommand{\lowerbounds}{\vmath{lowerbounds}} \newcommand{\upperbounds}{\vmath{upperbounds}} \newcommand{\posMin}{\Min} \newcommand{\posleq}{\preceq} \newcommand{\poslt}{\prec} \newcommand{\posgeq}{\succeq} \newcommand{\posA}{\mathcal{P}} \newcommand{\posAleq}{\mathrel{{\posleq_\posA}}} \newcommand{\posAMin}{\mathop{{\posMin_{\posAleq}}}} \newcommand{\posAmin}{\mathop{{\min_{\posAleq}}}} \newcommand{\posAmax}{\mathop{{\max_{\posAleq}}}} \newcommand{\posB}{\mathcal{Q}} \newcommand{\posBleq}{\mathrel{{\posleq_\posB}}} \newcommand{\posC}{\mathcal{R}} \newcommand{\lfp}{\vmath{lfp}} \newcommand{\prefixed}{\vmath{prefixed}} \newcommand{\CPOs}{\textsc{CPO}s\xspace} \newcommand{\CPO}{\textsc{CPO}\xspace} \newcommand{\DCPOs}{\textsc{DCPO}s\xspace} \newcommand{\DCPO}{\textsc{DCPO}\xspace} \newcommand{\antichains}{\vmath{A}} \newcommand{\upsets}{\vmath{U}} \newcommand{\downsets}{\vmath{D}} \newcommand{\upresleq}{\posleq_{\upressp}} \newcommand{\upressp}{\upsets\ressp} \newcommand{\allupsets}{\vmath{Up}} \newcommand{\upit}{{\uparrow\,}} \newcommand{\stupit}{\dot{\upit}} \newcommand{\posetwidth}{\vmath{width}} \newcommand{\posetheight}{\vmath{height}} \newcommand{\posdef}[1]{\mathcal{P}_{#1}} \newcommand{\MR}{\M{R}} \newcommand{\myacronym}[1]{\textsc{#1}\xspace} \newcommand{\T}[1]{\boldsymbol{{\mathsf{#1}}}} \newcommand{\Tel}[1]{{\mathsf{#1}}} \newcommand{\Te}[1]{\Tel{#1}} \newcommand{\M}[1]{\mathbf{#1}} \newcommand{\Mel}[1]{\mathrm{#1}} \newcommand{\aset}[1]{\mathscr{#1}} \newcommand{\agroup}[1]{\mathrm{#1}} \newcommand{\aseq}[1]{\boldsymbol{#1}} \newcommand{\aseqe}[1]{#1} \newcommand{\dummyIndices}{} \newcommand{\aword}[1]{\mathsf{#1}} \newcommand{\vmath}[1]{\aword{#1}} \newcommand{\codefunc}[1]{\texttt{#1}\xspace} \newcommand{\swpackage}[1]{\textsc{#1}\xspace} \newcommand{\MA}{\M{A}} \newcommand{\MB}{\M{B}} \newcommand{\MC}{\M{C}} \newcommand{\MG}{\M{G}} \newcommand{\MH}{\M{H}} \newcommand{\ML}{\M{L}} \newcommand{\MQ}{\M{Q}} \newcommand{\MP}{\M{P}} \newcommand{\MS}{\M{S}} \newcommand{\MSigma}{\M{\Sigma}} \newcommand{\MV}{\M{V}} \newcommand{\MW}{\M{W}} \newcommand{\SP}{P_{\text{s}}} \newcommand{\AP}{P_{\text{a}}} \newcommand{\SE}{E} \newcommand{\ER}{r} \newcommand{\HP}{\Theta} \newcommand{\np}{n} \newcommand{\ones}{\boldsymbol{1}} \newcommand{\idMat}{\M{I}} \newcommand{\matTrace}{\vmath{Tr}} \newcommand{\angleFun}{\angle} \newcommand{\flatten}{\mathsf{vec}} \newcommand{\batterymass}{{\colR{m}}} \newcommand{\batterycapacity}{{\colF{C}}} \newcommand{\batterycost}{{\colR{c}}} \newcommand{\specificenergy}{{\colR{\rho}}} \newcommand{\specificcost}{{\colR{\alpha}}} \newcommand{\D}{\,\textrm{d}} \newcommand{\ex}{\mathbb{E}} \newcommand{\AC}[1]{{\color{blue}AC: #1}} \newcommand{\JZ}[1]{{\color{olive}JZ: #1}} \newcommand{\fix}{\marginpar{FIX}} \newcommand{\new}{\marginpar{NEW}} \newcommand{\dynamical}{\mathcal{D}} \newcommand{\robot}{\mathcal{R}} \newcommand{\config}{\mathcal{Q}} \newcommand{\sensors}{\{z\}} \newcommand{\bandwidth}{\mathcal{B}} \newcommand{\computation}{\mathcal{C}} \newcommand{\memory}{\mathcal{M}} \newcommand{\actuators}{\mathcal{A}} \newcommand{\knowledge}{\mathcal{K}} \newcommand{\perception}{P} \newcommand{\control}{U} \newcommand{\actions}{\mathcal{U}} \newcommand{\operator}{T} \newcommand{\groups}{G} \newcommand{\groupalgebra}{\mathfrak{g}} \newcommand{\timespace}{\mathbb{T}} \newcommand{\environment}{E} \newcommand{\scene}{\xi} \newcommand{\scenespace}{\Xi} \newcommand{\universe}{U} \newcommand{\sensor}{\zeta} \newcommand{\sensorproj}{z} \newcommand{\sensorspace}{Z} \newcommand{\projection}{\pi} \newcommand{\projectionspace}{\Pi} \newcommand{\viewport}{v} \newcommand{\viewportspace}{\mathcal{V}} \newcommand{\dataspace}{\mathcal{X}} \newcommand{\data}{x} \newcommand{\dataproj}{\phi} \newcommand{\datakernel}{\psi} \newcommand{\outputy}{y} \newcommand{\outputspace}{\mathcal{Y}} \newcommand{\task}{T} \newcommand{\taskspace}{\mathcal{T}} \newcommand{\objective}{\mathcal{J}} \newcommand{\robotictask}{RT} \newcommand{\rules}{\Phi} \newcommand{\constraints}{\Lambda} \newcommand{\action}{u} \newcommand{\actionspace}{\mathcal{U}} \newcommand{\nuisance}{\nu} \newcommand{\place}{\eta} \newcommand{\image}{I} \newcommand{\noise}{n} \newcommand{\pose}{p} \newcommand{\shape}{S} \newcommand{\albedo}{\rho} \newcommand{\information}{\mathcal{I}} \newcommand{\expectation}{\mathbb{E}} \newcommand{\loss}{L} \newcommand{\AC}[1]{{\color{blue}AC: #1}} \newcommand{\JZ}[1]{{\color{olive}JZ: #1}} \newcommand{\fix}{\marginpar{FIX}} \newcommand{\new}{\marginpar{NEW}} \newcommand{\dynamical}{\mathcal{D}} \newcommand{\robot}{\mathcal{R}} \newcommand{\config}{\mathcal{Q}} \newcommand{\sensors}{\{z\}} \newcommand{\bandwidth}{\mathcal{B}} \newcommand{\computation}{\mathcal{C}} \newcommand{\memory}{\mathcal{M}} \newcommand{\actuators}{\mathcal{A}} \newcommand{\knowledge}{\mathcal{K}} \newcommand{\perception}{P} \newcommand{\control}{U} \newcommand{\actions}{\mathcal{U}} \newcommand{\operator}{T} \newcommand{\groups}{G} \newcommand{\groupalgebra}{\mathfrak{g}} \newcommand{\timespace}{\mathbb{T}} \newcommand{\environment}{E} \newcommand{\scene}{\xi} \newcommand{\scenespace}{\Xi} \newcommand{\universe}{U} \newcommand{\sensor}{\zeta} \newcommand{\sensorproj}{z} \newcommand{\sensorspace}{Z} \newcommand{\projection}{\pi} \newcommand{\projectionspace}{\Pi} \newcommand{\viewport}{v} \newcommand{\viewportspace}{\mathcal{V}} \newcommand{\dataspace}{\mathcal{X}} \newcommand{\data}{x} \newcommand{\dataproj}{\phi} \newcommand{\datakernel}{\psi} \newcommand{\outputy}{y} \newcommand{\outputspace}{\mathcal{Y}} \newcommand{\task}{T} \newcommand{\taskspace}{\mathcal{T}} \newcommand{\objective}{\mathcal{J}} \newcommand{\robotictask}{RT} \newcommand{\rules}{\Phi} \newcommand{\constraints}{\Lambda} \newcommand{\action}{u} \newcommand{\actionspace}{\mathcal{U}} \newcommand{\nuisance}{\nu} \newcommand{\place}{\eta} \newcommand{\image}{I} \newcommand{\noise}{n} \newcommand{\pose}{p} \newcommand{\shape}{S} \newcommand{\albedo}{\rho} \newcommand{\information}{\mathcal{I}} \newcommand{\expectation}{\mathbb{E}} \newcommand{\loss}{L} \newcommand{\aset}[1]{\mathcal{#1}} \newcommand{\amat}[1]{\mathbf{#1}} \newcommand{\avec}[1]{\mathbf{#1}} \newcommand{\rv}[1]{\boldsymbol{#1}} \newcommand{\definedas}{\triangleq} \newcommand{\nats}{\mathbb{N}} \newcommand{\ints}{\mathbb{Z}} \newcommand{\rats}{\mathbb{Q}} \newcommand{\reals}{\mathbb{R}} \newcommand{\comp}{\mathbb{C}} \newcommand{\Time}{\mathbb{T}} \newcommand{\SEthree}{\text{SE}(3)} \newcommand{\SEtwo}{\text{SE}(2)} \newcommand{\sethree}{\text{se}(3)} \newcommand{\setwo}{\text{se}(2)} \newcommand{\SOthree}{\text{SO}(3)} \newcommand{\pose}{\boldsymbol{q}} \newcommand{\state}{\boldsymbol{x}} \newcommand{\statesp}{\mathcal{X}} \newcommand{\bmu}{\boldsymbol{\mu}} \newcommand{\bSigma}{\boldsymbol{\Sigma}} \newcommand{\tup}[1]{\langle#1\rangle}$$

ROS Template

✎

Modified 2021-10-30 by liampaull

This section describes the basic procedure for making a submission with an agent using the Robot Operating System. It can be used as a starting point for any of the LF, LFV, and LFI challenges.

That you have setup your accounts.

That you meet the software requirement.

That you have a basic understanding of ROS.

You make a submission to all of the LF* challenges and can view their status and output.

AI-DO 5 (NeurIPS 2020) - Second Webinar — ROS template

Quickstart

✎

Modified 2020-11-11 by Liam Paull

Clone the template repo:

$ git clone git@github.com:duckietown/challenge-aido_LF-template-ros.git

Change into the directory:

$ cd challenge-aido-LF-template-ros

Either make a submission with:

$ dts challenges submit --challenge CHALLENGE_NAME

where you can find a list of the open challenges here.

Or, run local evaluation with:

$ dts challenges evaluate --challenge CHALLENGE_NAME

Verify the submission:

✎

Modified 2020-11-06 by Liam Paull

This will make a number of submissions (as described below). You can track the status of these submissions in the command line with:

$ dts challenges follow --submission SUBMISSION_NUMBER

or through your browser by navigating the webpage: https://challenges.duckietown.org/v4/humans/submissions/SUBMISSION_NUMBER

where SUBMISSION_NUMBER should be replaced with the number of the submission which is reported in the terminal output.

Anatomy of the submission

✎

Modified 2021-10-30 by liampaull

The submission consists of all of the basic files that required for a basic submission. Below we will highlight the specifics with respect to this template.

There are also a few other new files and folders in this submission:

launchers/
submission_ws/

and additionally the solution.py is inside the submission_ws folder and Dockerfile have changed. We will describe each of these in detail.

If you don’t care about the details, or just want to get started, you can start by adding new ROS packages into the submission_ws.

Dockerfile

✎

Modified 2021-10-30 by liampaull

The main update here is that we build your catkin workspace inside (the submission_ws folder) in the Dockerfile:

RUN . /opt/ros/${ROS_DISTRO}/setup.sh && \
    . ${CATKIN_WS_DIR}/devel/setup.bash  && \
    catkin build --workspace /code/submission_ws

Also note that instead of just running solution.py when we enter the container, we now run a “launcher” (in the launchers folder) called run_and_start.sh. For details see Subsection 2.2.4 - launchers/.

Also note that in this Dockerfile we are not copying the entire directory over, instead we are copying files individually (this is actually more efficient). So if you add new files that you are using that are outside of the submission_ws and launchers folders, you will have to add additional COPY commands.

`solution.py`

✎

Modified 2021-10-30 by liampaull

You probably don’t need to change this file.

We instantiate a ROSAgent() (see Subsection 2.2.3 - rosagent.py) and this becomes the object that handles interfacing with the ROS interface. This includes the publishing of imagery and encoder data to ROS:

def on_received_observations(self, data: DB20ObservationsWithTimestamp, context: Context):
        camera = data.camera
        odometry = data.odometry
        # context.info(f'received obs camera {camera.timestamp} odometry {odometry.timestamp}')

        if camera.timestamp != self.last_camera_timestamp or True:
            self.agent.publish_img(camera.jpg_data, camera.timestamp)
            self.agent.publish_info(camera.timestamp)
            self.last_camera_timestamp = camera.timestamp

        if odometry.timestamp != self.last_odometry_timestamp or True:
            self.agent.publish_odometry(
                odometry.resolution_rad, odometry.axis_left_rad, odometry.axis_right_rad, odometry.timestamp
            )
            self.last_odometry_timestamp = odometry.timestamp

Notice now that the protocol includes timestamps which are used to tag the data,
and that a new camera image is not published if the timestamp does not change.

`rosagent.py`

✎

Modified 2021-10-30 by liampaull

You probably don’t need to change this file.

rosagent.py sets up a class that can be used to interface with the rest of the ROS stack. It is for all intents and purposes a fully functional ROS node except that it isn’t launched through ROS, it is instantiated in code. This class takes care of a few useful things, such as getting the correct camera calibration files, subscribing to control commands and sending them to your robot (real or simulated), as well as retreiving the sensor data from the robot and publishing it to ROS.

The main functions are:

def publish_img(self, obs: bytes, timestamp: float):, which takes the camera observation from the environment, and publishes it to the topic that you specify in the constructor of the ROSAgent
def publish_odometry(self, resolution_rad: float, left_rad: float, right_rad: float, timestamp: float):, which take the encoder data from the robot, and publishes it to the topic specified in the constructor of the ROSAgent.
def _ik_action_cb(self, msg):, listens on the inverse kinematics action topic, and assigns it to self.action.

`launchers/`

✎

Modified 2021-10-30 by liampaull

The bash scripts in the launchers directory are there to help you get everything started when you run your container. In this template there is only run_and_start.sh:

#!/bin/bash

source /environment.sh

source /opt/ros/noetic/setup.bash
source /code/catkin_ws/devel/setup.bash --extend
source /code/submission_ws/devel/setup.bash --extend

set -eux

dt-exec-BG roscore

dt-exec-BG roslaunch --wait random_action random_action_node.launch
dt-exec-FG roslaunch --wait agent agent_node.launch || true

copy-ros-logs

You are free to modify this as you see fit, but a few things are important to consider.

The order that we source things matters. If we have a package with the same name in two workspaces, ROS will run whichever one got sourced last.
If you don’t put things in the background (with dt-exec-BG) then if those commands don’t end, subsequent commands will not get run.
The --wait flag in the roslaunch command is recommended so that roslaunch will wait until the roscore has finished initializing.

`submission_ws/`

✎

Modified 2021-10-30 by liampaull

This is a standard ROS catkin workspace. You can populate it with ROS packages. You will notice that the random_action package is already in the workspace. This can be used as a template for creating more packages. The main elements are launch files in the launch folder (you will see the random_action_node.launch which is launched by the run_and_start.sh launcher), the src folder which contains the ROS nodes, and the include folder which contains your python includes (you can also write nodes in C++ or other languages if you prefer).