$$\newcommand{\vmath}[1]{\mathsf{#1}} \newcommand{\mapsfrom}{\leftarrow\!\shortmid} \newcommand{\maps}{\mapsto} \newcommand{\exc}{\vmath{\colF{exec}}} \newcommand{\eval}{\vmath{\colR{eval}}} \newcommand{\ufloor}{{\vmath{\colL floor}}} \newcommand{\uceil}{{\vmath{\colU ceil}}} \newcommand{\triv}{\text{Triv}} \newcommand{\idFunc}{\mathrm{Id}} \newcommand{\reals}{\mathbb{R}} \newcommand{\One}{\mathbf{1}} \renewcommand{\mathbbm}[1]{\mathbb{#1}} \renewcommand{\mathscr}[1]{\mathcal{#1}} \newcommand{\varocircle}{⦾} \newcommand{\varotimes}{⊗} \newcommand{\varovee}{(\vee)} \newcommand{\colR}{\color{darkred}} \newcommand{\colF}{\color{darkgreen}} \newcommand{\colH}{\color{blue}} \newcommand{\colI}{\color{orange}} \newcommand{\rtof}{{\colH{\varphi}}} \newcommand{\ftor}{{\colH{h}}} \newcommand{\ftoR}{{\colH{H}}} \newcommand{\Rcomp}{{\mathbb{R}}^{*}_{\small+}} \newcommand{\nonNegRealsComp}{\Rcomp} \newcommand{\nonNegReals}{\mathbb{R}_+} \newcommand{\Rcpu}[1]{\Rcomp{}[\textrm{#1}]} \newcommand{\funsp}{{\colF{\mathscr{F}}}} \newcommand{\impsp}{{\colI{\mathscr{I}}}} \newcommand{\ressp}{{\colR{\mathscr{R}}}} \newcommand{\funleq}{\posleq_{\funsp}} \newcommand{\fun}{\vmath{\colF{f}}} \newcommand{\res}{\vmath{\colR{r}}} \newcommand{\funtop}{\top_\funsp} \newcommand{\funbot}{\bot_\funsp} \newcommand{\imp}{\vmath{i}} \newcommand{\paramsp}{\mathscr{P}} \newcommand{\resleq}{\posleq_{\ressp}} \newcommand{\restop}{\top_\ressp} \newcommand{\resbot}{\bot_\ressp} \newcommand{\resspleq}{\resleq} \newcommand{\tressp}{\trof(\ressp)} \newcommand{\trof}{\mathscr{T}} \newcommand{\tres}{T} \newcommand{\tresleq}{\leq_{\trof}} \newcommand{\trleq}{\leq_{\trof}} \newcommand{\dpisp}{\ensuremath{\vmath{DPI}}} \newcommand{\cdpisp}{\ensuremath{\vmath{CDPI}}} \newcommand{\dprobsp}{\ensuremath{\vmath{DP}}} \newcommand{\dprob}{\vmath{dp}} \newcommand{\dpseries}{\vmath{series}} \newcommand{\dppar}{\vmath{par}} \newcommand{\dploop}{\vmath{loop}} \newcommand{\dploopb}{\vmath{loopb}} \newcommand{\cdprobsp}{\ensuremath{\vmath{CDP}}} \newcommand{\cdprob}{\vmath{cdp}} \newcommand{\dpatoms}{\vmath{atoms}} \newcommand{\resMin}{{\Min_{\resleq}}} \newcommand{\unconnectedfun}{\mathsf{UF}} \newcommand{\unconnectedres}{\mathsf{UR}} \newcommand{\Aressp}{{\mathsf{\colR A}\colR\ressp}} \newcommand{\Afunsp}{{\mathsf{\colF A}\colF\funsp}} \newcommand{\udpa}{\boldsymbol{u}_a} \newcommand{\udpb}{\boldsymbol{u}_b} \newcommand{\udpL}{\boldsymbol{\mathsf{L}}} \newcommand{\udpU}{\boldsymbol{\mathsf{U}}} \newcommand{\udpsp}{\vmath{UDP}} \newcommand{\udpleq}{\posleq_\udpsp} \newcommand{\dpsp}{\vmath{DP}} \newcommand{\dpleq}{\posleq_\dpsp} \newcommand{\terms}{\vmath{Terms}} \newcommand{\udpsem}{\Phi} \newcommand{\dpsem}{\varphi} \newcommand{\atoms}{\mathcal{A}} \newcommand{\atree}{\boldsymbol{\vmath{T}}} \newcommand{\val}{\boldsymbol{v}} \newcommand{\ops}{\vmath{ops}} \newcommand{\ftorL}{\ftor_L} \newcommand{\ftorU}{\ftor_U} \newcommand{\acprod}{\mathbin{\boldsymbol{\times}}} \newcommand{\oploop}{\dagger} \newcommand{\opseries}{\mathbin{\varocircle}} \newcommand{\oppar}{\mathbin{\varotimes}} \newcommand{\opcoprod}{\mathbin{\varovee}} \newcommand{\UId}{\vmath{UId}} \newcommand{\vdc}{\vmath{vdc}} \newcommand{\makedp}{\Gamma} \newcommand{\colU}{\color{purple}} \newcommand{\colL}{\color{orange}} \newcommand{\R}[1]{{\colR{#1}}} \newcommand{\F}[1]{{\colF{#1}}} \newcommand{\I}[1]{{\colI{#1}}} \newcommand{\cdpiN}{\mathcal{V}} \newcommand{\cdpin}{v} \newcommand{\cdpinA}{v_1} \newcommand{\cdpinB}{v_2} \newcommand{\cdpiresind}{i} \newcommand{\cdpifunind}{j} \newcommand{\cdpiresindA}{i_1} \newcommand{\cdpifunindB}{j_2} \newcommand{\dpinumf}{\vmath{nf}} \newcommand{\dpinumr}{\vmath{nr}} \newcommand{\cdpinnumf}{\dpinumf_\cdpin} \newcommand{\cdpinnumr}{\dpinumr_\cdpin} \newcommand{\cdpiE}{\mathcal{E}} \newcommand{\subto}{\text{s.t.}} \newcommand{\with}{\text{using}} \newcommand{\pset}{\mathscr{P}} \DeclareMathOperator*{\Min}{Min} \DeclareMathOperator*{\Inf}{Inf} \DeclareMathOperator*{\Sup}{Sup} \DeclareMathOperator*{\Max}{Max} \newcommand{\lowerbounds}{\vmath{lowerbounds}} \newcommand{\upperbounds}{\vmath{upperbounds}} \newcommand{\posMin}{\Min} \newcommand{\posleq}{\preceq} \newcommand{\poslt}{\prec} \newcommand{\posgeq}{\succeq} \newcommand{\posA}{\mathcal{P}} \newcommand{\posAleq}{\mathrel{{\posleq_\posA}}} \newcommand{\posAMin}{\mathop{{\posMin_{\posAleq}}}} \newcommand{\posAmin}{\mathop{{\min_{\posAleq}}}} \newcommand{\posAmax}{\mathop{{\max_{\posAleq}}}} \newcommand{\posB}{\mathcal{Q}} \newcommand{\posBleq}{\mathrel{{\posleq_\posB}}} \newcommand{\posC}{\mathcal{R}} \newcommand{\lfp}{\vmath{lfp}} \newcommand{\prefixed}{\vmath{prefixed}} \newcommand{\CPOs}{\textsc{CPO}s\xspace} \newcommand{\CPO}{\textsc{CPO}\xspace} \newcommand{\DCPOs}{\textsc{DCPO}s\xspace} \newcommand{\DCPO}{\textsc{DCPO}\xspace} \newcommand{\antichains}{\vmath{A}} \newcommand{\upsets}{\vmath{U}} \newcommand{\downsets}{\vmath{D}} \newcommand{\upresleq}{\posleq_{\upressp}} \newcommand{\upressp}{\upsets\ressp} \newcommand{\allupsets}{\vmath{Up}} \newcommand{\upit}{{\uparrow\,}} \newcommand{\stupit}{\dot{\upit}} \newcommand{\posetwidth}{\vmath{width}} \newcommand{\posetheight}{\vmath{height}} \newcommand{\posdef}[1]{\mathcal{P}_{#1}} \newcommand{\MR}{\M{R}} \newcommand{\myacronym}[1]{\textsc{#1}\xspace} \newcommand{\T}[1]{\boldsymbol{{\mathsf{#1}}}} \newcommand{\Tel}[1]{{\mathsf{#1}}} \newcommand{\Te}[1]{\Tel{#1}} \newcommand{\M}[1]{\mathbf{#1}} \newcommand{\Mel}[1]{\mathrm{#1}} \newcommand{\aset}[1]{\mathscr{#1}} \newcommand{\agroup}[1]{\mathrm{#1}} \newcommand{\aseq}[1]{\boldsymbol{#1}} \newcommand{\aseqe}[1]{#1} \newcommand{\dummyIndices}{} \newcommand{\aword}[1]{\mathsf{#1}} \newcommand{\vmath}[1]{\aword{#1}} \newcommand{\codefunc}[1]{\texttt{#1}\xspace} \newcommand{\swpackage}[1]{\textsc{#1}\xspace} \newcommand{\MA}{\M{A}} \newcommand{\MB}{\M{B}} \newcommand{\MC}{\M{C}} \newcommand{\MG}{\M{G}} \newcommand{\MH}{\M{H}} \newcommand{\ML}{\M{L}} \newcommand{\MQ}{\M{Q}} \newcommand{\MP}{\M{P}} \newcommand{\MS}{\M{S}} \newcommand{\MSigma}{\M{\Sigma}} \newcommand{\MV}{\M{V}} \newcommand{\MW}{\M{W}} \newcommand{\SP}{P_{\text{s}}} \newcommand{\AP}{P_{\text{a}}} \newcommand{\SE}{E} \newcommand{\ER}{r} \newcommand{\HP}{\Theta} \newcommand{\np}{n} \newcommand{\ones}{\boldsymbol{1}} \newcommand{\idMat}{\M{I}} \newcommand{\matTrace}{\vmath{Tr}} \newcommand{\angleFun}{\angle} \newcommand{\flatten}{\mathsf{vec}} \newcommand{\batterymass}{{\colR{m}}} \newcommand{\batterycapacity}{{\colF{C}}} \newcommand{\batterycost}{{\colR{c}}} \newcommand{\specificenergy}{{\colR{\rho}}} \newcommand{\specificcost}{{\colR{\alpha}}} \newcommand{\D}{\,\textrm{d}} \newcommand{\ex}{\mathbb{E}} \newcommand{\aset}[1]{\mathcal{#1}} \newcommand{\amat}[1]{\mathbf{#1}} \newcommand{\avec}[1]{\mathbf{#1}} \newcommand{\rv}[1]{\boldsymbol{#1}} \newcommand{\definedas}{\triangleq} \newcommand{\nats}{\mathbb{N}} \newcommand{\ints}{\mathbb{Z}} \newcommand{\rats}{\mathbb{Q}} \newcommand{\reals}{\mathbb{R}} \newcommand{\comp}{\mathbb{C}} \newcommand{\Time}{\mathbb{T}} \newcommand{\SEthree}{\text{SE}(3)} \newcommand{\SEtwo}{\text{SE}(2)} \newcommand{\sethree}{\text{se}(3)} \newcommand{\setwo}{\text{se}(2)} \newcommand{\SOthree}{\text{SO}(3)} \newcommand{\pose}{\boldsymbol{q}} \newcommand{\state}{\boldsymbol{x}} \newcommand{\statesp}{\mathcal{X}} \newcommand{\bmu}{\boldsymbol{\mu}} \newcommand{\bSigma}{\boldsymbol{\Sigma}} \newcommand{\tup}[1]{\langle#1\rangle}$$

My First Duckietown Python Library

✎

Modified 2020-09-16 by liampaull

Learn how to do proper python development

Get the Duckietown library template

✎

Modified 2020-10-06 by arsimone

A boilerplate is provided by the library template repository.

The repository contains a lot of files, but do not worry, we will analyze them one by one. Click on the green button that says “Use this template”.

This will take you to a page that looks like the following:

Pick a name for your repository (say my-library) and press the button Create repository from template. Note, you can replace my-library with the name of the repository that you prefer.

This will create a new repository and copy everything from the repository template-library to your new repository. You can now open a terminal and clone your newly created repository.

laptop $ git clone https://github.com/YOUR_NAME/my-library
laptop $ cd my-library

Replace YOUR_NAME in the link above with your GitHub username.

Features of the library template

✎

Modified 2020-09-16 by liampaull

We have the following features:

Unit-tests using Nose.
Building/testing in Docker environment locally.
Integration with CircleCI for automated testing.
Integration with CodeCov for displaying coverage result.
Integration with Sphinx to build code docs. (So far, only built locally.)
Jupyter notebooks, which are run also in CircleCI as tests.
Version bump using Bumpversion.
Code formatting using Black.
Command-line program for using the library.

Anatomy of the library template

✎

Modified 2020-09-16 by liampaull

This repository describes a library called “duckietown_pondcleaner” and there is one command-line tool called dt-pc-demo.

Meta-files

✎

Modified 2020-09-16 by liampaull

.gitignore: Files ignore by Git.
.dtproject: Enables the project to be built and used by dts devel tools
.bumpversion.cfg: Configuration for bumpversion
Makefile: Build tools configuration with Make

Python packaging

✎

Modified 2020-10-25 by Fidel Esquivel

requirements.txt: Contains the pinned versions of your requirement that are used to run tests.
MANIFEST.in: Deselects the tests to be included in the egg.
setup.py: Contains meta information, definition of the scripts, and the dependencies information.

Python code

✎

Modified 2020-09-16 by liampaull

src/ - This is the path that you should set as “sources root” in your tool
src/duckietown_pondcleaner: Contains the code.
src/duckietown_pondcleaner/__init__.py: Contains the __version__ library.
src/duckietown_pondcleaner_tests: Contains the tests - not included in the egg.

Docker testing

✎

Modified 2020-09-16 by liampaull

These are files to build and run a testing container.

.dockerignore: Describes what files go in the docker container.
Dockerfile: …

Sphinx

✎

Modified 2020-09-16 by liampaull

src/conf.py: Sphinx settings
src/index.rst: Sphinx main file
src/duckietown_pondcleaner/index.rst: Documentation for the package

Coverage

✎

Modified 2020-09-16 by liampaull

.coveragerc: Options for code coverage.

Notebooks

✎

Modified 2020-09-16 by liampaull

notebooks: Notebooks that are run also as a test.
notebooks-extra: Other notebooks (not run as test)
notebooks/*.ipynb: The notebooks themselves.

Creating your Library

✎

Modified 2020-09-16 by Liam Paull

Using the repo you have already created:

Clone the newly created repository;
Place your Python packages inside src/;
List the python dependencies in the file dependencies.txt;
Update the appropriate section in the file setup.py;

Make sure that there are no other remains:

laptop $ grep -r . pondcleaner

Update the branch names in README.md.

Other set up (for admins)

✎

Modified 2020-09-16 by liampaull

The following are necessary steps for admins to do:

Activate on CircleCI. Make one build successful.
Activate on CodeCov. Get the CODECOV_TOKEN. Put this token in CircleCI environment.

How to use the utilities in the library template

✎

Modified 2020-09-16 by liampaull

Test the code

✎

Modified 2020-10-04 by arsimone

Test the code using Docker by:

laptop $ make test-docker

This runs the test using a Docker container built from scratch with the pinned dependencies in requirements.txt. This is equivalent to what is run on CircleCI.

To run the tests natively on your pc, use:

laptop $ make test

To do so you will need to have installed the libraries listed in the file requirements.txt on your computer.

For that we assume you have already setup a Python virtual environment.

To do so you will need to pip install virtualenv then virtualenv duckietown then source duckietown/bin/activate. In order to install the requirements to run the test do pip install -r requirements.txt.

Development

✎

Modified 2020-10-04 by arsimone

In the same virtual environment as above run:

laptop $ python setup.py develop

This will install the library in an editable way (rather than copying the sources somewhere else).

If you don’t want to install the deps, do:

laptop $ python setup.py develop  --no-deps

For example, this is done in the Dockerfile so that we know we are only using the dependencies in requirements.txt with the exact pinned version.

Adding tests

✎

Modified 2020-09-16 by liampaull

To add another tests, add files with the name test_*py in the package duckietown_podcleaner_tests. The name is important.

Tip: make sure that the tests are actually run looking at the coverage results.

Notes on using the notebooks

✎

Modified 2020-09-16 by liampaull

Always clean the notebooks before committing them:

laptop $ make -C notebooks cleanup

If you don’t think you can be diligent, then add the notebooks using Git LFS.

Releasing a new version

✎

Modified 2020-09-16 by liampaull

Updating the version

✎

The first step is to change the version and tag the repo. DO NOT change the version manually; use the CLI tool bumpversion instead.

The tool can be called by:

laptop $ make bump    # bump the version, tag the tree

If you need to include the version in a new file, list it inside the file .bumpversion.cfg using the syntax [bumpversion:file: <FILE_PATH >].

Releasing the package

✎

The next step is to upload the package to PyPy. We use twine. Invoke using:

laptop $ make upload  # upload to PyPI

For this step, uou need to have admin permissions on PyPy.