build details

Show: section status errors & todos local changes recent changes last change in-page changes feedback controls

Using the Evaluator

Modified 2018-10-09 by Andrea Censi

This section describes how to use the Challenges evaluators.

Evaluators

Modified 2018-10-09 by Andrea Censi

An evaluator is a machine that is in charge of evaluating the protocols.

Running your own evaluator

Modified 2020-11-07 by Andrea Censi

We have several evaluators online that process jobs.

If you want to avoid waiting in the queue for to long, you can run your own evaluator.

The command line is:

$ dts challenges evaluator --continuous

This evaluator will connect to the server and execute preferentially your submissions.

Advanced options for evaluator

Modified 2018-10-09 by Andrea Censi

Naming evaluator

Modified 2020-11-07 by Andrea Censi

Use the option --name to name the evaluator instance:

$ dts challenges evaluator --name a name

Otherwise the name is going to be autogenerated.

For example:

$ dts challenges evaluator --name Instance1 &
$ dts challenges evaluator --name Instance2 &

Run a specific submission

Modified 2020-11-07 by Andrea Censi

Run the evaluator on a specific submission:

$ dts challenges evaluator --submission ID

This evaluates a specific submission.

Note that to force re-evaluation of a submission, you must first reset the submission.

Also note that you cannot re-evaluate a submission that has been “retired”.