A tutorial presenting some background on temporal logic
testers and the syntax used in CPSGrader.
Download PDF Tutorial
Using CPSGrader with a Custom Simulator
We propose currently two methods:
File interface: The simplest method is to have
the custom simulator write traces into files and grade
them using the program CPSFileGrader.
C++ API interface: CPSGrader main class is
STLDriver, which exposes a simple API to run tests
defined in a test plan. The easiest way to get started
with the C++ API is to modify CPSFileGrader.cpp by
replacing the call to the read_data function by a call
to the simulator, or a wrapper function which will run a
simulation and write the result in a simple vector of
vector of double.
If these methods are not working for you, or if you encounter
any issue, please Contact us
will figure out a way to interface your simulator to
CPSGrader comes with a robotic simulator based on NI LabView
Robotics which was used in the MOOC EECS149.1x on edX.
Instructions on how use CyberSim+CPSGrader are still available
in the Virtual Lab section of the MOOC page: EECS 149.1x info page
Customizing assignments and test plans used for grading the
EECS149 labs is partially covered in the tutorial above.
Here is some information about using the NI Robotics Simulator:
CPSGrader also includes a virtual lab for students to test their C code robot (turtlebot) controller in simulation prior to actual testing in hardware. This enabled students to significantly reduce development time by quickly identifying bugs in their code in our virtual platform. The students’ programmed robots (turtlebots) are tested in Gazebo simulator running on the Robot Operating System (ROS). Different challenging environments are presented to turtlebots to achieve a specific task. These environments are modeled and generated using the scenario specification language Scenic
which is interfaced to the Gazebo simulator.
Furthermore, to provide helpful feedback to students to better identify issues in their code, the virtual lab comes with a framework for defining monitors using specifications in metric temporal logic (MTL). Monitors are executed on the logs produced during the simluation. The underlying tool used for defining the monitors is pyMTL
in combination with the discrete signals library
The VM image and more information and instructions on how to use ROS/Gazebo+CPSGrader can be found here:
Synthesis of Test Plans
Testers to be used with CPSGrader can also be automatically
synthesized using labeled examples of student
solutions/traces that exhibit certain desired behavior.
Details of the technique are in our EMSOFT'14
paper. We also provide a small module to demonstrate how
synthesis works. You will need to install the Breach MATLAB toolbox to run this
Download Synthesis Module (13 MB zip archive)