A tutorial presenting some background on temporal logic testers and the syntax used in CPSGrader.

Download PDF Tutorial

We propose currently two methods:

1. File interface: The simplest method is to have the custom simulator write traces into files and grade them using the program CPSFileGrader.

2. C++ API interface: CPSGrader main class is STLDriver, which exposes a simple API to run tests defined in a test plan. The easiest way to get started with the C++ API is to modify CPSFileGrader.cpp by replacing the call to the read_data function by a call to the simulator, or a wrapper function which will run a simulation and write the result in a simple vector of vector of double.

If these methods are not working for you, or if you encounter any issue, please Contact us and we will figure out a way to interface your simulator to CPSGrader.

CPSGrader comes with a robotic simulator based on NI LabView Robotics which was used in the MOOC EECS149.1x on edX. Instructions on how use CyberSim+CPSGrader are still available in the Virtual Lab section of the MOOC page: EECS 149.1x info page.

Customizing assignments and test plans used for grading the EECS149 labs is partially covered in the tutorial above.

Here is some information about using the NI Robotics Simulator:

• Overview of the LabVIEW Robotics Environment Simulator
• Getting Started with LabVIEW Robotics
• LabVIEW 2014 Robotics Module Help - Simulating Mobile Robots

CPSGrader also includes a virtual lab for students to test their C code robot (turtlebot) controller in simulation prior to actual testing in hardware. This enabled students to significantly reduce development time by quickly identifying bugs in their code in our virtual platform. The students’ programmed robots (turtlebots) are tested in Gazebo simulator running on the Robot Operating System (ROS). Different challenging environments are presented to turtlebots to achieve a specific task. These environments are modeled and generated using the scenario specification language Scenic which is interfaced to the Gazebo simulator.
Furthermore, to provide helpful feedback to students to better identify issues in their code, the virtual lab comes with a framework for defining monitors using specifications in metric temporal logic (MTL). Monitors are executed on the logs produced during the simluation. The underlying tool used for defining the monitors is pyMTL in combination with the discrete signals library.

The VM image and more information and instructions on how to use ROS/Gazebo+CPSGrader can be found here:

• VM image
• Lab book of EECS149. Check Chapter 8 for instructions on using the simulator and monitors.

Testers to be used with CPSGrader can also be automatically synthesized using labeled examples of student solutions/traces that exhibit certain desired behavior. Details of the technique are in our EMSOFT'14 paper. We also provide a small module to demonstrate how synthesis works. You will need to install the Breach MATLAB toolbox to run this module.

Download Synthesis Module (13 MB zip archive)