Learning Automation, Automated

Students on the ETH course PDM4AR (Planning and Decision Making for Autonomous Robots) are there to learn how to write algorithms to guide autonomous decision making – for instance, teaching an autonomous vehicle how to drive. Courses like this are traditionally taught using a lot of pen and paper exercises, solving a few iterations of an algorithm on paper. The PDM4AR course was taught this way too, at first. But that isn’t really close to what is needed for real work in robotics; and it’s not much fun. There is a better way to show students the real magic of these algorithms, and awaken their passion for the subject.

As a firm believer in the value of hands-on work, Professor Emilio Frazzoli decided to bring the experience closer to the real world and deliver learning benefits beyond the basic course material. He worked with fellow NCCR Automation member Alessandro Zanardi and software engineer Shengjie Hu to create an entirely new learning experience. Now, students need to transform the theory they have been taught into actual working code. They can access tools and test cases from a shared repository, code a solution, upload the finished work on Github – and have it benchmarked on secret test cases that they haven’t seen. Their grade is automatically generated based on how well it performs in these tests. 

While it may at first seem artificial to use automatic assessment, in fact, it is much more realistic than manual individual feedback. After all, it’s not uncommon for code to fail when put to the test, and as frustrating as that is, in the real world, programmers won’t get careful explanations of where they went wrong. You just have to figure it out.

“Way more inspiring than conventional coursework”

Alessandro, who was responsible for designing the test exercises, is enthusiastic about this innovative approach. “Actual coding is way tougher, yet more inspiring than the usual pen-and-paper exercises,” he says. “The best way to get that ‘wow’ effect is through struggling through the process of implementing the theory, and eventually seeing your code solve real problems at the scale that modern computers can. 

“And besides being exposed in this way to modern tools, like Docker and Github, students also benefit from many life lessons that are inherent in robotics.” 

Such as the difference between your test cases, and the real world. And the cost of deployment. 

“Field testing on real hardware is expensive. So you need to think through all the requirements, specifications and possible edge cases really carefully.” 

In this course, students in the Github classroom each get a private copy of the exercise repository and can take time to develop their own solutions. Only when they believe their code is ready do they commit it for evaluation, with results automatically posted on their own repository as well as a shared leaderboard. It’s possible to try again if the results are bad – but not endlessly: students are allowed a limited number of submissions for each exercise. That imposes a similar discipline to cost constraints.  

Another real-world skill that is stressed in this course is collaboration, with the last two assignments carried out in small groups. Universities tend to focus on evaluating individuals, but in industrial settings, teamwork is essential. Students here have the opportunity to practise working together on the same codebase, as they will have to do in commercial employment. 

As with so many innovations, this way of doing things seems obvious… once it’s already in place. But getting here wasn’t straightforward. Alessandro and Shengjie faced a daunting task in setting up the necessary software infrastructure to work reliably on all the various machines that students could be using, each with their own architecture and OS. And of course, designing exercises and scoring functions that were fair, while reasonably reflecting the students’ understanding of the material, demanded a great deal of attention and creativity. 

But it has been worth it. Zhengyu Fu, one of the guinea pigs who participated in the first run of this newly automated course, confirms that this “rewarding and engaging coding experience” taught him a lot. “You need to thoroughly understand every aspect of the material in order to transform it into working code. In addition, it requires students to write core parts from scratch rather than simply filling in the blanks.”

And the course designers are convinced of the value their hard work has created. 

“This is something that goes well beyond conventional robotics instruction,” says Emilio. “After my experience starting and growing my own robotics company, I wanted to make sure that students learned not only the theoretical foundations of the field, but also the skills to implement and deploy production-ready code using industry-standard tools. We aren’t just delivering course material: we are preparing students to make an impact in this field. It’s challenging, but it’s fun. And it doesn’t even have to stay within the walls of ETH; the materials are all there on Github. Anyone can try it out for themselves.”