Your edit is roughly correct. It's a bit odd that they're comparing a single human-made policy with the very best of the DrEureka outputs; in practice, I would expect to make multiple different reward functions and then do validation on the functions and hyperparameters based on the resulting trained models. However, their comparison isn't necessarily wrong, since it seems (I could be wrong) that they used the reward function from prior papers in the field of policy learning.

If you're interested in methods for actually learning policies for these sorts of dynamic motions, note that this paper is simply applying proximal-policy optimization. They're pulling in the training and implementation methods from Margolis's [1] and Shan's [2] work.

So, in sum, the contribution of this paper is exclusively the method for generating reward functions (which is still pretty cool!!!!!), not all the learning-based policy stuff.

[1]: https://web.archive.org/web/20220703005502id_/http://www.rob... [2]: https://arxiv.org/pdf/2309.06440