Some executors do not run the tests in a shell, and so assuming that
they can understand shell builtins is wrong. Instead, call Bash
directly to do what we need to do.
This still requires the executor to be able to run Bash, but at least
it does not require it to interpret command lines in the Bash language.