Best way to run many experiments at once

Question

Hello,

I am trying to run many successive bench experiments on the command line, using slightly different parameters one at a time. Do you have any suggestions for a good way to automate this process? Currently I am thinking of using a bash script.

Best,

Chang Hi Lee

Answer 1

You could do that in a bash script. Even better would be to include the iteration over parameters in your carputils script.

The basic bench example can be a good starting point:

• example
• code of the example

Comments

Thanks! I had not thought of using carputils. Just wondering - how big is the efficiency difference between the options? Is using carputils much faster than a bash script?(For 1000's of bench simulations)

---

Hi! There is no efficiency difference regarding bench, but you may gain something if you are also post-processing the results in python. In the end, it comes down to what you are more acustomed to.

For performance, the most efficient thing is to run multiple jobs (as many as you have CPU cores) asynchronously. This can be done both from python and from bash.

---

Indeed, being able to use multiple cores would be great! For bench simulations, is it possible to pass the - -np command? I saw that bench did not recognize it when running it on my installation.

---

I don't think it's supported out of the box but you could use python's multiprocessing module or something similar to spawn multiple bench instances concurrently: https://docs.python.org/3/library/multiprocessing.html