Guidelines on when to use bash vs python

[asunar]

Just wondering if you have any guidelines on when to use bash vs python. For example one can setup folder structure for lesson 1 homework using bash or python. I found that "!" commands are independent from each other and certain tasks are more readable in python.

Any preferences based on experience?

[ccrome]

Just wondering if you have any guidelines on when to use bash vs python. For example one can setup folder structure for lesson 1 homework using bash or python. I found that "!" commands are independent from each other and certain tasks are more readable in python.

I think the answer is "whatever's easiest for you". I use all of the above, depending on the task.

I often just use an emacs macro for doing things, other times I use the find command, and still others I'll actually use pytohn commands. I suspect it just depends on what you're comfortable with and know the best.

[jeremy]

I think it's good to have everything necessary to reproduce your analysis results in one place - so I like to have all the stuff for creating the validation and sample sets in the same notebook as the analysis.

If you need a more complex script, I'd suggest calling it from the notebook, so you still have a reproducible set of steps.

Reproducibility of analysis is something that I learned in my time competing on kaggle, and talking to the top competitors, is one of the most important things in getting good results.

[davecg]

Anything you can do in bash you can do in python, although usually in a more verbose manner.

If you want to automate bash calls from within a python script, check out the subprocess module, particularly check_call, check_output and Popen.