General ML/ DL tuning prolems

Hello,
One of the things that bothers me is that whether we need to retune parameters each time after generating new features. From one perspective, tuning parameters might help model to fit dataset well, thus give us more accruate results on the performance of new features, but from another perspective it it really frustrating to wait for a long while until training loops are completed. Any suggestions to solve this dilemma? Thanks a lot.