Imbalanced Classes in terms of Structured Data

I am working on Structured data, binary classification problem. The bincount of the two classes are 50000 and 4000. Can u give me some Tips and Tricks so that i can improve my f1 score. I tried up sampling the minority class…it gave very good f1 score on validation data but f1 score was very poor on test data.