Hi,
I already tried Gradient Boosted Trees, Random Forest, ExtraTreesRegressor and Adaboosted Decision trees from scikitlearn with paraneter tuning using GridSearchCV, RepeatedKFold cross validation, stratified k fold cross validation. The best RMSE I got there is 24.3, I want to see if with lesser effort and simpler code neural network to beat ensemble method, hence the try.
@ramesh, there is no other cod, this is all the code, I define a base model, pass it in kerasregressor wrapper and then using 10-fold cross validation on training data. As I told there are only 4500 rows in training data and ~900 features. There is absolutely no business knowledge available to feature engineer and basic engineering proven to be counterproductive. But I am using imputing missing data, standard scaling data etc already.
@both of you: The code tells it clearly that I am doing cross validation on training data already. My best performance using this model is 27 which is much worse than ensemble method so far.
I will post the code in gist.github.com