Lesson 14 wiki


(Jeremy Howard) #21

Thanks - that’s much nicer!


(Yash Katariya) #22

@jeremy @rachel I was implementing the Tiramisu code and got an error in the ‘color_label’ function. This is the link my notebook https://gist.github.com/yashk2810/0aa2aa0af70c5c23cb1fc20c01277e62


Also, which dataset are you using? I used this [https://github.com/mostafaizz/camvid](https://github.com/mostafaizz/camvid). The shape of your labels is (701, 360, 480) but mine is (701, 360, 480, 3).

Moreover, you created labels_int but replaced it with labels in the Test set section. So why are you creating that variable?


(Jeremy Howard) #23

Don’t take the details of that notebook too seriously, especially around how the labels are manipulated - I was using different labels from different places as I found various datasets. You should focus on what dataset you’re using, and figure out what steps you need to take - just use my code as a source of tips if you get stuck; don’t use it as something you just run through from top to bottom!


(Milton Wong) #25

Hi, all,

I try to run through “tiramisu-keras.ipynb”. However, I came across this error at model.fit_generator() function.

I guess that’s the Keras version mismatch problem. Could someone reply me the exact Keras version as well as tensorflow version for running “tiramisu-keras.ipynb” ?

(My system currently runs Keras 1.2.2 and tensorflow 1.0, in python 3.5)

Thanks!


(Constantin) #26

@amiltonwong, I don’t think this is a version conflict. You aren’t passing the right data to fit_generator. From your screenshot I can not tell what you are actually passing, but gen_test should be a generator which outputs a tuple of (X_val, y_val), being images and labels respectively. The value passed to nb_val_samples should be an integer of how many samples the generator will pass to the model.


(Charles Lu) #27

This is changed in Keras 2, I believe? Do you know how to modify the generators for the new fit_generator function?


(syed danish) #29

Did you get any solution?


(Yash Katariya) #30

@alibaba Yes. You can refer my code https://github.com/yashk2810/Semantic-Image-Segmentation


(syed danish) #31

Thanks !!


(Akshay) #32

@iNLyze I Was Able to get 87 % accuracy on train set and 84.06 % on Validation set…using the below Notebook with almost 500 Epochs…

Below is my link to the notebook

Pls Suggest…

@iNLyze if you could Share your notebook for reference…


(Constantin) #33

Does anyone know an automatic way of deploying a notebook with all its dependencies to GitHub? Checking manually is time consuming.


(Constantin) #34

@akshaylamba, done it. Let me know what you think.

here


(Sander Stepanov) #35

pla help understand how to find pages for another lectures like lesson 13


(alex) #36

When is the podcast coming out?! :slight_smile: :slight_smile: :slight_smile:


(Eric Perbos-Brinck) #38

Note: the complete collection of Part 2 video timelines is available in a single thread for keyword search.
Part 2: complete collection of video timelines

Lesson 14 video timeline:

00:01:25 Time-Series and Structured Data
& “Patient Mortality Risk Predictions in Pediatric Intensive Care, using RNN’s” (research)

00:07:30 Time-Series with Rossmann Store Sales (Kaggle)
& 3rd place solution with "a very uncool NN ^!^ ".

00:18:00 Implementing the Rossman solution with Keras + TensorFlow + Pandas + Sklearn
Building Tables & Exploratory Data Analysis (EDA)

00:27:15 Digress: categorical variable encodings and “Vtreat for R”

00:30:15 Back to Rossmann solution
& “Python for Data Analysis” (book)

00:36:30 What Jeremy does everytime he sees a ‘date’ in a structured ML model
& other tips

00:43:00 Dealing with duration of special events (holidays, promotions) in Time-Series

00:52:00 Using ‘inplace=True’ in .drop(), & a look at our final ‘feature engineering’ results

00:53:40 Starting to feed our NN
& using pickle.dump() for storage encodings

01:00:45 “Their big mistake” and how they could have won #1

01:05:30 Splitting into Training and Test, but not randomly

01:08:20 Why they modified their Sales Target with np.log()/max_log_y

01:11:20 A look at our basic model

01:14:45 Training our model and questions

01:16:45 Running the same model with XGBoost

01:20:10 “The really, really, really weird things here !”
& end of the Rossmann coverage :wink:

01:26:30 Taxi Trajectory Prediction (Kaggle) with “another uncool NN” Time-Series winner

01:38:00 “Start with a Conv layer and pass it to an RNN” question and research

01:42:40 The 100-layers Tiramisu: Fully Convolutional DenseNets, for Image Segmentation (Lesson 13 cont.)

01:58:00 Building and training the Tiramisu model

02:02:50 ENet and LINKNet models: better than the Tiramisu ?

02:04:00 Part 2: conclusion and next steps


(James Dietle) #39

Where do we find the data for “pkl_path = ‘/data/jhoward/github/entity-embedding-rossmann/’”?

  • We don’t seem to generate it in the code.
  • It’s not in the files downloaded.
  • I don’t see it in the Wiki.
  • Haven’t found it on Github.

Did I just miss it?


(Jaime) #40

The file is created with the code in prepare_features.py you can find in the original paper github repo


(Ben Johnson) #41

Hi All –

I have a question about the performance of @jeremy’s Rossmann model. It looks like he gets to loss=0.1108 w/ the neural network and loss=0.1138 w/ the GBM. However, on the Kaggle leaderboard, Neokami gets down to loss=0.10583.

Does anyone know what the explanation for the gap is? Are the train/eval datasets slightly different? Is the performance reported in the notebook actually from one of Jeremy’s variations on the original model? I haven’t had a chance to step through the code yet, but thought someone here may have some insight.

Thanks
Ben


(keyu nie) #42

He said that it might because of the third place removed all no-sales data point.


(keyu nie) #43

I have a question for Rossman’s 3rd place method. I checked their github code and I can successfully repeat the model training part for both version. But however I have difficulties on get the embedding layer weights.

I am trying to use some code like: model.layers[0].get_weights() to get embedding layer’s weights. it returns “[]”, an empty list. Any idea how can I get that weights part?

The model structure is:


Layer (type) Output Shape Param # Connected to

embedding_51 (Embedding) (None, 1, 50) 55750


reshape_51 (Reshape) (None, 50) 0


embedding_52 (Embedding) (None, 1, 6) 42


reshape_52 (Reshape) (None, 6) 0


dense_25 (Dense) (None, 1) 2


embedding_53 (Embedding) (None, 1, 2) 6


reshape_53 (Reshape) (None, 2) 0


embedding_54 (Embedding) (None, 1, 6) 72


reshape_54 (Reshape) (None, 6) 0


embedding_55 (Embedding) (None, 1, 10) 310


reshape_55 (Reshape) (None, 10) 0


embedding_56 (Embedding) (None, 1, 3) 12


reshape_56 (Reshape) (None, 3) 0


dense_26 (Dense) (None, 1) 2


embedding_57 (Embedding) (None, 1, 2) 50


reshape_57 (Reshape) (None, 2) 0


embedding_58 (Embedding) (None, 1, 1) 26


reshape_58 (Reshape) (None, 1) 0


embedding_59 (Embedding) (None, 1, 1) 4


reshape_59 (Reshape) (None, 1) 0


dense_27 (Dense) (None, 1) 2


embedding_60 (Embedding) (None, 1, 2) 10


reshape_60 (Reshape) (None, 2) 0


embedding_61 (Embedding) (None, 1, 3) 12


reshape_61 (Reshape) (None, 3) 0


embedding_62 (Embedding) (None, 1, 3) 12


reshape_62 (Reshape) (None, 3) 0


embedding_63 (Embedding) (None, 1, 4) 72


reshape_63 (Reshape) (None, 4) 0


embedding_64 (Embedding) (None, 1, 4) 32


reshape_64 (Reshape) (None, 4) 0


embedding_65 (Embedding) (None, 1, 6) 72


reshape_65 (Reshape) (None, 6) 0


embedding_66 (Embedding) (None, 1, 2) 106


reshape_66 (Reshape) (None, 2) 0


dense_28 (Dense) (None, 3) 12


dense_29 (Dense) (None, 3) 12


dense_30 (Dense) (None, 2) 6


dense_31 (Dense) (None, 1) 2


embedding_67 (Embedding) (None, 1, 4) 88


reshape_67 (Reshape) (None, 4) 0


embedding_68 (Embedding) (None, 1, 1) 8


reshape_68 (Reshape) (None, 1) 0


embedding_69 (Embedding) (None, 1, 1) 8


reshape_69 (Reshape) (None, 1) 0


embedding_70 (Embedding) (None, 1, 1) 8


reshape_70 (Reshape) (None, 1) 0


embedding_71 (Embedding) (None, 1, 1) 8


reshape_71 (Reshape) (None, 1) 0


embedding_72 (Embedding) (None, 1, 1) 3


reshape_72 (Reshape) (None, 1) 0


embedding_73 (Embedding) (None, 1, 1) 3


reshape_73 (Reshape) (None, 1) 0


embedding_74 (Embedding) (None, 1, 1) 8


reshape_74 (Reshape) (None, 1) 0


embedding_75 (Embedding) (None, 1, 1) 8


reshape_75 (Reshape) (None, 1) 0


dense_32 (Dense) (None, 1) 2


dense_33 (Dense) (None, 1) 2


dropout_3 (Dropout) (None, 131) 0 merge_3[0][0]


dense_34 (Dense) (None, 1000) 132000 dropout_3[0][0]


activation_7 (Activation) (None, 1000) 0 dense_34[0][0]


dense_35 (Dense) (None, 500) 500500 activation_7[0][0]


activation_8 (Activation) (None, 500) 0 dense_35[0][0]


dense_36 (Dense) (None, 1) 501 activation_8[0][0]


activation_9 (Activation) (None, 1) 0 dense_36[0][0]

Total params: 689,773
Trainable params: 689,773
Non-trainable params: 0