Lesson 14 wiki

(Jeremy Howard) #21

Thanks - that’s much nicer!

(Yash Katariya) #22

@jeremy @rachel I was implementing the Tiramisu code and got an error in the ‘color_label’ function. This is the link my notebook https://gist.github.com/yashk2810/0aa2aa0af70c5c23cb1fc20c01277e62

Also, which dataset are you using? I used this [https://github.com/mostafaizz/camvid](https://github.com/mostafaizz/camvid). The shape of your labels is (701, 360, 480) but mine is (701, 360, 480, 3).

Moreover, you created labels_int but replaced it with labels in the Test set section. So why are you creating that variable?

(Jeremy Howard) #23

Don’t take the details of that notebook too seriously, especially around how the labels are manipulated - I was using different labels from different places as I found various datasets. You should focus on what dataset you’re using, and figure out what steps you need to take - just use my code as a source of tips if you get stuck; don’t use it as something you just run through from top to bottom!

(Milton Wong) #25

Hi, all,

I try to run through “tiramisu-keras.ipynb”. However, I came across this error at model.fit_generator() function.

I guess that’s the Keras version mismatch problem. Could someone reply me the exact Keras version as well as tensorflow version for running “tiramisu-keras.ipynb” ?

(My system currently runs Keras 1.2.2 and tensorflow 1.0, in python 3.5)


(Constantin) #26

@amiltonwong, I don’t think this is a version conflict. You aren’t passing the right data to fit_generator. From your screenshot I can not tell what you are actually passing, but gen_test should be a generator which outputs a tuple of (X_val, y_val), being images and labels respectively. The value passed to nb_val_samples should be an integer of how many samples the generator will pass to the model.

(Charles Lu) #27

This is changed in Keras 2, I believe? Do you know how to modify the generators for the new fit_generator function?

(syed danish) #29

Did you get any solution?

(Yash Katariya) #30

@alibaba Yes. You can refer my code https://github.com/yashk2810/Semantic-Image-Segmentation

(syed danish) #31

Thanks !!

(Akshay) #32

@iNLyze I Was Able to get 87 % accuracy on train set and 84.06 % on Validation set…using the below Notebook with almost 500 Epochs…

Below is my link to the notebook

Pls Suggest…

@iNLyze if you could Share your notebook for reference…

(Constantin) #33

Does anyone know an automatic way of deploying a notebook with all its dependencies to GitHub? Checking manually is time consuming.

(Constantin) #34

@akshaylamba, done it. Let me know what you think.


(Sander Stepanov) #35

pla help understand how to find pages for another lectures like lesson 13

(alex) #36

When is the podcast coming out?! :slight_smile: :slight_smile: :slight_smile:

(Eric Perbos-Brinck) #38

Note: the complete collection of Part 2 video timelines is available in a single thread for keyword search.
Part 2: complete collection of video timelines

Lesson 14 video timeline:

00:01:25 Time-Series and Structured Data
& “Patient Mortality Risk Predictions in Pediatric Intensive Care, using RNN’s” (research)

00:07:30 Time-Series with Rossmann Store Sales (Kaggle)
& 3rd place solution with "a very uncool NN ^!^ ".

00:18:00 Implementing the Rossman solution with Keras + TensorFlow + Pandas + Sklearn
Building Tables & Exploratory Data Analysis (EDA)

00:27:15 Digress: categorical variable encodings and “Vtreat for R”

00:30:15 Back to Rossmann solution
& “Python for Data Analysis” (book)

00:36:30 What Jeremy does everytime he sees a ‘date’ in a structured ML model
& other tips

00:43:00 Dealing with duration of special events (holidays, promotions) in Time-Series

00:52:00 Using ‘inplace=True’ in .drop(), & a look at our final ‘feature engineering’ results

00:53:40 Starting to feed our NN
& using pickle.dump() for storage encodings

01:00:45 “Their big mistake” and how they could have won #1

01:05:30 Splitting into Training and Test, but not randomly

01:08:20 Why they modified their Sales Target with np.log()/max_log_y

01:11:20 A look at our basic model

01:14:45 Training our model and questions

01:16:45 Running the same model with XGBoost

01:20:10 “The really, really, really weird things here !”
& end of the Rossmann coverage :wink:

01:26:30 Taxi Trajectory Prediction (Kaggle) with “another uncool NN” Time-Series winner

01:38:00 “Start with a Conv layer and pass it to an RNN” question and research

01:42:40 The 100-layers Tiramisu: Fully Convolutional DenseNets, for Image Segmentation (Lesson 13 cont.)

01:58:00 Building and training the Tiramisu model

02:02:50 ENet and LINKNet models: better than the Tiramisu ?

02:04:00 Part 2: conclusion and next steps

(James Dietle) #39

Where do we find the data for “pkl_path = ‘/data/jhoward/github/entity-embedding-rossmann/’”?

  • We don’t seem to generate it in the code.
  • It’s not in the files downloaded.
  • I don’t see it in the Wiki.
  • Haven’t found it on Github.

Did I just miss it?

(Jaime) #40

The file is created with the code in prepare_features.py you can find in the original paper github repo

(Ben Johnson) #41

Hi All –

I have a question about the performance of @jeremy’s Rossmann model. It looks like he gets to loss=0.1108 w/ the neural network and loss=0.1138 w/ the GBM. However, on the Kaggle leaderboard, Neokami gets down to loss=0.10583.

Does anyone know what the explanation for the gap is? Are the train/eval datasets slightly different? Is the performance reported in the notebook actually from one of Jeremy’s variations on the original model? I haven’t had a chance to step through the code yet, but thought someone here may have some insight.


(keyu nie) #42

He said that it might because of the third place removed all no-sales data point.

(keyu nie) #43

I have a question for Rossman’s 3rd place method. I checked their github code and I can successfully repeat the model training part for both version. But however I have difficulties on get the embedding layer weights.

I am trying to use some code like: model.layers[0].get_weights() to get embedding layer’s weights. it returns “[]”, an empty list. Any idea how can I get that weights part?

The model structure is:

Layer (type) Output Shape Param # Connected to

embedding_51 (Embedding) (None, 1, 50) 55750

reshape_51 (Reshape) (None, 50) 0

embedding_52 (Embedding) (None, 1, 6) 42

reshape_52 (Reshape) (None, 6) 0

dense_25 (Dense) (None, 1) 2

embedding_53 (Embedding) (None, 1, 2) 6

reshape_53 (Reshape) (None, 2) 0

embedding_54 (Embedding) (None, 1, 6) 72

reshape_54 (Reshape) (None, 6) 0

embedding_55 (Embedding) (None, 1, 10) 310

reshape_55 (Reshape) (None, 10) 0

embedding_56 (Embedding) (None, 1, 3) 12

reshape_56 (Reshape) (None, 3) 0

dense_26 (Dense) (None, 1) 2

embedding_57 (Embedding) (None, 1, 2) 50

reshape_57 (Reshape) (None, 2) 0

embedding_58 (Embedding) (None, 1, 1) 26

reshape_58 (Reshape) (None, 1) 0

embedding_59 (Embedding) (None, 1, 1) 4

reshape_59 (Reshape) (None, 1) 0

dense_27 (Dense) (None, 1) 2

embedding_60 (Embedding) (None, 1, 2) 10

reshape_60 (Reshape) (None, 2) 0

embedding_61 (Embedding) (None, 1, 3) 12

reshape_61 (Reshape) (None, 3) 0

embedding_62 (Embedding) (None, 1, 3) 12

reshape_62 (Reshape) (None, 3) 0

embedding_63 (Embedding) (None, 1, 4) 72

reshape_63 (Reshape) (None, 4) 0

embedding_64 (Embedding) (None, 1, 4) 32

reshape_64 (Reshape) (None, 4) 0

embedding_65 (Embedding) (None, 1, 6) 72

reshape_65 (Reshape) (None, 6) 0

embedding_66 (Embedding) (None, 1, 2) 106

reshape_66 (Reshape) (None, 2) 0

dense_28 (Dense) (None, 3) 12

dense_29 (Dense) (None, 3) 12

dense_30 (Dense) (None, 2) 6

dense_31 (Dense) (None, 1) 2

embedding_67 (Embedding) (None, 1, 4) 88

reshape_67 (Reshape) (None, 4) 0

embedding_68 (Embedding) (None, 1, 1) 8

reshape_68 (Reshape) (None, 1) 0

embedding_69 (Embedding) (None, 1, 1) 8

reshape_69 (Reshape) (None, 1) 0

embedding_70 (Embedding) (None, 1, 1) 8

reshape_70 (Reshape) (None, 1) 0

embedding_71 (Embedding) (None, 1, 1) 8

reshape_71 (Reshape) (None, 1) 0

embedding_72 (Embedding) (None, 1, 1) 3

reshape_72 (Reshape) (None, 1) 0

embedding_73 (Embedding) (None, 1, 1) 3

reshape_73 (Reshape) (None, 1) 0

embedding_74 (Embedding) (None, 1, 1) 8

reshape_74 (Reshape) (None, 1) 0

embedding_75 (Embedding) (None, 1, 1) 8

reshape_75 (Reshape) (None, 1) 0

dense_32 (Dense) (None, 1) 2

dense_33 (Dense) (None, 1) 2

dropout_3 (Dropout) (None, 131) 0 merge_3[0][0]

dense_34 (Dense) (None, 1000) 132000 dropout_3[0][0]

activation_7 (Activation) (None, 1000) 0 dense_34[0][0]

dense_35 (Dense) (None, 500) 500500 activation_7[0][0]

activation_8 (Activation) (None, 500) 0 dense_35[0][0]

dense_36 (Dense) (None, 1) 501 activation_8[0][0]

activation_9 (Activation) (None, 1) 0 dense_36[0][0]

Total params: 689,773
Trainable params: 689,773
Non-trainable params: 0