You can pip install isoweek, or include it in the pip section of your environment.yml if you use one.
1 Like
Okey I solved it with pip install from the terminal with the env activated. From the nb it let me install but was not importing.
I have two questions. Do we ever normalise our embeddings, or is that useless since they anyway end up as input to the batch norm layer?
second is, how can we save these embeddings using Fastai?
1 Like
Hi Lawrie, are you installing v1 using a peterjc123 build? If so are you compiling yourself or is there a .whl available?
1 Like
I was building from source when the course first started however the pytorch team began running nightly builds for windows a few weeks later. Here is the current discussion. The peterjc123 builds seem to be failing at the moment but the most recent successful builds appear to be working fine. I am using the conda-3.6 cuda-90 build myself. You can download the tarball from his github page or follow the links to here to download directly.
Hope that helps.
2 Likes
I understand that in NLP and vision, we generally want to maximize the GPU memory and utilization during training. Is that idea generally the same for collaborative filtering and tabular data like the Rossmann competition, or do we often want to have small batch sizes even if they appear inefficient in terms of GPU utilization?
Yes, just pip install. No need for VS but you will need to pip install any dependencies first along with cuda if you are using a gpu.
Besides regularization affect, data augmentation also makes the CNN more robust by increasing its translation and scale invariance,
Yes, bias is also a “learned” parameter that is generally initialized as 0 or a small number in case of relu activations. Generally not randomly initialized as the random initialization of the weights suffices.
1 Like
I had a scenario for tabular data where all the features were continuous. So what feature engineering I can do to prepare such datasets.
@jeremy Could you turn Lesson 6 official resources and updates into wiki so others can contribute or edit? Thanks.
1 Like
Yes. Because we simply create and use pandas categorical variables, you can manually cast a column, including passing ordered=True.
https://pandas.pydata.org/pandas-docs/stable/categorical.html
3 Likes
You shouldn’t ever (AFAICT) need or want to use get_dummies, since that does 1-hot encoding, which is the thing that embeddings allows you to avoid (remember, they are just a computational shortcut for that).
It is not different - if you check the code, you’ll see that we’re simply using nn.Dropout to create that layer.
Great question. In general, no, these particular augmentation methods are unlikely to work well for (say) spectograms. E.g. it doesn’t make sense to flip a spectogram or use perspective warping. So you’ll need to come up with your own domain-specific augmentation methods. It’s easy to create new augmentation functions - but we won’t cover that until part 2, so in the meantime have a look at the implementation of (e.g.) brightness or flip in fastai.
6 Likes
There are only two types of numbers in a neural net: activations (stuff that’s calculated by a layer) and parameters (stuff that’s learned through gradient descent). Since bias isn’t calculated, you know it must be a parameter, so it’s being learned.
2 Likes
Would that just be in the case of time ordinal data since we have a specific month order or any groups - such as a column that has groups of symptoms for cancer patients? Wouldn’t it then make sense to expand to show whether or not a certain category is present or not?