Live coding 15

[jeremy (Jeremy Howard)]

This topic is for discussion of the 15th live coding session.

<<< session 14 ｜ session 16 >>>

• Recording

Links from the walk-thru

• WeightedDL source code
• fastai issue created by Jeremy to add DataBlock.weighted_dataloaders
• How to Install Xrdp Server (Remote Desktop) on Ubuntu 20.04

What was covered

• Xrdp
• tmux
• Weighted data loaders

Video timeline by @mattr and @kurianbenoy

00:00 - Questions
02:00 - Running notebooks in the background from scripts
04:00 - Connecting to your server using Xrdp
07:00 - Can you connect to Paperspace machines remotely?
06:30 - Installing Xrdp
13:30 - Dealing with startup issue in Paperspace
16:20 - Native windows in tmux with tmux -CC a
18:30 - Getting mouse support working in tmux. To be able to scroll the buffer within tmux with your mouse create a config file in your home directory using touch ~/.tmux.config and then within this file include the line set -g mouse on to enable mouse support. Once saved you’ll need to close and reopen tmux to enable the new configuration. Now you will be able to use the mouse to scroll the buffer and even navigate between panes.
20:00 - Experimenting with closing notebooks while fine tuning
24:30 - Progressive resizing recap
26:00 - Building a weighted model
29:00 - Picking out the images that are difficult to classify (Lost Nick in discussion)
33:30 - Weighting with 1/np.sqrt(value_counts())
36:00 - Merging the weights with the image description dataframe
38:00 - Building a datablock
40:20 - How we want weighted dataloaders to work
41:30 - Datasets for indexing into to get an x,y pair
44:00 - Sorting list of weights and image files to match them up
47:30 - Batch transforms not applied because datasets didn’t have this method
49:00 - Reviewing errors
55:00 - Python debugger pdb %debug
56:50 - List comprehension to assign weight to images
59:00 - Use set_index instead of sort_values
59:30 - Review weighted dataloaders with show_batch
1:00:40 - Review of how weighted sampling will work
1:03:30 - Updating the fastai library to make weighted sampling easier
1:04:47 - Is WeightedDL a callback?
1:06:20 - modifying weighted_dataloaders function
1:10:00 - fixing tests for weighted_dataloaders
1:13:40 - editable install of pip in fastai
1:16:15 - modifying PETS notebooks to work on splitter
1:18:07 - How does splitters work in datablock
1:20:25 - modifying weighted dataloader by using weights from Dataset
1:23:36 - running tests in fastai and creating github issues
1:24:33 - fixing the failing test
1:29:40 - creating a commit to fix the issue
1:31:30 - nbdev hook to clean notebooks

[radek]

another amazing walk-thru!

I can’t wrap my head around that setting it all up to RDP into your box is so easy!

I still plan to mostly use scripts for longer-running training (just a plan for now, haven’t made the jump yet ) but it is amazing to have this ability to execute nbs in the browser on the same machine that they are run!!!

One interesting thing I noticed is that the screen draw time was very slow, I think that is because I don’t have display drivers installed and everything is running on the CPU? I wonder if that would be different if I had them installed

But the nice upside here is that I don’t have anything eating up my GPU RAM! (which was a slight concern when running Ubuntu Desktop with a single GPU)

one other neat thing that I discoverd today is argcomplete! you can have tab completion for your Python scripts… wow

[mike.moloch (github:aeamaea)]

@radek

One interesting thing I noticed is that the screen draw time was very slow, I think that is because I don’t have display drivers installed and everything is running on the CPU? I wonder if that would be different if I had them installed

I have noticed this when RDP’ing into my Linux machine from two different OSes … in my case I noticed my RDP from the macOS was slower than when I RDP’d into it from a windows machine. So, believe it or not, I’d RDP from my Macbook into my Windows machine (on the same remote network as the linux box with GPU+Conda+Pytorch+fastai) and then from that machine RDP into the linux box.

I haven’t noticed too much GPU usage on the Linux local side but then again, I use XFCE because at the time I started doing it, my linux version had problems allowing RDP on Ubuntu (so I used Linux Mint + XFCE specifically so I could easily setup RDP) Things may have changed now.

I think if your monitor (if you have any on the remote linux box) is not attached to a port on your GPU card, you won’t see Ubuntu use GPU resources. The graphics chip on your motherboard is good enough to draw a GUI so I wouldn’t attach the monitor to the GPU on the remote linux machine with GPUS (unless of course you use it for gaming as well, in which case you sort of have to take the hit )

[brismith (Brian Smith)]

I’m just catching up on the recording - another great session! .tmux.config and ‘mouse on’ will get me back to using tmux - the scroll wheel behaviour was driving me mad. Interesting observation from Jeremy on the remote desktop client in the Mac being better than that in Windows - we had the same conversation internally at Microsoft recently wondering why that was ;). Having just become the owner of my first Mac I am appreciating it too. Now deciding if I should re-image my old Windows box (i9 processor and 1080ti) with Linux. And maybe add another GPU as the prices come down.

[fmussari (Francisco Mussari)]

From Paddy Competition, some pictures that are almost identical (found manually).
There are others that are very similar but a little bit different.

• Label “Hispa”, variety “ADT45”, age 70

• 100804.jpg
• 100867.jpg
• 100918.jpg

• Label “Normal”, variety “ADT45”, age 60

• 100506.jpg
• 100583.jpg
• 101426.jpg

• Label “Bacterial Leaf Blight”, variety “ADT45”, age 55

• 100447.jpg
• 101169.jpg
• 102166.jpg

• Label “Bacterial Leaf Streak”, variety “Zonal”, age 70

• 100552.jpg
• 101461.jpg
• 101600.jpg
• 102137.jpg

From the test set

• 200748.jpg
• 200760.jpg

• 201084.jpg
• 200634.jpg

[radek]

This, my friends, is madness

Apparently, I now have a full-fledged desktop on my ubuntu server!

I had some network issues where I had to hook up a monitor to my rig, and lo and behold, there is a desktop there now!

@radek

I still plan to mostly use scripts for longer-running training (just a plan for now, haven’t made the jump yet ) but it is amazing to have this ability to execute nbs in the browser on the same machine that they are run!!!

Turns out there is no delay in drawing the screen, it was just probably the high resolution of the screen being sent over by RDP that was causing the draw issues.

Thx to a post by Rachel I switched to using a trackball (mx log trackball + keyboard) where I can switch Bluetooth inputs on the keyboard and the mouse, so I now have “something” like a KVM setup!

I am still not planning to use the desktop (too much hassle switching back and forth), but that is really damn cool. I have a place where I can run i3wm if I ever want to, and this might be super useful down the road. You might say – why not use ubuntu desktop? It has been super painful to me with all the weird small issues between using the GPU for deeplearning and as a graphics card (slowness, restarts, issues with sleeping and waking from sleep, some of these were motherboard and not GPU related). I ultimately resolved most of the problems, but for foreseeable future I am sticking to just Ubuntu Server.

And I can see how being able to have such a setup (especially with a one-button KVM) can be super extremely useful for productivity and workflow So happy I now know this exists!

[jeremy (Jeremy Howard)]

If anyone has a few minutes to spare today, I’d be really grateful if you could add the timestamps for the last few minutes of the timeline (see first post in the topic).

[kurianbenoy]

Have added the time stamps for the last portion also. Thanks @Mattr for creating the earlier time stamps of these walk-thrus.

[fmussari (Francisco Mussari)]

Checking the images that are (h=480, w=640), we can see that they capture the same part of the paddy with some differences in angle, composition or bluring:

bacterial_leaf_blight

ADT45, Age: 50

• bacterial_leaf_blight/100622.jpg
• bacterial_leaf_blight/103734.jpg

Very similar.

brown_spot

ADT45, Age: 70

• brown_spot/103050.jpg
• brown_spot/103343.jpg

Similar, but not that much.

So they are like hand made (or wind made) augmentations of one-another.

[nikem (Niyazi Kemer)]

After Jeremy’s fix to weighted_dataloaders, I’ve tried the upgrade the fastai but couldn’t manage to do it. /mambaforge/lib/python3.9/site-packages/fastai/callback/data.py is still the same. What is the proper way to upgrade fastai?

[FraPochetti (Francesco Pochetti)]

did you try with pip install ... --upgrade?
Actually, now that I think about it. Did he actually publish to pip or is the change only available on the master branch?

[jeremy (Jeremy Howard)]

@nikem

What is the proper way to upgrade fastai

I only just released the new version today.

[nikem (Niyazi Kemer)]

Yes I tried that way, eventually, It could be considered a crime but I manually replace data.py file by downloading it from Github and it works. Yes, you are right, Jeremy’s published it just today. I didn’t know the process before, I’ve been thinking that it would’ve been pushed automatically. Thanks

[nikem (Niyazi Kemer)]

Ok great, I replicate codes after walkthroughs and couldn’t do it for Walkthru-15 after the fix part. Now I understand, thanks.

[Daniel (深度碎片)]

Walkthru 15 a rough detailed note in the form of questions and images

Built upon very nice video timeline by @mattr and @kurianbenoy

02:00 - Can we sleep while GPU is working on the models without losing everything in the morning

04:00 - Connecting to your server using Xrdp

07:00 - Can you connect to Paperspace machines remotely?

06:30 - Installing Xrdp

13:30 - Dealing with startup issue in Paperspace

15:52 Can we sleep and let paperspace working on? (paid user should be ok)

16:20 - Native windows in tmux with tmux -CC a

18:30 - Getting mouse support working in tmux

20:00 - Experimenting with closing notebooks while fine tuning

24:30 How to use a extra large model to apply progressive resizing?

How does ( with halved learning rate) Gradient accumulation can help you train with bigger images with less time?

Images

Why do we need to set gradient accumulation to 4 on Kaggle instead of 2 on Jeremy’s computer?

Images

26:00 - Building a model with weighted training set

duplicate Road to top 2 notebook for weighted model 28:15

32:35 Students said applying the techniques like confusion matrix etc learned in lecture 1-2 for paddy has not been easy

33:10 Why or when should we use weighted_dataloaders to a dataset

hint: some classes are harder; or some are less common

33:33 How to create the weights for data to show less common ones more often and vice verse in both a dramatic method and less dramatic method?

Images

34:40 Do we face the danger of overfitting model trained with weighted dataset to think the less common classes are more common?

36:14 How to give weights to each item based on the item’s class/disease names (including the weights into the dataframe of train.csv).

Images

37:41 According to where WeightedDataLoader is defined, we have to build a DataBlock first, and then create a datasets upon dblock

Images

42:41 How Jeremy figure out the way to use Weighted_dataloaders using docs and source?

Images

43:49 How to map the dataframe where weights are stored to the files returned from get_image_files?

How Jeremy used a lazier method (sorting) to do the mapping above? 44:46?

Images

46:52 How Jeremy explore and experiment to make Weighted_dataloaders work in real time?

#explore

47:48 How did Jeremy explore and experiment to force Datasets.weighted_dataloaders to apply batch_tfms and item_tfms which are actually used only happen in DataBlock.dataloaders?

#jeremy-explore-source

Images

55:04 How to use pdb to debug an error directly? (weights should only include training set not validation set)

h: for help

w: where in the stack you are in

p self.n: to print out self.n

q: to exit

What exactly is the problem?

Images

56:30 How to pick out weights only for the training set using pandas index func?

Images

MISSING IMAGE: good-mix

[/details]

1:00:55 How the batch is created in weighted dataset?

hint: a 64 images are sampled from a dataset weighted by the weights, not the normal dataset

What does model on weighted dataset see in one epoch?

no longer the full dataset once, but a lot more less-common items, so overfitting is a danger

Why we should expect the error rate of weighted model to be worse?

1:02:26 When to use weighted dataset to train the model is more appropriate?

you could train the model well with proper dataset enough times and then use weighted dataset to train only a few epochs

other ways of using weighted dataset?

1:03:24 How should weighted_dataloaders work?

Images

1:03:54 How to fix up weighted_dataloaders and update the fastai library?

step 1: git pull to get the latest fastai lib

step 2: go to fastai/nbs/14a_callback.data.ipynb and find weighted_dataloaders

step 3: make changes

1:04:44 Is Weighted_dataloaders a callback?

it is a dataloader and a patch to a datasets

1:05:32 How does @patch add a method/function weighted_dataloaders to a class Datasets?

Images

1:06:02 How to create a weighted_dataloaders for DataBlock class and another weighted_dataloaders for Datasets class?

Images

1:09:23 How doe @delegates(Datasets.dataloaders) pass all its parameters to return self.dataloaders(....) and then to return dss.weighted_dataloaders(...)

Images

1:09:47 How to create/fix the tests

Images

1:13:23 How to export and try out the updated fastai lib?

How to export the library?

Images

How update and use the new lib locally?

Images

1:14:37 How Jeremy would like splitter to work nicely with DataBlock.weighted_dataloaders

Images

1:17:56 How does splitter work inside Datasets to split training and validation sets

Images

1:21:19

The final version of DataBlock.weighted_dataloaders

Images

Now, it all works

Images

1:23:24 run test and create an issue about updating lib

How to run test?

Images

What’s the error in the test of DataBlock.weighted_dataloaders

Images

How to create an issue from terminal?

Images

#question where are Jeremy’s dotfiles?

1:24:33 To fix the test error by converting a list into an np.array when wgts is a list

How to check whether wgts is a Tensor or np.array with hasattr?

Images

How to update or fix the issue

Images

1:29:58 How to use nbdev hook to clean notebooks

[jeremy (Jeremy Howard)]

@Daniel we’ve written an app for you!:

Daniel Img Fix - a Hugging Face Space by jph00

See the details in the latest live-coding session. I ran it on the above summary and edited your post with the results. Basically, you just paste the forum markdown into “dst” and the list of image links into “src” and click “submit”. Then copy the output into the forum post.

Feel free to fork and edit as needed, or let me know if you’d like us to make changes.

[Daniel (深度碎片)]

@jeremy

Basically, you just paste the forum markdown into “dst” and the list of image links into “src” and click “submit”. Then copy the output into the forum post.

Wow! this works great! Thank you Jeremy and everyone!

[Daniel (深度碎片)]

Walkthru 15 a rough detailed note in the form of questions and images

Built upon very nice video timeline by @mattr and @kurianbenoy

02:00 - Can we sleep while GPU is working on the models without losing everything in the morning

04:00 - Connecting to your server using Xrdp

07:00 - Can you connect to Paperspace machines remotely?

06:30 - Installing Xrdp

13:30 - Dealing with startup issue in Paperspace

15:52 Can we sleep and let paperspace working on? (paid user should be ok)

16:20 - Native windows in tmux with tmux -CC a

18:30 - Getting mouse support working in tmux

20:00 - Experimenting with closing notebooks while fine tuning

24:30 How to use a extra large model to apply progressive resizing?

How does ( with halved learning rate) Gradient accumulation can help you train with bigger images with less time?

Images

Why do we need to set gradient accumulation to 4 on Kaggle instead of 2 on Jeremy’s computer?

Images

26:00 - Building a model with weighted training set

duplicate Road to top 2 notebook for weighted model 28:15

32:35 Students said applying the techniques like confusion matrix etc learned in lecture 1-2 for paddy has not been easy

33:10 Why or when should we use weighted_dataloaders to a dataset

hint: some classes are harder; or some are less common

33:33 How to create the weights for data to show less common ones more often and vice verse in both a dramatic method and less dramatic method?

Images

34:40 Do we face the danger of overfitting model trained with weighted dataset to think the less common classes are more common? yes

36:14 How to give weights to each item based on the item’s class/disease names (including the weights into the dataframe of train.csv)

Images

37:41 According to where WeightedDataLoader is defined, we have to build a DataBlock first, and then create a datasets upon dblock

Images

42:41 How Jeremy figure out the way to use Weighted_dataloaders using docs and source?

Images

43:49 How to map the dataframe where weights are stored to the files returned from get_image_files?

How Jeremy used a lazier method (sorting) to do the mapping above? 44:46?

Images

46:52 How Jeremy explore and experiment to make Weighted_dataloaders work in real time?

#explore

47:48 How did Jeremy explore and experiment to force Datasets.weighted_dataloaders to apply batch_tfms and item_tfms which are actually used only happen in DataBlock.dataloaders?

#jeremy-explore-source

Images

55:04 How to use pdb to debug an error directly? (weights should only include training set not validation set)

h: for help

w: where in the stack you are in

p self.n: to print out self.n

q: to exit

What exactly is the problem?

Images

56:30 How to pick out weights only for the training set using pandas index func?

Images

MISSING IMAGE: good-mix

[/details]

1:00:55 How the batch is created in weighted dataset?

hint: a 64 images are sampled from a dataset weighted by the weights, not the normal dataset

What does model on weighted dataset see in one epoch?

no longer the full dataset once, but a lot more less-common items, so overfitting is a danger

Why we should expect the error rate of weighted model to be worse?

1:02:26 When to use weighted dataset to train the model is more appropriate?

you could train the model well with proper dataset enough times and then use weighted dataset to train only a few epochs

other ways of using weighted dataset?

1:03:24 How should weighted_dataloaders work?

Images

1:03:54 How to fix up weighted_dataloaders and update the fastai library?

step 1: git pull to get the latest fastai lib

step 2: go to fastai/nbs/14a_callback.data.ipynb and find weighted_dataloaders

step 3: make changes

1:04:44 Is Weighted_dataloaders a callback?

it is a dataloader and a patch to a datasets

1:05:32 How does @patch add a method/function weighted_dataloaders to a class Datasets?

Images

1:06:02 How to create a weighted_dataloaders for DataBlock class and another weighted_dataloaders for Datasets class?

Images

1:09:23 How doe @delegates(Datasets.dataloaders) pass all its parameters to return self.dataloaders(....) and then to return dss.weighted_dataloaders(...)

Images

1:09:47 How to create/fix the tests

Images

1:13:23 How to export and try out the updated fastai lib?

How to export the library?

Images

How update and use the new lib locally?

Images

1:14:37 How Jeremy would like splitter to work nicely with DataBlock.weighted_dataloaders

Images

1:17:56 How does splitter work inside Datasets to split training and validation sets

Images

1:21:19

The final version of DataBlock.weighted_dataloaders

Images

Now, it all works

Images

1:23:24 run test and create an issue about updating lib

How to run test?

Images

What’s the error in the test of DataBlock.weighted_dataloaders

Images

How to create an issue from terminal?

Images

#question where are Jeremy’s dotfiles?

1:24:33 To fix the test error by converting a list into an np.array when wgts is a list

How to check whether wgts is a Tensor or np.array with hasattr?

Images

How to update or fix the issue

Images

1:29:58 How to use nbdev hook to clean notebooks

[lucasvw (Lucas)]

On the class imbalances issue and the weighted dataloaders solution for that: it bugs me a bit that an epoch is no longer guaranteed to seeing every image in the training set ( I think I heard that an epoch is defined as training on as many samples as there are in the training set, with a normal dataloader that would make sure that you see each sample once, but with the weighted dataloader we chose training samples stochastically, so there is no guarantee that we see each sample).

I was wondering whether the following approach would also work:

1. identify a (number) of categories that are underrepresented in the data
2. duplicate the images (perhaps multiple times) in the training set that are associated with these categories
3. do this possibly with a symlink to make sure you don’t use any diskspace

[lucasvw (Lucas)]

And another question which plays a role with imbalanced classes: what guarantee do we have that each class is both represented in the training and validation set? Is fastai using some kind of a stratified split?