For your first question, have you tried plotting the entire (not truncated) graph?
Well he could certainly do that and maybe he would see an upward slope. But that would mean that the optimal learning rate is smaller than 10E-6. This seems suspiciously small given the learning rates we usually use.
It suggests you don’t need to unfreeze and fine-tune the lower layers. Is that what you found when you tried?
I wonder why we download and use only .jpg images? Could we use .png or other types of images mixed together or even only .png type images? What is the difference and couldn’t we just somehow convert type from .png to .jpg?
That’s an interesting idea. But we might end up with too many categories (28 * 4).
I’ve no experience in running with that many categories. But I have seen some good results even with such an amount in some other threads.
Give it a try
Lot of work already done in this area.
One example. [Project] Fully convolutional watermark removal
There are more resources that make it easy to remove watermarks.
You could. I believe Resnet works with both image types. This article may help
Nothing changes, actually. It just happens that the loss surface is a N-dimensional hypersurface embedded in a N+1-dimensional euclidean space.
It still got maxima and minima (but see below), and it’s generally quite bumpy. For, we make good use of cyclical learning rates and restarts: they’ll get you out of bad local minima.
Another important consideration is that as the dimensionality increases, almost every stationary point is a saddle point (= ability to escape).
This is not difficult to believe: intuitively, imagine a stationary point on a thousand-dimensional surface. For it to be a saddle point, it just suffices that it’s not a maximum/minimum with respect to one coordinate axis.
Yes. The research community is starting to think about the loss surface not as embedded in a euclidean space (and thus inheriting its metric), but as a standalone non-euclidean manifold with its own intrinsic metric.
It would be great if @rachel could write an article about it.
I think that as long as the image can be turned in a torch tensor, you’ll be fine. Maybe the focus is on .jpgs since the huge vision datasets like Imagenet are a bit old, and back then .jpg was the most common compressed format.
jpg are compressed images (a lossy compression at that) which makes their size on servers smaller. they have 3 channels namely RGB
pngs have 4 channels namely RGBA. where the last layer is alpha which defines transparency.
if you open a 600 X 400 png image using PIL like:
and then see it as numpy array, its shape will be seen as:
np.array(image).shape>>>>(960, 1440, 4)
Depending upon how the image was constructed you could get a surprise as described here:
Thanks for you reply.
I’m not yet so into the math behind deep-learning, but your words are a good starting point for me to start operating deeper study
Here is a simple ‘for loop’ for helping make image folders more automatically.
All you need to do is create a “data/urls” folder where your’re running your notebook and upload all the link files into the urls folder.
Folder names will be named whatever you name the url txt files.
Note: This isn't to take away from highlighting the fact that you can run things out of order in notebooks, I just got a bit lazy in changing lots of code blocks, so this 'for loop' was born.
EDIT: variable path to images is now "pathI" instead of "path" just fyi.
folderNames =  pathU = 'data/urls/' files = os.listdir(pathU) for name in files: folderNames.append(os.path.splitext(name)) pathI = Path('data/images') for title in folderNames: dest = pathI/title filename = title + '.txt' dest.mkdir(parents=True, exist_ok=True) download_images('data/urls/' + filename, dest, max_pics=200,max_workers=0)
EDIT: oops! the code above didnt show the line for “folderNames = ”
Yes I would say stop at epoch 2. But look at the accuracy…you can do the same flipping a coin. GL
Guys, is there an SGD with restarts available in fastai v.1?
Yes it is,I guess, in
learn.fit(); however I have no clue how to use it.
It should be
Hello! I am going through the Lesson 2 training and thinking about the time required to complete each epoch. I would love to see the amount of time required to complete each epoch included in the table showing learning rate, accuracy and loss by epoch. What could I add to this function to see this additional data?
How could we incorporate a “none of the above” option for classification? We regularly see mis-classified images where the image does not really belong to any of the classification options. But since “none of the above” is not really an option the image gets mis-classified. Any ideas on this? It seems like it should be a “one minus” type function…
If you’ve been using the
FileDeleter widget to clean your data, some changes came out today that may interest you. source
FileDeleter was removed in favor of two new widgets:
ImageDeleter is essentially the same as the old
ImageRelabeler renders the files you pass in and lets you re-label them. This is to find mis-categorized images in your data directory. NOTE:
ImageRelabeler currently only works with files where labels were created from the names of their parent directory (i.e. with .from_folder()). The widget moves mislabeled photos from the incorrect parent directory to the properly-labeled parent directory.
To relabel an image, just click the proper label in the widget dropdown.
Both widgets take a formatted dataset from
DatasetFormatter. You can easily specify which dataset you’d like to render by passing in
DatasetType.Test to the
ds_type keyword arg in
All three classes are available in
ds, idxs = DatasetFormatter().from_toplosses(learn, ds_type=DatasetType.Train) ImageRelabeler(ds, idxs) # or ImageDeleter(ds, idxs)
Real docs for these are coming soon!
Lesson 3 In-Class Discussion ✅
Lesson 2 Errors: FileDeleter and 'defaults" attribute
Lesson 2 In-Class Discussion ✅
Lesson 3 In-Class Discussion ✅
Jupyter plugin to fix mislabeled data