I changed the one line in predict with test_dl yes, to put num_workers=0.
1 Like
Would adding albumentations to fastai be as simple as creating a new Pipeline object and using that as training augmentation?
A new Transform I believe. I actually will try to add an example of this in the tutorial I was mentioning before.
1 Like
Great, thanks. This is something I was planning on trying to do and adding a new tutorial to the library or write a blog 
I’ll move on to something else if you’re already doing this.
Also, planning to try to add Fmix or CutMix as part of our source code study group. Would the best way be to follow the implementation of Mixup in the library?
If it needs to change the loss function, yes (can’t remember if it does at the top of my mind). Otherwise, maybe a Transform would work better.
1 Like
I was looking at ClassificationInterpretation and am interested in adding multi-label support. However, am not sure of the best way to do it, or if it’d even work.
My thought was to make another class MultiClassificationInterpretation(ClassificationInterpretation) which within init modified targs and decoded to look like TensorCategory from TensorMultiCategory.
The other change is there could be multple TensorCategory for one TensorMultiCategory
If there are any thoughts, that’d be appreciated.
have you verified fast.ai v2 builds/runs on python 3.7x ?
Just checking today … not sure why but its returning an empty dictionary for pred_idx and probs (but it is returning the correct pred_class)
pred_class, pred_idx, probs = inf_learn.predict(img_bytes)
# grizzly, {}, {}
So something still looks off.
That is really weird. Not sure I’ll have to look at it today though.
1 Like
Technically speaking TF Lite uses their own C++ code. It can “delegate” the work to something like Metal Performance Shaders (GPU code). In the future TF Lite will also be able to delegate to Core ML but currently it doesn’t.
EDIT: I take that back, Core ML support for TF Lite was released an hour or so ago.
Don’t bother looking … the problem is with the FastAPI framework. See the solution here.
In a nutshell, it doesn’t like tensor objects … but instead of throwing an exception indicating it can’t return them as part of the response, it just returns them as {}. Bonkers.
Your initial fix with setting num_workers=0 seems to have done the trick after all!
Oh so that’s why. Good to know!
Just a small question how can I connect Jupyter notebook or Google Colab to my Flask application???
To connect a Jupyter Notebook in Colab, you need to expose your Flask application as an external web service. There are many options to do that.
If you want to run your web application locally, and test things out before you deploy it, then you can just use a service like ngrok to expose your local web service temporarily. You can then make HTTP requests to your Flask app.
Thanks a lot!
Is it the same if I’m using Jupyter on Paperspace?
Yeah., all you are doing is making a HTTP request at that point.
1 Like
Automatically Test Multiple Architectures and Batch sizes against your Learner
Since V1 of the course I’ve been looking for a way to do this, and thanks to @jeremy 's implementation of the GPU Out of Memory Crash Recover-er. I’ve now been able to. Using this code obviously does NOT substitute for studying the architectures and selecting the appropriate one for you model.
I have two gists available:
architectureRunner.py - Contains the code and
Architecture Runner Example.ipynb - Shows an example notebook
You need both to try this. Only the first gist is public, I’m hoping that by giving the links above you can get both. Let me know if you can’t and I’ll make the notebook public too
I think it’s quite nice library to look at for cuda mem problems… According to the toma page it’s small elegant library.
A collection of helpers to make it easier to write code that adapts to
the available (CUDA) memory. Specifically, it retries code that fails due to
OOM (out-of-memory) conditions and lowers batchsizes automatically.
To avoid failing over repeatedly, a simple cache is implemented that memorizes
that last successful batchsize given the call and available free memory.Does anyone have experience deploying models that run batch inference? I’m wondering about the best way to handle the actual batching process. I’m aware of Apache Kafka for building queue systems, but I’m curious if there are better tools for handing batch ML inference.
In fastai v1 the default for TTA is 4((number of augmented images used during TTA) I guess. Played with it during a competition(scene classification) which has mostly rectangular images & increasing it has increased the inference time but we got the improvement on numbers in validation dataset when TTA was 8 and after 8 there the increment in metric was not so much. So I inferred when the most of the images are mostly rectangular, may be few images have information loss when it gets transformed. Hence it could be useful in those cases
1 Like