I implemented this paper which introduces a regularization term for RNNs. It proposes that the term introduced keeps the norms of the hidden state stable.
ROCKET: a new SOTA in Time Series Classification, now with multivariate and GPU support
Last week, there was a major milestone in the area of Time Series Classification.
A new method, called ROCKET (RandOm Convolutional KErnel Transform) developed by Dempster et al. was released (ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels, paper) together with the code they used.
This new method not only beat the previous recognized state of the art (HIVE-COTE) on a TSC benchmark, but it does it in record time, many orders of magnitude faster than any other method.
I’ve been using it for a couple of days and the results are IMPRESSIVE!!
The release code however has 2 limitations:
it can only handle univariate time series
it doesn’t support GPU
I have developed ROCKET in Pytorch and you can now use it with univariate of multivariate time series, and with a GPU. I have shared a notebook that explains how you can use this new method.
If you are interested in this, you can find a more information here, in the time series/ sequential data study group thread.
Hi guys! I’ve finally deployed my superres model in a form of a telegram bot.
You can check it here: @PimpMyResBot or https://t.me/PimpMyResBot
You can either upscale the picture and run it through the model or run it without upscaling (sharpen) - it sometimes works for blurry images you’ve found on the internetz, that someone has tried to upscale before.
It’s based on https://github.com/jantic/DeOldify and the superres lessons from fastai, of course.
I’ve managed to deploy the model onto algorithmia, which is a serverless hosting, so I won’t run out of money unless someone starts to use the bot =) As for the bot, I’ve used my vps on digitalocean.
I can drop a quick tutorial on how to do that, if someone’s interested.
I’ve run into some RAM problems, as the fastai unet with self-attention takes 11gb ram for a 1500x1500 output image, so the input has to be somewhere around 700x700. I’ve managed to cap it so that the server won’t die every time, but the results for upscaling, say, 1200x1200 to 1500x1500 with an x2 model are overly sharpened and messy.
I’d appreciate any help on quantizing unet, or converting it to TF (I’ve heard it has better production deployment options/model optimization for production)
Just sharing a dirty hack in notebook2script.py (@jeremy you are just a genius) to manually enter names of output py.
#!/usr/bin/env python
import json,fire,re
from pathlib import Path
def is_export(cell):
if cell['cell_type'] != 'code': return False
src = cell['source']
if len(src) == 0 or len(src[0]) < 7: return False
#import pdb; pdb.set_trace()
return re.match(r'^\s*#\s*export\s*$', src[0], re.IGNORECASE) is not None
def getSortedFiles(allFiles, upTo=None):
'''Returns all the notebok files sorted by name.
allFiles = True : returns all files
= '*_*.ipynb' : returns this pattern
upTo = None : no upper limit
= filter : returns all files up to 'filter' included
The sorting optioj is important to ensure that the notebok are executed in correct order.
'''
import glob
ret = []
if (allFiles==True): ret = glob.glob('*.ipynb') # Checks both that is bool type and that is True
if (isinstance(allFiles,str)): ret = glob.glob(allFiles)
if 0==len(ret):
print('WARNING: No files found')
return ret
if upTo is not None: ret = [f for f in ret if str(f)<=str(upTo)]
return sorted(ret)
def notebook2script(fname=None, allFiles=None, upTo=None, fnameout=None):
'''Finds cells starting with `#export` and puts them into a new module
+ allFiles: convert all files in the folder
+ upTo: convert files up to specified one included
ES:
notebook2script --allFiles=True # Parse all files
notebook2script --allFiles=nb* # Parse all files starting with nb*
notebook2script --upTo=10 # Parse all files with (name<='10')
notebook2script --allFiles=*_*.ipynb --upTo=10 # Parse all files with an '_' and (name<='10')
notebook2script --fnameout='test_25.py'
'''
# initial checks
if (allFiles is None) and (upTo is not None): allFiles=True # Enable allFiles if upTo is present
if (fname is None) and (not allFiles): print('Should provide a file name')
if not allFiles: notebook2scriptSingle(fname, fnameout)
else:
print('Begin...')
[notebook2scriptSingle(f, fnameout) for f in getSortedFiles(allFiles,upTo)]
print('...End')
def notebook2scriptSingle(fname, *fname_out):
"Finds cells starting with `#export` and puts them into a new module"
fname = Path(fname)
if (fname_out[0]==None):
fname_out = f'nb_{fname.stem.split("_")[0]}.py'
else: fname_out = fname_out[0]
#print(f"fname_out {fname_out}")
main_dic = json.load(open(fname,'r',encoding="utf-8"))
code_cells = [c for c in main_dic['cells'] if is_export(c)]
module = f'''
#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: {fname.name}
'''
for cell in code_cells: module += ''.join(cell['source'][1:]) + '\n\n'
# remove trailing spaces
module = re.sub(r' +$', '', module, flags=re.MULTILINE)
if not (fname.parent/'exp').exists(): (fname.parent/'exp').mkdir()
output_path = fname.parent/'exp'/fname_out
#print(f"output_path {output_path}")
open(output_path,'w').write(module[:-2])
print(f"Converted {fname} to {output_path}")
if __name__ == '__main__': fire.Fire(notebook2script)
Usage !python notebook2script.py --fnameout="df_12_ZGFM_25625_1year.py" "12 - ZGFM 25625 - load data from ACCDB - save to DF.ipynb"
Converted 12 - ZGFM 25625 - load data from ACCDB - save to DF.ipynb to exp\df_12_ZGFM_25625_1year.py
I have seen that notebook2script.py doesn’t work great when extended characters are used (such as accents éàè) complaining about UTF-8. Has anyone fixed it?
Hello everyone,
Hope y’all are enjoying the course as much as I am. I am currently on lesson10 and I’ve written 3 articles so far. Just wanted to share them here. Happy learning.
Hey Guys! just started the course a few weeks ago.
This is my first deep learning model.
To recognize adidas, nike , puma sneakers inspired from lesson one.
deployed here: https://sneakernet-v1.onrender.com/
I wrote 2 articles on Python and one on Jupyter notebooks. All of them are based on things I have learnt from this course. Hope y’all find them useful.
I was on and off but got excited when the MULTIFiT model was announced. I trained an Arabic MULTIFiT model based on Arabic Wikipedia and a bunch of books (to get words to 100m). Then, I fine-tuned hotel reviews sentiment classification. After some struggle (MultiFiT Inference and SentencePiece Hardcoded tmp path), I published the app: Arabic Sentiment Analyzer
Sadly I don’t remember these experiments very well, but they were a bust. The spike happens because of the inherent shift in the the weights during backprop. The “expected” std changes at each batch, and therefore using the expected std, will eventually just cause your network to blow up. Changing it so that it is calculated at every batch just causes it to be “batch norm”-light.
Dividing by a fixed std,over time the fixed std will either be very much too large or very much too small, causing your weights to either explode(std<1) or disappear(std>1).
Was a great learning experience though. I think everyone should do things like this, because the experience of being excited about your work allows you to be super excited, and it is either going to work, or your going to find out where your understanding of deep learning is the weakest.
Since then I have worked on other projects, but never felt like sharing them, as they didn’t have results. I am currently working on a project that I am fairly happy about and will be sharing soon, after I fix one of my last headaches. I already have results from it that I want to share, I’m just in the “cleaning up” phase.
For this project, I collected geospatial data related to U.S. land-based Wind Farms and regional wind speed classes in order to engineer an image data set for training a multi-label model using the EfficientNet b1 architecture.
The model infers Wind Farm capacity for existing wind farms and potential capacity for unoccupied plots of land.
Didn’t get the job but I learned a lot! It’s far from perfect, but was able to run small tests (e.g., EfficientNet v InceptionV3, Python vs Swift-Python hybrid on speed and memory performance), develop a grid search helper for selecting an optimal weight decay hp, and deployed the model to inference engines for both web and NVIDIA Jetson edge devices (a flask API and also a Kitura API).
When I have more time, I’m hoping to train a GAN to draw potential WFs and deploy the model to an additional endpoint in the API.
and the repo that includes notebooks in python and swift (API deployments can be found on different branches but have not been updated with the multilabel models yet): https://github.com/codeamt/WindFarmSpotter``
Hi all,
I wanted to share my web app Plants Parts Detector at http://detect.plantbook.io
I’m going to continue working in plant pictures detection and use knowledge and skills acquired at fast.ai in my project Plantbook.io
Huge thanks for such an amazing AI course and FastAI library.
I used Bing Image API to get pictures as recommended in this post Tips for building large image datasets and then manually labeled images using Mac’s Finder with some tricks to quickly assign tags and then extracted these tags into CSV file suitable for fastai library.