Fastai v2 chat

Is there any recent changes or updates in dependency and libs ? By the way I’m using root and I don’t have environments and It was working fine… I’ll further try other stuff… thank you for investing time on it…

No the deps are the same. Just redo the pip install -e . in both repos to make sure you have a dev install.

Ok, thank you didn’t help … I should try other things…

@sgugger I’m noticing an issue within colab when trying to do a show_doc on some code:

from nbdev.showdoc import *
show_doc(Lookahead)

The error:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/nbdev/export.py in get_nbdev_module()
    258     try:
--> 259         spec = importlib.util.spec_from_file_location(f"{Config().lib_name}._nbdev", Config().lib_path/'_nbdev.py')
    260         mod = importlib.util.module_from_spec(spec)

8 frames
/usr/local/lib/python3.6/dist-packages/nbdev/imports.py in __init__(self, cfg_name)
     39         self.config_file = cfg_path/cfg_name
---> 40         assert self.config_file.exists(), "Use `Config.create` to create a `Config` object the first time"
     41         self.d = read_config_file(self.config_file)['DEFAULT']

AssertionError: Use `Config.create` to create a `Config` object the first time

During handling of the above exception, another exception occurred:

AssertionError                            Traceback (most recent call last)
<ipython-input-18-076f8277c624> in <module>()
----> 1 show_doc(Lookahead)

/usr/local/lib/python3.6/dist-packages/nbdev/showdoc.py in show_doc(elt, doc_string, name, title_level, disp, default_cls_level)
    234     if inspect.isclass(elt):
    235         if is_enum(elt.__class__):   name,args = _format_enum_doc(elt, qname)
--> 236         else:                        name,args = _format_cls_doc (elt, qname)
    237     elif callable(elt):  name,args = _format_func_doc(elt, qname)
    238     else:                            name,args = f"<code>{qname}</code>", ''

/usr/local/lib/python3.6/dist-packages/nbdev/showdoc.py in _format_cls_doc(cls, full_name)
    224     parent_class = inspect.getclasstree([cls])[-1][0][1][0]
    225     name,args = _format_func_doc(cls, full_name)
--> 226     if parent_class != object: args += f' :: {doc_link(get_name(parent_class))}'
    227     return name,args
    228 

/usr/local/lib/python3.6/dist-packages/nbdev/showdoc.py in doc_link(name, include_bt)
     57     if is_lib_module(name): return f"[{cname}]({Config().doc_baseurl}{'_'.join(name.split('.'))})"
     58     #Link to local functions
---> 59     try_local = source_nb(name, is_name=True)
     60     if try_local:
     61         page = _re_digits_first.sub('', try_local).replace('.ipynb', '')

/usr/local/lib/python3.6/dist-packages/nbdev/sync.py in source_nb(func, is_name, return_all, mod)
     33     "Return the name of the notebook where `func` was defined"
     34     is_name = is_name or isinstance(func, str)
---> 35     if mod is None: mod = get_nbdev_module()
     36     index = mod.index
     37     name = func if is_name else qual_name(func)

/usr/local/lib/python3.6/dist-packages/nbdev/export.py in get_nbdev_module()
    261         spec.loader.exec_module(mod)
    262         return mod
--> 263     except: return _EmptyModule()
    264 
    265 # Cell

/usr/local/lib/python3.6/dist-packages/nbdev/export.py in __init__(self)
    249     def __init__(self):
    250         self.index,self.modules = {},[]
--> 251         self.doc_url,self.git_url = f"{Config().doc_host}{Config().doc_baseurl}",Config().git_url
    252 
    253     def custom_doc_links(self, name): return None

/usr/local/lib/python3.6/dist-packages/nbdev/imports.py in __init__(self, cfg_name)
     38         while cfg_path != Path('/') and not (cfg_path/cfg_name).exists(): cfg_path = cfg_path.parent
     39         self.config_file = cfg_path/cfg_name
---> 40         assert self.config_file.exists(), "Use `Config.create` to create a `Config` object the first time"
     41         self.d = read_config_file(self.config_file)['DEFAULT']
     42         add_new_defaults(self.d, self.config_file)

AssertionError: Use `Config.create` to create a `Config` object the first time

It seems it doesn’t know that the Config file exists? Is there a step I’m missing?

Is it with a dev install or a pip install?

Pip install, version is 0.0.5

I can try with the dev real quick too if that would help :slight_smile:

YEs please, I think it’s because settings.ini is not in the MANIFEST. Just fixed that but it will need a new release to be working.

1 Like

Still not working :confused: here is how I’m installing (I also restarted the instance):

!pip install -q git+https://github.com/fastai/fastai2 --upgrade

And running:

from fastai2.basics import *
from fastai2.vision.all import *
from fastai2.callback.all import *
from nbdev.showdoc import *
show_doc(untar_data)
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-3-71804236c9be> in <module>()
----> 1 show_doc(untar_data)

1 frames
/usr/local/lib/python3.6/dist-packages/nbdev/showdoc.py in show_doc(elt, doc_string, name, title_level, disp, default_cls_level)
    245         s = inspect.getdoc(elt)
    246         # doc links don't work inside markdown pre/code blocks
--> 247         s = f'```\n{s}\n```' if Config().get('monospace_docstrings') == 'True' else add_doc_links(s)
    248         doc += s
    249     if disp: display(Markdown(doc))

/usr/local/lib/python3.6/dist-packages/nbdev/imports.py in __init__(self, cfg_name)
     38         while cfg_path != Path('/') and not (cfg_path/cfg_name).exists(): cfg_path = cfg_path.parent
     39         self.config_file = cfg_path/cfg_name
---> 40         assert self.config_file.exists(), "Use `Config.create` to create a `Config` object the first time"
     41         self.d = read_config_file(self.config_file)['DEFAULT']
     42         add_new_defaults(self.d, self.config_file)

AssertionError: Use `Config.create` to create a `Config` object the first time

Weird. What version of nbdev?

It’s running 0.2.5

I tried doing the dev (0.2.6) and upon import I’ll get the config error as well

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-2-e0e43e5288cc> in <module>()
----> 1 from nbdev.showdoc import *

2 frames
/usr/local/lib/python3.6/dist-packages/nbdev/showdoc.py in <module>()
      8 from .imports import *
      9 from .export import *
---> 10 from .sync import *
     11 from nbconvert import HTMLExporter
     12 

/usr/local/lib/python3.6/dist-packages/nbdev/sync.py in <module>()
     41 
     42 # Cell
---> 43 _re_default_nb = re.compile(f'File to edit: {Config().nbs_path.relative_to(Config().config_file.parent)}/(\\S+)\\s+')
     44 _re_cell = re.compile(r'^# Cell|^# Comes from\s+(\S+), cell')
     45 

/usr/local/lib/python3.6/dist-packages/nbdev/imports.py in __init__(self, cfg_name)
     38         while cfg_path != Path('/') and not (cfg_path/cfg_name).exists(): cfg_path = cfg_path.parent
     39         self.config_file = cfg_path/cfg_name
---> 40         assert self.config_file.exists(), "Use `Config.create` to create a `Config` object the first time"
     41         self.d = read_config_file(self.config_file)['DEFAULT']
     42         add_new_defaults(self.d, self.config_file)

AssertionError: Use `Config.create` to create a `Config` object the first time

Note that the Cuda transform has been replaced since we now use the device attribute of DataLoader. By default:

  • a DataLoader (or any subclass like TfmdDL) don’t do anything in terms of device placement (pass along a device to change this)
  • a DataBunch (or any subclass) uses default_device(), again pass along the device you want at creation (either with a .databunch or the regular init/factory methods).
5 Likes

Torch Core

Class ArrayImageBW

Class ArrayImageBW, it seems, doesn’t cast/convert an RGB image to 1-channel, only sets cmap='Greys'.

The impact: when one tries to cast an opened RGB image to ArrayImageBW, it will stay RGB and show() displays it in color, no matter what cmap is set innernally in _show_args or passed via an argument.

Nota bene: You need to use .convert(mode='L') when opening a PIL image if you intend it to be displayed as Greys (by default):
image
or use explicitly cmap='Greys_r':
image

Note also this comment by @jeremy regarding to why some monochrome images look “inverted”. Spoiler: it is not a bug.

1 Like

Like the latest torchvision release, the last PyTorch release is not compatible with fastai v2 (yet). I have constrained the requirements while we figure out a solution.

3 Likes

@sgugger, when you have time can you explain or improve docs about TransformBlock, defining Transform, linking them and how they work together …

I try to define BlockFloat but not sure where to start… I try to do image regression. I did it with TransformBlock and works fine… but I want to improve it with float and may be other types to use mix models (tabular, image, etc.)

@s.s.o Once I finish the heatmap implementation I’m going to release a tutorial video of sorts showing what looked at and how I implemented it which may be of some help to you and anyone looking to add custom implementations to the DataBlock API

(Trying to do this sooner than later)

2 Likes

@muellerzr, thank you… I’ll wait for it. It may help to solve show_batch problems as well.

@s.s.o For a hint, I’d peek at how CategoryBlock’s show looks. I was debating on making a FloatBlock as compared to other implementations it’s an ‘easier’ one so I’d be happy to help :slight_smile:

I tried to do:

d = {‘files’: [“file 1”, “file 2”], ‘ca’: [3.2, 4.4]}
data = pd.DataFrame(data=d)
data[‘files’] = data[‘files’].apply(lambda x: path_img+"/{}{}".format(x, ‘.jpg’)); data
class ToFloatTensor(Transform):
   "Transform to float tensor"
    order = 10 #Need to run after PIL transforms on the GPU
    def __init__(self, split_idx=None, as_item=True):
        super().__init__(split_idx=split_idx,as_item=as_item)

    def encodes(self, o): return o.astype(np.float32)
    def decodes(self, o): return o
def FloatBlock(vocab=None, add_na=False):
   "`TransformBlock` for single-label float targets"
   return TransformBlock(type_tfms=ToFloatTensor())
den_db = DataBlock(blocks=(ImageBlock, FloatBlock),
                   get_x=lambda x:x[0],
                   get_y=lambda x:x[1],
                   splitter=RandomSplitter())
item_tfms = [Resize(224),ToTensor()]
batch_tfms =[Normalize.from_stats(*imagenet_stats)]
dbunch = den_db.databunch(data, path=path_img, bs=128,  num_workers=0, item_tfms=item_tfms, batch_tfms=batch_tfms)
dbunch.c = 1

this seems to work… but still dbunch.show_batch() gives error but shows image:

AttributeError: ‘Tensor’ object has no attribute 'show’

also,

show_image_batch(xys) gives warning and gives wrong image coloring:

Clipping input data to the valid range for imshow with RGB data ([0…1] for floats or [0…255] for integers).

Any suggestions?

Hi! I am trying to port a project to v2 (mainly because of data loader improvements since I have a huge dataset). I am trying right now to understand some imports (still a bit allergic to import *, sorry :sweat_smile:).

Could anyone point out how to import RandomSplitter? So far I have found it at fastai_dev/dev/local/data/core.py but nowhere within fastaiv2.

from fastai2.text.all import *
does the trick, but I can’t figure out how.

The second import I can’t figure out is “Cuda”, as in
self.data = dsrc.databunch(bs=batch_size, val_bs=batch_size, after_batch=Cuda)

Thanks!

http://dev.fast.ai/data.transforms#RandomSplitter

The page has a search function that depending on your window size or addblocker might not show directly.

PS Using the * does work quite well

1 Like