Path.BASE_PATH no info available?

thetj09 · November 29, 2020, 9:30pm

Can’t find any information on Path.BASE_PATH online. Why? What am I missing?

I checked the pathlib documentation here and found nothing on BASE_PATH attribute. Why? What am I doing wrong?

PalaashAgrawal · December 1, 2020, 8:54pm

@thetj09
BASE_PATH is a fastai thing, which is defined in the fastcore library, which is nothing but a collection of slight modifications of already existing Python libraries and functions, to make life a little bit easier (and a few functions which dont exist anywhere else)

github.com

fastai/fastcore/blob/28721554e0eab1f2974237c502912f8ba42a44a0/fastcore/xtras.py#L136


    "Save a pickle file, to a file name or opened file"
    with open_file(fn, 'wb') as f: pickle.dump(o, f)
# Cell
def load_pickle(fn):
    "Load a pickle file from a file name or opened file"
    with open_file(fn, 'rb') as f: return pickle.load(f)
# Cell
@patch
def __repr__(self:Path):
    b = getattr(Path, 'BASE_PATH', None)
    if b:
        try: self = self.relative_to(b)
        except: pass
    return f"Path({self.as_posix()!r})"
# Cell
@contextmanager
def maybe_open(f, mode='r', **kwargs):
    "Context manager: open `f` if it is a path (and close on exit)"

You can see here, the repr is modified to print the path without the BASE_PATH part.

thetj09 · December 1, 2020, 9:55pm

This is great. Thanks.

So I looked at the Path object. I checked the location of Path by using Shift tab. It didn’t appear to be modified (File: /opt/conda/envs/fastai/lib/python3.8/pathlib.py). So I assumed it is the real pathlib without any modifications.

How to I go from looking at Path in a sheet to this page? How do I systematically go there? can you please help me?

PalaashAgrawal · December 2, 2020, 6:54am

I’m no expert, but the way I thought of this was:

I first checked the Pathlib documentation and source code - could’nt find anything about the BASE_PATH variable.
Then I thought - what if BASE_PATH is just a dummy name. What if you can attribute a string to any variable name, and it would still work? For example, I tried, Path.BASE = ‘path/…’. But that didnt work. So BASE_PATH definitely must be hard coded somewhere.
So it must be within fastai. So just to be sure, I reloaded the notebook, and tried doing this without importing fastai or fastcore. That didnt work, so I knew it was definitely a fastai thing. And not any fastai module, it must be within fastcore, because thats what its meant to do.
So it must be stored in a place where all other Path modifications are stored. I know one of them - Path.ls() (this too is a fastai thing). So I did ??Path.ls and saw that it is stored in a file inside fastcore, called xtras.py. So its quite likely that this too would be in the same file.
So i went there on github, did a quick CTRL+F BASE_PATH, and sure it was there…

Hope this helps you develop an idea of searching through source codes.
Cheers and stay safe

thetj09 · December 2, 2020, 6:14pm

Amazing. But I can’t believe it is this convoluted to find what is happening where.

vivekharshey1 · July 23, 2021, 4:20am

With the limited experience of fastai which I have, I can suggest the following thing.

If any constant or method or function that you can not find with single ? or double ??, in Python standard library then it must have to be a fast.ai thing.
Now you can go to fastai repo in GitHub and search in the repo. This will give you the file name where it is defined.
Then either you can read the source code in GitHub or you refer to the documentation of fastai or fastcore.
Most of the time it has to be in the fastcore.

I too struggled a lot initially, but now I am able to find the many things with relative ease. however this time I stumbled on this thing and had to do a Google search to find this post BASE_PATH.

CreatorCreation · September 15, 2021, 10:56am

Thank you for sharing!

CreatorCreation · September 15, 2021, 11:05am

Jeremy briefly touches upon it here (Lesson 3 - Deep Learning for Coders (2020) - YouTube at time stamp 1:08:02)

oduvan · December 12, 2021, 7:00pm

@PalaashAgrawal

2.5 - google and find this post

oduvan · December 12, 2021, 7:11pm

@jeremy is using a lot of dirty patches, which is not very cool for educational purposes, and I’ve seen a lot of criticism of fastai because of that.

He has his point on that, at the end of the day it is his library and his rule, but I think fastai could be much more popular if one uses more stand way of development.

oduvan · December 12, 2021, 7:19pm

@CreatorCreation the video gives a wrong impression of pathlib

for example @jeremy said that pathlib doesn’t have ls, it doesn’t, but it has glob function, which has more functions than its own ls implementation + it returns generator instead of list, which is better by definition