This is exactly what I want to achieve. Thanks for help clarify. I will try to be clearer next time.
Try this…
MARGIN=150
blocks = """
if to is None:
to_f,from_f = f.__base__.__init__,f.__init__
else:
to_f,from_f = to.__init__ if isinstance(to,type) else to,f
"""
lst = blocks.split('\n')
maxlen = max(map(lambda l : len(l) , lst ))
indent = MARGIN - maxlen
for l in lst:
print(' '*indent + format(l))
2 Likes
Thanks a lot! This approach looks a lot more promising than mine, and I will try it later and get back to you.
Yes, this works! Thanks
Can we align a colored text all the way to the right too? I can’t figure this one out.
I didn’t know the answer, but it was an interesting challenge, so I’ll step you through how I found out.
- Identified root cause - the colour escape characters are being counted in the maxlen calculation, but not appearing in the printed output, so the indent is short.
- Reword root cause into required fix - strip colour escape characters from the maxlen calculation
- Search for fix… python strip colour control characters from string
- Fifth result showed several examples to strip ansi
- Copy-paste strip_ansi() from Example 1…
import re
MARGIN=150
def strip_ansi(source):
return re.sub(r'\033\[(\d|;)+?m', '', source)
def alignright(blocks):
lst = blocks.split('\n')
maxlen = max(map(lambda l : len(strip_ansi(l)) , lst ))
indent = MARGIN - maxlen
for l in lst:
print(' '*indent + format(l))
alignright("this is me")
alignright("\x1b[91mthis is me\x1b[0m")
1 Like
This might not be a python question but fastai in general.
I am writing the loss for object detection and it contains the location_loss and classification_loss, which will be summed up to get the final loss. I want to print out each loss after each epoch but don’t really know how. I think I can recalculate each loss in Metrics but we will have duplications.
I now declaring a global variable in loss_function for counting when I need to print the loss, as below:
It somehow solves my problem but not totally. I want to average the losses in an epoch and only print loss in validation set instead of training set
What is a cleaner way to solve this?
Thanks
2 Likes
If you have not read Fastai callbacks, maybe it can be helpful to your problem
2 Likes
How to open a jupyter notebook locally by calling a function from another notebook?
I want to use a function call like openNB(nbname) in one jupyter notebook to open another with the name nbname.
However, I have not found anything useful in google, so I put jupyter notebook command into python script with !.
#| export
def openNB(nbname):
folder ='/Users/Natsume/Documents/fastdebug/nbs/demos/'
for f in os.listdir(folder):
if f.endswith(".ipynb"):
if nbname in f:
file_name = folder + f
!jupyter notebook {file_name}
But this solution is problematic, it is not only verbose, but I can’t use the notebook anymore after calling the function above. (see the * symbol on most left of the cell)
How to create a function to open a notebook without having the problems above? Could you help me? Thanks!
I found a solution to the problem above by copying some code from colab_link from nbdev/showdoc.py
However, in order to make the function more robust, I wonder whether I need to generate the root_server string automatically by calling another function, like get_local_jupyter_notebook_server_string()? What do you think? Thanks
Maybe something like this will find the port too?..
1 Like
Thanks Jeremy! I found a solution online by adding ‘port’ to my search question.
Although another problem appeared (there is a second port found which I am not using at all), I think it won’t block me from doing what I want.
Not sure if this is directly useful to you, or just broader background info…
I wanted to get the current notebook name so I could tag my kaggle submissions with this, but I struggled to get any of the answers I saw using IPython/Javascript like the following to work in JupyterLab on Paperspace…
%%javascript
IPython.notebook.kernel.execute('nb_name = "' + IPython.notebook.notebook_name + '"')
Then starting with this little bit on CLI management of workspaces I discovered a way to get current notebook name for JupyterLab.
def get_jupyterlab_notebook_name():
import json
sh = ! jupyter lab workspaces export
j = json.loads(str(sh[0]))
return j['data']['layout-restorer:data']['main']['current'].split(":")[1].split(".")[0]
2 Likes
I have too found a solution to this yesterday without using ! (because nbdev does not like ! in the source code I guess) when I try to convert current jupyter notebook into a md file by running a function inside the current notebook. I have added my source code below too.
from fastdebug.utils import *
import inspect
nb_name()
'Untitled.ipynb'
nb_path()
'/Users/Natsume/Documents/fastdebug/nbs/2022part1/Untitled.ipynb'
nb_url()
'http://localhost:8888/notebooks/nbs/2022part1/Untitled.ipynb?kernel_name=python3'
print(inspect.getsource(nb_name))
def nb_name():
"run this func to get nb_path of this current notebook"
import ipyparams
return eval("ipyparams.notebook_name")
print(inspect.getsource(nb_path))
def nb_path():
"run this func to get nb_path of this current notebook"
import ipyparams
return eval("os.path.join(os.getcwd(), ipyparams.notebook_name)")
print(inspect.getsource(nb_url))
def nb_url():
"run this func to get nb_url of this current notebook"
import ipyparams
return eval("ipyparams.raw_url")
print(inspect.getsource(ipy2md))
def ipy2md(db=True):
"convert the current notebook to md"
import ipyparams
import os
path = nb_path()
name = nb_name()
url = nb_url()
obs_path = "/Users/Natsume/Documents/divefastai/Debuggable/jupytext"
obs_output_path = "/Users/Natsume/Documents/divefastai/Debuggable/nbconvert"
mds_path = path.replace("nbs", "mds").split(name)[0]
mds_output = "/Users/Natsume/Documents/fastdebug/mds_output"
# https://stackabuse.com/executing-shell-commands-with-python/
os.system(f"jupytext --to md {path}")
os.system(f"cp {path.split('.ipynb')[0]+'.md'} {obs_path}")
if db: print(f'cp to : {obs_path}')
os.system(f"mv {path.split('.ipynb')[0]+'.md'} {mds_path}")
if db: print(f'move to : {mds_path}')
os.system(f"jupyter nbconvert --to markdown {path}")
os.system(f"cp {path.split('.ipynb')[0]+'.md'} {mds_output}")
os.system(f"mv {path.split('.ipynb')[0]+'.md'} {mds_output}")
if db: print(f'move to : {mds_output}')
ipy2md()
[jupytext] Reading /Users/Natsume/Documents/fastdebug/nbs/2022part1/Untitled.ipynb in format ipynb
[jupytext] Writing /Users/Natsume/Documents/fastdebug/nbs/2022part1/Untitled.md
cp to : /Users/Natsume/Documents/divefastai/Debuggable/jupytext
move to : /Users/Natsume/Documents/fastdebug/mds/2022part1/
[NbConvertApp] Converting notebook /Users/Natsume/Documents/fastdebug/nbs/2022part1/Untitled.ipynb to markdown
[NbConvertApp] Writing 2700 bytes to /Users/Natsume/Documents/fastdebug/nbs/2022part1/Untitled.md
move to : /Users/Natsume/Documents/fastdebug/mds_output
1 Like
Hi guys,
Is there a way to wrap lines of codes which include % or %% into a function without nbdev or else reporting invalid syntax?
#| export
def reload():
%load_ext autoreload
%autoreload 2
reload()
(base) 10:31 ~/Documents/fastdebug > nbdev_export
Traceback (most recent call last):
File "/Users/Natsume/mambaforge/bin/nbdev_export", line 8, in <module>
sys.exit(nbdev_export())
File "/Users/Natsume/mambaforge/lib/python3.9/site-packages/fastcore/script.py", line 119, in _f
return tfunc(**merge(args, args_from_prog(func, xtra)))
File "/Users/Natsume/mambaforge/lib/python3.9/site-packages/nbdev/doclinks.py", line 135, in nbdev_export
_build_modidx()
File "/Users/Natsume/mambaforge/lib/python3.9/site-packages/nbdev/doclinks.py", line 97, in _build_modidx
res['syms'].update(_get_modidx((dest.parent/file).resolve(), code_root, nbs_path=nbs_path))
File "/Users/Natsume/mambaforge/lib/python3.9/site-packages/nbdev/doclinks.py", line 75, in _get_modidx
for tree in ast.parse(cell.code).body:
File "/Users/Natsume/mambaforge/lib/python3.9/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
File "<unknown>", line 2
%load_ext autoreload
^
SyntaxError: invalid syntax
Yes, you are right. Using exec like below won’t cause error for nbdev, but the function won’t work as there is syntax error. Do you know how can I make it work?
#| export
def reload():
exec("%load_ext autoreload")
exec("%autoreload 2")
reload()
Traceback (most recent call last):
File ~/mambaforge/lib/python3.9/site-packages/IPython/core/interactiveshell.py:3398 in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
Input In [16] in <cell line: 1>
reload()
Input In [15] in reload
exec("%load_ext autoreload")
File <string>:1
%load_ext autoreload
^
SyntaxError: invalid syntax
1 Like
I believe these magic commands are tied to the ipython shell and cannot be referenced in a function defined in a .py file/module.
https://ipython.readthedocs.io/en/stable/config/extensions/index.html#using-extensions
https://ipython.readthedocs.io/en/stable/config/extensions/autoreload.html
I think you can create a ipython config file and set autoreload there
https://ipython.readthedocs.io/en/stable/config/index.html#
1 Like
Thanks a lot!
Yes, I have tried it before but I was not sure whether it is really working. Since you mentioned this method here, I decided to try it thoroughly today and I think it really works.
Below is some detail steps I copied from stackoverlow (I forgot the url of the answer) and it works for me.
If you want it to automatically start every time you launch ipython, you need to add it to the ipython_config.py startup file:
It may be necessary to generate one first:
ipython profile create
Then include these lines in ~/.ipython/profile_default/ipython_config.py:
c.InteractiveShellApp.exec_lines = []
c.InteractiveShellApp.exec_lines.append('%load_ext autoreload')
c.InteractiveShellApp.exec_lines.append('%autoreload 2')
Hi Mat, I think I managed to find a way to do it in a function!
#| export
def autoreload():
from IPython.core.interactiveshell import InteractiveShell
get_ipython().run_line_magic(magic_name="load_ext", line = "autoreload")
get_ipython().run_line_magic(magic_name="autoreload", line = "2")
2 Likes