Share your nbdev project 📜

I created DS Blocks based on nbdev, please check it here:

The objective of this library is to facilitate writing modular, compact, and decoupled data science pipelines, while getting rid of boiler-plate associated with common operations such as chekpointing, profiling, data conversion, and other functionalities. In particular, it also contains extensions to nbdev that allow to build a test suite to be consumed by pytest, as explained here:

and to freeze and unfreeze code cells to avoid re-running them, while preserving their outputs.

4 Likes

Started a TIL blog, it utilizes Code Notes to privide the explanation more than the text in the notebook so that code can be copied and glanced at quickly :slight_smile:

https://muellerzr.github.io/til

6 Likes

I wrote my first open-source library using nbdev: streamlit-jupyter

It allows for seamless creation of streamlit apps in jupyter notebooks.

A nifty little alternative to voila, one might say.

The way this works is it detects your environment, and if the script is run from a jupyter notebook, then it wraps existing streamlit methods to replace them with ipywidgets alternatives - that way you can code as you would in streamlit, and then displays your page as-you-code.

Then you export your notebook to a .py file, and run it via streamlit as usual (no change in behaviour there)

Install

pip install streamlit_jupyter

How to use it

Take a look at our example notebook

The main idea is for you experiment and develop in your notebook,
visually see all the pieces, and then convert the notebook to .py to
be run by streamlit

start by importing streamlit and patching it with streamlit-jupyter:


import streamlit as st

from streamlit_jupyter import StreamlitPatcher, tqdm

StreamlitPatcher().jupyter()  # register streamlit with jupyter-compatible wrappers

And now develop your notebook as usual, but with the ability to use
Streamlit widgets and components.

See how it works below, and check out the example (it also incudes a nifty alternative to nbconvert that strips out all the magics and other irrelevant stuff)

Demonstration

Jupyter Streamlit
Markdown and headings
Interactive data entry
Pick and choose alt alt
Dataframes, caching and progress bars alt alt
Plots

ps

This is a very early beta and my first try at open source, so please share your thoughts and reach out with any questions/critiques!

7 Likes

I adapted and expanded a pre-existing project creating a Stata jupyter kernel and have really enjoyed using nbdev:

One thing I’ve been stuck on is getting the conda publish to work.

4 Likes

I implemented RLHF using nbdev: GitHub - xrsrke/instructGOOSE: Implementation of Reinforcement Learning from Human Feedback (RLHF)

1 Like

Let me know if you want help with that - it’s something I largely created for myself, so there’s rough edges.

2 Likes

This sounds amazing! How are you finding it to use, in practice?

2 Likes

@ddobrinskiy I wonder if you might be open to writing a blog post about your project? I’d love to help share your project with the world and I think a post describing what you’ve built and why it’s helpful would be great! Especially if it discussed a bit about what Streamlit and nbdev are, and why you like them.

1 Like

This sounds amazing! How are you finding it to use, in practice?

Thank you for the kind words, this means a lot coming from you :slight_smile:

I had an app with multiple notebooks running at work without any trouble.

Combined with nbdev, this is a great experience where iterating on data exploration and sharing the results with stakeholders meld together and save me a lot of time.

So far, the toy example from main codebase runs smoothly at https://ddobrinskiy-jupyter.streamlit.app/

2 Likes

That would be wonderful!

Please find the blog post here: David’s Blog - New library streamlit-jupyter, a new way to develop streamlit apps in jupyter notebooks

Also, my twitter handle is @daviddobrinskiy

2 Likes

Looks good @ddobrinskiy ! I strongly suggest adding a preview image to your post. It makes it stand out far more on Twitter et al.

Also, a bit of personal background might make it more readable. I.e who are you and why did you make this, and what impact has this project had on your work?

2 Likes

@jeremy Thanks for the feedback!

The updated version is up now, including both motivation and preview

3 Likes

Published my first Python package developed with nbdev:

It implements the simple observer pattern from the JS Svelte Framework in less than 200 lines of code.

3 Likes

Hi!

Inspired by FastAPI, we built a framework for writing Kafka services using nbdev.

It allows you to build a complete ML serving process in about 50 lines of code. It has an integrated testing framework as well as automatic documentation generation from the code.

Please take a look at it and let me know what you think.

Davor

3 Likes

I implemented ToolFormer paper using nbdev

Paper: [2302.04761] Toolformer: Language Models Can Teach Themselves to Use Tools
Implementation: GitHub - xrsrke/toolformer: Implementation of Toolformer: Language Models Can Teach Themselves to Use Tools


2 Likes

This is really cool!! How was your experience doing this in nbdev?

3 Likes

Nbdev is great, but I separated all the tests into a separate file and used VSCode to run them. This way, I can easily run a specific test when needed, and have a clear overview of all the tests in my codebase, making it easier to manage and identify any missing tests!!

6 Likes

This is interesting, thanks for sharing! Did you consider writing the tests in a notebook and exporting those to a file – or would that add too much friction?

4 Likes

I apologize for the delayed response. I just tried it, and it works great (VSCode found the test). Thank you!!

3 Likes

Such a beautiful UI design :slightly_smiling_face:. I am absolutely loving this kind of UI design and it shows the power of quarto+nbdev.

1 Like