[fastpages] GitHub Pages Blog Using Nbdev

nbdev
1

Jeremy announced recently that we added Jupyter notebook automation into fast_template. We also added similar automation for word docs.

UPDATE: The new repo is up here: https://github.com/fastai/fastpages

We would love to your input, and have you try it out! Please give it a go!

Bad News: We recently took all the automation out. Because we found the extra setup steps required to make this work properly (creating an SSH key, adding it to your secrets, etc may be too cumbersome and confusing for beginners).

Good News: We are rebuilding a “power user” version of fast_template, called fastpages with all of the Automation and more exciting features. We are building it from the ground up to be automated and modular enough so you can take the automation to any other Jekyll theme of your choice. The reason for doing this is it will allow us to have the following features:

  • Automatic Jupyter / Word doc to blog post conversion without making your local copy’s master branch out of sync (which would otherwise require you to keep pulling before committing).
  • More transparent build status of your GitHub pages - you will get to see the full logs
  • Flexibility to use whatever plugins, gems, etc. that you want even if they aren’t whitelisted by GitHub Pages.
  • Jupyter blog posts retain interactivity where possible (ex: Altair Charts).
  • Additional CI and Tests that will help both maintainers and users
  • Modularity such that you can take this automation to a different Pages theme, including any third party theme so you can customize the look and feel of your blog but still retain automated Jupyter and Word doc to blog post conversion.

This is an open thread to discuss the development of fastpages

Constraints:

  • We will probably stick with Jekyll, even there are more robust static site generators like Gatsby, in order to leverage the strengths of nbdev and the knowledge of this community.

Some preliminary ideas of things to discuss (Feel free to discuss anything else):

  • What plugins do we want ( might be good for testing purposes to know )
  • Are there any features that would be particularly useful for fastai students?
  • What CSS to include? Perhaps we drop Primer and use something more lightweight that doesn’t clobber other themes. We could, for example, borrow some of the css you are using for the docs sites that allow for the useful things like callouts and such. ( I think it is defined here).
  • Ideas for tests: The only kind of tests I can think of is to kick off a jekyll build (without committing the build) and see if that provides an error - this might help catch errors on PRs to the template.

cc: @jeremy @xnutsive

21 Likes
2

Some initial thoughts on areas we need help (feel free to propose more):

  1. Adding collapsable code cells ( would need a contribution to nbdev to enable this )
  2. Adding support for other interactive charting libraries like bokeh
  3. Make code highlighting prettier
  4. Make notebook rendering prettier in general
  5. More integration between nbdev and fastpages, such as for showdoc and jekyll notes ( I don’t think its compatible because the CSS is not compatible, perhaps someone that is good at that kind of thing might be able to help)
  6. More tests, CI to help maintainers catch errors, a way to see a preview of the website from PRs so we don’t have to clone forks and preview
  7. Adding documentation

Those are the things I can think of off hand, but will keep adding to this list as I think of more. Feel free to jump in and add suggestions on things you would like to see!

5 Likes
3

Could you tell how to test the project locally?

or

Should it be run on the git page all the time?

1 Like
4

also, it looks like we have twitter preview
but do you think we need the samething for facebook and other websites in general?

5

Hi @chansung18, I’m working on setting up tools for easy local preview, will be doing that next week. Right now you have to call bundle exec jekyll serve (See these docs https://jekyllrb.com/docs/) to test locally.

Happy to take PRs for other things or suggestions, if you know of any other tools that may be helpful. Do you often want facebook embedded posts in your blog?

Thanks

2 Likes
6

Thanks @hamelsmu. I now got my local environment working.

I do not share a facebook post as a part of a blog, but I do share often with people in a chat app. so I thought it could be useful somewhat

7

@hamelsmu

I am trying to convert notebook to post in my local environment.

As far as I understand, nb2post.py under _action_files directory does this.
However, when I run it python nb2post.py from the root directory, I get a message like below.

converting: _notebooks/test.ipynb
Use `Config.create` to create a `Config` object the first time

And it doesn’t create a post for me.
How could I test nb2post feature locally?

8

Config.create is likely an ‘old’ reference - seems there is another function in nbdev:

 from nbdev.imports import *
 create_config??
1 Like
9

@hnrk

You are right, but I just use nbdev library from pip install.
It means it doesn’t matter which version I have.

All I do in fastpages's nb2post.py is calling notebook2html function defined in nbdev.
Underlying procedure should work fine since I got nbdev from PIP.

10

Hi @chansung18 the best thing to do is to copy settings.ini from here https://raw.githubusercontent.com/fastai/nbdev/master/settings.ini

Into to directory from where you are running nbdev. These are part of the little things I plan to automate next week or make easier for users to develop locally … but since you are ahead of the game I think that’s all you need to get it started!

Please let me know if you run into any more issues

1 Like
11

Thank you @hamelsmu.

I got it working fine now!
one issue is that settings.ini seems to be placed under the root directory not _action_files for now. (even though there is a separate settings.ini in _action_files directory.

12

@chansung18 Yeah that was just a hack to get you started, the automated system uses the settings.ini file in _action_files/

Next week I will be preparing a shell script with a docker compose, etc. that will put you into a development environment with one command and everything pre-loaded.

2 Likes
13

@chansung18 I’ve finished updating the docs, please give it a look when you have a chance: https://github.com/fastai/fastpages

4 Likes
14

Thanks for your hard work @hamelsmu !
I will check the repo :slight_smile:

15

Thank you @hamelsmu for the awesome job.
I think there is a link missing link at the 4th point of the Setup Instructions section (see here below).

Setup Instructions

  1. Follow these instructions to create an ssh-deploy key. Make sure you select Allow write access when adding this key to your GitHub account.
  2. Follow these instructions to upload your deploy key as an encrypted secret on GitHub. Make sure you name your key SSH_DEPLOY_KEY .
16

Hi @farid I just made it bold - there is no link :slight_smile:

However if you are able to go through the instructions and can think of ways to make it better please let me know!

1 Like
17

Great work @hamelsmu thanks!
There is indeed a typo in the deploy-key link, I made a PR earlier.

One other note is the convert-docker-compose.yml which exposes port 8888 - not sure it’s needed. And if you run a local jupyter notebook it is sort of in the way. As far as I can see, the port is not needed - can it be removed or uncommented?

Otherwise the local dev workflow seems great. :+1:

18

I was referring to

  1. Follow these instructions to create an ssh-deploy key

I was expecting a link there like the one at the 5th point but I noticed the latter point out to the instructions mentioned in the 4th point if I understand it correctly :slightly_smiling_face:

19

@farid you are correct. @hnrk just fixed that with a pull request

20

Hmm re: port 8888 I left it open incase someone wants to start a Jupyter notebook inside the container, but now that you mention it I don’t think its obvious. I’ll make a PR with an explicit command and get your review on it