Jekyll scholar , I believe can’t be standardized as I understand, becoz GitHub doesn’t support the plugin. You either do a local build or use Netlify
. That’s how I managed to do it.
1 Like
@rahuketu86 This is not true - the plugin is supported in fastpages because we materialize the static site ourselves in GitHub Actions before deploying to GitHub Pages.
@drscotthawley I am not well educated on the best citation workflow. Is there something in your testing that works with the least effort in fastpages? if so, if you want to propose a recommendation that would be helpful and perhaps you can submit a blog post onto the fastpages site dedicated to citations, and how to do them? What do you think?
I’m happy to give it a start. I just recently had the idea of wanting to do citations from bibtex, but hadn’t actually tried it yet.
When you say “the plugin is supported in fastpages” – does that mean that I don’t have bother trying to install jekyll-scholar myself? ( I followed jekyll-scholar’s ‘gem install…’ instructions a couple days ago and it built a bunch of things on my laptop; but I’m not sure how to get GitHub Actions to reliably do the same. )
If not, is there a place with instructions on 'adding plugins to fastpages"? If not I’ll just muck about as if fastpages were any other jekyll blog.
@drscotthawley you don’t have to do anything 
especially if it is complicated. I’m looking for the easiest path for people to accomplish citations, in a consistent way across markdown and Jupyter
Ok, I’m on it. Will write back in a bit…
@hamelsmu Regaring “you don’t have to do anything”:
I just added jekyll-scholar to the list of plugins in the config file. Then running make server produced an error:
jekyll_1 | /usr/local/bundle/ruby/2.6.0/gems/jekyll-4.0.0/lib/jekyll/external.rb:60:in require’: cannot load such file – jekyll-scholar (LoadError)
`
And that error persists even after I do a sudo gem install jekyll-scholar on my laptop, and I can see a jekyll-scholar-6.8.0/ listed in my /var/lib/gems/2.7.0/gems directory.
…I’m trying to attach my new Markdown blog entry describing my whole process, but the Fast.ai forum won’t allow the file type.
So here’s a Dropbox link:
1 Like
Hmm not sure, you might have to try bundle exec … see the Jekyll Docs
Welcome to my rabbit hole
1 Like
Huh, I didn’t have bundle installed on my system, but I do now.
…Oh, weird: Seems there are two sets of ruby & gems on my system. sudo placed everything in /var where I have 2.7.0 version numbers, but…
if I look at the output of make server, all the references are to things in /usr/local/bundle/ruby/2.6.0/
…arg, I don’t know Ruby at all, but running bundle install finally dies with an error about conflicts involving jemoji and nokogiri.
…Yeah, too rich for me. This was why I asked. Switching back to doing science! If someone else can figure this out, great! 
1 Like
Totally understand.
Ok wait, I have a hard time giving up.
Oddly, there is no /usr/local/bundle/ruby/2.6.0/ on my system.
So… references to that are then inside the Docker container that make server manages?
i.e. so installing stuff on my laptop’s hard drive is having absolutely zero effect on my Fastpages build?
@hamelsmu I got it!! I now have a working Fastpages blog with bibtex-style citations, and I can even change the citation format!!
Turns out bundle install pretty much tells you (if I were to have read it more carefully) exactly what one needes to run to fix the dependencies.
…the path I took to get this working was a bit long and circuitous (and involved submitting a couple Issues on the jekyll-scholar GitHub site), so I’m not ready to post a “minimal path” yet. But I am at least noting here that IT IS POSSIBLE.
Correct. You must replicate the dependency inside Docker. However if you update your Gemfile, etc it should take care of itself … hopefully when you run Docker, if not you will have to debug …
Victory! Minimal changes, done in-place on a pre-existing Fastpages blog living on GitHub Pages. (i.e. anybody can do this, no admin access or Docker skills required – you were right re. the “you don’t have to do anything…”. Although I DID have to edit the Gemfile too.)
Working example & instructions here on the following publicly-available post. Feel free to share.
1 Like
Excellent work! Would you mind adding this to the README of fastpages? Thank you
1 Like
Thanks. And by the way… the ‘labor saving’ motivation behind this maybe wasn’t obvious, so I edited the instructions to clarify:
This allows you to use one references file for your entire blog, so you can just throw any reference you might ever think to use into references.bib. And then, rather than pasting full references into each individual post or notebook manually, now all your notebooks and Markdown posts can just have little cite tags, and the system reads from the single (massive) ‘repository’ of reference information and fills in allllll your posts (and can even sequentially number the citations in each post, if you’re into that)!
Furthermore, if you ever wanted to change your citation format for the entire blog, rather than having to edit each post where you’d (previously) pasted the full references, now it’s literally just a change of one line in _config.yml.
Similarly if you’re citing one paper on multiple pages, you just correct the entry in the .bib file and that correction propagates everywhere, without you having to edit multiple individual pages.
First of all thank you to @drscotthawley and @hamelsmu to include jekyll-scholar and finally coming up with a unified workflow for generating the citation. However, I am still not fully satisfied. Here are my pain points. We now have 2 solutions for citation.
a. @ducha-aiki solution using minimal latex plugin in jupyter environment. Here are few advantages which I find useful in my workflow
-
I can use this solution with nbdev generated package documentation and fastpages. It works by using a minimal hack ( regex substitution ) on nbdev side.
-
I can see my references in my jupyter notebooks. This has some orthogonal advantages.
-
A lot of times blog post is not just a blog post . It could be a tutorial with link to colab/ binder. For this reason I often find it useful to have references at the bottom of page
-
This is also useful in context of generating pdf’s out of notebooks . Now I realize focus for fastpages is blogging but then again . I don’t want to redo one version for blog another version for printout a third version for nbdev.
b. Current jekyll-scholar solution - This one actually works well. I have taken similar approaches in past , albeit with a nbdev package . I did this by taking the build step out of github action and putting it on netlify side.
- nbdev currently has a much simpler build setup which works on windows / linux with nbdev_build_lib . I really prefer not to complicate it docker images and containers.
I believe a more robust solution still lies with a more robust integration of cite2c/ cite2js on nbdev side of things ;
- This will solve all the downstream issues
- Make things more interactive and visual on jupyter side which is our user interface.
- Help integrating things like Zotero/ mendley eventually ( plugin exists on jupyter side)
Problem
- Markdown post. From what I understand on inner working of nbdev/ fastpages. This is a 2 step process converting nb -> html/md (nbdev); rendering md -> html using jekyll build. We should probably explore something like this https://github.com/mwouts/jupytext. ( This again is currently based around jekyll-scholar)
Perhaps solution lies on nbdev side. Let me know what you think ?
Thanks for engaging and putting so much thought into your remarks, and for trying to find a robust solution. Yea, my jekyll-scholar solution, even though this discussion has taken place on this nbdev thread, was really for Fastpages, which while there’s overlap,…is hopefully not (see #2 below) all the same thing.
Some of the things you want to do, I don’t have as goals, and some of the things I want, maybe aren’t goals of yours. So the following responses are merely one users’s opinions: (Presumably Hamel and/or Jeremy will weigh in with something more measured & authoritative. 
)
-
It seems to me that multiple ways of doing things could be good – again, for Fastpages, not for fast.ai documentation. It may be that a one-size-fits-all solution isn’t suitable for all bloggers but that that they could be presented with a few options.
-
I need the non-Jupyter Markdown support for my current blogging activities; some of this came about after struggling (as Hamel can attest) with nbdev’s mangling of my added HTML formatting & javascript, and so needing to bypass nbdev. (And also, the extra Jupyter binder & colab links are overkill for these posts.) As an academic I’m used to using LaTeX and not seeing the bibliography until the document is built; so the desire to have WYSIWG for references in Jupyter is of no concern for me personally. I do however prefer the WYSIWG editing of Markdown via Typora as opposed to the wait-until-you-press-Shift-Enter constraints of Jupyter which (personal taste) I find to be uncomfortably ‘clunky’.
A couple other responses:
“I really prefer not to complicate it docker images and containers.”
Agreed. Although the jekyll-scholar solution merely involves changing one line in _config.yml and one line in the Gemfile,…so I don’t find that to be complicated.
“Help integrating things like Zotero/ mendley eventually ( plugin exists on jupyter side)”
I actually use Zotero! Both it and Mendeley will export BibTeX files.
Juptext does look really interesting.
Is jekyll-scholar still working for people using fastpages? With the latest version of fastpages, my References / bibliography section ends up completely blank. Probably user error on my part somehow, but…just checking.
What I did:
- git-clone the latest fastpages.
- modify the Gemfile and _config.yml as per the official instructions I wrote myself (and diff these files to compare with my older, working version(s) to check for any important omissions in my modifications)
- copy over my notebooks, Markdown posts, bibliography, CSL file, etc from my older version of blog
- run make server
Scrolling through the make server log (pasted below), I see only good things, and no bad things: I see that ruby is installing the jekyll-scholar plugin and a bunch of latex…stuff… and no errors are ever generated. …But in the end, no references for posts that used to have them.
?
— log:
$ make server
#chmod -R u+rw .
docker-compose down --remove-orphans || true;
Removing scottergories_jekyll_1 ... done
Removing scottergories_watcher_1 ... done
Removing scottergories_fastpages_1 ... done
Removing scottergories_notebook_1 ... done
Removing scottergories_converter_1 ... done
Removing network scottergories_default
docker-compose up
Creating network "scottergories_default" with the default driver
Creating scottergories_jekyll_1 ... done
Creating scottergories_notebook_1 ... done
Creating scottergories_watcher_1 ... done
Creating scottergories_fastpages_1 ... done
Creating scottergories_converter_1 ... done
Attaching to scottergories_watcher_1, scottergories_notebook_1, scottergories_fastpages_1, scottergories_converter_1, scottergories_jekyll_1
converter_1 | Agent pid 7
converter_1 | === Running Locally: All assets expected to be in the directory /data ===
fastpages_1 | Agent pid 7
fastpages_1 | === Running Locally: All assets expected to be in the directory /data ===
watcher_1 | Agent pid 13
watcher_1 | === Running Locally: All assets expected to be in the directory /data ===
notebook_1 | [I 16:06:34.712 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
fastpages_1 | No notebooks were modified
converter_1 | No notebooks were modified
watcher_1 | No notebooks were modified
scottergories_fastpages_1 exited with code 0
scottergories_converter_1 exited with code 0
notebook_1 | [W 16:06:35.808 NotebookApp] All authentication is disabled. Anyone who can connect to this server will be able to run code.
notebook_1 | [I 16:06:35.824 NotebookApp] Serving notebooks from local directory: /data
notebook_1 | [I 16:06:35.824 NotebookApp] The Jupyter Notebook is running at:
notebook_1 | [I 16:06:35.824 NotebookApp] http://9a8e0a1356bd:8080/
notebook_1 | [I 16:06:35.825 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
jekyll_1 | Fetching gem metadata from https://rubygems.org/.......
jekyll_1 | Using concurrent-ruby 1.1.7
jekyll_1 | Using i18n 1.8.5
jekyll_1 | Using minitest 5.14.2
jekyll_1 | Using thread_safe 0.3.6
jekyll_1 | Using tzinfo 1.2.7
jekyll_1 | Using zeitwerk 2.4.0
jekyll_1 | Using activesupport 6.0.3.3
jekyll_1 | Using public_suffix 4.0.6
jekyll_1 | Using addressable 2.7.0
jekyll_1 | Fetching latex-decode 0.3.2
jekyll_1 | Installing latex-decode 0.3.2
jekyll_1 | Fetching bibtex-ruby 6.0.0
jekyll_1 | Installing bibtex-ruby 6.0.0
jekyll_1 | Using bundler 2.1.4
jekyll_1 | Fetching namae 1.1.1
jekyll_1 | Installing namae 1.1.1
jekyll_1 | Fetching citeproc 1.0.10
jekyll_1 | Installing citeproc 1.0.10
jekyll_1 | Fetching csl 1.5.2
jekyll_1 | Installing csl 1.5.2
jekyll_1 | Fetching citeproc-ruby 1.1.13
jekyll_1 | Installing citeproc-ruby 1.1.13
jekyll_1 | Using colorator 1.1.0
jekyll_1 | Fetching csl-styles 1.0.1.10
jekyll_1 | Installing csl-styles 1.0.1.10
jekyll_1 | Using eventmachine 1.2.7
jekyll_1 | Using http_parser.rb 0.6.0
jekyll_1 | Using em-websocket 0.5.1
jekyll_1 | Using execjs 2.7.0
jekyll_1 | Using multipart-post 2.1.1
jekyll_1 | Using faraday 0.17.3
jekyll_1 | Using ffi 1.13.1
jekyll_1 | Using forwardable-extended 2.6.0
jekyll_1 | Using gemoji 3.0.1
jekyll_1 | Fetching mini_portile2 2.5.0
jekyll_1 | Installing mini_portile2 2.5.0
jekyll_1 | Fetching racc 1.5.2
jekyll_1 | Installing racc 1.5.2 with native extensions
jekyll_1 | Fetching nokogiri 1.11.1 (x86_64-linux)
jekyll_1 | Installing nokogiri 1.11.1 (x86_64-linux)
jekyll_1 | Using html-pipeline 2.14.0
jekyll_1 | Using sassc 2.4.0
jekyll_1 | Using jekyll-sass-converter 2.1.0
jekyll_1 | Using rb-fsevent 0.10.4
jekyll_1 | Using rb-inotify 0.10.1
jekyll_1 | Using listen 3.2.1
jekyll_1 | Using jekyll-watch 2.2.1
jekyll_1 | Using rexml 3.2.4
jekyll_1 | Fetching kramdown 2.3.1
jekyll_1 | Installing kramdown 2.3.1
jekyll_1 | Using kramdown-parser-gfm 1.1.0
jekyll_1 | Using liquid 4.0.3
jekyll_1 | Using mercenary 0.4.0
jekyll_1 | Using pathutil 0.16.2
jekyll_1 | Using rouge 3.23.0
jekyll_1 | Using safe_yaml 1.0.5
jekyll_1 | Using unicode-display_width 1.7.0
jekyll_1 | Using terminal-table 1.8.0
jekyll_1 | Using jekyll 4.1.1
jekyll_1 | Using jekyll-feed 0.15.0
jekyll_1 | Using sawyer 0.8.2
jekyll_1 | Using octokit 4.18.0
jekyll_1 | Using jekyll-gist 1.5.0
jekyll_1 | Using octicons 11.0.0
jekyll_1 | Using jekyll-octicons 11.0.0
jekyll_1 | Using jekyll-paginate 1.1.0
jekyll_1 | Using jekyll-relative-links 0.6.1
jekyll_1 | Using rubyzip 2.3.0
jekyll_1 | Using jekyll-remote-theme 0.4.2
jekyll_1 | Fetching jekyll-scholar 7.0.0
jekyll_1 | Installing jekyll-scholar 7.0.0
jekyll_1 | Using jekyll-seo-tag 2.6.1
jekyll_1 | Using jekyll-sitemap 1.4.0
jekyll_1 | Using jekyll-toc 0.14.0
jekyll_1 | Using jekyll-twitter-plugin 2.1.0
jekyll_1 | Using jemoji 0.12.0
jekyll_1 | Using katex 0.6.0
jekyll_1 | Using kramdown-math-katex 1.0.1
jekyll_1 | Using minima 2.5.1
jekyll_1 | Bundle complete! 19 Gemfile dependencies, 67 gems now installed.
jekyll_1 | Use `bundle info [gemname]` to see where a bundled gem is installed.
jekyll_1 | ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux-musl]
jekyll_1 | Configuration file: /data/_config.yml
jekyll_1 | Remote Theme: Using theme jekyll/minima
jekyll_1 | Source: /data
jekyll_1 | Destination: /data/_site
jekyll_1 | Incremental build: disabled. Enable with --incremental
jekyll_1 | Generating...
jekyll_1 | Remote Theme: Using theme jekyll/minima
jekyll_1 | Jekyll Feed: Generating feed for posts
watcher_1 | Agent pid 20
watcher_1 | === Running Locally: All assets expected to be in the directory /data ===
jekyll_1 | done in 64.336 seconds.
jekyll_1 | Auto-regeneration: enabled for '/data'
jekyll_1 | Server address: http://0.0.0.0:4000/scottergories/
jekyll_1 | Server running... press ctrl-c to stop.
watcher_1 | No notebooks were modified
^CGracefully stopping... (press Ctrl+C again to force)
Stopping scottergories_watcher_1 ... done
Stopping scottergories_jekyll_1 ... done
Stopping scottergories_notebook_1 ... done
(fastai) shawley@oryxpro:~/ClassificationBook/scottergories$ vi _config.yml
(fastai) shawley@oryxpro:~/ClassificationBook/scottergories$ make server
#chmod -R u+rw .
docker-compose down --remove-orphans || true;
Removing scottergories_converter_1 ... done
Removing scottergories_fastpages_1 ... done
Removing scottergories_watcher_1 ... done
Removing scottergories_jekyll_1 ... done
Removing scottergories_notebook_1 ... done
Removing network scottergories_default
docker-compose up
Creating network "scottergories_default" with the default driver
Creating scottergories_fastpages_1 ... done
Creating scottergories_jekyll_1 ... done
Creating scottergories_notebook_1 ... done
Creating scottergories_converter_1 ... done
Creating scottergories_watcher_1 ... done
Attaching to scottergories_watcher_1, scottergories_fastpages_1, scottergories_converter_1, scottergories_notebook_1, scottergories_jekyll_1
converter_1 | Agent pid 7
converter_1 | === Running Locally: All assets expected to be in the directory /data ===
fastpages_1 | Agent pid 7
fastpages_1 | === Running Locally: All assets expected to be in the directory /data ===
watcher_1 | Agent pid 13
watcher_1 | === Running Locally: All assets expected to be in the directory /data ===
fastpages_1 | No notebooks were modified
converter_1 | No notebooks were modified
notebook_1 | [I 16:52:27.548 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
watcher_1 | No notebooks were modified
scottergories_fastpages_1 exited with code 0
scottergories_converter_1 exited with code 0
notebook_1 | [W 16:52:28.743 NotebookApp] All authentication is disabled. Anyone who can connect to this server will be able to run code.
notebook_1 | [I 16:52:28.761 NotebookApp] Serving notebooks from local directory: /data
notebook_1 | [I 16:52:28.761 NotebookApp] The Jupyter Notebook is running at:
notebook_1 | [I 16:52:28.762 NotebookApp] http://e62845ca2e2e:8080/
notebook_1 | [I 16:52:28.762 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
jekyll_1 | Fetching gem metadata from https://rubygems.org/.......
jekyll_1 | Using concurrent-ruby 1.1.7
jekyll_1 | Using i18n 1.8.5
jekyll_1 | Using minitest 5.14.2
jekyll_1 | Using thread_safe 0.3.6
jekyll_1 | Using tzinfo 1.2.7
jekyll_1 | Using zeitwerk 2.4.0
jekyll_1 | Using activesupport 6.0.3.3
jekyll_1 | Using public_suffix 4.0.6
jekyll_1 | Using addressable 2.7.0
jekyll_1 | Fetching latex-decode 0.3.2
jekyll_1 | Installing latex-decode 0.3.2
jekyll_1 | Fetching bibtex-ruby 6.0.0
jekyll_1 | Installing bibtex-ruby 6.0.0
jekyll_1 | Using bundler 2.1.4
jekyll_1 | Fetching namae 1.1.1
jekyll_1 | Installing namae 1.1.1
jekyll_1 | Fetching citeproc 1.0.10
jekyll_1 | Installing citeproc 1.0.10
jekyll_1 | Fetching csl 1.5.2
jekyll_1 | Installing csl 1.5.2
jekyll_1 | Fetching citeproc-ruby 1.1.13
jekyll_1 | Installing citeproc-ruby 1.1.13
jekyll_1 | Using colorator 1.1.0
jekyll_1 | Fetching csl-styles 1.0.1.10
jekyll_1 | Installing csl-styles 1.0.1.10
jekyll_1 | Using eventmachine 1.2.7
jekyll_1 | Using http_parser.rb 0.6.0
jekyll_1 | Using em-websocket 0.5.1
jekyll_1 | Using execjs 2.7.0
jekyll_1 | Using multipart-post 2.1.1
jekyll_1 | Using faraday 0.17.3
jekyll_1 | Using ffi 1.13.1
jekyll_1 | Using forwardable-extended 2.6.0
jekyll_1 | Using gemoji 3.0.1
jekyll_1 | Fetching mini_portile2 2.5.0
jekyll_1 | Installing mini_portile2 2.5.0
jekyll_1 | Fetching racc 1.5.2
jekyll_1 | Installing racc 1.5.2 with native extensions
jekyll_1 | Fetching nokogiri 1.11.1 (x86_64-linux)
jekyll_1 | Installing nokogiri 1.11.1 (x86_64-linux)
jekyll_1 | Using html-pipeline 2.14.0
jekyll_1 | Using sassc 2.4.0
jekyll_1 | Using jekyll-sass-converter 2.1.0
jekyll_1 | Using rb-fsevent 0.10.4
jekyll_1 | Using rb-inotify 0.10.1
jekyll_1 | Using listen 3.2.1
jekyll_1 | Using jekyll-watch 2.2.1
jekyll_1 | Using rexml 3.2.4
jekyll_1 | Fetching kramdown 2.3.1
jekyll_1 | Installing kramdown 2.3.1
jekyll_1 | Using kramdown-parser-gfm 1.1.0
jekyll_1 | Using liquid 4.0.3
jekyll_1 | Using mercenary 0.4.0
jekyll_1 | Using pathutil 0.16.2
jekyll_1 | Using rouge 3.23.0
jekyll_1 | Using safe_yaml 1.0.5
jekyll_1 | Using unicode-display_width 1.7.0
jekyll_1 | Using terminal-table 1.8.0
jekyll_1 | Using jekyll 4.1.1
jekyll_1 | Using jekyll-feed 0.15.0
jekyll_1 | Using sawyer 0.8.2
jekyll_1 | Using octokit 4.18.0
jekyll_1 | Using jekyll-gist 1.5.0
jekyll_1 | Using octicons 11.0.0
jekyll_1 | Using jekyll-octicons 11.0.0
jekyll_1 | Using jekyll-paginate 1.1.0
jekyll_1 | Using jekyll-relative-links 0.6.1
jekyll_1 | Using rubyzip 2.3.0
jekyll_1 | Using jekyll-remote-theme 0.4.2
jekyll_1 | Fetching jekyll-scholar 7.0.0
jekyll_1 | Installing jekyll-scholar 7.0.0
jekyll_1 | Using jekyll-seo-tag 2.6.1
jekyll_1 | Using jekyll-sitemap 1.4.0
jekyll_1 | Using jekyll-toc 0.14.0
jekyll_1 | Using jekyll-twitter-plugin 2.1.0
jekyll_1 | Using jemoji 0.12.0
jekyll_1 | Using katex 0.6.0
jekyll_1 | Using kramdown-math-katex 1.0.1
jekyll_1 | Using minima 2.5.1
jekyll_1 | Bundle complete! 19 Gemfile dependencies, 67 gems now installed.
jekyll_1 | Use `bundle info [gemname]` to see where a bundled gem is installed.
jekyll_1 | ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux-musl]
jekyll_1 | Configuration file: /data/_config.yml
jekyll_1 | Remote Theme: Using theme jekyll/minima
jekyll_1 | Source: /data
jekyll_1 | Destination: /data/_site
jekyll_1 | Incremental build: disabled. Enable with --incremental
jekyll_1 | Generating...
jekyll_1 | Remote Theme: Using theme jekyll/minima
jekyll_1 | Jekyll Feed: Generating feed for posts
watcher_1 | Agent pid 20
watcher_1 | === Running Locally: All assets expected to be in the directory /data ===
jekyll_1 | done in 64.181 seconds.
jekyll_1 | Auto-regeneration: enabled for '/data'
jekyll_1 | Server address: http://0.0.0.0:4000/scottergories/
jekyll_1 | Server running... press ctrl-c to stop.
watcher_1 | No notebooks were modified