How to include citation in nbdev exported html?

I think you have to use html syntax here to make it work.

You’ll need to run jekyll yourself to do this, and use this package:

1 Like

Links in quotes in general will work - just not in that specific location, since that’s where the page summary goes.

Thanks @jeremy and @sgugger

Since I am new to jekyll world, exploring these options is going to take some time. I have reviewed following things so far :-

  • Idea 1: Starting from scratch and learning basics of jekyll template and jekyll plugins. Online recommendations include following
    • Jekyll scholar is not included on github pages so either use it to build _site locally or use a service which can let you build the same. Netlify seems to have worked for people. I prefer building _site from netlify deploy settings but havenot succeeded in doing the same so far.

    • Building with jekyll locally on windows is giving me a lot of pain. I could use linux VM to learn the basics which I think I should do .But I want to figure out a windows solution first as most of my team is on windows machine with GPU.[GPU won’t work on VM side]

  • Idea 2: Hack a command which can do some preprocessing before using nbdev_build_docs

    • Looking at generated html from nbdev_build_docs in my docs folder; I find html has been stripped of all the metadata which includes metadata added by from cite2c plugin from jupyter; which means whatever I do these will not work
      image

    • Read a bit of source code to figure out what is happening in notebook2html function. I am still not certain of all the things HTMLParsers are doing . This would probably require much more effort.

    • I like how I can dynamically see references getting included by cite2c plugin in rendering in jupyter notebook. In above approach there might be some hack to read metadata -> find all ‘cite’ tags using regex -> do some external call cite2proc-js -> replace string output -> call nbdev_build_docs. However this would require much more time to explore which I currently lack.

I think I have now spend more time on this then I intended. I will keep learning & exploring on and off whenever I can afford some more time.

Ok Got a tentative solution working with nbdev using jekyll scholar as @jeremy suggested . Steps to make it work

  • If you are on windows , make sure ruby 2.6.2 is installed and not 2.7

  • Update Gemfile as follows

source "https://rubygems.org"
gem "rake"
gem "jekyll-scholar", group: :jekyll_plugins
gem 'wdm', '>= 0.1.0' if Gem.win_platform?
gem 'github-pages', group: :jekyll_plugins

# Added at 2019-11-25 10:11:40 -0800 by jhoward:
gem "jekyll", "~> 3.7"
  • Run bundle install . This will update dependencies and Gemfile.lock

  • Add _plugins/jekyl_scholar.rb in docs folder with following content
    `

require ‘jekyll/scholar’

  • Add a _bibiliography/references.bib . This file contains all the citations.

  • Update _config.yml as follows

scholar:
   source: _bibliography
  • Add line {% bibliography %} or {% bibliography --cited %} in one of the markdown cells in notebook

  • Hosting site on netlify, So Git push works. [Don’t have to build locally. I am still struggling with viewing local docs , sidebar links not working at all]

  • Here is the final result

Hi,

I would like to propose my solution for citations, https://github.com/ducha-aiki/nbdev/blob/latex_envs_citations/nbs/03_export2html.ipynb

Specifically, it consists of two stages:

  1. Install jupyter extension https://github.com/jfbercher/jupyter_latex_envs
    Then one could use latex-style \cite{Ref1} in markdown + bibtex file

It also adds 3 buttons to jupyter notebook (at the right):

After click on “Refresh” and “Book” buttons above, one gets following rendering in jupyter notebook

And the reference section in the end of notebook, like this:

  1. export to html with modified nbdev, specifically, I have added the following

The final result of fastpages-generated is below, where links are same-page links to and from references.

If anyone is interested, I can PR that branch into nbdev. @sgugger

P.S. You can see result here: https://ducha-aiki.github.io/wide-baseline-stereo-blog/2020/03/27/intro.html

1 Like

@ducha-aiki There is already a way to do this in fastpages! See https://github.com/fastai/fastpages/blob/master/_fastpages_docs/NOTEBOOK_FOOTNOTES.md

Detailed Guide To Footnotes in Notebooks

Notebook -> HTML Footnotes don’t work the same as Markdown. There isn’t a good solution, so made these Jekyll plugins as a workaround

This adds a linked superscript {% fn 15 %}

{{ "This is the actual footnote" | fndetail: 15 }}

You can have links, but then you have to use single quotes to escape the link.

This adds a linked superscript {% fn 20 %}

{{ 'This is the actual footnote with a [link](www.github.com) as well!'  | fndetail: 20 }}

However, what if you want a single quote in your footnote? There is not an easy way to escape that. Fortunately, you can use the special HTML character ' (you must keep the semicolon!). For example, you can include a single quote like this:

This adds a linked superscript {% fn 20 %}

{{ 'This is the actual footnote; with a [link](www.github.com) as well! and a single quote ' too!'  | fndetail: 20 }}

1 Like

Will copy an extended answer from twitter for the sake of completeness:

footnote-based solution does not fit me because of :

  1. Things mentioned by @rahuketu86

During this effort , following features where a big boost in productivity

  • Create a separate bib file
  • Include the citation anywhere in rmarkdown file, from figure caption to text, just by calling a [@bibkey] was a significant boost in productivity
  • Generate a section at the bottom of rendered html with all the cited reference
  1. I generate not only blog post from the notebook, but also latex source for my thesis. Making this two things separately (nicely formatted blog and nicely formatted latex->pdf) creates too much burden.

@ducha-aiki thanks for sharing. Haven’t thought about multiple target outputs. It looks like you may have a plan on how to do that via nbdev, so look forward to watching that thread.

Well,I already have implemented it, see above.

It works in fastpages locally, but fails on github, I suppose because I am failed to make github actions custom docker image instead of standard

Yeah I can help with that. Why don’t you submit a PR to nbdev? Looks like a reasonable set of features to me? Based on my experience, this is something that has a reasonable probability of being merged. ( @sgugger of course will review)

Once that is done, I can bump all the official images for fastpages to include the changes.

Sure. The reason I haven’t submitted PR yet is because I am following the contributing guide)))

“Once your approach has been discussed and confirmed on the forum, you are welcome to push a PR, including a complete description of the new feature and an example of how it’s used. Be sure to document your code in the notabook.”

All I would advise is to make sure you include tests ( maybe you already have them ) in your changes

I guess another thing you want to check for is there are no collisions with other features, somehow. I have some suspicion that this could collide with other magic that handles links, But I don’t think it will - nothing seems off to me, but just asking to make sure

Yes, I have one, although not sure is it is exported to the tests:

#hide
cell = {'cell_type': 'markdown', 'source': r"""This is cited multireference \cite{Frob1, Frob3}.
And single \cite{Frob2}."""}
expected=r"""This is cited multireference [<a class="latex_cit" id="call-Frob1" href="#cit- 
Frob1">Frob1</a>,<a class="latex_cit" id="call-Frob3" href="#cit-Frob3">Frob3</a>].
And single [<a class="latex_cit" id="call-Frob2" href="#cit-Frob2">Frob2</a>]."""
test_eq(_cite2link(cell)["source"], expected)
'''
1 Like

Following up on this, a few months later:
@hamelsmu Is there now any consensus for how can we best (i.e. most easily) incorporate citations from a bibtex file into our Fastpages blogs? I’m fine with trying to plug in jekyll-scholar, just wasn’t sure if this was still a “you’re on your own” kind of thing or if it’s now been ‘standardized’ somehow. (I realize such a feature is probably beyond the scope of your original vision for this increasingly successful blogging platform! :wink: )

Btw, I’m enjoying writing ‘pure Markdown’ posts using Typora (which I find to be a ‘beautiful’ editor), rather than writing in Jupyter. Alternatively, I could switch to doing these “Markdown-only” posts in just one big Jupyter cell and then use @ducha-aiki’s method. But…that would not be my preferred method.

(If it ends up being that I’d need to run some sort of manual script after editing, such that the build is not fully automated, I’m actually ok with that.)

Jekyll scholar , I believe can’t be standardized as I understand, becoz GitHub doesn’t support the plugin. You either do a local build or use Netlify
. That’s how I managed to do it.

1 Like

@rahuketu86 This is not true - the plugin is supported in fastpages because we materialize the static site ourselves in GitHub Actions before deploying to GitHub Pages.

@drscotthawley I am not well educated on the best citation workflow. Is there something in your testing that works with the least effort in fastpages? if so, if you want to propose a recommendation that would be helpful and perhaps you can submit a blog post onto the fastpages site dedicated to citations, and how to do them? What do you think?

I’m happy to give it a start. I just recently had the idea of wanting to do citations from bibtex, but hadn’t actually tried it yet.

When you say “the plugin is supported in fastpages” – does that mean that I don’t have bother trying to install jekyll-scholar myself? ( I followed jekyll-scholar’s ‘gem install…’ instructions a couple days ago and it built a bunch of things on my laptop; but I’m not sure how to get GitHub Actions to reliably do the same. )

If not, is there a place with instructions on 'adding plugins to fastpages"? If not I’ll just muck about as if fastpages were any other jekyll blog.