Example: nbdev on Gitlab

Hi all,

after posting a problem here about the source links on Gitlab pages (link), i thought maybe I can share my current setup for nbdev on Gitlab:

Note:

Steps

I created a new empty repo, cloned it and set up a venv with pipenv. Then I installed nbdev and all other necessary packages.

After that I ran nbdev_new (with some warning messages because the default github stuff is not available)

Then I changed settings.ini and _quarto.yml for the use with gitlab

Changes made in settings.ini

  • set doc_path = public
  • set branch = main instead of master
  • change doc_host to your gitlab pages url, e.g. https://{userid}.github.io/{reponame}
  • change git_url to your gitlab repo url, e.g. https://gitlab.com/{userid}/{reponame}

Changes in _quarto.yml

  • repo-branch: main
  • site-url same as above e.g. https://{userid}.gitlab.io/{reponame}
  • repo-url dito https://gitlab.com/{userid}/{reponame}

Note: I just blindly changed every line containing “github”, I really don’t know which of these is relevant. But it works :slight_smile:

And to use Gitlab pages you need to adapt the CI-pipeline, here is my (very simple) gitlab-ci.yml. I am using pipenv here, so you may have to remove/adapt the pipenv related statements.

# The Docker image that will be used to build your app
image: python:3.8-bullseye
# Functions that should be executed before the build script is run
before_script:
  - apt install wget
  - wget "https://github.com/quarto-dev/quarto-cli/releases/download/v1.1.149/quarto-1.1.149-linux-amd64.deb"
  - dpkg -i quarto-1.1.149-linux-amd64.deb
  - pip3 install pipenv
  - pipenv install --dev
pages:
  script:
    - pipenv run nbdev_install
    - pipenv run nbdev_docs
  artifacts:
    paths:
      # The folder that contains the files to be exposed at the Page URL
      - public
  rules:
    # This ensures that only pushes to the default branch will trigger
    # a pages deploy
    - if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH

The nbdev command nbdev_publish doesn’t work here, you can simply push your changes and the docs will be build by the CI-pipeline.

So, I hope this helps maybe one or the other

6 Likes

Rob Johnson shared this also.
Maybe you can use his docker image:
image: robtheoceanographer/nbdev2:latest
The following doc_host was working in nbdev1
doc_host = https://%(user)s.pages.%(company_name)s.de/%(repo_name)s/

1 Like

Thanks for sharing this very helpful

1 Like

Based on these examples (thanks to @tom_500 and Rob), here are my steps to make it happen in my context:

The key part is in .gitlab-ci.yml where I want 4 steps:

  • test
  • build
  • build_doc
  • deploy_artifactory (when tag is set)

The 1st steps are identical to github’s:

default:
  image: 'docker.artifactory.acme.com/acme/hub/ubuntu20.04:latest'
  tags:
    - k8s
  interruptible: true
  retry:
    max: 2
    when:
      - runner_system_failure
      - stuck_or_timeout_failure

# Functions that should be executed before the build script is run
before_script:
  - apt -y install wget
  - wget "https://github.com/quarto-dev/quarto-cli/releases/download/v1.1.189/quarto-1.1.189-linux-amd64.deb"
  - dpkg -i quarto-1.1.189-linux-amd64.deb
  - apt -y install python3-pip
  - pip3 install nbdev
  - nbdev_install

stages:
  - test
  - build_doc
  - build
  - deploy_artifactory

tests:
  stage: test
  script:
    - nbdev_test

pages:
  stage: build_doc
  script:
    - nbdev_docs
  artifacts:
    paths:
      # The folder that contains the files to be exposed at the Page URL
      - public
  rules:
    # This ensures that only pushes to the default branch will trigger
    # a pages deploy
    - if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH

wheel:
  stage: build
  script:
    - mkdir -p public
    - echo "Build wheel with python version `python3 --version`:"
    - pip install -U setuptools wheel 
    - pip install -e .
    - python3 setup.py bdist_wheel
    - mkdir -p packages && mv dist/* packages/
  artifacts:
    when: always
    paths:
      - packages/

publish:
  stage: deploy_artifactory
  dependencies:
    - wheel
  only:
    - tags
  script:
    # create credential config file
    - >
      if [ -f '.pypirc' ]; then
        echo "Information: .pypirc file is not mandatory anymore." && cp .pypirc ~/
      else
        echo "[distutils]
        index-servers = local
        [local]
        repository: https://artifactory.acme.com/api/pypi/pypi
        username: <id>
        password: <secret>" > ~/.pypirc
      fi
    - pip install -U twine
    - pip index versions nbdev_gitlab || true
    - echo 'If the "twine upload" command below failed with a 403 status code, please check that the version is not already uploaded on artifactory (see versions of nbdev_git above).'
    - twine upload --verbose -r local packages/*

I am quite sure this is highly sub optimal, it was my first time using CI/CD in gitlab.
But at the end I have documentation available in gitlab pages, (and even a nice badge in gitlab pointing to documentation) and lib available in artifactory meaning that anyone at acme can pip install it.

4 Likes

You can add company_name = acme to your settings.ini file and then just write %(company_name)s where you were writting acme before.

2 Likes