Managing secrets in nbdev

Hey fastai & nbdev community

Curious to see if anyone else has a view on managing secrets within nbdev, specifically when testing inline during development?

Here is background and rationale for this question:

  • I first started playing with nbdev to make a package that was used in multiple microservices. This was to avoid repeating code used within several of the services. A few of the functions and classes within this repo used secrets/keys - connecting to databases, calling a paid API, etc. In order to properly test these within nbdev, the keys need to be made available - however, in development they change a lot (certainly the database passwords), so I can’t just “set it and forget it” on each machine.
  • Inline testing is my favourite feature of nbdev. But when creating packages or scripts that rely on secrets/keys (as mentioned above) especially in development/iteration, I can’t currently use nbdev with 100% confidence for fear that I will leave a key exposed (a key that is needed for inline testing).
  • I hate setting environment vars - can never remember the syntax (I switch between Mac and PC a lot). I forget how to find which ones I’ve set, which names I used, etc.

I acknowledge I went against the recommendations of the contribution guidelines to create a mockup of what this could look like - this was more for my own understanding of how nbdev works than to actually come up with a legit solution on my own without validation of the issue from others. It doesn’t work 100% as I want but I’m posting this before I spend (waste?) any more time on it.

The gist is here for anyone curious.

Thanks for any feedback or input

3 Likes

I’m currently struggling with secrets as well, and while I’m importing them as individual files, tests this breaks if I’m using relative paths.
Your suggestion seems very interesting. I’d love to see it adapted to the current version of nbdev.