Collaborative note creation and way to proceed?

Don’t get me wrong. I am all for it. We should definitely try it out and see how people react.

Most likely we need to twist the version control strategy a bit. The big idea itself is on the mark.

I’d avoid using Jupyter notebooks as a collaborative tool for multiple concurrent editors like a plague. Unless every participant wants to learn how to merge and resolve conflicts in json files, and do it a lot. That wouldn’t be very encouraging for people to want to contribute. You will have to learn how to use nbdime, which let’s you merge json notebook files in a visual way.

Collaborating on nbs is ok, if you use a token approach which we more or less use for the fastai docs. i.e. I check out the latest docs, quickly edit them and push my changes in. We try to avoid situations where 2 people edit the same notebook at the same time. It’s also very easy to make a mistake and overwrite someone’s changes when doing the merging. This is easy to catch with code, but almost impossible to catch with prose.

I recommend to stick to simple formats that are easy to edit and merge, especially so at the beginning of creation. If you have to include runnable-code probably plain python scripts is the best way. And if really desired there could be a simple tool that combines the final pieces together into a notebook, i.e. combine the text with python scripts and automatically run the outcome like we do with the docs. And once the notes are more or less complete it’s OK to continue with the finished combined notebook, since the future changes would be infrequent and thus avoid merge conflicts.

So you use .md files under git and if you need to include runnable code with outputs, you simply add: code/lesson_8_example_1.py inside .md file, and add that python script to the repo. And then at the end of the note writing period, when there almost no changes left to add, write a script to split that md file into text cells and code cells (sucking all .py files’ content in) and save them into a notebook.

4 Likes

The other approach is to have a note captain for each class and have contributors contribute edits, suggestions and improvements directly to the captain and she/he does all the editing and supervision. No source control issues. I did that while writing my first book and it worked great.

And again, once the notes are complete, then they can be released as a notebook and then it shouldn’t be a problem to add occasional fixes.

I think you’d want to have a coordinator/supervisor for each class anyway, otherwise it might be tricky if there is a disagreement on what Jeremy really meant when he said “Ugh!”

5 Likes

If you consider using discourse wiki, I’d first investigate it’s reliability as a concurrent editing tool. I have seen at least once when my changes were overwritten by another editor who was editing at the same time. We both started with the same edit, then both were editing concurrently, I saved first, he did second, my changes were replaced by his changes. Not good.

But I haven’t investigated whether it’s a bug in discourse or it was a fluke (if there is such a thing in software engineering). I just know now not to edit wiki posts in discourse if the system is telling me someone else is either already editing it or have just started editing, since I no longer trust it.

2 Likes

Maybe an alternative would be using google docs which is suitable for collaborative editing + comments etc. and also possible to web publish…

3 Likes

Maybe an alternative would be using google docs which is suitable for collaborative editing + comments etc. and also possible to web publish…

google docs is indeed a good one for collaborating.

Also I see that people do send PRs with fixes to Hiromi’s notes:


so perhaps nothing needs to be changed, other than making it easy on Hiromi so that others could have commit access and help with PRs.

1 Like

I will attempt to summarize your points here so that I can make sure I get what you mean. Please let me know if I misunderstood anything again.

  • Jupyter Notebook is not a good collaborative tool for multiple concurrent editors. Too high a collaboration overhead.
  • Follow the token approach we used for fastai docs.
  • Stick to simple file formats that are easy to collaborate on, like markdown with code cell annotation, then turn them into Jupyter Notebooks at the end using automated workflow.
  • Have a note captain in charge of each class and serve as the conflict resolver and final arbitrator.
  • Discourse wiki might have some issue with concurrent editing.
2 Likes

Actually, now thinking about it, why not give Google Colab a try? It is Jupyter Notebook interface but with the collaborative editing support of Google docs.

Maybe we can just let the note captains decide what approach they would like to take, and see how things work out.

3 Likes

I vote for google docs. We can collaborate on the notes and after Part 2 is complete and we’ve created highly polished notes-then we upload them to a GH repo.

I was worried about someone accidentally deleting notes/editing them-Google docs show the complete history so that wouldn’t be a problem.

Are there any other shortcoming in this approach?

5 Likes

Bookdown seems to be a nice tool https://bookdown.org/yihui/bookdown/

It provides means for collaboration

Regardless of tool choice which is very important . I agree that we need to have a person or steering committee to be in charge of this project…

1 Like

is this a way to deliver lesson notes?

https://colab.research.google.com/github/tensorflow/swift/blob/master/docs/site/tutorials/walkthrough.ipynb

1 Like

Colab seems worth a shot - COLLABoration in Jupyter Notebooks is what it’s designed for :smile:

1 Like

I know, right? :wink:

This looks pretty good to me!

Yeah even better with dark mode :slight_smile:

1 Like

Google docs, the easier the better and it’s pretty nice for real-time coordination of a group while taking notes during the lecture. Collaborating on notebooks would be a separate topic, for notes during lessons I would avoid all github complexities, even for using MD files.

Just to start out, other tools (probably MD on github) should follow.

Naming coordinator / coordinator per lesson could follow along the way, I would try (small timebudget) to help. If @hiromi is in this thread, maybe even she is most the qualified (based on her super cool md lesson write ups ;-)) and her github repo could help distributing this.

I guess one question here is, if we write down word by word (like Hiromis repo) or add & rephrase and restructure as we like. Or we do both and mark the difference. Would tend to think only writing down the exact words is a task soon to be done by a bot :slight_smile: and hence using bulletpoints, adding links and remarks maybe (qualified as such) etc. should be good

Makes sense?

1 Like

Hey @Benudek, thanks a lot for your input! It is great to hear from your experience what would work well and what would not.

Right now we are just going to create a signup thread (soon), and people who wish to take the lead could propose in the top wiki post. Then the fellow students can choose whatever proposal they find appealing to contribute to.

1 Like

ok, sure. thats fine - I took out the google doc and we could add it later.

As I take notes anyways and hugely benefitted from e.g. Hiromis Notes, keep me posted so can try helping.

1 Like

Guys, let me direct your attention to Collaborative Lesson Notes . For all of you who would like to be coordinators, please fill your name in the signup sheet along with other necessary information.

1 Like