Weights and Biases Introduction

Hello,

I am Thomas, I work at Weights and Biases. We wrote with @jeremy the hyperparameter search for fine-tuning the image models that he showed on the walkthrus. Sweeps are great to explore different training recipes and configurations, so…

I think it’s due time for an introduction to experiment tracking with W&B :sweat_smile:

I am planning on doing an intro to Weights and Biases for fastai users. The idea is to give students a simple walkthru on how to use W&B with fastai. If you want to start reading about W&B and fastai you can start with:

24 Likes

Thanks for organizing

I wonder if Jeremy changed his opinion about experiments tracking; as (if I am not mistaken) it was mentioned during one of the lectures that sometimes it is too distracting and making you watching training curves instead of preparing next experiments. (Though I like W&B and used it quite a lot.)

Though I guess it depends on use-case. Running multiple experiments in parallel, i.e., like you did for picking the best fine-tuning architecture, is definitely a good idea.

1 Like

Yes I still think that’s a danger. I don’t look at the runs while they’re happening or use any of the GUI features. I just used it for the same purpose as fastgpu, basically (i.e managing a list of jobs to run and putting them onto free GPUs as they’re ready). For doing sweeps over a cartesian product of configuration options it’s a bit easier to do in wandb than fastgpu at this stage.

I know a lot of folks really like the full experiment tracking system and it seems to be very well designed in wandb. It’s just not something that I have need of myself.

9 Likes

Thanks Thomas.

I’ll join if I can (from Canberra). A colleague introduced me to wandb a few months ago, but I have yet to use it fully. I am pretty sure some form of sweeps are on the card. I can be choosy about the level of logging/provenance I need depending on the task, but in this case I’ll need something like wand. Having written in-house metaheuristics frameworks logging system back a decade ago, I certainly am keen to use existing offerings this time around…

Wish I were (back) in the Dauphine.

1 Like

I would not say that Sweeps is our most powerful feature. it’s cool, but we have other more important features. As Jeremy said, we could have done this cartesian product of parameters with a couple of for loops. The async of the sweep controller (enabling you to run agents in parallel) can be very practical.

Most of the value I saw from W&B was in my previous company, when I needed to present analysis to managers/stakeholders that did not know anything about programming but understood the problem we were solving. Being able to share results embedded in a WYSWYG format was life-changing. I could stay on notebooks, and they could edit and present the data on a Report.

6 Likes

Hi Thomas,
Could you go through what you did for Jeremy and the setup in wand?
I published a paper previously. It is a standard question for reviewers to ask benchmarking our results against multiple architectures. Different journal/conference has a different preference for architecture. It was a time-consuming task. Also, is any logging system to list out the final output together with all the hyperparameters?
I am a beginner and would like to join in person. I am in Perth, Western Australia (GMT+8).

1 Like

This is an important point – I don’t have this use-case myself, and it’s a critical foundation of what wandb is all about, so I’m not really a great person to comment on whether it does it well or not!

4 Likes

I have not fully explored this myself yet, but this sounds like a nice way of causing some mayhem on social media :slightly_smiling_face: Presentation of results goes a long way!

1 Like

This will be so helpful as I have been struggling to understand how to set it up! Thank you so much!

1 Like

Looking forward to this. I’ll try my best to join live if it’s held 3pm+ CET sometime early-mid week.

2 Likes

Yeah, that’s a good point. Interesting enough that at my work, we mostly use DIY reporting solutions. I guess that some years ago W&B / Streamlit / etc. didn’t exist or weren’t yet that advanced. But I like the reporting features and dashboards provided by W&B. Would be great to see if this keeps evolving into even more flexible visualization toolkit.

1 Like

I am planning on doing this intro on Friday, June 24, at 10am CET (18h Brisbane Time). I may do a second one for our US folks, later on.

It will be a chilled-out zoom call with time for questions and demos. My idea is not to overwhelm people with W&B features, just give some value straightaway for fastai users.

Link to the meeting: Launch Meeting - Zoom

The password is fastai creator’s last name.

6 Likes

Had a great time discussing W&B with everyone. For the folks in America (not only USA) do we book another session on a more friendly time schedule?

4 Likes
1 Like

@tcapelle can you share the notebooks you had used during the presentation to showcase how to use fast.ai with Weights and Biases?

I got my first wandb project working with fast.ai and I am planning to use it to log my experiments for Paddy competition in Kaggle.

2 Likes

I shared the official wandb fastai example:

4 Likes

Thanks for sharing :pray:

I’m lagging a bit on the forums recently. Any chances these sessions are/will be recorded & published later ?

3 Likes

I will re do this today, and I will record it this time =P

Link: Launch Meeting - Zoom

This is Today Jun 30, at 17:30 CET (8:30 am Los Angeles Time)

password: is fastai creators lastname

3 Likes