I want the WMAE error to be displayed each time I call the fit function.
I have a column holiday_week which checks for holiday, but wondering how do I access this column holiday_week while the model is training?

Just having trouble understanding this.
I looked at Rossmann code. For Rossman it was RMSE and it was passed via metrics.

There was no weighted mean in exp_rmspe. But here we need to check if each prediction is a holiday_week and accordingly add a weight to the formula.

My question is how do I check/access this holiday_week inside each prediction? Not sure if I am doing this correctly, as I am not able to understand how to do this at all. Also, not sure what y_range is doing in the Rossmann code.

Thanks @ecdrid and @anandsaha … Am not sure of the formula muself, but good idea to check via excel and see. Will do that.

I’m lost on how to get holiday_week from y_pred inside the function? The actual code while fitting the model. Or do I make another df containing only weights and pass that along into exp_wmae function?

I know this is an old thread, but I found it useful, so in case anyone needs a little more help, I have written the code for the WMAE based on the excel document posted by @anandsaha

Hi,
after reading different posts and docs I finally was able to implement the Weighted MAE in a callback class that I can pass to the learner and calculate the metric at training time!!!
My version doesn’t use any passed parameters but gets the needed information directly from the training input data.
Here the code:

class Weighted_Mae(LearnerCallback):
_order=-20 # Needs to run before the recorder
def __init__(self, learn):
super().__init__(learn)
self.w_sum = 0
self.numerator_sum = 0
self.output_list = []
self.target_list = []
self.w_mae = 0
self.input_list = []
def on_epoch_begin(self, **kwargs):
self.w_sum = 0
self.numerator_sum = 0
self.output_list = []
self.target_list = []
self.input_list = []
self.w_mae = 0
def on_batch_begin(self, last_input, last_target, **kwargs):
# Here I'm taking only the categorical variables from the input tensor
# last_input[0] = categorical variables
# last_input[1] = continuos variables
self.input_list.append(last_input[0])
def on_batch_end(self, last_output, last_target, **kwargs):
self.output_list.append(last_output.flatten())
self.target_list.append(last_target)
def on_train_begin(self, **kwargs):
self.learn.recorder.add_metric_names(['weighted_mae'])
def on_epoch_end(self, last_metrics, **kwargs):
self.output_list = torch.cat(self.output_list)
self.target_list = torch.cat(self.target_list)
self.input_list = torch.cat(self.input_list)
for pred,targ,inp in zip(self.output_list, self.target_list, self.input_list):
w = 1
if inp[3]: # the third categorical variable is IsHoliday
w = 5
self.w_sum += w
self.numerator_sum += w*(abs(targ-pred))
self.w_mae = self.numerator_sum/self.w_sum
return add_metrics(last_metrics, [float(self.w_mae)])