# Lesson 5: Create his own Linear Layer

Hi guys,

During the fasi.ai course Lesson 5 https://youtu.be/CJKnDu2dxOE at 1:30:30 he suggest to try to create his own Linear cald My_Linear for example.
To better understand how fast.ai work I tried.

``````class My_layer(nn.Module):
__constants__ = ['bias']

def __init__(self, in_features, out_features,bias=True):
super().__init__()
self.in_features = in_features
self.out_features = out_features
self.weight = Parameter(torch.Tensor(out_features, in_features))
self.bias = Parameter(torch.Tensor(out_features))
self.reset_parameters()

@weak_script_method
def forward(self, input):
y_hat = input@self.weight.t()
y_hat = y_hat+self.bias
return y_hat

def reset_parameters(self):
init.kaiming_uniform_(self.weight, a=math.sqrt(5))
if self.bias is not None:
fan_in, _ = init._calculate_fan_in_and_fan_out(self.weight)
bound = 1 / math.sqrt(fan_in)
init.uniform_(self.bias, -bound, bound)

class My_layer(nn.Module):
__constants__ = ['bias']

def __init__(self, in_features, out_features,bias=True):
super().__init__()
self.in_features = in_features
self.out_features = out_features
self.weight = Parameter(torch.Tensor(out_features, in_features))
self.bias = Parameter(torch.Tensor(out_features))
self.reset_parameters()

@weak_script_method
def forward(self, input):
y_hat = input@self.weight.t()
y_hat = y_hat+self.bias
return y_hat

def reset_parameters(self):
init.kaiming_uniform_(self.weight, a=math.sqrt(5))
if self.bias is not None:
fan_in, _ = init._calculate_fan_in_and_fan_out(self.weight)
bound = 1 / math.sqrt(fan_in)
init.uniform_(self.bias, -bound, bound)
model = Mnist_Logistic().cuda()

def update(x,y,lr):
wd = 1e-5
y_hat = model(x)
# weight decay
w2 = 0.
for p in model.parameters(): w2 += (p**2).sum()
loss = loss_func(y_hat, y) + w2*wd
loss.backward()
for p in model.parameters():
return loss.item()

losses = [update(x,y,lr) for x,y in data.train_dl]
``````

And I have this error

``````Traceback (most recent call last):
File "/opt/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 677, in __del__
Traceback (most recent call last):
File "/opt/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 677, in __del__
self._shutdown_workers()
self._shutdown_workers()
File "/opt/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 659, in _shutdown_workers
File "/opt/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 659, in _shutdown_workers
w.join()
File "/opt/anaconda3/lib/python3.7/multiprocessing/process.py", line 138, in join
w.join()
File "/opt/anaconda3/lib/python3.7/multiprocessing/process.py", line 138, in join
assert self._parent_pid == os.getpid(), 'can only join a child process'
assert self._parent_pid == os.getpid(), 'can only join a child process'
AssertionError: can only join a child process
``````

What I understood it is that the weight are normalize during the init part and during the training (forward) the weight are updated.
I had an error before (tensor size mismatch) but it was because I forgot to transport self.weight.
But then this error happen but I don’t understand what I missed.

1 Like

It seems like this is probably an error with PyTorch and that the actual code might be fine.

See here and here.

1 Like

Yeah I saw this post and it look like I won’t be able to resolve that.
Thanks for your help Hi, in lesson 5 I’m not able to download the dataset ML-100k and I’m using google colab.
How can i get the dataset?
@ilovescience

Check your system path with pwd.

Normally in Colab I change my system path by:

``````import os
os.chdir('/content/gdrive/My Drive/Colab Notebooks')
``````

From there it is easier to check the file.

1 Like

@AjayStark the dowloaded file is not downloaded in the right directory that is Config.data_path(). Use this in line 13 of your code in the above screenshot:

``````path = Config.data_path()
! wget -P {path} http://grouplens.org/datasets/movielens/ml-100k.zip
``````

After this u can carry after this.

1 Like

@dhoa Hi, it worked. Thanks for the response Hi @umaidzz it works, but when i give the names(user, item,ratinig,timestamp), it says user is not defined.

@AjayStark the elements: user and item in [user, item, ‘rating’, ‘timestamp’] are not in single quotes which passes them as variables instead of string which have not been defined. Replace:

``````name = [ user, item, 'rating', 'timestamp']
``````

with
`name = [ 'user', 'item', 'rating', 'timestamp']`

or add this line at the top of this cell:
user, item = ‘user’, ‘item’

1 Like

I restarted runtime and now i’m not even able to unzip
@umaidzz

@AjayStark If u restart your kernel, you don’t need to unzip the folder again because the data was downloaded once and then u unzipped it before. So now u need to skip these 2 steps as they are not required because data is already present

1 Like

@AjayStark If you want to understand whats happening in these lines. The lines with exclamation mark at the start indicate that it is a terminal command so in your terminal of jupyter instance go to the data directory by running

``````cd .fastai/data
``````

Then print the directory with

``````ls
``````

You will find your ml-100k.zip file there then use unzip

``````unzip ml-100k
``````

A directory by the name of ml-100k will be created

These steps are already done by you once so the directory has already been created by you the first time you ran the commands

1 Like

@umaidzz Oh, i understand.
Thanks for the detailed explanation 1 Like

Hi,
In lesson 5, movielens, how to create dataset using datablock api?
I tried this:

np.random.seed(42)
data=(CollabList.from_df(ratings_movie)
.split_by_rand_pct(0.2)
)
But how to set item_name=title?
@umaidzz

just do .from_df(… item_name = 'title …)

1 Like

@umaidzz yeah got it, thanks
And I’m trying to write nn.Linear(from below) from scratch.

class mnnist_logistic(nn.Module):
def init(self):
super().init()
self.lin1= nn.Linear(784,10, bias=True)

def forward(self, xb):
return self.lin1(xb)

In this nn.Linear performs: x@a+b and I have x, how do it get a and b (i.e the embedding and bias)?

You create a a tensor and initialize them with ones or zeros by using the function torch.zeros or torch.ones()

1 Like

oh ok and the nn.Linear does the same thing?

Yes, you can look up the pytorch docs u will find the same thing

1 Like