Hello,

I’m working on Kaggle and currently, I’m playing with models by trying to “unbox” them, in particular, to get the weights they store.

## TL;DR

My questions are:

- How can I get all the weights and parameters of, for instance, the first trained model of Chapter 10 of the Fastbook
- How can I do this with the help of the doc or
`??foo`

(as I will explain below, I’m quite at loss to find the attribute lists)

Basically, it’s less a post about finding the weights than a **post about the method of reading and find one’s way about the doc**, so now I include more precise examples

## Long version + more specific questions

**You can find the questions by searching the string “question:”

**Context** just after the first training of Chapter 10, i.e. just after the cell:

```
learn.fit_one_cycle(1, 2e-2) # ~22 minutes to run of Kaggle
```

So now, I have my trained model and I want to unbox pretty much anything there is in it. So let’s start:

```
learn.model
```

gives me:

```
SequentialRNN(
(0): AWD_LSTM(
(encoder): Embedding(60008, 400, padding_idx=1)
(encoder_dp): EmbeddingDropout(
(emb): Embedding(60008, 400, padding_idx=1) )
(rnns): ModuleList(
(0): WeightDropout(
(module): LSTM(400, 1152, batch_first=True) )
(1): WeightDropout(
(module): LSTM(1152, 1152, batch_first=True) )
(2): WeightDropout(
(module): LSTM(1152, 400, batch_first=True) ))
(input_dp): RNNDropout()
(hidden_dps): ModuleList(
(0-2): 3 x RNNDropout() ))
(1): LinearDecoder(
(decoder): Linear(in_features=400, out_features=60008, bias=True)
(output_dp): RNNDropout() ))
```

Then, I focus on the first element of the list:

```
mod0 = learn.model[0]
```

- First attribute:

```
mod0.encoder
```

I get `Embedding(60008, 400, padding_idx=1)`

so I can write `learn.model.encoder.weight.data`

to get the weights of the encoder. Thins work also fine with `mod0.input_dp`

and `mod0.hidden_dps`

(4th and 5th attributes)

**Second attribute (fail)**: the cell above has suggested that`learn.model`

has also`encoder_dp`

and an attribute, but:

```
mod0.encoder_dp
```

gives `AttributeError: 'AWD_LSTM' object has no attribute 'encoding_dp'`

**Question 1**: why is this so? Whereas in the cell `learn.model`

, `encoder`

and `encoder_dp`

appear in a similar fashion?

**Question 2**: if I understand correctly, AWD_LSTM is a fastai (and not a Pytorch) feature. So I looked at this page of the Fastai doc. There is no indication on the possible attribute that I can read from AWD_LSTM. Where shoud I find them? What am I missing?

- 3rd attribute:

```
mod0.rnns
```

works fine and I get

```
ModuleList(
(0): WeightDropout(
(module): LSTM(400, 1152, batch_first=True) )
(1): WeightDropout(
(module): LSTM(1152, 1152, batch_first=True) )
(2): WeightDropout(
(module): LSTM(1152, 400, batch_first=True) ))
```

So, now, I’ll have a look at the element of rank 0 of this `ModuleList`

:

```
mods0.rns[0]
```

I get:

```
ModuleList(
(0): WeightDropout(
(module): LSTM(400, 1152, batch_first=True))
(1): WeightDropout(
(module): LSTM(1152, 1152, batch_first=True))
(2): WeightDropout(
(module): LSTM(1152, 400, batch_first=True) ))
```

So, I write

```
lstm0 = mod0.rnns[0].module ; lstm0
```

which displays `LSTM(400, 1152, batch_first=True)`

**Question 3**: how do I get the weights in`lstm0`

? Are they any?**Question 4**: how is this information indicated for instance in the Pytorch doc for`LSTM`

? Is this page the right place to get this info?

I tried the thing which seems rational to me, i.e. doing `??lstm0`

and looking at the Pytorch doc page on LSTM which in the **Variables* section, suggested that I could get the attribute `weight_ih_l[k] `

.

Thus I wrote

```
lstm0.weight_ih_l[0]
```

which gives me the error: `AttributeError: 'LSTM' object has no attribute 'weight_ih_l' `

**Question 5** What am I reading wrong in the pytorch doc? How should I get the weights of `lstm0`

?