How to use your finished trained network incorporated into a program?

OK… (this is great and i dont want anyone to think its not, as i get distracted when things get exciting like this!)

the challenge puts forth a page, and not much i the way of data…
you have to figure out, using what little they give you, how to get the data you need…
ie. you have to take what small code framework they give you, and change it to not only make queries, but to format data, and obtain it, as they do not provide the data in any easy format…

i was able to do that, and was able to create several datasets (at one time) of over 100,000 queries to their system to make a training, val, test pool… 70k, 20k, 10k…

with that data then formatted and dropped into a file to read, i then was able to construct a tabular net that gave the results you can see above… wonderful… (and probably if i worked at it more, i might get it higher than accuracy .982500 - or maybe thats the top… )

this tool then constructed has to be taken back to the original small piece of code they provide, and inserted to take the data they are providing without answer in the first step, insinuate itself to then provide an answer, and replace the guessed answer of the 2nd step…

(the first step, provides one sample, putting that into the 2nd step with a random answer, gives you the answer of whether your right, and the right answer in return… putting the two together makes a complete target/data sample unit… run it 100k times an you get your pool)…

once you have this accurate ‘guess’. you feed it into the 2nd step…
once you get a large number of answers in a row (over 500), the system replies with a hash code to validate your achieving the goal… you then provide the code you wrote, the saved file, and everything along with a resume to get a potential interview stop… there are 4 or 5 other challenges in different domains… this is the first one i will hit… you only have to hit one… but i will try to hit all of them except for the electronics challenge, because without a job currently i am not spending cash on raspberry pi and that stuff…

i hope i explained that well enough…

so in essence, what i need to do is to take the structure and rebuild it the same way as it was or is in my training series… load in the saved data of the weights so that this structure works… then be able to feed it a given data sample, and get the result… (all this of course in their framework)… this result then answers the question of the series for real instead of a false guess, and allows you to accurately answer the series a large number of times in a row.

:slight_smile:

note that i am also trying not to cheat by giving more information than i need to get the piece working
i am quite honest and so forth, and do not want to game their system outside of what they are trying to accomplish… (nor provide a way for hundreds of other people to do so with little effort)

also please forgive the language issue… ie. with only three days in, my terms are not yet going to be right or classical in terms of NN, but are going to be from the other knowlege domains i work in… i can only do so much in a few days… ha ha

2 Likes

for the interest of clarity, here is a pseudo code
this is ALL you get for the challenge… nothing else… the rest is up to you

the quote code removes the tab formatting i had… sorry

import necessary python modules

create loggging…

create a server object
in server object:
define the base URL target
define self session variables

def a request
while true
process in a try exception the asking for
a unit of data to analyse and predict
[if you hit their limit, wait 60 seconds and try again]

def a get(self) that uses the request and puts data into variables and returns

def a post(self) that that sends the answer back and uses client session state
to know what data was sent in the get step

create a main loop
instantiate server object

for the target number of steps:
do a get()

post() back a random answer (until you can change it to send back a real one)
get response that says whether the answer was correct, and gives the correct answer

While i am kind of sort of waiting…
i am obtaining more data, and trying training on a different representation of the same data
one or the other may raise my accuracy beyond .982500… (or may not)

Not easy to understand, however if you have to predict a number of results in a batch, go with the get_preds method already mentioned; if you have to predict one at a time, https://docs.fast.ai/basic_train.html#Learner.predict . The rest is programming.

1 Like

need one at a time after i do all the learning (which is batch).
thanks… i will read and hope i can put it together from that…

do note that so far, there isnt one complete example of end to end code for after the training
your sugestion of predict is just one tiny piece of such an example, which would come after building and loading the nn with the saved weights…
after all if your going to deploy, deploying is not going to be with the test data there just so you can get through the code but with a clean tabular nn and load, and then letting it work…

IF i can get this right, i would not mind sharing my code so that a clean example would exist
then doing the same for other networks as i complete them (or rather use them)

this is my first project
the 2nd one is finance as i have a great large dataset
the 3rd i know HOW to do, detecting/locating a firearm without xrays or magnetics
[but there is no way for me to get the data i would need for that at this point (and i mean generate it myself)]

1 Like

thanks to VDM!!! i am making progress… and progress is good…

while i have the training network and such in place:

df.iloc[3457]
pred = learn.predict(df.iloc[3457])
pred

With pred[0] containing the output i am looking for…
Huzzah!!!

this is probably enough to get me where i am going in this project…
however, i would like to write a nice end to end example everyone can use
not just ‘get it working’
regardless… i will get to that point one way or another…
when i do i will then use the tutorial version to write my example
its the least i can do
:slight_smile:

Hi Artfldgr Best of luck with your future endeavors!

I haven’t built a tabular app otherwise I would send you my code.

I you want an intro to fastai2 here’s a course that is running however its changing daily so I am not using it in production yet. A walk with fastai2 - Vision - Study Group and Online Lectures Megathread

however, i would like to write a nice end to end example everyone can use
not just ‘get it working’
regardless… i will get to that point one way or another…
when i do i will then use the tutorial version to write my example
its the least i can do

This notebook has some notes on deployment that could give you some ideas its for fastai2 so would so may need some adapting for fastai 1.

https://colab.research.google.com/github/muellerzr/Practical-Deep-Learning-for-Coders-2.0/blob/master/Computer%20Vision/02_Deployment.ipynb#scrollTo=J-FaHcuTzdq8

There are a number of people on this forum who have are trying to create a simple starter app for each ML approach in fastai. Hopefully when version 2 is released we will be able to complete this. The above notebook created by @muellerzr is a step in the right direction.

Cheers mrfabulous1 :smiley: :smiley:

1 Like

I’ll be trying to get tabular working here in the near future as well, but if you’re trying to do batch predictions you should build a test dataloader of some form (there’s numerous examples here on the forum on this, and I’ve made some too) for tabular so you can do them faster than learn.predict()

1 Like

ok… here is what i did…

i created a very small sample filler file of data, just so that i could repeat the train code
but without any intent to use it… (call this the quick and dirty form till I figure out how to do without)
i imported all the same modules then

path = Path(“a/”)
df = pd.read_csv(path / ‘verysmallsamplefiller.txt’)
dep_var = ‘Target’
cat_names = [‘col1’,‘col2’,‘col3’,‘col4’,‘col5’,‘col6’,‘col7’,‘col8’,‘col9’,‘col10’,‘col11’,‘col12’,‘col13’,‘col14’,‘col15’,‘col16’,‘col17’,‘col18’,‘col19’,‘col20’,‘col21’,‘col22’,‘col23’,‘col24’,‘col25’,‘col26’,‘col27’,‘col28’,‘col29’,‘col30’,‘col31’,‘col32’,‘col33’,‘col34’,‘col35’,‘col36’,‘col37’,‘col38’,‘col39’,‘col40’,‘col41’,‘col42’,‘col43’,‘col44’,‘col45’,‘col46’,‘col47’,‘col48’,‘col49’,‘col50’,‘col51’,‘col52’,‘col53’,‘col54’,‘col55’,‘col56’,‘col57’,‘col58’,‘col59’,‘col60’,‘col61’,‘col62’,‘col63’,‘col64’]
cont_names = []
procs = [FillMissing, Categorify, Normalize]
testpct = int(len(df) * .10)
valpct = int(len(df)* .20)
trainpct = int(len(df) * .70)

test = TabularList.from_df(df.iloc[-testpct:-1].copy(), path=path, cat_names=cat_names, cont_names=cont_names)

data = (TabularList.from_df(df, path=path, cat_names=cat_names, cont_names=cont_names, procs=procs)
.split_by_idx(list(range(valpct,trainpct)))
.label_from_df(cols=dep_var)
.add_test(test)
.databunch())

learn = tabular_learner(data, layers=[250,400], metrics=accuracy)
learn = learn.load(“Filename”)

now all the above works in the original notebook i developed and trained in
however, when i try to load the weights in the copy it gives an error
[the SAME code in the original notebook does not give an error!!]

To make sure it worked, i copied the original code to a new notebook
shut down jupyter, and then used the new notebook to reconstruct the net
and use the pred = learn.predict(df.iloc[3227]) code, and it worked…

i will work on the error, but here it is just so others can see what happened…
remember, i copied everything from a working model, including the sizes and even loaded in a tiny bit of data (just to get by without changing code)

RuntimeError Traceback (most recent call last)
in
----> 1 learn = learn.load(“Filename”)

C:\Anaconda3\lib\site-packages\fastai\basic_train.py in load(self, file, device, strict, with_opt, purge, remove_module)
271 model_state = state[‘model’]
272 if remove_module: model_state = remove_module_load(model_state)
–> 273 get_model(self.model).load_state_dict(model_state, strict=strict)
274 if ifnone(with_opt,True):
275 if not hasattr(self, ‘opt’): self.create_opt(defaults.lr, self.wd)

C:\Anaconda3\lib\site-packages\torch\nn\modules\module.py in load_state_dict(self, state_dict, strict)
828 if len(error_msgs) > 0:
829 raise RuntimeError(‘Error(s) in loading state_dict for {}:\n\t{}’.format(
–> 830 self.class.name, “\n\t”.join(error_msgs)))
831 return _IncompatibleKeys(missing_keys, unexpected_keys)
832

RuntimeError: Error(s) in loading state_dict for TabularModel:
size mismatch for embeds.0.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([133, 25]).
size mismatch for embeds.1.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([145, 26]).
size mismatch for embeds.2.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([147, 26]).
size mismatch for embeds.3.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([143, 26]).
size mismatch for embeds.4.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([138, 25]).
size mismatch for embeds.5.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([146, 26]).
size mismatch for embeds.6.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([148, 26]).
size mismatch for embeds.7.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([144, 26]).
size mismatch for embeds.8.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([145, 26]).
size mismatch for embeds.9.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([139, 25]).
size mismatch for embeds.10.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([149, 26]).
size mismatch for embeds.11.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([139, 25]).
size mismatch for embeds.12.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([148, 26]).
size mismatch for embeds.13.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([152, 27]).
size mismatch for embeds.14.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([145, 26]).
size mismatch for embeds.15.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([143, 26]).
size mismatch for embeds.16.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([140, 25]).
size mismatch for embeds.17.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([153, 27]).
size mismatch for embeds.18.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([137, 25]).
size mismatch for embeds.19.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([152, 27]).
size mismatch for embeds.20.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([149, 26]).
size mismatch for embeds.21.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([145, 26]).
size mismatch for embeds.22.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([141, 26]).
size mismatch for embeds.23.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([143, 26]).
size mismatch for embeds.24.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([145, 26]).
size mismatch for embeds.25.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([147, 26]).
size mismatch for embeds.26.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([141, 26]).
size mismatch for embeds.27.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([149, 26]).
size mismatch for embeds.28.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([143, 26]).
size mismatch for embeds.29.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([144, 26]).
size mismatch for embeds.30.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([135, 25]).
size mismatch for embeds.31.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([147, 26]).
size mismatch for embeds.32.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([164, 28]).
size mismatch for embeds.33.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([143, 26]).
size mismatch for embeds.34.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([146, 26]).
size mismatch for embeds.35.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([137, 25]).
size mismatch for embeds.36.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([144, 26]).
size mismatch for embeds.37.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([149, 26]).
size mismatch for embeds.38.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([143, 26]).
size mismatch for embeds.39.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([143, 26]).
size mismatch for embeds.40.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([147, 26]).
size mismatch for embeds.41.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([143, 26]).
size mismatch for embeds.42.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([149, 26]).
size mismatch for embeds.43.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([135, 25]).
size mismatch for embeds.44.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([148, 26]).
size mismatch for embeds.45.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([136, 25]).
size mismatch for embeds.46.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([140, 25]).
size mismatch for embeds.47.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([143, 26]).
size mismatch for embeds.48.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([153, 27]).
size mismatch for embeds.49.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([133, 25]).
size mismatch for embeds.50.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([144, 26]).
size mismatch for embeds.51.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([151, 27]).
size mismatch for embeds.52.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([141, 26]).
size mismatch for embeds.53.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([137, 25]).
size mismatch for embeds.54.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([152, 27]).
size mismatch for embeds.55.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([147, 26]).
size mismatch for embeds.56.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([154, 27]).
size mismatch for embeds.57.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([139, 25]).
size mismatch for embeds.58.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([140, 25]).
size mismatch for embeds.59.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([150, 26]).
size mismatch for embeds.60.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([149, 26]).
size mismatch for embeds.61.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([136, 25]).
size mismatch for embeds.62.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([133, 25]).
size mismatch for embeds.63.weight: copying a param with shape torch.Size([257, 36]) from checkpoint, the shape in current model is torch.Size([142, 26]).
size mismatch for layers.0.weight: copying a param with shape torch.Size([250, 2304]) from checkpoint, the shape in current model is torch.Size([250, 1656]).

and i am NOT trying to do batch…
each request is singular… so i cant batch them…
i have to answer single requests in a row…
so the code above will work…
its just not pretty…
(once i figure out why the load is giving the error… and yes, i know it says its different, but its not, its an exact copy from the working model, and matches a copy of a copy to make sure)

thanks…
this is great…
so close, but yet so far

This could be do to how your splits happened to be done and how the categories lied or something, basically your procs didn’t quite match up perfectly and so the size of your categorical embeddings was slightly different (I’ve faced this quite a bit)

why are you loading without saving in advance?
By the way, use the export methid to save a model for deployment.

1 Like

i saved previously…
so what i am trying to do is take what was created in one notebook
and install just the code i need in another…

i am trying to be clear that what i am doing is making a deployed version
not a deploy at the end of a training session

so in another notebook i did all the work and trained and saved versions till i got “the best”
now i am trying to get the code into another program in which there is no training ever

just using the finished net for single predicting… (not batch)
one query at a time…

FOUND THE ERROR!!

the error was caused by the small file i was using as a placeholder to get past not knowing how to build the network and load without the train, test, validate data being present…

i took it all apart and then deleted and updated items in a separate notebook till i could reproduce it
now i am working on making a new clean tiny file so i can get past this point…

though what i was looking for was how to create the net without the test, validate, train data anymore, just load the weights and use the single use predict (not batch)

==========================================================================

ok… now its using the training data to discard potential solutions…
so i have to make a tiny file to take up space that has all the possible outcomes…
[which is so dirty compared to being able to instantiate a nn, load in weights, and predict]
i have changed the names of the items from the category list because of the challenge nature of this.

C:\Anaconda3\lib\site-packages\fastai\data_block.py:537: UserWarning: You are labelling your items with CategoryList.
Your valid set contained the following unknown labels, the corresponding items have been discarded.
xx1, xx2, xx3
if getattr(ds, ‘warn’, False): warn(ds.warn)
C:\Anaconda3\lib\site-packages\fastai\basic_data.py:248: UserWarning: Your training dataloader is empty, you have only 6 items in your training set.
Your batch size is 64, you should lower it.
Your batch size is {self.train_dl.batch_size}, you should lower it.""")

eventually this will work… but not because its right, but because i wrestled it and cheated around the thing to get something that works… not the right thing that works…

==========================================================================

ok… so now i have to put in a 56 meg file with 100,000 test cases just to get it to work…
how messy is that? good thing i didnt use a larger file yet… (this kind of sucks)
:slight_smile:

now that it works with copying in a 56 meg file just so i can load the weights

i cant just feed it the new data, because thats in a dataframe which also has the ‘answer’ from training
and of course what you WANT to do is feed it the data only, without the target and without headers and get an answer…

to think that this is an afterthought is amazing…

==========================================================================

basically to get this to work (in this dirty fashion)
i have to get this

11110000,10010001,00000000,00000000,00101101,10000001,00111110,10000001,00110001,10000011,00100000,10000011,10001111,01011111,10000000,10010011,00000000,00000000,10000000,10010001,00000000,00000000,10000110,00110001,00000100,11110000,00000000,11000000,10001111,01011111,00000000,11000000,10000000,11100000,10000011,00110000,00000100,11110000,00010000,10010010,00000000,00000000,00010000,10010010,00000000,00000000,00000000,11000000,00010000,10010010,00000000,00000000,00010000,10010010,00000000,00000000,00000000,11000000,11100000,10010001,00000000,00000000,11110000,10010001

into this

Target ans01
col1 100000
col2 10100000
col3 0
col4 11000110

col60 10010100
col61 10000
col62 100000
col63 0
col64 10001
Name: 3457, Length: 65, dtype: object

i guess i can make a file with the column heads…
read that file, write it out to a work file
then append the data
and read it back in
then voila… i have a single line copy of the data…
of course, this is soooo much easier than just being able to give it a comma delimited list or such!!!
sometimes python really doesnt make things easier…

1 Like

thanks mr fabulous…
but the link was not productive…
https://colab.research.google.com/github/muellerzr/Practical-Deep-Learning-for-Coders-2.0/blob/master/Computer%20Vision/02_Deployment.ipynb#scrollTo=J-FaHcuTzdq8

i dont need to create a web application that retrieves a batch file…
heck… i can do that in tons of other languages a lot easier to use…

what i need, and seems no one has or can give me

is how to create a nn without training data that loads the weights and can put out a prediction one at at time!!!

its like everyone worked on a car engine, but forgot you need seats, tires, brakes and all the other stuff to put it on the road… that there is no way to professionally take what you create and use it in any easy way without the training wheels in place… it boggles the mind…

sadly… it makes this into a really cool toy for braniacs…
you can make nets, you can train them, you can even use them to predict in situ

but what you cant do is instantiate the structure, load in the weights, and use the completed item as a real tool without everything else in any reasonably easy fashion!!!

in 3.5 days i was able to get the toy to take in data, and make a net that had a high success rate
its appears its going to take more than a week or so to use it in any real fashion even in an ugly way loading 56 megs of data i dont need any more and wont be used Just to get it to work…

not to mention not being able to hand it data in easy ways…
as an example…

The data i trained it on has 65 columns, 1 being with a target answer and in a dataframe with headers

a completed tool would take 64 columns or data, and spit out an answer… without needing the headers, without needing the data you used to develop on to prevent errors, without needing to give it batch files or all that.

my finance application has 5 million records in 700+ gig file…
the fact i am looking at having to load that in just to use the net makes me depressed…

this reminds me of the history of canning…
they made the cans years before they made can openers…
you could get the food into the cans, and have them last, but you had no way to get it out
people used hammers, and chisels, and so on…

fastai seems similar…
very very depressing…

This is discussed, and has been brought up multiple times. You should follow the Lesson 2 example from the course about deployment. You need to export your model, via learn.export() and then you can load in the data in a variety of ways in which we have provided here.

What was communicated to us at first was in fact what sounded like a web application, not applying inference upon some dataset.

Again, there are deployment examples in which they all use data without the training code, this is why we are showing them.

I am confused on why this is not the case here? This is up to how you design the website and UI. fastai simply expects an input of a numpy array with your procs already stated and applied in there how your tabular model expects.

fastai is a library for training models, and then how you perform inference has been helped by methods and wrapped and documented by the course, users, and by the documentation as well so there are numerous examples with this. There have been many users now who are trying their best to help you, please take that into consideration :slight_smile:

Now, for your predictions you should be looking at generating a test set to pass into your model. This can be done after exporting your model via learn.export() and loading back in. There are examples here: https://docs.fast.ai/tutorial.inference.html from the documentation

1 Like

i think there is a huge communication gulf here…
i am not using this on a website… never said website… go search the thread and no one said website
so that is assuming (remember the trope that goes with it and seems to be true here)

maybe there are deployment examples…
but so far i have not found the core of it…

i saved my net… with learn.save(‘filename’)
now i should have used learn.export? ok. fine

but now… where is the example code of instantiating the empty net and loading it?
if it was so easy… why hasnt anyone in this whole thread pointed to the 25 lines of code?

to create a nn, load in data, and restore its weights takes only this

import torch
import pandas as pd
from fastai import *
from fastai.tabular import *
from pathlib import Path
path = Path(“a/”)
df = pd.read_csv(path / ‘lg_binaryol.txt’)
dep_var = ‘Target’
cat_names = [‘col1’,‘col2’,‘col3’,‘col4’,‘col5’,‘col6’,‘col7’,‘col8’,‘col9’,‘col10’,‘col11’,‘col12’,‘col13’,‘col14’,‘col15’,‘col16’,‘col17’,‘col18’,‘col19’,‘col20’,‘col21’,‘col22’,‘col23’,‘col24’,‘col25’,‘col26’,‘col27’,‘col28’,‘col29’,‘col30’,‘col31’,‘col32’,‘col33’,‘col34’,‘col35’,‘col36’,‘col37’,‘col38’,‘col39’,‘col40’,‘col41’,‘col42’,‘col43’,‘col44’,‘col45’,‘col46’,‘col47’,‘col48’,‘col49’,‘col50’,‘col51’,‘col52’,‘col53’,‘col54’,‘col55’,‘col56’,‘col57’,‘col58’,‘col59’,‘col60’,‘col61’,‘col62’,‘col63’,‘col64’]
cont_names = []
procs = [FillMissing, Categorify, Normalize]
testpct = int(len(df) * .10)
valpct = int(len(df)* .20)
trainpct = int(len(df) * .70)
test = TabularList.from_df(df.iloc[-testpct:-1].copy(), path=path, cat_names=cat_names, cont_names=cont_names)
data = (TabularList.from_df(df, path=path, cat_names=cat_names, cont_names=cont_names, procs=procs)
.split_by_idx(list(range(valpct,trainpct)))
.label_from_df(cols=dep_var)
.add_test(test)
.databunch())

learn = tabular_learner(data, layers=[250,400], metrics=accuracy)
learn = learn.load(savenetII01")

less than 25 lines to be ready to make it learn more, or play with it, and such

it should take LESS than that to import, create a empty net of structure, load structure, load weights and be ready to answer… by sending just a list the same size as the data trained (without a target).

it shouldn’t matter if i am using it in a web app, or on my desk, or in a phone, etc…
because with that core, you could use it anywhere you could run python…

that’s all i have been asking for since the beginning…
because that core can be inserted where its needed…
in my case, inside someone elses framework i have to use
in another case, maybe as a python module called by a coldfusion/java program on a website
in another case, on a phone… or tablet phablet…

all the other stuff around that request has nothing to do with the essential
and thats what i been trying to get at.
how to make a empty shell that matches the training structure, load in whats needed, and then be able to query that to get outputs…

i will look at lesson two…
hopefully the 15 line of code i need are there… (given you can amazingly do most everything else with less than 25 lines… thats incredible!!! )… maybe you guys are overthinking what should be an easy answer… what should be a tiny how to post…

once i figure it out, i will do that post…
but i have to get there from here…

thanks, you guys been great…
but it still boggles the mind (of someone who has been writing code since the late 1970s professionally)

=========================

imagine this all was completed… there is no more training… there is no more test set…
there is only what core remains to be used… THAT is what i am looking for…
of course the first post apologizes that i have not learned the lingo in 3.5 days
so i dont know to use the word inference…
or that learn.save and learn.export are two different things (are they?)

but whatever is around the core code that does the work AFTER all the creation and testing is done
is not relevant… that, as you say, is up to the person deploying that core…

but so far, i have not found that easy answer without all the other stuff around it
its like you cant tell me how to install a water heater without first telling me how to build a house!

and i made a mistake above… been writing code since the late 1970s…
been doing it professionally since about 1984-1985 - about 35 years…
so i am not a dunce when it comes to this stuff… as i am still writing code and learning new stuff

dont get me wrong… what is here is amazing… really it is… (and i am a tough critic)
but whats missing is the easy core without the distracting costume on the outside
the naked person, not the person in a suit, a sari, a samurai outfit, etc…

thanks… i do appreciate it… i am just a bit frustrated…

=======================================

i just looked at Create a Learner for inference…
its still about validation and training… not consumption…

i went through all the text of lesson 2… nothing about after all the work is done what to do to have a naked system to wrap with what you need…

Create a Learner for inference…

third sentence in the tutorial
Now that our data has been properly set up, we can train a model. We already did in the look at your data tutorial so we’ll just load our saved results here.

i dont want to train it… its trained… i dont want to test it, its tested
i want what comes AFTER that… using it without training or testing any more…

at the bottom is the tabular example…
it shows how to load in your whole thing again, with training and test

notice data on top has valid_idx = range (800,1000) which is for the adult database it says above it


it has learn fit after that… which trains the network, no?

then it says learn.export()… should that be alone or learn.export(‘filename’)
because after that, it says load learner adult…
above it adult is the file adult = untar_data(URLs.ADULT_SAMPLE) from adult
why is that needed for deploy?

and then it calse predict… but its looking for one of the loaded adult data…
its not a simple string someone puts in to get an answer…

so the example you told me to look at doesnt have what you claim it has…
i wouldnt be bothering anyone if it did…

All the parts for inference without any data natively begin after the learn.export() step. To try this, try doing learn.export() on your model, then with a seperate learner, IE learn2 = load_learner(export.pkl) attempt to do your single prediction inference. This is what it’s talking about. No data (IE rows or our training/validation sets) is saved with the model, and no data is needed to bring the model in

ok, so this is basically it…

with the caveat that data has to be in what format?
the dataframe format with the column heads?
[sorry, python renames things that ppl been using for decades
its only been two weeks so havent learned the new lingo]

or can you give it a list?

lets say your data is like mine… 65 columns with the target
so without the target its 64 columns…

would a dataframe of 64 columns be ok?
how about a list with 64 items?
how about a comma delimited string of entries?

the documentation and such is not very clear on it…
which is why i have been having a problem
its as if its written for people who already know, not people who dont know

predict(self, item:ItemBase, return_x:bool=False, batch_first:bool=True, with_dropout:bool=False, **kwargs)

And looking up itembase isnt helpful either…

a sentence would go a long way in the docs with a complete excised example as i show…

basically what i show with some data defined in some way that is acceptable, and a sentence that would say it has to be in the same format minus the target… with the target… or different format with the right number would do, etc…

this is why things are terribly unclear…
this whole thread would never have happened if there was a page that it connected to that said, this is what you do to use it after you do the export from the prior page… the data has to be in x format… minus the target… or with a target that is ignored… etc…

even now i am going to have to try to figure out what format the data has to be in

if the original data was a data frame of
target, col1, col2 col3
“something”,0001,1010,1111

do i need a dataframe like this?
target, col1, col2 col3
“”,0001,1010,1111

or a dataframe like this
col1, col2 col3
0001,1010,1111

or
or a dataframe like this
0001,1010,1111

or would a list do?

this is why its confusing to me…
its clear to people who wrote it, and know whats expected
its not clear to someone who arrived from planet zed and has to use that documentation

thanks…
i will keep plugging at it till i hit the magic combination of data format…
i will first try to convert my comma delimited string into a dataframe with headers that matches the originally loaded data… (that in itself will keep me busy at my level of python!!! :stuck_out_tongue_winking_eye:

There is an example with a dataframe in the Inference documentation. Pass a row from the dataframe, which is read in via Pandas. All that matters is that the columns have the same name as what you trained on as your x labels

2 Likes

thats what i thought…
but a single sentence that says:

The data that gets passed has to be in the same data frame format as the data the net was trained in, it will ignore the target column…

would go a long long way as would breaking the docs into
page 1) preparing the model for deploying
page 2) using the prepared model in a deploy

with the above sentence that makes it clear as to the format for either a single prediction or using the other term, a multiple prediction…

though if your using a dataframe like the original that went in, why have a single and a batch statement as a single is just a batch with one entry then…

formatting my data is my problem, not yours…
so thanks for your infinite patience…

in a few days i will figure out how to change the format of what i have into what i need without having to save it to a file and read it in which is how it got changed in the first place… tomorrow is my interview and i have to get ready for that too… so cant keep on this… solutions architect is the position… thanks again

1 Like

It says so here, btw:

“And we can predict on a row of dataframe that has the right cat_names and cont_names .”, also any mention of “dataframe” is 99% of the time a reference to a Pandas dataframe

2 Likes

I told you before. Use the export method. And, please recognize that 3 days is a too short timespan to learn what is needed, even with Fastai :wink:

1 Like