#seprating images into folders according to labels:
I am using this code to seprate images of HAM10000 but it is showing this error...
#Sort images to subfolders first
import pandas as pd
import os
import shutil
# Dump all images into a folder and specify the path:
data_dir = os.getcwd() + "/data/all_images/"
# Path to destination directory where we want subfolders
dest_dir = os.getcwd() + "/data/reorganized/"
# Read the csv file containing image names and corresponding labels
skin_df2 = pd.read_csv('data/HAM10000/HAM10000_metadata.csv')
print(skin_df['dx'].value_counts())
label=skin_df2['dx'].unique().tolist() #Extract labels into a list
label_images = []
# Copy images to new folders
for i in label:
os.mkdir(dest_dir + str(i) + "/")
sample = skin_df2[skin_df2['dx'] == i]['image_id']
label_images.extend(sample)
for id in label_images:
shutil.copyfile((data_dir + "/"+ id +".jpg"), (dest_dir + i + "/"+id+".jpg"))
label_images=[]
ERROR:
File “C:\Users\Sam\PycharmProjects\pythonProject6\main.py”, line 8
data_dir = os.getcwd() + “/data/all_images/”
^
SyntaxError: (unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3: truncated \UXXXXXXXX escape
Hi @samely, Since the above error message indicates it occurs on Line 8 I’m confused that you include all that code after Line 8 in your query. Do you think that code may have an impact on the error? Including all that additional code makes it harder to answer you.
Tip: To make it easier to answer you, please trim down your code to a Minimal Example. That is, in your notebook, remove everything after Line 8, and see if the error remains. If the error goes away, restore lines until the error comes back.
Line 8 also seems to rely only on “import os”, and on no other lines prior to Line 8, so delete those preceeding lines. I suspect you’ll end up trimming your notebook down to as small as this…
import os
dest_dir = os.getcwd() + “/data/reorganized/”
with the error still occuring. This tiny bit code has a much smaller “surface area” that is easier to reason about. If you get trimmed down to this…
and the error occurs on line 3, that means line 2 was okay. Since these lines are very similar, a Minimal Example facilitates a close examination for what is different between the working and non-working lines, which may highlight what may be causing the error. At ths point, try googling for your error message.
p.s. For problems in my own code, I often find the process of paring down my code getting ready to ask a question on the forum, causes the answer leap out at me. Please follow up with your observations from running your Minimal Example. This will provide good info so readers can try to assist your further.
Hello! I’m Rayan. I’m a beginning to deep learning but have a pretty good understanding of Python. I’ve started to read the book (Deep Learning for Coders) and I’m kind of confused. I tried reading the book before watching the videos on YouTube. However, the code (2022 version) in the video is completely different from the book’s code. Also, I’m not sure where to find the exact same code used in the book.
I’m intimidated by all the code and stuff. Does the book (or video course) explain all the imports afterwards or does it assume any prior knowledge of the libraries used?
Can anyone guide me through the course? Also, I think most of the code and versions of the libraries used in the book have changed, when will the second edition of the book be released?
Hi Rayan
I suggest you do the video first and probably watch each one three times. You will then have a 70% understand of Deep Learning. Next read the book and it will make 100% sense. There is an on-line erratum for the book which desribes the mistakes and changes to the code. All the lessson are available as Juptyter notebooks which you should follow.
Regards Conwyn
Greetings all! Not sure if this is the best place to ask this question, but figure it’s “general” enough. I’ve been lurking around fast.ai for sometime even working on some lessons and such. In recent days I’ve become really enamored with the capabilities of LLMs and the likes of GPT-4 for “copilot” usage, etc. Part 2 of the course looks awesome working with Stable Diffusion, but does it give any insight into how one would develop/train an LLM for use as a personalized copilot? I’d be grateful for any feedback from those that might have completed Part 2 or anyone that has knowledge of this. Thanks in advance for any help!
In case anyone else is wondering about this, Jeremy’s appearance on this episode of the Gradient Dissent podcast, seems to have answered the question. This is discussed around the 32:00 minute mark. Paraphrasing, it seems like working with LLMs might come in a future part of the course as it’s different than image generation models. Although it was back in January, I assume this to still be relevant today. However, it does appear that Stabiliy AI has recently released its own LLM which perhaps may be a bit of foreshadowing regarding what students might get to tinker with soon. Cheers.
I made a notebook with a nice widget-thing to extract transcriptions from Jeremy’s videos and make them available for gpt prompt engineering - it’s called ‘Lazy Student’. There’s also an (ai-)auto-chapter feature and everything is yaml, oh yeah, it’s bilingual too.
Have a look. I have an animated gif on the github page.
In the following example of birds vs forest prediction, why does it always return ‘bird’, even if the probability is low? thank you in advance for your help.
This is the most common confusion from the first lesson. To make it easier to digest on a a first reading, is hardcoded to always output the probability of a bird (i.e. probs[0] is always first label) even when the prediction is not a bird. You need to make use of that ignored result…
Posting this here in case it’s useful for someone as beginner as I am (and also because I’m quite happy my Docker image got leaner).
I’ve been struggling deploying a Docker image that is not too large for a web app’ that provides a predict endpoint for inference using a fastai exported model; I’ve solved this by :
Hi all. I know that LLMs aren’t really part of the fast.ai curriculum but as I wrestle with hype vs facts, I can’t help but think about the ethics surrounding the use of a few, mostly proprietary models, with the approach taught here of building our own models, fine-tuned to each domain and unique data set. After reading Jeremy’s recent blog post on AI Safety and the Age of Dislightenment, I struggle with how much of my time to invest in leveraging the ecosystem and evolving stack around LLMs or whether to invest more time in the fast.ai courses. I know they’re different approaches and useful for different scenarios, but I worry about the black boxes that are LLMs. Further, it’s concerning to me that data security and privacy concerns aren’t more prevalent in the discussions of adopting usage of these models. I guess in other words, a lot of what I’ve learned from this community, and Jeremy and Rachel in particular, seem to fly in the face of the hype machine around LLMs. I’m just wondering if others are wrestling with the same conflicts and how they are navigating these waters. TIA!
