problem with Quantization convertion. "SOLVED"
for anyone who has problem too!
i haven't loaded them, but i was having problem converting them to q8 or anything via "llama.cpp". it would gave me error:
"INFO:hf-to-gguf:Loading model: gemma-3-27b-it ERROR:hf-to-gguf:Model Gemma3ForConditionalGeneration is not supported"
i updated them around 15 hours ago. but i fined i have to do this too:
pip install git+https://github.com/huggingface/[email protected]
after that, i just updated everything else too, including llama.cpp .
pip install --upgrade huggingface-hub
pip install --upgrade datasets huggingface-hub
pip install numpy pandas
pip install --upgrade datasets transformers huggingface-hub
python -m venv venv
venv\Scripts\activate
then it start the conversion (Quantization !). like this, for q8_0:
python convert_hf_to_gguf.py "D:\AI\gemma-3-27b-it" --outfile "C:\ai\llama.cpp\new_model\new.gguf" --outtype q8_0
hope it was helpful.
Thanks for the confirmation @josef2600 .
@Renu11
, thank you.
also, i now confirm that it worked! right now i am working with it to code with Arduino. i am using converted to 8bit (q8) and if i do put the correct and clear instructions, it does a good job at coding, at least so far for testing!although it does hallucinate a bit, for my specific codes, i think because its database is older than 1 month for the codes! since it wasnt properly converted into Arduino but it was in espressif library's.
also, a big thanks to google and everybody who was and is involved in this project,
and also who ever else is helping everyone for free.