16
The Distill Template
š
Craft Beautiful Blogs
Brigitte Tousignant
BrigitteTousi
AI & ML interests
None yet
Recent Activity
liked
a Space
about 2 hours ago
lvwerra/distill-blog-template
replied to
burtenshaw's
post
1 day ago
everybody and their dog is fine-tuning Gemma 3 today, so I thought I'd do a longer post on the tips and sharp edges I find. let's go!
1. has to be install everything form main and nightly. this is what I'm working with to get unsloth and TRL running
```txt
git+https://github.com/huggingface/transformers@main
git+https://github.com/huggingface/trl.git@main
bitsandbytes
peft
```
plus this with `--no-deps`
```txt
git+https://github.com/unslothai/unsloth-zoo.git@nightly
git+https://github.com/unslothai/unsloth.git@nightly
```
2. will brown's code to turn GSM8k into a reasoning dataset is a nice toy experiment https://gist.github.com/willccbb/4676755236bb08cab5f4e54a0475d6fb
3. with a learning rate of 5e-6 rewards and loss stayed flat for the first 100 or so steps.
4. so far none of my runs have undermined the outputs after 1 epoch. therefore, I'm mainly experimenting with bigger LoRA adapters.
```python
from trl import GRPOConfig
training_args = GRPOConfig(
learning_rate = 5e-6,
adam_beta1 = 0.9,
adam_beta2 = 0.99,
weight_decay = 0.1,
warmup_ratio = 0.1,
lr_scheduler_type = "cosine",
optim = "adamw_8bit",
logging_steps = 1,
per_device_train_batch_size = 2,
gradient_accumulation_steps = 1,
num_generations = 2,
max_prompt_length = 256,
max_completion_length = 1024 - 256,
num_train_epochs = 1,
max_steps = 250,
save_steps = 250,
max_grad_norm = 0.1,
report_to = "none",
)
```
5. vision fine-tuning isn't available in TRL's GRPOTrainer, so stick to text datasets. but no need to load the model differently in transformers or Unsloth
```python
from transformers import AutoModelForImageTextToText
model = AutoModelForImageTextToText.from_pretrained("google/gemma-3-4b-it)
```
if you want an introduction to GRPO, check out the reasoning course, it walks you through the algorithm, theory, and implementation in a smooth way.
https://huggingface.co./reasoning-course
Organizations
BrigitteTousi's activity
liked
a
Space
about 2 hours ago
reacted to
ginipick's
post with š„
1 day ago
Post
2770
š GraphMind: Phi-3 Instruct Graph Explorer
āØ Extract and visualize knowledge graphs from any text in multiple languages!
GraphMind is a powerful tool that leverages the capabilities of Phi-3 to transform unstructured text into structured knowledge graphs, helping you understand complex relationships within any content.
ginigen/Graph-Mind
š Key Features
Multi-language Support š: Process text in English, Korean, and many other languages
Instant Visualization š§©: See extracted entities and relationships in an interactive graph
Entity Recognition š·ļø: Automatically identifies and categorizes named entities
Optimized Performance ā”: Uses caching to deliver faster results for common examples
Intuitive Interface š: Simple design makes complex graph extraction accessible to everyone
š” Use Cases
Content Analysis: Extract key entities and relationships from articles or documents
Research Assistance: Quickly visualize connections between concepts in research papers
Educational Tool: Help students understand the structure of complex texts
Multilingual Processing: Extract knowledge from content in various languages
š§ How It Works
Enter any text in the input field
Select a model from the dropdown
Click "Extract & Visualize"
Explore the interactive knowledge graph and entity recognition results
GraphMind bridges the gap between raw text and structured knowledge, making it easier to identify patterns, extract insights, and understand relationships within any content. Try it now and transform how you interact with textual information!
#NLP #KnowledgeGraph #TextAnalysis #Visualization #Phi3 #MultilingualAI
āØ Extract and visualize knowledge graphs from any text in multiple languages!
GraphMind is a powerful tool that leverages the capabilities of Phi-3 to transform unstructured text into structured knowledge graphs, helping you understand complex relationships within any content.
ginigen/Graph-Mind
š Key Features
Multi-language Support š: Process text in English, Korean, and many other languages
Instant Visualization š§©: See extracted entities and relationships in an interactive graph
Entity Recognition š·ļø: Automatically identifies and categorizes named entities
Optimized Performance ā”: Uses caching to deliver faster results for common examples
Intuitive Interface š: Simple design makes complex graph extraction accessible to everyone
š” Use Cases
Content Analysis: Extract key entities and relationships from articles or documents
Research Assistance: Quickly visualize connections between concepts in research papers
Educational Tool: Help students understand the structure of complex texts
Multilingual Processing: Extract knowledge from content in various languages
š§ How It Works
Enter any text in the input field
Select a model from the dropdown
Click "Extract & Visualize"
Explore the interactive knowledge graph and entity recognition results
GraphMind bridges the gap between raw text and structured knowledge, making it easier to identify patterns, extract insights, and understand relationships within any content. Try it now and transform how you interact with textual information!
#NLP #KnowledgeGraph #TextAnalysis #Visualization #Phi3 #MultilingualAI
- 1 reply
replied to
burtenshaw's
post
1 day ago
brb making a PR to include dog emoji reaction
reacted to
burtenshaw's
post with š„
1 day ago
Post
1124
everybody and their dog is fine-tuning Gemma 3 today, so I thought I'd do a longer post on the tips and sharp edges I find. let's go!
1. has to be install everything form main and nightly. this is what I'm working with to get unsloth and TRL running
plus this with
2. will brown's code to turn GSM8k into a reasoning dataset is a nice toy experiment https://gist.github.com/willccbb/4676755236bb08cab5f4e54a0475d6fb
3. with a learning rate of 5e-6 rewards and loss stayed flat for the first 100 or so steps.
4. so far none of my runs have undermined the outputs after 1 epoch. therefore, I'm mainly experimenting with bigger LoRA adapters.
5. vision fine-tuning isn't available in TRL's GRPOTrainer, so stick to text datasets. but no need to load the model differently in transformers or Unsloth
if you want an introduction to GRPO, check out the reasoning course, it walks you through the algorithm, theory, and implementation in a smooth way.
https://huggingface.co./reasoning-course
1. has to be install everything form main and nightly. this is what I'm working with to get unsloth and TRL running
git+https://github.com/huggingface/transformers@main
git+https://github.com/huggingface/trl.git@main
bitsandbytes
peftplus this with
--no-depsgit+https://github.com/unslothai/unsloth-zoo.git@nightly
git+https://github.com/unslothai/unsloth.git@nightly2. will brown's code to turn GSM8k into a reasoning dataset is a nice toy experiment https://gist.github.com/willccbb/4676755236bb08cab5f4e54a0475d6fb
3. with a learning rate of 5e-6 rewards and loss stayed flat for the first 100 or so steps.
4. so far none of my runs have undermined the outputs after 1 epoch. therefore, I'm mainly experimenting with bigger LoRA adapters.
from trl import GRPOConfig
training_args = GRPOConfig(
learning_rate = 5e-6,
adam_beta1 = 0.9,
adam_beta2 = 0.99,
weight_decay = 0.1,
warmup_ratio = 0.1,
lr_scheduler_type = "cosine",
optim = "adamw_8bit",
logging_steps = 1,
per_device_train_batch_size = 2,
gradient_accumulation_steps = 1,
num_generations = 2,
max_prompt_length = 256,
max_completion_length = 1024 - 256,
num_train_epochs = 1,
max_steps = 250,
save_steps = 250,
max_grad_norm = 0.1,
report_to = "none",
)5. vision fine-tuning isn't available in TRL's GRPOTrainer, so stick to text datasets. but no need to load the model differently in transformers or Unsloth
from transformers import AutoModelForImageTextToText
model = AutoModelForImageTextToText.from_pretrained("google/gemma-3-4b-it)if you want an introduction to GRPO, check out the reasoning course, it walks you through the algorithm, theory, and implementation in a smooth way.
https://huggingface.co./reasoning-course
-
- 2 replies
reacted to
fdaudens's
post with š„
2 days ago
Post
986
Ever wanted 45 min with one of AIās most fascinating minds? Was with
@thomwolf
at HumanX Vegas. Sharing my notes of his Q&A with the pressācompletely changed how I think about AIās future:
1ļøā£ The next wave of successful AI companies wonāt be defined by who has the best model but by who builds the most useful real-world solutions. "We all have engines in our cars, but thatās rarely the only reason we buy one. We expect it to work well, and thatās enough. LLMs will be the same."
2ļøā£ Big players are pivoting: "Closed-source companiesāOpenAI being the firstāhave largely shifted from LLM announcements to product announcements."
3ļøā£ Open source is changing everything: "DeepSeek was open source AIās ChatGPT moment. Basically, everyone outside the bubble realized you can get a model for freeāand itās just as good as the paid ones."
4ļøā£ Product innovation is being democratized: Take Manus, for exampleāthey built a product on top of Anthropicās models thatās "actually better than Anthropicās own product for now, in terms of agents." This proves that anyone can build great products with existing models.
Weāre entering a "multi-LLM world," where models are becoming commoditized, and all the tools to build are readily availableājust look at the flurry of daily new releases on Hugging Face.
Thom's comparison to the internet era is spot-on: "In the beginning you made a lot of money by making websites... but nowadays the huge internet companies are not the companies that built websites. Like Airbnb, Uber, Facebook, they just use the internet as a medium to make something for real life use cases."
Love to hear your thoughts on this shift!
1ļøā£ The next wave of successful AI companies wonāt be defined by who has the best model but by who builds the most useful real-world solutions. "We all have engines in our cars, but thatās rarely the only reason we buy one. We expect it to work well, and thatās enough. LLMs will be the same."
2ļøā£ Big players are pivoting: "Closed-source companiesāOpenAI being the firstāhave largely shifted from LLM announcements to product announcements."
3ļøā£ Open source is changing everything: "DeepSeek was open source AIās ChatGPT moment. Basically, everyone outside the bubble realized you can get a model for freeāand itās just as good as the paid ones."
4ļøā£ Product innovation is being democratized: Take Manus, for exampleāthey built a product on top of Anthropicās models thatās "actually better than Anthropicās own product for now, in terms of agents." This proves that anyone can build great products with existing models.
Weāre entering a "multi-LLM world," where models are becoming commoditized, and all the tools to build are readily availableājust look at the flurry of daily new releases on Hugging Face.
Thom's comparison to the internet era is spot-on: "In the beginning you made a lot of money by making websites... but nowadays the huge internet companies are not the companies that built websites. Like Airbnb, Uber, Facebook, they just use the internet as a medium to make something for real life use cases."
Love to hear your thoughts on this shift!
- 1 reply
reacted to
thomwolf's
post with š„š
2 days ago
Post
2002
We've kept pushing our Open-R1 project, an open initiative to replicate and extend the techniques behind DeepSeek-R1.
And even we were mind-blown by the results we got with this latest model we're releasing: ā”ļøOlympicCoder ( open-r1/OlympicCoder-7B and open-r1/OlympicCoder-32B)
It's beating Claude 3.7 on (competitive) programming āa domain Anthropic has been historically really strong atā and it's getting close to o1-mini/R1 on olympiad level coding with just 7B parameters!
And the best part is that we're open-sourcing all about its training dataset, the new IOI benchmark, and more in our Open-R1 progress report #3: https://huggingface.co./blog/open-r1/update-3
Datasets are are releasing:
- open-r1/codeforces
- open-r1/codeforces-cots
- open-r1/ioi
- open-r1/ioi-test-cases
- open-r1/ioi-sample-solutions
- open-r1/ioi-cots
- open-r1/ioi-2024-model-solutions
And even we were mind-blown by the results we got with this latest model we're releasing: ā”ļøOlympicCoder ( open-r1/OlympicCoder-7B and open-r1/OlympicCoder-32B)
It's beating Claude 3.7 on (competitive) programming āa domain Anthropic has been historically really strong atā and it's getting close to o1-mini/R1 on olympiad level coding with just 7B parameters!
And the best part is that we're open-sourcing all about its training dataset, the new IOI benchmark, and more in our Open-R1 progress report #3: https://huggingface.co./blog/open-r1/update-3
Datasets are are releasing:
- open-r1/codeforces
- open-r1/codeforces-cots
- open-r1/ioi
- open-r1/ioi-test-cases
- open-r1/ioi-sample-solutions
- open-r1/ioi-cots
- open-r1/ioi-2024-model-solutions
reacted to
clefourrier's
post with š
2 days ago
Post
1530
Gemma3 family is out! Reading the tech report, and this section was really interesting to me from a methods/scientific fairness pov.
Instead of doing over-hyped comparisons, they clearly state that **results are reported in a setup which is advantageous to their models**.
(Which everybody does, but people usually don't say)
For a tech report, it makes a lot of sense to report model performance when used optimally!
On leaderboards on the other hand, comparison will be apples to apples, but in a potentially unoptimal way for a given model family (like some user interact sub-optimally with models)
Also contains a cool section (6) on training data memorization rate too! Important to see if your model will output the training data it has seen as such: always an issue for privacy/copyright/... but also very much for evaluation!
Because if your model knows its evals by heart, you're not testing for generalization.
Instead of doing over-hyped comparisons, they clearly state that **results are reported in a setup which is advantageous to their models**.
(Which everybody does, but people usually don't say)
For a tech report, it makes a lot of sense to report model performance when used optimally!
On leaderboards on the other hand, comparison will be apples to apples, but in a potentially unoptimal way for a given model family (like some user interact sub-optimally with models)
Also contains a cool section (6) on training data memorization rate too! Important to see if your model will output the training data it has seen as such: always an issue for privacy/copyright/... but also very much for evaluation!
Because if your model knows its evals by heart, you're not testing for generalization.
liked
a
Space
2 days ago
Nice! Love the layout
reacted to
openfree's
post with š¤ā¤ļøššš„
2 days ago
Post
4079
Huggingface Space Leaderboard š
Hello Huggingface Community!
VIDraft/Space-Leaderboard
We are excited to introduce the Huggingface Space Leaderboard, a service that lets you view the latest trending Spaces on the Huggingface platform at a glance. This service helps you quickly explore a wide range of creative projects and will spark new inspiration for your own ideas. š
Detailed Feature Overview
1. Real-time Trend Reflection
Automated Aggregation: Analyzes and ranks over 500 popular Spaces on Huggingface in real time.
Accurate Ranking: Combines various metrics such as likes, engagement, and creation time to accurately reflect the latest trends.
Instant Updates: Data is continuously updated, so you always see the most current popular Spaces.
2. Intuitive Preview
70% Scaled Preview: Each Space is displayed at 70% scale, providing a neat and clear preview at a glance.
Easy Visual Comparison: View multiple Spaces side by side to easily compare their designs and functionalities.
Error Handling: In case of loading issues, a clear error message with a direct link is provided to help resolve any problems.
3. Creator Statistics
Top 30 Creators Analysis: A chart visualizes the number of Spaces created by the most active creators, giving you a clear view of the communityās top contributors. š
Data-driven Insights: Analyze the activity trends of each creator to gain fresh insights and inspiration.
Collaboration Opportunities: Use the statistics to easily identify potential collaborators within the community.
Why Choose the Huggingface Space Leaderboard?
š Fast and Reliable: Real-time data updates deliver the latest trends instantly, ensuring you gain insights without any delays.
š Easy Search Functionality: Easily find the Space youāre looking for with filters by name, owner, or tags.
š” Intuitive Design: A clean, user-friendly interface makes it simple for anyone to navigate and explore.
Hello Huggingface Community!
VIDraft/Space-Leaderboard
We are excited to introduce the Huggingface Space Leaderboard, a service that lets you view the latest trending Spaces on the Huggingface platform at a glance. This service helps you quickly explore a wide range of creative projects and will spark new inspiration for your own ideas. š
Detailed Feature Overview
1. Real-time Trend Reflection
Automated Aggregation: Analyzes and ranks over 500 popular Spaces on Huggingface in real time.
Accurate Ranking: Combines various metrics such as likes, engagement, and creation time to accurately reflect the latest trends.
Instant Updates: Data is continuously updated, so you always see the most current popular Spaces.
2. Intuitive Preview
70% Scaled Preview: Each Space is displayed at 70% scale, providing a neat and clear preview at a glance.
Easy Visual Comparison: View multiple Spaces side by side to easily compare their designs and functionalities.
Error Handling: In case of loading issues, a clear error message with a direct link is provided to help resolve any problems.
3. Creator Statistics
Top 30 Creators Analysis: A chart visualizes the number of Spaces created by the most active creators, giving you a clear view of the communityās top contributors. š
Data-driven Insights: Analyze the activity trends of each creator to gain fresh insights and inspiration.
Collaboration Opportunities: Use the statistics to easily identify potential collaborators within the community.
Why Choose the Huggingface Space Leaderboard?
š Fast and Reliable: Real-time data updates deliver the latest trends instantly, ensuring you gain insights without any delays.
š Easy Search Functionality: Easily find the Space youāre looking for with filters by name, owner, or tags.
š” Intuitive Design: A clean, user-friendly interface makes it simple for anyone to navigate and explore.
-
- 1 reply
reacted to
jasoncorkill's
post with š
2 days ago
Post
2092
Benchmarking Google's Veo2: How Does It Compare?
The results did not meet expectations. Veo2 struggled with style consistency and temporal coherence, falling behind competitors like Runway, Pika, Tencent, and even Alibaba. While the model shows promise, its alignment and quality are not yet there.
Google recently launched Veo2, its latest text-to-video model, through select partners like fal.ai. As part of our ongoing evaluation of state-of-the-art generative video models, we rigorously benchmarked Veo2 against industry leaders.
We generated a large set of Veo2 videos spending hundreds of dollars in the process and systematically evaluated them using our Python-based API for human and automated labeling.
Check out the ranking here: https://www.rapidata.ai/leaderboard/video-models
Rapidata/text-2-video-human-preferences-veo2
The results did not meet expectations. Veo2 struggled with style consistency and temporal coherence, falling behind competitors like Runway, Pika, Tencent, and even Alibaba. While the model shows promise, its alignment and quality are not yet there.
Google recently launched Veo2, its latest text-to-video model, through select partners like fal.ai. As part of our ongoing evaluation of state-of-the-art generative video models, we rigorously benchmarked Veo2 against industry leaders.
We generated a large set of Veo2 videos spending hundreds of dollars in the process and systematically evaluated them using our Python-based API for human and automated labeling.
Check out the ranking here: https://www.rapidata.ai/leaderboard/video-models
Rapidata/text-2-video-human-preferences-veo2
reacted to
AdinaY's
post with š„
2 days ago
Post
1253
R1-Omniš„RLVR-Powered Multimodal LLM released by Alibaba
Model: StarJiaxing/R1-Omni-0.5B
Paper: R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcing Learning (2503.05379)
āØ0.5B with Apache2.0
āØ Improve emotion recognition with visual and audio cues
Model: StarJiaxing/R1-Omni-0.5B
Paper: R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcing Learning (2503.05379)
āØ0.5B with Apache2.0
āØ Improve emotion recognition with visual and audio cues
- 1 reply
reacted to
AdinaY's
post with š„
2 days ago
Post
1176
Spark TTS šNew OPEN TTS model that can generate any voice with just seconds of audio!
Released by SparkAudio communityš„
Modelš SparkAudio/Spark-TTS-0.5B
Paperš Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens (2503.01710)
āØ Supports English & Chinese
āØ BiCodec Speech Codec: Enables precise voice control by separating semantics & speaker attributes
Released by SparkAudio communityš„
Modelš SparkAudio/Spark-TTS-0.5B
Paperš Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens (2503.01710)
āØ Supports English & Chinese
āØ BiCodec Speech Codec: Enables precise voice control by separating semantics & speaker attributes
reacted to
eliebak's
post with š„
2 days ago
Post
1344
Google just dropped an exciting technical report for the brand-new Gemma3 model! š Here are my personal notes highlighting the most intriguing architectural innovations, design choices, and insights from this release:
1) Architecture choices:
> No more softcaping, replace by QK-Norm
> Both Pre AND Post Norm
> Wider MLP than Qwen2.5, ~ same depth
> SWA with 5:1 and 1024 (very small and cool ablation on the paper!)
> No MLA to save KV cache, SWA do the job!
2) Long context
> Only increase the rope in the global layer (to 1M)
> Confirmation that it's harder to do long context for smol models, no 128k for the 1B
> Pretrained with 32k context? seems very high
> No yarn nor llama3 like rope extension
3) Distillation
> Only keep te first 256 logits for the teacher
> Ablation on the teacher gap (tl;dr you need some "patience" to see that using a small teacher is better)
> On policy distillation yeahh (by
@agarwl_
et al), not sure if the teacher gap behave the same here, curious if someone have more info?
4) Others
> Checkpoint with QAT, that's very cool
> RL using improve version of BOND, WARM/WARP good excuse to look at
@ramealexandre
papers
> Only use Zero3, no TP/PP if i understand correctly ?
> Training budget relatively similar than gemma2
1) Architecture choices:
> No more softcaping, replace by QK-Norm
> Both Pre AND Post Norm
> Wider MLP than Qwen2.5, ~ same depth
> SWA with 5:1 and 1024 (very small and cool ablation on the paper!)
> No MLA to save KV cache, SWA do the job!
2) Long context
> Only increase the rope in the global layer (to 1M)
> Confirmation that it's harder to do long context for smol models, no 128k for the 1B
> Pretrained with 32k context? seems very high
> No yarn nor llama3 like rope extension
3) Distillation
> Only keep te first 256 logits for the teacher
> Ablation on the teacher gap (tl;dr you need some "patience" to see that using a small teacher is better)
> On policy distillation yeahh (by
@agarwl_
et al), not sure if the teacher gap behave the same here, curious if someone have more info?
4) Others
> Checkpoint with QAT, that's very cool
> RL using improve version of BOND, WARM/WARP good excuse to look at
@ramealexandre
papers
> Only use Zero3, no TP/PP if i understand correctly ?
> Training budget relatively similar than gemma2
- 1 reply
reacted to
freddyaboulton's
post with š
3 days ago
Post
1655
Privacy matters when talking to AI! š
We've just added a microphone mute button to FastRTC in our latest update (v0.0.14). Now you control exactly what your LLM hears.
Plus lots more features in this release! Check them out:
https://github.com/freddyaboulton/fastrtc/releases/tag/0.0.14
We've just added a microphone mute button to FastRTC in our latest update (v0.0.14). Now you control exactly what your LLM hears.
Plus lots more features in this release! Check them out:
https://github.com/freddyaboulton/fastrtc/releases/tag/0.0.14