Fine tuning GPT-OSS 20b open source model from OpenAI #openai #gpt #finetuning

Details

Description

In this video, let's look into fine-tuning the open-source 20b parameters GPT-OSS model. We will use the HugginFace ecosystem for this. It includes the transformers, peft, dataset, and the related libraries.

The fine-tuning process is mostly following a boilerplate code other than that we need to be wary of what datasets we use, what hardware resources we need, and at times, the parameters we need to set before starting the training.

Hope it's useful!

Github Notebook: https://github.com/ai-bites/generative-ai-course (look for gpt-oss-ft.ipynb)
Fine-tuned model - https://huggingface.co/ai-bites/gpt-oss-20b-multilingual-reasoner
GPT-OSS - https://openai.com/index/introducing-gpt-oss
HF dataset - https://huggingface.co/datasets/HuggingFaceH4/Multilingual-Thinking
OpenAI's cookbook - https://cookbook.openai.com/articles/gpt-oss/fine-tune-transfomers

AI BITES KEY LINKS
Website: https://www.ai-bites.net
YouTube: https://www.youtube.com/@AIBites
Twitter: https://twitter.com/ai_bites​
Patreon: https://www.patreon.com/ai_bites​
Github: https://github.com/ai-bites​