1Masalah: Fine-Tuning itu Boros Banget
Fine-tuning large pre-trained models is an effective transfer mechanism in NLP. However, in the presence of many downstream tasks, fine-tuning is parameter inefficient: an entire new model is required for every task.
Dulu, kalau kita punya model AI kayak BERT dan mau dipake buat 10 tugas beda (misal: analisis sentimen, klasifikasi spam, dll), kita harus bikin 10 copy model itu dan dilatih ulang semuanya. Bayangin satu model ukurannya bergiga-giga, kalau ada 100 tugas, storage kita bisa jebol!
Dulu, kalau kita punya model AI kayak BERT dan mau dipake buat 10 tugas beda (misal: analisis sentimen, klasifikasi spam, dll), kita harus bikin 10 copy model itu dan dilatih ulang semuanya. Bayangin satu model ukurannya bergiga-giga, kalau ada 100 tugas, storage kita bisa jebol!
Fine-tuning large pre-trained models is an effective transfer mechanism in NLP. However, in the presence of many downstream tasks, fine-tuning is parameter inefficient: an entire new model is required for every task.
Kayak tiap kali lo mau masak menu baru, lo harus beli satu set dapur baru lengkap sama kompor dan kulkasnya. Padahal kan sebenernya tinggal ganti resep atau bumbunya doang.