Yo'qotish funksiyasi va optimizator

Modelda parametrlar bor, lekin u hali nima o'rganishi kerakligini bilmaydi. Buning uchun ikkita narsa kerak:

• Yo'qotish funksiyasi - «javobing qanchalik noto'g'ri» degan o'lchov.
• Optimizator - o'sha xatoni kamaytirish uchun parametrlarni o'zgartiruvchi qoida.

Yo'qotish: regressiya uchun #

Uzluksiz sonni bashorat qilayotganda (narx, harorat, og'irlik) ikkita asosiy tanlov bor:

import torch
import torch.nn as nn

bashorat = torch.tensor([2.5, 0.0, 2.1, 7.8])
haqiqiy  = torch.tensor([3.0, -0.5, 2.0, 7.0])

mse = nn.MSELoss()
mae = nn.L1Loss()

print("MSE:", round(mse(bashorat, haqiqiy).item(), 4))
print("MAE:", round(mae(bashorat, haqiqiy).item(), 4))

MSE: 0.2875
MAE: 0.475

Ichida nima bo'layotganini ko'rish uchun qo'lda hisoblaymiz:

qol = bashorat - haqiqiy
print("qo'lda MSE:", round((qol ** 2).mean().item(), 4))
print("qo'lda MAE:", round(qol.abs().mean().item(), 4))

qo'lda MSE: 0.2875
qo'lda MAE: 0.475

Hech qanday sehr yo'q: MSE - farqlar kvadratining o'rtachasi, MAE - modullarining o'rtachasi.

Yo'qotish: tasniflash uchun #

Sinfni bashorat qilayotganda CrossEntropyLoss ishlatiladi:

logit = torch.tensor([[2.0, 0.5, -1.0],
                      [0.1, 3.0,  0.2]])
nishon = torch.tensor([0, 1])

ce = nn.CrossEntropyLoss()
print("CrossEntropy:", round(ce(logit, nishon).item(), 4))
print("softmax:", torch.softmax(logit, dim=1).round(decimals=3).tolist())

CrossEntropy: 0.1755
softmax: [[0.786, 0.175, 0.039], [0.049, 0.896, 0.054]]

Birinchi namunada model 0-sinfga 78.6% ishondi, ikkinchisida 1-sinfga 89.6%. Ikkalasi ham to'g'ri, shuning uchun yo'qotish kichik.

Optimizatorning uch qadami #

Optimizator har doim bir xil uchlikda ishlatiladi:

torch.manual_seed(42)

model = nn.Linear(1, 1)
print("boshlang'ich w =", round(model.weight.item(), 4), "| b =", round(model.bias.item(), 4))

opt = torch.optim.SGD(model.parameters(), lr=0.1)
x = torch.tensor([[1.0]])
y = torch.tensor([[2.0]])

for qadam in range(3):
    opt.zero_grad()
    yoqotish = mse(model(x), y)
    yoqotish.backward()
    print(f"  {qadam+1}: yo'qotish={yoqotish.item():.4f}  w.grad={model.weight.grad.item():.4f}  w={model.weight.item():.4f}")
    opt.step()

print("oxirgi w =", round(model.weight.item(), 4))

boshlang'ich w = 0.7645 | b = 0.83
  1: yo'qotish=0.1644  w.grad=-0.8109  w=0.7645
  2: yo'qotish=0.0592  w.grad=-0.4865  w=0.8456
  3: yo'qotish=0.0213  w.grad=-0.2919  w=0.8943
oxirgi w = 0.9235

Yo'qotish 0.16 dan 0.02 gacha tushdi, w esa o'sib bordi. Gradient manfiy edi - demak w ni oshirish kerak, optimizator aynan shuni qildi.

SGD va Adam #

Optimizatorlar gradientdan qanday foydalanishi bilan farq qiladi. Ularni halqa shaklidagi sun'iy masalada solishtiramiz:

X = torch.randn(500, 2)
y = ((X[:, 0] ** 2 + X[:, 1] ** 2) < 1.0).long()
kriteriya = nn.CrossEntropyLoss()

def tarmoq():
    torch.manual_seed(42)
    return nn.Sequential(nn.Linear(2, 16), nn.ReLU(),
                         nn.Linear(16, 16), nn.ReLU(),
                         nn.Linear(16, 2))

for nom, yasa in [("SGD (lr=0.05)", lambda p: torch.optim.SGD(p, lr=0.05)),
                  ("SGD + momentum", lambda p: torch.optim.SGD(p, lr=0.05, momentum=0.9)),
                  ("Adam (lr=0.05)", lambda p: torch.optim.Adam(p, lr=0.05))]:
    model = tarmoq()
    opt = yasa(model.parameters())
    for _ in range(100):
        opt.zero_grad()
        yoqotish = kriteriya(model(X), y)
        yoqotish.backward()
        opt.step()
    aniqlik = (model(X).argmax(dim=1) == y).float().mean().item()
    print(f"{nom:16s} yo'qotish = {yoqotish.item():.4f}   aniqlik = {aniqlik:.1%}")

SGD (lr=0.05)    yo'qotish = 0.5934   aniqlik = 62.6%
SGD + momentum   yo'qotish = 0.0886   aniqlik = 99.0%
Adam (lr=0.05)   yo'qotish = 0.0112   aniqlik = 100.0%

Bir xil model, bir xil urug', bir xil qadamlar soni - lekin natija juda boshqacha. Oddiy SGD 100 qadamda deyarli hech narsa o'rganmadi, Adam esa masalani to'liq yechdi.

O'rganish tezligi #

lr (learning rate) - eng muhim sozlama. U har qadamda qanchalik katta siljish qilinishini belgilaydi:

Xulosa #

• Yo'qotish funksiyasi modelning javobi qanchalik noto'g'riligini bitta songa aylantiradi.
• Regressiya uchun MSELoss, chetdagi qiymatlar ko'p bo'lsa L1Loss.
• MSE xatoni kvadratga ko'taradi - shuning uchun katta xatolarga o'ta sezgir.
• Tasniflash uchun CrossEntropyLoss ishlatiladi.
• U softmax ni o'z ichida bajaradi - modelning oxiriga softmax qo'shmang.
• Optimizator har qadamda uchta amalni bajaradi: zero_grad(), backward(), step().
• Bu tartibni buzsangiz, model eski gradient bilan yangilanadi.
• Adam ko'p qatlamli masalalarda deyarli sozlashsiz ishlaydi - birinchi tanlov shu.
• Oddiy chiziqli masalada SGD tezroq bo'lishi mumkin - «eng yaxshi optimizator» degan narsa yo'q.
• lr eng muhim sozlama: Adam uchun 1e-3 dan boshlang, kerak bo'lsa 10 barobarga o'zgartiring.

Keyingi bo'limda ma'lumotni modelga paket-paket yetkazib beradigan Dataset va DataLoader bilan tanishamiz.