Giperparametrlarni sozlash va Pipeline - Mashinali o'qitish asoslari

Har bir modelda sozlanadigan "murvatlar" bor. Ularni qo'lda tanlash - vaqtni behuda sarflash. Buni avtomatlashtiramiz.

Parametr va giperparametr #

	Parametr	Giperparametr
Kim belgilaydi	Model o'zi o'qish paytida	Siz, o'qishdan oldin
Misol	`coef_`, `intercept_`	`max_depth`, `C`, `n_estimators`
Qachon	`fit()` paytida	`fit()` dan oldin

Python

model = RandomForestClassifier(
    n_estimators=100,      # giperparametr
    max_depth=5,           # giperparametr
)
model.fit(X, y)
print(model.feature_importances_)   # parametr - o'qish natijasi

GridSearchCV #

Barcha kombinatsiyalarni sinab ko'radi.

Python

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.ensemble import RandomForestClassifier

X, y = load_breast_cancer(return_X_y=True)
X_o, X_s, y_o, y_s = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

tor = {
    "n_estimators": [50, 100, 200],
    "max_depth": [3, 5, 10, None],
    "min_samples_leaf": [1, 2, 4],
}

qidiruv = GridSearchCV(
    RandomForestClassifier(random_state=42),
    tor,
    cv=5,
    scoring="f1",
    n_jobs=-1,
    verbose=1,
)
qidiruv.fit(X_o, y_o)

print(f"Eng yaxshi parametrlar: {qidiruv.best_params_}")
print(f"Eng yaxshi CV natija:   {qidiruv.best_score_:.4f}")
print(f"Sinov natijasi:         {qidiruv.score(X_s, y_s):.4f}")

Natija

Fitting 5 folds for each of 36 candidates, totalling 180 fits
Eng yaxshi parametrlar: {'max_depth': None, 'min_samples_leaf': 1, 'n_estimators': 200}
Eng yaxshi CV natija:   0.9724
Sinov natijasi:         0.9722

Kombinatsiyalar soni tez o'sadi

Natija

3 × 4 × 3 = 36 kombinatsiya
36 × 5 fold = 180 marta o'qitish

Yana bitta parametr (5 qiymat) qo'shsangiz - 900 marta. n_estimators=200 bilan har biri 1 soniya bo'lsa - 15 daqiqa.

Natijalarni tahlil qilish #

Python

import pandas as pd

natijalar = pd.DataFrame(qidiruv.cv_results_)

muhim = natijalar[[
    "param_n_estimators", "param_max_depth", "param_min_samples_leaf",
    "mean_test_score", "std_test_score", "mean_fit_time",
]].sort_values("mean_test_score", ascending=False)

print(muhim.head(5).round(4).to_string(index=False))

Natija

 param_n_estimators param_max_depth param_min_samples_leaf  mean_test_score  std_test_score  mean_fit_time
                200            None                      1           0.9724          0.0148         0.3120
                100            None                      1           0.9711          0.0161         0.1580
                200              10                      1           0.9709          0.0155         0.2980
                100              10                      2           0.9698          0.0143         0.1490
                 50            None                      1           0.9686          0.0172         0.0810

Eng yaxshi emas, eng amaliyni tanlang

Birinchi qator 0.9724, beshinchi qator 0.9686 - farq 0.4 foiz. Lekin beshinchisi to'rt barobar tezroq o'qiydi.

Standart og'ish 0.015 - ya'ni bu farq statistik jihatdan ahamiyatsiz.

Ishlab chiqarish uchun tezroq modelni tanlash to'g'riroq qaror.

RandomizedSearchCV #

Barcha kombinatsiyalar o'rniga tasodifiy tanlanganlarini sinaydi.

Python

from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import randint, uniform

taqsimot = {
    "n_estimators": randint(50, 500),
    "max_depth": [3, 5, 10, 20, None],
    "min_samples_leaf": randint(1, 10),
    "max_features": uniform(0.1, 0.9),
}

qidiruv = RandomizedSearchCV(
    RandomForestClassifier(random_state=42),
    taqsimot,
    n_iter=50,              # faqat 50 ta kombinatsiya
    cv=5,
    scoring="f1",
    n_jobs=-1,
    random_state=42,
)
qidiruv.fit(X_o, y_o)

print(f"Eng yaxshi: {qidiruv.best_params_}")
print(f"CV natija:  {qidiruv.best_score_:.4f}")

Odatda parametrlarning faqat bir-ikkitasi haqiqatan muhim

Qaysi birini tanlash?

Vaziyat	Tavsiya
Kam parametr (2-3), kam qiymat	`GridSearchCV`
Ko'p parametr, keng oraliq	`RandomizedSearchCV`
Vaqt cheklangan	`RandomizedSearchCV`
Aniq oraliq ma'lum	`GridSearchCV`

Amaliy usul: avval keng oraliqda RandomizedSearchCV, keyin topilgan hudud atrofida aniq GridSearchCV.

HalvingGridSearchCV #

Yomon nomzodlarni erta tashlab yuboradi:

Python

from sklearn.experimental import enable_halving_search_cv
from sklearn.model_selection import HalvingGridSearchCV

qidiruv = HalvingGridSearchCV(
    RandomForestClassifier(random_state=42),
    tor,
    cv=5,
    factor=3,
    n_jobs=-1,
    random_state=42,
)
qidiruv.fit(X_o, y_o)

Birinchi bosqichda barcha nomzodlar kichik ma'lumotda sinaladi. Eng yaxshi uchdan biri keyingi bosqichga o'tadi va ko'proq ma'lumot oladi.

Pipeline bilan sozlash #

Python

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.svm import SVC

quvur = Pipeline([
    ("masshtab", StandardScaler()),
    ("pca", PCA()),
    ("model", SVC()),
])

tor = {
    "pca__n_components": [5, 10, 15, 20],
    "model__C": [0.1, 1, 10, 100],
    "model__gamma": ["scale", 0.01, 0.1],
}

qidiruv = GridSearchCV(quvur, tor, cv=5, n_jobs=-1)
qidiruv.fit(X_o, y_o)

print(f"Eng yaxshi: {qidiruv.best_params_}")
print(f"Sinov:      {qidiruv.score(X_s, y_s):.3f}")

Ikki pastki chiziq

"pca__n_components" - pca qadamining n_components parametri.

Ichma-ich bo'lsa, davom etadi: "model__estimator__max_depth"

Modelning o'zini ham tanlash #

Python

from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier

quvur = Pipeline([
    ("masshtab", StandardScaler()),
    ("model", SVC()),
])

tor = [
    {
        "model": [SVC()],
        "model__C": [0.1, 1, 10],
        "model__kernel": ["rbf", "linear"],
    },
    {
        "model": [RandomForestClassifier(random_state=42)],
        "model__n_estimators": [100, 200],
        "model__max_depth": [5, 10, None],
    },
    {
        "model": [LogisticRegression(max_iter=1000)],
        "model__C": [0.01, 0.1, 1, 10],
    },
    {
        "model": [KNeighborsClassifier()],
        "model__n_neighbors": [3, 5, 7, 11],
    },
]

qidiruv = GridSearchCV(quvur, tor, cv=5, scoring="f1", n_jobs=-1)
qidiruv.fit(X_o, y_o)

print(f"Eng yaxshi model: {qidiruv.best_params_['model']}")
print(f"CV natija:        {qidiruv.best_score_:.4f}")

Natija

Eng yaxshi model: SVC(C=10)
CV natija:        0.9781

Bu juda kuchli imkoniyat

Bitta buyruq bilan to'rtta turli model va ularning giperparametrlari sinaladi. Barchasi bir xil kross-validatsiya bilan - natijalar solishtirish uchun to'g'ri.

ColumnTransformer #

Turli ustunlarga turli tayyorlash:

Python

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder
from sklearn.impute import SimpleImputer

uylar = pd.DataFrame({
    "maydon":   [65, 90, 120, 45, np.nan],
    "xonalar":  [2, 3, 4, 1, 3],
    "yosh":     [12, 5, 2, 25, 8],
    "tuman":    ["Chilonzor", "Sergeli", None, "Chilonzor", "Yunusobod"],
    "holati":   ["yangi", "o'rta", "eski", "o'rta", "yangi"],
    "narx":     [620, 780, 1150, 380, 810],
})

sonli    = ["maydon", "xonalar", "yosh"]
toifali  = ["tuman"]
tartibli = ["holati"]

sonli_quvur = Pipeline([
    ("toldirish", SimpleImputer(strategy="median")),
    ("masshtab", StandardScaler()),
])

toifali_quvur = Pipeline([
    ("toldirish", SimpleImputer(strategy="most_frequent")),
    ("kodlash", OneHotEncoder(handle_unknown="ignore", drop="first")),
])

tartibli_quvur = Pipeline([
    ("toldirish", SimpleImputer(strategy="most_frequent")),
    ("kodlash", OrdinalEncoder(categories=[["eski", "o'rta", "yangi"]])),
])

tayyorlovchi = ColumnTransformer([
    ("sonli", sonli_quvur, sonli),
    ("toifali", toifali_quvur, toifali),
    ("tartibli", tartibli_quvur, tartibli),
])

X = uylar.drop(columns=["narx"])
y = uylar["narx"]

X_tayyor = tayyorlovchi.fit_transform(X)
print(f"Natija shakli: {X_tayyor.shape}")
print(tayyorlovchi.get_feature_names_out())

Natija

Natija shakli: (5, 6)
['sonli__maydon' 'sonli__xonalar' 'sonli__yosh'
 'toifali__tuman_Sergeli' 'toifali__tuman_Yunusobod' 'tartibli__holati']

Bu tuzilma ma'lumot sizib chiqishining oldini oladi

To'liq ish oqimi #

Python

from sklearn.compose import ColumnTransformer

toliq_quvur = Pipeline([
    ("tayyorlash", tayyorlovchi),
    ("model", RandomForestClassifier(random_state=42)),
])

tor = {
    "tayyorlash__sonli__toldirish__strategy": ["mean", "median"],
    "model__n_estimators": [100, 200],
    "model__max_depth": [5, 10, None],
}

qidiruv = GridSearchCV(toliq_quvur, tor, cv=5, n_jobs=-1)
qidiruv.fit(X_o, y_o)

Hatto tayyorlash strategiyasi ham sozlanadi

"tayyorlash__sonli__toldirish__strategy" - bo'sh qiymatlarni o'rtacha bilan yoki mediana bilan to'ldirish yaxshiroqmi degan savolga model o'zi javob beradi.

O'z transformeringiz #

Python

from sklearn.base import BaseEstimator, TransformerMixin

class NarxBelgilari(BaseEstimator, TransformerMixin):
    """Uy ma'lumotidan yangi belgilar yasaydi."""

    def __init__(self, xona_maydoni_qoshilsin=True):
        self.xona_maydoni_qoshilsin = xona_maydoni_qoshilsin

    def fit(self, X, y=None):
        return self

    def transform(self, X):
        X = X.copy()
        X["yangi_uy"] = (X["yosh"] < 5).astype(int)

        if self.xona_maydoni_qoshilsin:
            X["xona_maydoni"] = X["maydon"] / X["xonalar"].replace(0, 1)

        return X


quvur = Pipeline([
    ("belgilar", NarxBelgilari()),
    ("tayyorlash", tayyorlovchi),
    ("model", RandomForestClassifier(random_state=42)),
])

# Yangi belgi foydalimi - buni ham sinash mumkin
tor = {"belgilar__xona_maydoni_qoshilsin": [True, False]}

BaseEstimator va TransformerMixin

Bu ikki asosiy klassdan meros olsangiz, sizning transformeringiz scikit-learn ekotizimida to'liq ishlaydi: Pipeline, GridSearchCV, cross_val_score - hammasi.

Faqat fit va transform metodlarini yozish yetarli.

Modelni saqlash #

Python

import joblib

# Butun quvurni saqlash
joblib.dump(qidiruv.best_estimator_, "modellar/uy_narxi_v1.joblib")

# Yuklash
model = joblib.load("modellar/uy_narxi_v1.joblib")

yangi_uy = pd.DataFrame({
    "maydon": [85], "xonalar": [3], "yosh": [7],
    "tuman": ["Chilonzor"], "holati": ["yangi"],
})

print(model.predict(yangi_uy))

Versiyalarni ham saqlang

Python

import sklearn, json
from datetime import date

metama = {
    "sana": str(date.today()),
    "sklearn": sklearn.__version__,
    "parametrlar": qidiruv.best_params_,
    "cv_natija": float(qidiruv.best_score_),
    "belgilar": list(X.columns),
}

with open("modellar/uy_narxi_v1.json", "w", encoding="utf-8") as f:
    json.dump(metama, f, ensure_ascii=False, indent=2)

scikit-learn versiyasi mos kelmasa, saqlangan model yuklanmasligi mumkin.

Amaliy maslahatlar #

Sozlash tartibi

Asos model quring - hech qanday sozlashsiz
Eng muhim parametrlardan boshlang:
- Daraxtlar: max_depth, n_estimators
- SVM: C, gamma
- Neyron tarmoq: learning_rate, qatlamlar soni
Avval keng oraliqda tasodifiy qidiruv
Keyin topilgan hudud atrofida aniq grid
Har bir qadamda yaxshilanishni o'lchang

Haddan tashqari sozlashdan ehtiyot bo'ling

Natija

Asos model:      0.9520
Sozlashdan keyin: 0.9538

0.18 foiz yaxshilanish uchun 3 soat sarflash mantiqiymi?

Ko'pincha yangi belgi qo'shish yoki ko'proq ma'lumot yig'ish giperparametr sozlashdan ancha ko'proq foyda beradi.

Validatsiyaga qayta o'qish

Yuzlab kombinatsiyani sinasangiz, ulardan biri tasodifan yaxshi natija ko'rsatadi.

Bu "validatsiya to'plamiga qayta o'qish" deb ataladi. Shuning uchun yakuniy bahoni alohida sinov to'plamida olish shart.

Bayes optimizatsiyasi #

Terminal

pip install optuna

Python

import optuna
from sklearn.model_selection import cross_val_score

def maqsad(sinov):
    parametrlar = {
        "n_estimators": sinov.suggest_int("n_estimators", 50, 500),
        "max_depth": sinov.suggest_int("max_depth", 3, 30),
        "min_samples_leaf": sinov.suggest_int("min_samples_leaf", 1, 10),
        "max_features": sinov.suggest_float("max_features", 0.1, 1.0),
    }

    model = RandomForestClassifier(**parametrlar, random_state=42, n_jobs=-1)
    return cross_val_score(model, X_o, y_o, cv=5, scoring="f1").mean()


tadqiqot = optuna.create_study(direction="maximize")
tadqiqot.optimize(maqsad, n_trials=50, show_progress_bar=True)

print(f"Eng yaxshi: {tadqiqot.best_params}")
print(f"Natija:     {tadqiqot.best_value:.4f}")

Optuna nima uchun aqlliroq?

Grid va Random qidiruv oldingi natijalarni hisobga olmaydi.

Optuna esa har bir urinishdan keyin "qaysi hududda yaxshi natija bor?" degan modelni yangilaydi va keyingi urinishni shu hududda tanlaydi.

Natijada u kamroq urinishda yaxshiroq natijaga yetadi.

Amaliy topshiriq

RandomForestClassifier uchun GridSearchCV yozing.
cv_results_ ni jadvalga aylantirib, eng yaxshi 5 tasini ko'ring.
Natija va o'qitish vaqtini solishtiring - qaysi birini tanlaysiz?
RandomizedSearchCV bilan 50 urinish qiling va natijani solishtiring.
Pipeline quring: masshtablash, PCA, model.
pca__n_components ni ham sozlanadigan qiling.
To'rtta turli modelni bitta GridSearchCV da sinang.
ColumnTransformer bilan aralash ma'lumotni tayyorlang.
O'z transformeringizni yozing (yangi belgi yasovchi).
Yakuniy modelni joblib bilan saqlang va qayta yuklab tekshiring.

Xulosa #

Parametr - model o'zi o'rganadi, giperparametr - siz belgilaysiz.
GridSearchCV barcha kombinatsiyalarni sinaydi - kam parametrda yaxshi.
RandomizedSearchCV tasodifiy tanlaydi - ko'p parametrda samaraliroq.
cv_results_ ni tahlil qiling: eng yaxshi emas, eng amaliyni tanlang.
Pipeline ichida giperparametrlar qadam__parametr ko'rinishida beriladi.
Bitta GridSearchCV da bir nechta modelni solishtirish mumkin.
ColumnTransformer har bir ustun turiga o'z tayyorlash yo'lini beradi.
BaseEstimator + TransformerMixin bilan o'z transformeringizni yozing.
joblib bilan butun quvurni saqlang, versiyalarni ham yozib qo'ying.
Giperparametr sozlash odatda kichik foyda beradi - avval ma'lumot va belgilar ustida ishlang.

Keyingi bo'limda neyron tarmoqlar bilan tanishamiz.

Parametr va giperparametr #

GridSearchCV #

Natijalarni tahlil qilish #

RandomizedSearchCV #

HalvingGridSearchCV #

Pipeline bilan sozlash #

Modelning o'zini ham tanlash #

ColumnTransformer #

To'liq ish oqimi #

O'z transformeringiz #

Modelni saqlash #

Amaliy maslahatlar #

Bayes optimizatsiyasi #

Xulosa #

Xatolik topdingizmi?