Models/Decision Tree Models/Gradient Boosting Models
XGBRegressor (XGBoost sklearn API)
What is XGBRegressor?
XGBRegressor is the scikit-learn compatible regressor from the XGBoost library. It provides the same power and speed as XGBoost, but with a familiar sklearn interface.
Common uses:
- Large tabular datasets
- When you want to use XGBoost with sklearn pipelines
Why Use XGBRegressor?
- All the benefits of XGBoost
- Seamless integration with sklearn
- Works for both regression and classification
Key Parameters
| Parameter | Purpose |
|---|---|
n_estimators | Number of boosting rounds |
learning_rate | Step size shrinkage |
max_depth | Maximum depth of a tree |
subsample | Fraction of samples used per tree |
colsample_bytree | Fraction of features used per tree |
Step-by-Step Example: XGBRegressor for Regression
from xgboost import XGBRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
import numpy as np
# Example data
y = np.array([1, 2, 3, 4, 5])
X = np.arange(5).reshape(-1, 1)
# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
# Create and fit XGBRegressor
xgb_reg = XGBRegressor(n_estimators=100, learning_rate=0.1, max_depth=3, random_state=0)
xgb_reg.fit(X_train, y_train)
# Predict and evaluate
preds = xgb_reg.predict(X_test)
mse = mean_squared_error(y_test, preds)
print('Predictions:', preds)
print('MSE:', mse)Explanation:
XGBRegressor: XGBoost's sklearn-compatible regressor.fit(X_train, y_train): Trains the model.predict(X_test): Makes predictions.
When to Use XGBRegressor
- When you want XGBoost power with sklearn compatibility
- When using sklearn pipelines