R Notebook

XGBoost Model evaluation

Forward Model

league_spi_pred%>%
  pluck("fit",1)%>%
  extract_fit_parsnip()%>%
  vip::vip(geom = "col", 
           num_features = 20,
           aesthetics = list(alpha = 0.8)) +
  scale_y_continuous(expand = c(0, 0))+
  ggtitle("XGB Variable Importance - FW")+
  theme_minimal()

# Top 10 
league_spi_pred%>%
  pluck("fit",1)%>%
  extract_fit_parsnip()%>%
  vip::vip(geom = "col", 
           num_features = 10,
           aesthetics = list(fill = brewer.pal(10,"RdBu"), alpha = 0.8)) +
  scale_y_continuous(expand = c(0, 0))+
  ggtitle("Top 10 FW VIP")+
  theme_minimal()

Midfield Model

league_spi_pred%>%
  pluck("fit",2)%>%
  extract_fit_parsnip()%>%
  vip::vip(geom = "col", 
           num_features = 20,
           aesthetics = list(alpha = 0.8)) +
  scale_y_continuous(expand = c(0, 0))+
  ggtitle("XGB Variable Importance - MF")+
  theme_minimal()

# Top 10 
league_spi_pred%>%
  pluck("fit",2)%>%
  extract_fit_parsnip()%>%
  vip::vip(geom = "col", 
           num_features = 10,
           aesthetics = list(fill = brewer.pal(10,"RdBu"), alpha = 0.8)) +
  scale_y_continuous(expand = c(0, 0))+
  ggtitle("Top 10 MF VIP")+
  theme_minimal()

Midfield Model

league_spi_pred%>%
  pluck("fit",3)%>%
  extract_fit_parsnip()%>%
  vip::vip(geom = "col", 
           num_features = 20,
           aesthetics = list(alpha = 0.8)) +
  scale_y_continuous(expand = c(0, 0))+
  ggtitle("XGB Variable Importance - DF")+
  theme_minimal()

# Top 10 
league_spi_pred%>%
  pluck("fit",3)%>%
  extract_fit_parsnip()%>%
  vip::vip(geom = "col", 
           num_features = 10,
           aesthetics = list(fill = brewer.pal(10,"RdBu"), alpha = 0.8)) +
  scale_y_continuous(expand = c(0, 0))+
  ggtitle("Top 10 DF VIP")+
  theme_minimal()

For Goalkeeper Model

Visualising lambda coefficients

Log lambda curve can be seen below:

gk_fit<-league_spi_pred%>%
  pluck("fit",4)%>%
  extract_fit_parsnip()%>%
  pluck("fit")

gk_fit%>%
  plot(label = T, xvar = "lambda")

Plot paired with explanation by print shows, from left to right, how the paths of the coefficient changes against the \(\ell_1\)-norm, as \(\lambda\) increases.

The number of non-zero coefficients is the effective degrees of freedom.

1. The number of nonzero coefficients (Df)
2. The percent (of null) deviance explained (%dev)

gk_fit%>%
  print()

Call:  glmnet::glmnet(x = maybe_matrix(x), y = y, family = "gaussian",      alpha = ~1) 

   Df  %Dev Lambda
1   0  0.00 5.2730
2   1  3.57 4.8040
3   2  6.61 4.3780
4   3 10.32 3.9890
5   3 13.99 3.6340
6   3 17.04 3.3110
7   3 19.56 3.0170
8   3 21.66 2.7490
9   3 23.41 2.5050
10  3 24.85 2.2820
11  3 26.06 2.0800
12  3 27.05 1.8950
13  3 27.88 1.7270
14  3 28.57 1.5730
15  3 29.14 1.4330
16  3 29.61 1.3060
17  3 30.01 1.1900
18  4 30.36 1.0840
19  4 30.87 0.9880
20  4 31.30 0.9003
21  4 31.66 0.8203
22  4 31.95 0.7474
23  4 32.20 0.6810
24  4 32.40 0.6205
25  4 32.57 0.5654
26  4 32.71 0.5152
27  4 32.82 0.4694
28  4 32.92 0.4277
29  4 33.00 0.3897
30  5 33.08 0.3551
31  5 33.15 0.3235
32  6 33.22 0.2948
33  7 33.33 0.2686
34  7 33.62 0.2447
35  7 33.86 0.2230
36  7 34.05 0.2032
37  7 34.22 0.1851
38  7 34.36 0.1687
39  7 34.47 0.1537
40  7 34.56 0.1401
41  7 34.64 0.1276
42  7 34.71 0.1163
43  7 34.76 0.1059
44  7 34.81 0.0965
45  7 34.85 0.0880
46  7 34.88 0.0801
47  8 34.91 0.0730
48  8 34.94 0.0665
49  8 34.96 0.0606
50  8 34.98 0.0552
51  8 35.00 0.0503
52  8 35.01 0.0459
53  8 35.02 0.0418
54  8 35.03 0.0381
55  8 35.04 0.0347
56  8 35.05 0.0316
57  8 35.05 0.0288
58  8 35.06 0.0262
59  8 35.06 0.0239
60  8 35.07 0.0218
61  8 35.07 0.0198
62  8 35.07 0.0181
63  8 35.07 0.0165
64  8 35.07 0.0150
65  8 35.07 0.0137
66  8 35.08 0.0125
67  8 35.08 0.0114
68  8 35.08 0.0104
69  8 35.08 0.0094
70  8 35.08 0.0086
71  8 35.08 0.0078
72  8 35.08 0.0071
73  8 35.08 0.0065

gk_fit%>%
  plot(label = T, xvar = "dev")

This is percent deviance explained on the training data, and is a measure of complexity of the model. We see that toward the end of the path, %deviance is not changing much but the coefficients are “blowing up” a bit. This enables us focus attention on the parts of the fit that matter.

Obtaining the model coefficients

league_spi_pred%>%
  pluck("fit",4)%>%
    tidy()%>%
    rename(lambda=penalty)