Football is becoming more predictable; network analysis of 88 thousand matches in 11 major leagues

4.2.1. Network model

To overcome the above-mentioned limitation and come up with a scoring system that is less sensitive to the set of teams that each team has played against, we build a directed network of all the matches within the training window, in which the edges point from the loser to the winner, weighted by the number of points the winner earned. In the next step, we calculate the network eigenvector centrality score for all the teams. The recursive definition of eigenvector centrality, that is that the score of each node depends on the score of its neighbours that send a link to it, perfectly solves the problem of the dyadic scoring system mentioned above [17,32,33]. An example of such network and calculated scores are presented in figure 3.

• Download figure
• Open in new tab
• Download PowerPoint

We can calculate the score difference between the two competing teams for any match after the Nth match. We will have (T − N) matches with their respective outcomes and score differences, for both models. We then fit a logistic regression model of the outcome (as a categorical y variable) over the score differences according to equation (4.1). Without loss of generality, we always calculate the point score difference as x = home team score − away team score and assume y = 1/0 if the home/away team wins.

where μ and s are model parameters than can be obtained by ordinary least-squares methods.

Finally, the logistic regression model will provide the probabilistic assessment of each score system (dyadic and network) for each match, allowing us to understand how correctly the outcomes are being split as a function of the pre-match score difference. See figure 4 for an example.

• Download figure
• Open in new tab
• Download PowerPoint

4.2.2. Quantifying home advantage

In figure 4, we see that the fitted sigmoid is not symmetric and a shift in the probable outcome (i.e. when Prob(y = 1) ≥ 0.50) happens at an x-value smaller than zero, meaning even for some negative values of x, which represents cases where the visiting team’s score is greater than the home team’s score, we still observe a higher chance for a home team victory. This is not surprising considering the existence of the notion of home-field advantage: a significant and structural difference in outcome contingent on the locality in which a sports match is played—favouring the team established in such locality, has been exhaustively uncovered between different team sports and between different countries both for men’s and women’s competitions [34]. Home-field advantage was first formalized for North-American sports [35] and then documented specifically for association football [36].

4.2.3. Benchmark model

We benchmark our models against the implicit prediction model provided by the betting market pay-offs (Bet 365). When betting houses allow players to bet on an event outcome, they are essentially providing an implicit probabilistic assessment of that event through the pay-off values. We calculate the probabilities of different outcomes as the following:

where h is monetary units for a home team win, a units for an away team win. The experienced gambler knows that in the real world, on any given betting house’s game, the sum of the outcomes’ probabilities never does equal to 1, as the game proposed by the betting houses is not strictly fair. However, we believe this set of probabilities give us a good estimate of the state-of-the-art predictive models in the market. In figure 4, the data and corresponding fits to all the three models are provided.

4.2.4. The Elo ranking system

The problem of ranking teams/players within a tournament based not only on their victories and defeats but also on how hard those matches were expected to be beforehand is rather explored throughout the literature. Arpad Elo [37] proposed what became the first ranking system to explicitly consider such caveat, as follows: new players/teams start with a numeric rating (R_i) of 1500. Prior to each match we calculate the expected result for each team (E_i), which is a quasi-normal function of the difference between the teams’ current rankings (R_i and R_j)

After the match, we update the rankings given the difference between the expected outcome and the actual onewhere S_i is the binary outcome of the match between teams/players i and j, and E_i is the expected outcome for player/team i, as determined by equation (4.3). Note a K-factor is proposed, which serves as a weight on how much the result will impact the new rating for the teams/players. Elo himself proposed two values of K when ranking chess players, depending on the player’s classification (namely 16 for a chess master and 32 for lower-ranked players). Despite the rating itself being composed of a prediction, we applied the same dynamics under which we analysed the prior models: we calculated the rating for both teams prior to each match and used the difference between ratings (home minus away) to fit a logistic curve (with 1 for a home win, 0 for an away win).

4.3.1. Loss function: the Brier score

Proposed in [38], the Brier score is a loss function that measures the accuracy of probabilistic predictions. It applies to tasks in which predictions must assign probabilities to a set of mutually exclusive discrete outcomes. The set of possible outcomes can be either binary or categorical in nature, and the probabilities assigned to this set of outcomes must sum to one (where each individual probability is in the range of 0 to 1).

Suppose that on each of t occasions an event can occur in only one of r possible classes or categories and on one such occasion, i, the forecast probabilities are f_i1, f_i2, …, f_ir, that the event will occur in classes 1, 2, …, r, respectively. The r classes are chosen to be mutually exclusive and exhaustive so that $\sum _{\hspace{0.17em}j=1}^r{f}_{ij}=1,i=1,2,3,\dots ,t$. Hence, the Brier score is defined as

where E_ij takes the value 1 or 0 according to whether the event occurred in class j or not. In our case, we have: r = 2 classes, i.e. either the home team won or not; and (1 − n)T matches to be predicted, transforming equation (4.5) intoWe see, from equation (4.6), how the Brier score is an averaged measure over all the predicted matches, i.e. each league–season combination will have its own single Brier score. The lower the Brier score, the better (as in the more assertive and/or correct) our prediction model. In electronic supplementary material, figure S7, the distributions of Brier scores for different models are shown.

4.3.2. Accuracy: receiver operating characteristic curve

A receiver operating characteristic curve, or ROC curve, is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied, which allows us to generally assess a prediction method’s performance for different thresholds. When using normalized units, the area under the ROC curve (often referred to as simply the AUC) is equal to the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one [39].

The machine learning community has historically used this measure for model comparison [40], though the practice has been questioned given AUC estimates are quite noisy and suffer from other problems [41–43]. Nonetheless, the coherence of AUC as a measure of aggregated classification performance has been vindicated, in terms of a uniform rate distribution [44], and AUC has been linked to several other performance metrics, including the above-mentioned Brier score [45]. In electronic supplementary material, figure S8, the distributions of Brier scores for different models are shown.

To compare our two models with the benchmark model, we calculate a loss function in the form of the Brier score for all the (1 − n)T matches in different leagues and years from 2005 to 2018 and n = 0.5. The distributions of scores are reported in electronic supplementary material, figure S1 and the average scores are shown in figure 5a (note that the smaller the Brier score, the better fit of the model). Progressing on the loss function to actual prediction accuracy, for each match, we considered a home team win prediction whether Prob(home win) ≥ π and an away team win prediction otherwise. We applied this prediction rule for all three models (dyadic, network and the betting houses market benchmark) and compared the predictions with the outcomes, calculating how many predictions matched the outcome. Note that the betting market prediction data are only available for more recent years (13 years), therefore our comparison is limited to those years. We change the threshold of π and calculate the average AUC score of ROC curve for each league, shown in figure 5b. The complete set of distributions for the accuracy indicators is depicted in electronic supplementary material, figures S1 and S2. By increasing n, naturally the accuracy of predictions increases. For the rest of the paper, however, we focus on n = 0.5. The comparisons for different values of n are presented in electronic supplementary material, figures S9–S12.

• Download figure
• Open in new tab
• Download PowerPoint

Even though our model underperforms the market on average, t-tests for the difference between the Brier score and AUC distributions of our models and the betting houses market are statistically indistinguishable at the 2% significance level for the majority of year-leagues (see electronic supplementary material, tables S1 and S2 for details). One should note that the goal here is not to develop a better prediction model than the one provided by the betting houses market but to obtain a comparable, consistent and simple-to-calculate model. Market models evolve throughout time, which would bias the assessment of a single prediction tool throughout the years. In that capacity, we are looking for models that provide the time-robust tool we need to assess whether football is becoming more predictable over the years.

To compare the network model with the Elo-based model, we calculated the AUC for the Elo-based model for each country–season combination and plotted it against the network model. The results are surprisingly similar, but the network model outperforms an Elo-based model (with a K-factor of 32), as we can see from electronic supplementary material, figure S13. Specifically, the network model scores a higher AUC value in roughly 61% of cases (170 of 280). Nevertheless, the results for the trends in predictability hold for the Elo-based model too. However, as said above, what matters the most is the ease in training of the model and the consistency of its predictions rather than slight advantages in performance.