This is the third installment in the 'How Its Made' series of articles - see Article 1
and Article 2
. Using the information held within these pages it is possible to construct your own fantasy price prediction framework. Of course you will need to understand the techniques used, implement your own coding, configure the numerous components, buy a domain and set up a web site. This update addresses a number of common questions recently sent into the web-site:
- What statistical tools are used to predict the price changes
- What is 'pattern matching'
- Why do the target prediction values jump for some players, in particular at the end of a game week.
There are currently four price change predictions engines in use. Each engine is optimised across differing player enviromental ranges, e.g. normal operation, heavy wild card usage (current and historic), game week change, low ownership, etc, etc. A brief outline of each engine is given below:
- This engine is similar to the historic method of predicting price changes, i.e. the approach in use prior to the changes implemented by the main site in August 2014. See this FISO link for details. Following the changes it was updated to include the effects of non effective transfers. This engine's output provides a 'deviation centre' and an associated 'confidence interval' for each player's Target%. These are useful in determining the extent to which existing and any new price changes rules together with the impact of non effective transfers pull the price change thresholds away from the norm. Large deviations are an indication of: i) a significant change to the players environmental parameters, e.g. wild cards; ii) a potential error resulting from an incorrect engine prediction; or iii) if common across a number of players a change to the price change rules - the set of impacted players gives an insight into the likely rule change.
This engine utilises 'pattern matching' to predict the effects of non-effective transfers. It makes use of the adjustment formulae described in this link and the behaviour patterns of the observed transfers according to different scenarios when using that formulae.
Adjustment Factor =
|Effective Transfers + Non-Effective Transfers
As an example, if you consider the situation when the effective transfers are equal in magnitude to the non effective transfers, but opposite in sign. In this scenario the adjustment factor can become infinite, i.e. the true effective volume of transfers is much greater than the apparent transfer rate. If we now consider what happens to a players transfers in this scenario if the magnitude of non effective transfers is varied slightly then the apparent transfer rate for that player switches in sign. The 'pattern matching engine' looks for player exhibiting this transfer behaviour. When such a player is identified it is flagged as a player with a potentially high transfer rate. The danger with this approach, of course, is that the players transfer rate may be simply just changing in sign If however, this behaviour is exhibited consistently over an extended period then the likelihood is enhanced. Many other patterns exist and the engine tracks each player for such behaviours. The results are combined to produce a matrix of potential transfer probabilities for each player.
The 'pattern matching engine' was the first engine used to successfully determine the transfer probabilities following the changes to the main site in 2014.
- This engine uses correlation search tools, i.e. it attempts to find correlations between any factor that influences the price changes. The functions are variables either from within the website database (e.g. statistics, wild card usage, teams with either a blank or double game week, etc., etc.) and also from variables external to the web site (for example volume of incidences of sold next to a players name on a forum) . All correlations are given a weight, conditions under which they will apply and finally a life expectancy. The resultant correlation predictions however are typically non linear in nature and require re calibration during the course of the season.
- This engine is only used on the day of game week switch over. It splits the transfers before and after the game week end in to two distinct population blocks for further analysis. Each population block has different correlation dependencies and pattern characteristics. Consequently each population is used in conjunction with the other three engines outlined above. Its predictions are given higher weight at the start of a new game week
All of these engines compete to show a final prediction value. The actual value chosen is determined using 'reinforcement machine learning' techniques somewhat similar to the schematic below.
Designs for a further engine have been in place for over a year. However, I've not had sufficient time to commence development.
Prediction Target % Jumps
The above approach can result in jumps of the target percentage value over time due to:
- A player switching between different prediction engines, for example due to a significant change in a players status;
- A reversal in the value of the net transfers. This may impact the choice of engine. But it will certainly impact the confidence interval and its position within the interval;
- Time delays before some individual correlations can be applied, for example although the impact of net transfers can be applied almost immediately the determination of the effectiveness of each transfer can lag by up to a couple of hours. Each element change type has a different application phase and update frequency. The update frequency can range from once per day to every thirty minutes; and
- Additional Target% jump occur at the commencement of a new game week. These are a direct consequence of the baselines for each player being readjusted to reflect 'discarded transfers' at a game week end. These adjustments can be significant as much as 83% of a players net transfers.
Price Change Prediction Accuracy
The accuracy of the prediction engines are each sensitive to the volume of non-effective transfers. Whenever the total volume of wild card transfers for a player exceeds the normal, (i.e. effective) transfers for that player, then accurate predictions for that player become almost impossible. In effect, accuracy reduces and overly enthusiastic predictions result for that player. The loss in accuracy is a direct consequence of the statistical mathematics utilised in making the price predictions. These inaccuracies are removed whenever a players transfers are reset, e.g. following a price change for that player, however if more wild card transfers are made then the inaccuracies quickly build up again. The inaccuracies are reduced, but not completely eliminated, following a game week change.
Other factors that adversely impact accuracy include: i) low overall transfer volumes (e.g. during international breaks; ii) predictions made on the day that includes the late start of a new game week; iii) periods of exceedingly low wild card usage; and iv) players with a ownership of less than 10k - this is due to data restrictions introduced during the 16/17 season. Periods of exceedingly low wild card usage should not reduce accuracy and highlights a potential algorithmic error in one or more of the engines. The combined cumulative effects of the previous inaccuracies make predictions on the final but one game week (i.e 37) especially challenging.
A key stage within the reinforcement learning module is to identify the internal accuracy of the individual prediction values for all players within the system.
Internal accuracy should not be confused with how well the system is ultimately performing at predicting price changes. Rather this is an estimation of how confident the system is internally with each individual prediction.
All the players are ranked and categorised according to their internal accuracy.
Each engine utilises this value as a further input parameter and where possible attempts to improve it, e.g. by additional iterations using alternate correlations.
Something exciting is currently planned. However, it could possibly take three months (or more) to develop and a further twelve months to test. But the good news is that the designs are in place and complete. If it works ......... then playing fantasy will truely be an armchair expereince. I just need the time to do it!!!!!!