Warning: define(): Argument #3 ($case_insensitive) is ignored since declaration of case-insensitive constants is no longer supported in /home/u742613510/domains/strategy-at-risk.com/public_html/wp-content/plugins/wpmathpub/wpmathpub.php on line 65
Forecast – Strategy @ Risk

Tag: Forecast

  • The Most Costly Excel Error Ever?

    The Most Costly Excel Error Ever?

    This entry is part 2 of 2 in the series Spreadsheet Errors

     

    Efficient computing tools are essential for statistical research, consulting, and teaching. Generic packages such as Excel are not sufficient even for the teaching of statistics, let alone for research and consulting (ASA, 2000).

    Introduction

    Back early in 2009 we published a post on the risk of spreadsheet errors. The reference above is taken from that post, but it seems even more relevant today as show in the following.

    Growth in a Time of Debt

    In 2010, economists Reinhart and Rogoff released a paper, “Growth in a Time of Debt.” Their “main result was:

    1. Average growth rates for countries with public debt over 90% of GDP are roughly 4% lower than when debt is low (under 30% of GDP).
    2. Median growth rates for countries with public debt over 90% of GDP are roughly 2.6% lower than the when debt is low (under 30% of GDP).
    3.   Countries with debt-to-GDP ratios above 90 percent have a slightly negative average growth rate (-0.1%).

    The paper has been widely cited by political figures around the world, arguing the case for reduced government spending and increased taxes and ultimately against government efforts to boost the economy and create jobs. All based on the papers conclusion that any short-term benefit in job creation and increased growth would come with a high long-term cost.

    Then in 2013, Herndon, Ash and Pollin (Herndon et. al., 2013) replicated the Reinhart and Rogoff study and found that it had:

    1. Coding errors in the spreadsheet programming,
    2. Selective exclusion of available data, and
    3. Unconventional weighting of summary statistics.

    All this led to serious errors that inaccurately estimated the relationship between public debt and GDP growth among 20 advanced economies in the post-war period. Instead they found that when properly calculated:

    That the average real GDP growth rate for countries carrying a public-debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0:1 percent as published in Reinhart and Rogoff.

    That is, contrary to the Reinhart and Rogoff study – average GDP growth at public debt/GDP ratios over 90 percent is not dramatically different than when debt/GDP ratios are lower.

    Statistics and the use of Excel

    Even if the coding error only accounted for a small part of the total error, “everyone” knows that excel is error-prone in a way that any programming language or statistical package is not; it mixes data and code and makes you do things by hand that would be automatically done in the other settings.

    Excel is good for ad-hoc calculations where you’re not really sure what you’re looking for, or for a first quick look at the data, but once you really start analyzing a dataset, you’re better off using almost anything else.

    Basing important decisions on excel models or excel analysis only is very risky – unless it has been thoroughly audited and great effort has been taken to ensure that the calculations are coherent and consistent.

    One thing is certain, serious problems demands serious tools. Maybe it is time to reread the American Statistical Association (ASA) endorsement of “Guidelines for Programs and Departments in Undergraduate Mathematical Sciences”

    References

    Herndon, T., Ash, M. and Pollin, R. (April 15, 2013). Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff, PERI, University of Massachusetts, Amherst. http://www.peri.umass.edu/fileadmin/pdf/working_papers/working_papers_301-350/WP322.pdf

    American Statistical Association (ASA) (2000).  Endorsement of the Mathematical Association of America (MAA): “Guidelines for Programs and Departments in Undergraduate Mathematical Sciences” http://www07.homepage.villanova.edu/michael.posner/sigmaastated/ASAendorsement2.html

    Baker, D. (16 April 2013) How much unemployment did Reinhart and Rogoff’s arithmetic mistake cause? The Guardian. http://www.guardian.co.uk/commentisfree/2013/apr/16/unemployment-reinhart-rogoff-arithmetic-cause

    Reinhart, C.M. & Rogoff, K.S., (2010). Growth in a time of Debt, Working Paper 15639 National Bureau of Economic Research, Cambridge. http://www.nber.org/papers/w15639.pdf

  • Budgeting Revisited

    Budgeting Revisited

    This entry is part 2 of 2 in the series Budgeting

     

    Introduction

    Budgeting is one area that is well suited for Monte Carlo Simulation. Budgeting involves personal judgments about future values of large number of variables like; sales, prices, wages, down- time, error rates, exchange rates etc. – variables that describes the nature of the business.

    Everyone that has been involved in a budgeting process knows that it is an exercise in uncertainty; however it is seldom described in this way and even more seldom is uncertainty actually calculated as an integrated part of the budget.

    Good budgeting practices are structured to minimize errors and inconsistencies, drawing in all the necessary participants to contribute their business experience and the perspective of each department. Best practice in budgeting entails a mixture of top-down guidelines and standards, combined with bottom-up individual knowledge and experience.

    Excel, the de facto tool for budgeting, is a powerful personal productivity tool. Its current capabilities, however, are often inadequate to support the critical nature of budgeting and forecasting. There will come a point when a company’s reliance on spreadsheets for budgeting leads to severely ineffective decision-making, lost productivity and lost opportunities.

    Spreadsheets can accommodate many tasks – but, over time, some of the models running in Excel may grow too big for the spreadsheet application. Programming in a spreadsheet model often requires embedded assumptions, complex macros, creating opportunities for formula errors and broken links between workbooks.

    It is common for spreadsheet budget models and their intricacies to be known and maintained by a single person who becomes a vulnerability point with no backup. And there are other maintenance and usage issues:

    A.    Spreadsheet budget models are difficult to distribute and even more difficult to collect and consolidate.
    B.    Data confidentiality is almost impossible to maintain in spreadsheets, which are not designed to hide or expose data based upon each user’s role.
    C.    Financial statements are usually not fully integrated leaving little basis for decision making.

    These are serious drawbacks for corporate governance and make the audit process more difficult.

    This is a few of many reasons why we use a dedicated simulation language for our models that specifically do not mix data and code.

    The budget model

    In practice budgeting can be performed on different levels:
    1.    Cash Flow
    2.    EBITDA
    3.    EBIT
    4.    Profit or
    5.    Company value.

    The most efficient is on EBITDA level, since taxes, depreciation and amortization on the short-term is mostly given. This is also the level where consolidation of daughter companies easiest is achieved. An EBITDA model describing the firm’s operations can again be used as a subroutine for more detailed and encompassing analysis thru P&L and Balance simulation.

    The aim will then to estimate of the firm’s equity value and is probability distribution. This can again be used for strategy selection etc.

    Forecasting

    In today’s fast moving and highly uncertain markets, forecasting have become the single most important element of the budget process.

    Forecasting or predictive analytics can best be described as statistic modeling enabling prediction of future events or results, using present and past information and data.

    1. Forecasts must integrate both external and internal cost and value drivers of the business.
    2. Absolute forecast accuracy (i.e. small confidence intervals) is less important than the insight about how current decisions and likely future events will interact to form the result.
    3. Detail does not equal accuracy with respect to forecasts.
    4. The forecast is often less important than the assumptions and variables that underpin it – those are the things that should be traced to provide advance warning.
    5.  Never relay on single point or scenario forecasting.

    All uncertainty about the market sizes, market shares, cost and prices, interest rates, exchange rates and taxes etc. – and their correlation will finally end up contributing to the uncertainty in the firm’s budget forecasts.

    The EBITDA model

    The EBITDA model have to be detailed enough to capture all important cost and value drivers, but simple enough to be easy to update with new data and assumptions.

    Input to the model can come from different sources; any internal reporting system or spread sheet. The easiest way to communicate with the model is by using Excel  spread sheet – templates.

    Such templates will be pre-defined in the sense that the information the model needs is on a pre-determined place in the workbook.  This makes it easy if the budgets for daughter companies is reported (and consolidated) in a common system (e.g. SAP) and can ‘dump’ onto an excel spread sheet. If the budgets are communicated directly to head office or the mother company then they can be read directly by the model.

    Standalone models and dedicated subroutines

    We usually construct our EBITDA models so that they can be used both as a standalone model and as a subroutine for balance simulation. The model can then be used both for short term budgeting and long-term EBITDA forecasting and simulation and for short/long term balance forecasting and simulation. This means that the same model can be efficiently reused in different contexts.
    Rolling budgets and forecast

    The EBITDA model can be constructed to give rolling forecast based on updated monthly or quarterly values, taking into consideration the seasonality of the operations. This will give new forecasts (new budget) for the remaining of the year and/or the next twelve month. By forecasts we again mean the probability distributions for the budget variables.

    Even if the variables have not changed, the fact that we move towards the end of the year will reduce the uncertainty of if the end year results and also for the forecast for the next twelve month.

    Uncertainty

    The most important part of budgeting with Monte Carlo simulation is assessment of the uncertainty in the budgeted (forecasted) cost and value drivers. This uncertainty is given as the most likely value (usually the budget figure) and the interval where it is assessed with a high degree of confidence (approx. 95%) to fall.

    We will then use these lower and upper limits (5% and 95%) for sales, prices and other budget items and the budget values as indicators of the shape of the probability distributions for the individual budget items. Together they described the range and uncertainty in the EBITDA forecasts.

    This gives us the opportunity to simulate (Monte Carlo) a number of possible outcomes – by a large number of runs of the model, usually 1000 – of net revenue, operating expenses and finally EBITDA. This again will give us their probability distributions

    Most managers and their staff have, based on experience, a good grasp of the range in which the values of their variables will fall. It is not based on any precise computation but is a reasonable assessment by knowledgeable persons. Selecting the budget value however is more difficult. Should it be the “mean”
    or the “most likely value” or should the manager just delegate fixing of the values to the responsible departments?

    Now we know that the budget values might be biased by a number of reasons – simplest by bonus schemes etc. – and that budgets based on average assumptions are wrong on average .

    This is therefore where the individual mangers intent and culture will be manifested, and it is here the greatest learning effect for both the managers and the mother company will be, as under-budgeting  and overconfidence  will stand out as excessive large deviations from the model calculated expected value (probability weighted average over the interval).

    Output

    The output from the Monte Carlo simulation will be in the form of graphs that puts all run’s in the simulation together to form the cumulative distribution for the operating expenses (red line):

    In the figure we have computed the frequencies of observed (simulated) values for operating expenses (blue frequency plot) – the x-axis gives the operating expenses and the left y-axis the frequency. By summing up from left to right we can compute the cumulative probability curve. The s-shaped curve (red) gives for every point the probability (on the right y-axis) for having an operating expenses less than the corresponding point on the x-axis. The shape of this curve and its range on the x-axis gives us the uncertainty in the forecasts.

    A steep curve indicates little uncertainty and a flat curve indicates greater uncertainty.  The curve is calculated from the uncertainties reported in the reporting package or templates.

    Large uncertainties in the reported variables will contribute to the overall uncertainty in the EBITDA forecast and thus to a flatter curve and contrariwise. If the reported uncertainty in sales and prices has a marked downside and the costs a marked upside the resulting EBITDA distribution might very well have a portion on the negative side on the x-axis – that is, with some probability the EBITDA might end up negative.

    In the figure below the lines give the expected EBITDA and the budget value. The expected EBIT can be found by drawing a horizontal line from the 0.5 (50%) point on the y-axis to the curve and a vertical line from this point on the curve to the x-axis. This point gives us the expected EBITDA value – the point where it is 50% probability of having a value of EBITDA below and 100%-50%=50% of having it above.

    The second set of lines give the budget figure and the probability that it will end up lower than budget. In this case it is almost a 100% probability that it will be much lower than the management have expected.

    This distributions location on the EBITDA axis (x-axis) and its shape gives a large amount of information of what we can expect of possible results and their probability.

    The following figure that gives the EBIT distributions for a number of subsidiaries exemplifies this. One wills most probable never earn money (grey), three is cash cows (blue, green and brown) and the last (red) can earn a lot of money:

    Budget revisions and follow up

    Normally – if something extraordinary does not happen – we would expect both the budget and the actual EBITDA to fall somewhere in the region of the expected value. We have however to expect some deviation both from budget and expected value due to the nature of the industry.  Having in mind the possibility of unanticipated events or events “outside” the subsidiary’s budget responsibilities, but affecting the outcome this implies that:

    • Having the actual result deviating from budget is not necessary a sign of bad budgeting.
    • Having the result close to or on budget is not necessary a sign of good budgeting.

    However:

    •  Large deviations between budget and actual result needs looking into – especially if the deviation to expected value also is large.
    • Large deviation between budget and expected value can imply either that the limits are set “wrong” or that the budget EBITDA is not reflecting the downside risk or upside opportunity expressed by the limits.

    Another way of looking at the distributions is by the probabilities of having the actual result below budget that is how far off line the budget ended up. In the graph below, country #1’s budget came out with a probability of 72% of having the actual result below budget.  It turned out that the actual figure with only 36% probability would have been lower. The length of the bars thus indicates the budget discrepancies.

    For country# 2 it is the other way around: the probability of having had a result lower than the final result is 88% while the budgeted figure had a 63% probability of having been too low. In this case the market was seriously misjudged.

    In the following we have measured the deviation of the actual result both from the budget values and from the expected values. In the figures the left axis give the deviation from expected value and the bottom axis the deviation from budget value.

    1.  If the deviation for a country falls in the upper right quadrant the deviation are positive for both budget and expected value – and the country is overachieving.
    2. If the deviation falls in the lower left quadrant the deviation are negative for both budget and expected value – and the country is underachieving.
    3. If the deviation falls in the upper left quadrant the deviation are negative for budget and positive for expected value – and the country is overachieving but has had a to high budget.

    With a left skewed EBITDA distribution there should not be any observations in the lower right quadrant that will only happen when the distribution is skewed to the right – and then there will not be any observations in the upper left quadrant:

    As the manager’s gets more experienced in assessing the uncertainty they face, we see that the budget figures are more in line with the expected values and that the interval’s given is shorter and better oriented.

    If the budget is in line with expected value given the described uncertainty, the upside potential ratio should be approx. one. A high value should indicate a potential for higher EBITDA and vice versa. Using this measure we can numerically describe the managements budgeting behavior:

    Rolling budgets

    If the model is set up to give rolling forecasts of the budget EBITDA as new and in this case monthly data, we will get successive forecast as in the figure below:

    As data for new month are received, the curve is getting steeper since the uncertainty is reduced. From the squares on the lines indicating expected value we see that the value is moving slowly to the right and higher EBITDA values.

    We can of course also use this for long term forecasting as in the figure below:

    As should now be evident; the EBITDA Monte Carlo model have multiple fields of use and all of them will increases the managements possibilities of control and foresight giving ample opportunity for prudent planning for the future.

     

     

  • The Uncertainty in Forecasting Airport Pax

    The Uncertainty in Forecasting Airport Pax

    This entry is part 3 of 4 in the series Airports

     

    When planning airport operations, investments both air- and land side or only making next years budget you need to make some forecasts of what traffic you can expect. Now, there are many ways of doing that most of them ending up with a single figure for the monthly or weekly traffic. However we do know that the probability for that figure to be correct is near zero, thus we end up with plans based on assumptions that most likely newer will happen.

    This is why we use Monte Carlo simulation to get a grasp of the uncertainty in our forecast and how this uncertainty develops as we go into the future. The following graph (from real life) shows how the passenger distribution changes as we go from year 2010 (blue) to 2017 (red). The distribution moves outwards showing an expected increase in Pax at the same time it spreads out on the x-axis (Pax) giving a good picture of the increased uncertainty we face.

    Pax-2010_2017This can also be seen from the yearly cumulative probability distributions given below. As we move out into the future the distributions are leaning more and more to the right while still being “anchored” on the left to approximately the same place – showing increased uncertainty in the future Pax forecasts. However our confidence in that the airport will reach at least 40M Pax during the next 5 years is bolstered.

    Pax_DistributionsIf we look at the fan-chart for the Pax forecasts below, the limits of the dark blue region give the lower (25%) and upper (75%) quartiles for the yearly Pax distributions i.e. the region where we expect with 50% probability the actual Pax figures to fall.

    Pax_Uncertainty

    The lower und upper limits give the 5% and 95% percentiles for the yearly Pax distributions i.e. we can expect with 90% probability that the actual Pax figures will fall somewhere inside these three regions.

    As shown the uncertainty about the future yearly Pax figures is quite high. With this as the backcloth for airport planning it is evident that the stochastic nature of the Pax forecasts has to be taken into account when investment decisions (under uncertainty) are to be made. (ref) Since the airport value will relay heavily on these forecasts it is also evident that this value will be stochastic and that methods from decision making under uncertainty have to be used for possible M&R.

    Major Airport Operation Disruptions

    Delays – the time lapse which occurs when a planned event does not happen at the planned time – are pretty common at most airports Eurocontrol  estimates it on average to approx 13 min on departure for 45%  of the flights and approx 12 min for arrivals in 42% of the flights (Guest, 2007). Nevertheless the airport costs of such delays are small; it can even give an increase in revenue (Cook, Tanner, & Anderson, 2004).

    We have lately in Europe experienced major disruptions in airport operations thru closing of airspace due to volcanic ash. Closed airspace give a direct effect on airport revenue and a higher effect the closer it is to an airport. Volcanic eruptions in some regions might be considered as Black Swan events to an airport, but there are a large number of volcanoes that might cause closing of airspace for shorter or longer time. The Smithsonian Global Volcanism Program lists more than 540 volcanoes with previous documented eruption.

    As there is little data for events like this it is difficult to include the probable effects of closed airspace due to volcanic eruptions in the simulation. However, the data includes effects of the 9/11 terrorist attack and the left tails of the yearly Pax distributions will be influenced by this.

    References

    Guest, Tim. (2007, September). A Matter of time: air traffic delay in Europe. , EUROCONTROL Trends in Air Traffic I, 2.

    Cook, A., Tanner, G., & Anderson, S. (2004). Evaluating the true cost to airlines of one minute of airborne or ground delay: final report. [University of Westminster]. Retrieved from, www.eurocontrol.int/prc/gallery/content/public/Docs/cost_of_delay.pdf

  • Airport Simulation

    Airport Simulation

    This entry is part 1 of 4 in the series Airports

     

    The basic building block in airport simulation is the passenger (Pax) forecast. This is the basis for subsequent estimation of aircraft movements (ATM), investment in terminal buildings and airside installations, all traffic charges, tax free sales etc. In short it is the basic determinant of the airport’s economics.

    The forecast model is usually based on a logarithmic relation between Pax, GDP and airfare price movement. ((Manual on Air Traffic Forecasting. ICAO, 2006)), ((Howard, George P. et al. Airport Economic Planning. Cambridge: MIT Press, 1974.))

    There has been a large number of studies over time and across the world on Air Travel Demand Elasticities, a good survey is given in a Canadian study ((Gillen, David W.,William G. Morrison, Christopher Stewart . “Air Travel Demand Elasticities: Concepts, Issues and Measurement.” 24 Feb 2009 http://www.fin.gc.ca/consultresp/Airtravel/airtravStdy_-eng.asp)).

    In a recent project for an European airport – aimed at establishing an EBITDA model capable of simulating risk in its economic operations – we embedded the Pax forecast models in the EBITDA model. Since the seasonal variations in traffic are very pronounced and since the cycles are reverse for domestic and international traffic a good forecast model should attempt to forecast the seasonal variations for the different groups of travellers.

    int_dom-pax

    In the following graph we have done just that, by adding seasonal factors to the forecast model based on the relation between Pax and change in GDP and air fare cost. We have however accepted the fact that neither is the model specification complete, nor is the seasonal factors fixed and constant. We therefore apply Monte Carlo simulation using estimation and forecast errors as the stochastic parts. In the figure the green lines indicate the 95% limit, the blue the mean value and the red the 5% limit. Thus with 90% probability will the number of monthly Pax fall within these limits.

    pax

    From the graph we can clearly se the effects of estimation and forecast “errors” and the fact that it is international travel that increases most as GDP increases (summer effect).

    As an increase in GDP at this point of time is not exactly imminent we supply the following graph, displaying effects of different scenarios in growth in GDP and air fare cost.

    pax-gdp-and-price

    References