A service of the

Content

Volume 52, July/August 2017, Number 4 · pp. 247-250

Articles

Welcome to the Club – On the Accuracy of Growth Forecasts for China

Ullrich Heilemann

Forecasts for the Chinese economy are often seen as unreliable. This impedes a proper assessment of the country's course, and given that China is the world's second-largest economy, it also has implications for international projections. This article examines the accuracy of real GDP forecasts by three international forecasters and two German forecasters over the period 1991-2015. While forecasters had considerable difficulties in tracking the large upswings in China in the 1990s and early 2000s, they were quite accurate for the period 2004-2015, particularly when comparing the IMF's forecasts for China with its forecasts for the United States or Germany. The forecast accuracy found in the Chinese Five-Year Plans is also surprisingly high.

Ullrich Heilemann, Leipzig University, Germany; and Fraunhofer Center for International Management and Knowledge Economy, Leipzig, Germany.

In the winter of 2015-16, China reported 6.8% growth of its real gross domestic product (GDP) for 2015, and forecasters expected 6.5% growth in 2016. However, after years of double-digit growth rates, worrying signals from major indicators and the widespread belief that Chinese data was manipulated and forecasts were unreliable led to increased fears that China's economy would face "a hard landing" in 2016, which would likely trigger another worldwide recession.1 The past has its limits as a guide for the future, but it was hard to understand why these fears were not examined in light of the record of economic forecasts for China.

The accuracy of forecasts for China has recently gained increased attention, particularly since the Great Recession (2007-2010) and primarily for forecasts by international organisations or by Asian forecasters.2 Similar studies of the Chinese economy by European forecasters, however, are still rare. This paper examines the accuracy of recent growth predictions for China by international as well as European and German forecasters, primarily from the perspective of a forecast user. Issues related to the quality of Chinese data are not addressed.3 The analysis focuses on forecasts by three international institutions – the International Monetary Fund (IMF), the Organisation for Economic Development and Co-operation (OECD) and the European Commission (EC) – and two German institutions – the Joint Diagnosis (JD), a government-sponsored group of five major German economic research institutes, and the Council of Economic Experts (CEE), a major government advisory institution.4 Forecasts produced by Chinese institutions such as the Chinese Academy of Social Sciences and the National Development and Reform Commission, which operates as China's government planning commission, were unavailable.5 In addition, the implicit annual growth rates expected according to the tenets of China's Five-Year Plans are examined.

With the exception of the CEE, all institutions present at least semiannual forecasts for the current and subsequent years. Only the autumn forecasts for the subsequent year – the most important forecasts from a policy perspective – are examined here. The IMF publishes these in September, the JD in October, and the EC and OECD in November.6 Comparisons are limited to 2004-2015; IMF forecasts are also examined for the period 1991-2015 (most other forecasts examined here did not begin being published until the early 2000s) and compared with the IMF forecasts for Germany and the United States.

Accuracy of growth forecasts for China

As Figure 1 displays, the accuracy of the five forecasts was hardly impressive. The high growth rates during both upswings and downturns throughout three long growth cycles were severely underestimated by all five forecasters (as well as by the Five-Year Plans),7 and turning points were often recognised only with considerable lags.

Figure 1 (back to the text)
Growth forecasts for China, 1991-2015

in %

Growth forecasts for China, 1991-2015

Notes: Real GDP, percentage change against previous year.

Sources: IMF; OECD; European Commission; RWI Leibniz Institute for Economic Research; M. Aoki, J. Wu (eds.): The Chinese Economy: A New Transition, Basingstoke 2012, Palgrave Macmillan UK; and J. Casey, K. Koleski: Backgrounder: China's 12th Five-Year Plan, U.S.-China Economic & Security Review Commission, 2011.

As Table 1 shows, the values of the mean absolute error (MAE), i.e. the mean absolute difference between a forecast and actual rates, of the five forecasts are basically the same (about one percentage point).8 An average MAE of one percentage point is large from the perspective of G7 countries – an autumn forecast of two per cent growth for the next year may result in a growth rate between one and three per cent. For China, however, with its much higher growth rates, a one percentage point MAE is small, as illustrated by the mean average percentage errors (MAPE) of the IMF forecasts over the period 1991-2015 for China, Germany and the United States (see Table 2).

Table 1 (back to the text)
Accuracy measures for selected growth forecasts of China's economy, 2004-2015

IMF

JD

CEE

EC

OECD

Average

China

Five-Year Plans

MAE

1.1

1.0

0.9

0.9

1.0

1.0

1.9

MAPE

11.7

10.8

9.1

9.2

10.5

10.2

19.0

Bias

-0.5

-0.5

-0.4

-0.5

-0.2

-0.4

-1.8

U

0.2

0.1

0.1

0.1

0.1

0.1

0.2

Notes: EC forecast for 2009 not available.

Sources: Author's computations; IMF; OECD; European Commission; RWI Leibniz Institute for Economic Research; and M. Aoki, J. Wu (eds.): The Chinese Economy. A New Transition, Basingstoke 2012, Palgrave Macmillan UK.

Table 2 (back to the text)
Accuracy measures for the International Monetary Fund's growth forecasts, 1991-2015

China Five-Year Plans

China IMF

Germany IMF

United States IMF

1991-2015

MAE

1.9

1.7

1.3

1.0

MAPE

18.3

16.5

195.2

47.7

Bias

-1.8

-1.2

0.6

0.0

U

0.3

0.3

0.8

0.4

RMSE/σ

1.4

1.4

1.1

0.8

2004-2015

MAE

1.9

1.1

1.2

0.7

MAPE

19.0

11.7

50.9

30.9

Bias

-1.8

-0.5

0.0

0.3

U

0.2

0.2

0.8

0.3

RMSE/σ

1.6

1.0

0.9

0.6

Sources: Author's computations; IMF; OECD; European Commission; RWI Leibniz Institute for Economic Research; and M. Aoki, J. Wu (eds.): The Chinese Economy. A New Transition, Basingstoke 2012, Palgrave Macmillan UK.

As to turning point errors, there are no differences between the five forecasts for China, but when compared with IMF forecasts for Germany and the United States, China's share of total forecast errors is four and six times higher, respectively.9 To a large degree, this is due to the fact that the predicted turning points in China failed to happen, while such errors were not made for the United States or Germany. Directional errors in the forecasts for China were only half as large as those for the other two countries. It should also be noted that the recessions in most G7 countries were recognised only when they were already underway.10

The bias – that is, the cumulative balance of over- and underestimations – is negative and within small bands; for the longer sample, it is twice as large for China as for Germany, and for the shorter sample, it is twice as large for China as for the United States, though given China's high rates of growth, it should also be normalised.11 A statistical explanation is that forecasters failed to recognise the strength and duration of the long upswings of 1991-1993 and 2001-2006, which were beyond China's (and the world's) experience. An economic explanation would be that the development was primarily due to the Communist Party's and the Chinese government's new direction and "the relatively direct correlation between government policy and investment activity".12 However, while China's Five-Year Plans were rather accurate in general and differed from the IMF's forecasts, there were also considerable positive surprises, for example in 1991-1995 and again during the Great Recession (see Table 3).

Table 3 (back to the text)
Forecast accuracy of China's Five-Year Plans, 1991-2015

1991-1995 (8)

1996-2000

(9)

2001-2005 (10)

2006-2010 (11)

2011-2015 (12)

1991-2015

MAE

3.5

0.7

1.8

2.5

0.8

1.9

MAPE

29.6

8.4

19.1

24.5

9.9

18.3

Bias

-3.5

-0.3

-1.8

-2.5

-0.8

-1.8

U

0.4

0.1

0.2

0.3

0.1

0.2

RMSE/σ

1.8

1.0

2.1

2.7

1.4

1.8

Notes: The numbers of the Five-Year Plans are listed in brackets, while the final column contains data for the average of the five Plans.

Sources: Author's computations; IMF; OECD; European Commission; RWI Leibniz Institute for Economic Research; and M. Aoki, J. Wu (eds.): The Chinese Economy. A New Transition, Basingstoke 2012, Palgrave Macmillan UK.

 

From a methodical perspective, the picture also looks favourable, although this perspective may be of greater interest to forecast producers than to forecast users. Results for Theil's inequality coefficient (U), which compares the actual forecasts with a so-called naïve/"no change" forecast13 – indeed a very naïve forecast – reveal that all five forecasts were much more accurate than "no change" forecasts. They were also more accurate than the naïve forecasts for Germany and the United States.

As noted, the accuracy of the IMF's forecasts differs little from those for Germany and the United States or G7 countries.14 In relative terms, they are even more accurate, and the picture does not change much when the GDP variability of the three countries is taken into account.15

With respect to the five forecasters, the accuracy of the European and German forecasts is much the same as that of the international organisations – despite the fact that the publication of the forecasts stretches over a period of three months, which naturally impacts how up to date their data sets are.16 "Home disadvantages" of German and European Commission forecasters during the period examined are hard to detect. This also holds for the longer sample and for a broader set of countries.17 As with many other industries, the forecasting industry thus seems to be rather location independent.

Summary and conclusions

While the accuracy of macroeconomic forecasts leaves much to be desired, growth forecasts for China from 2004 to 2015 were comparatively accurate in absolute terms and even more so in relative terms. Differences between the forecasts of various institutions were very small. The magnitude and persistence of the common forecast errors in the 1990s and early 2000s, however, remain a puzzle. The accuracy of forecasts for China did not differ much from the IMF forecasts for Germany, the United States or other G7 countries, and in relative terms, they were even more accurate.

Forecast users would be well advised to realise that major institutions' autumn forecasts of China's growth for the next year currently have mean average errors of about one percentage point – which is about the same as the forecasting errors for the G7 and OECD countries – and underestimate growth by 0.5 percentage points. To put it simply, with regard to the accuracy of its growth forecasts, China has joined the club of advanced economies. Things look different, of course, with respect to international growth or world trade. Given China's current weight in the world economy and its high growth rates, forecast errors should be weighted similarly to those of the European Union or the United States.

Revisiting the forecasts made in winter 2015-16: China's GDP growth in 2016 turned out to be 6.8%, or 0.3 percentage points higher than predicted. Of course, a "hard landing" of the Chinese economy in 2016 could not have been ruled out. However, given the accuracy and the pessimistic bias of forecasts for China, its economy in 2016 should have been viewed less anxiously than was the case.

 

Thanks for helpful comments go to Philipp Böing and Roland Döhrn. For skilful research assistance, I thank Mirja Hälbig.


  • 1The Economist: Fighting the next recession. Unfamiliar ways forward, 20 February 2016, available at https://www.economist.com/news/briefing/21693205-policymakers-rich-economies-need-consider-some-radical-approaches-tackling-next. Simulations using the National Institute Global Econometric Model, a quarterly structural macroeconometric model of the world economy, suggested that a one-time decline of China's real GDP by three per cent – about the decline that China experienced in the Great Recession – would reduce world GDP by 0.9 percentage points, that of the euro area by 1.5 percentage points and that of Germany by 2.5 percentage points (three-year averages). K.-J. Gern, P. Hauber, G. Potjagailo: Auswirkungen einer harten Landung in China, in: Wirtschaftsdienst, Vol. 95, No. 10, 2015, pp. 719-720.

  • 2 See, for example, H.O. Stekler, H. Zhang: An evaluation of Chinese economic forecasts, in: Journal of Chinese Economic and Business Studies, Vol. 11, No. 4, 2013, pp. 251-259; Q. Chen, M. Costantini, B. Deschamps: How accurate are professional forecasts in Asia?, in: International Journal of Forecasting, Vol. 32, No. 1, 2016, pp. 154-167; P. Higgins, T. Zha, K. Zhong: Forecasting China's economic growth and inflation, Working Paper 2016-7, Federal Reserve Bank of Atlanta, July 2016; Independent Evaluation Office of the International Monetary Fund (IEO): IMF Forecasts: Process, Quality, and Country Perspectives, 2014; N. Pain, C. Lewis, T.T. Dang, J. Jin, P. Richardson: OECD Forecasts During and After the Financial Crisis: A Post Mortem, OECD Economics Department Working Papers, No. 1107, 2014.

  • 3 In contrast to widely held beliefs, in a recent wide-ranging study of official Chinese GDP statistics, Holtz found few statistical anomalies. See C.A. Holtz: The quality of China's GDP statistics, in: China Economic Review, Vol. 30, 2014, pp. 309-338.

  • 4 For the IMF forecasts, see http://www.imf.org/external/ns/cs.aspx?id=29. The OECD forecasts are available at http://www.oecd.org/eco/economicoutlook.htm. The European Commission's forecasts are available at http://www.ec.europa.eu/economy_finance/publications/european_economy/forecasts/index_en.htm. For the JD and CEE forecasts, see http://www.rwi-essen.de/forschung-und-beratung/wachstum-konjunktur-oeffentliche-finanzen/projekte/gemeinschaftsdiagnose. Actuals are from IMF, WEO database, available at https://www.imf.org/external/pubs/ft/weo/data/WEOhistorical.xlsx.

  • 5 Forecasts from Consensus Economics (CF), representing monthly averages of forecasts from private institutions, were not included due to the difficulties of precise matching with the forecast calendar of the other institutions. When made comparable, CF's forecasts have not been much different from those by the IMF, OECD and EC; see Independent Evaluation Office of the International Monetary Fund, op. cit.; N. Pain et al., op. cit.; and L. González Cabanillas, A. Terzi: The accuracy of the European Commission's forecasts re-examined, European Economy – Economic Papers 476, 2012.

  • 6 Growth (and inflation) forecasts by the IMF, the JD and the OECD for 19 countries for the period 2001-2015 are examined in depth in U. Heilemann, K. Müller: Wenig Unterschiede: Zur Treffsicherheit internationaler Prognosen, Leipzig University Faculty of Economics and Management Science, Working Paper 149, July 2017. For forecasts by the IMF, OECD and CF, see also H.O. Stekler, H. Zhang, op. cit.

  • 7 China experienced ten growth cycles between 1991 and 2015. See Economic Cycle Research Institute (ECRI): Business Cycle Peak and Trough Dates, 21 Countries, 1948-2016, available at https://www.businesscycle.com/. For a brief history of these cycles and policy reactions, see, for example, N. Yamasawa: China's Business Cycles and Early Warning Indicators, JCER Discussion Paper No. 115, Japan Center for Economic Research, 2008.

  • 8 Formal tests (Diebold–Mariano efficiency test) did not reject the hypothesis of equal accuracy of IMF, JD and OECD forecasts during 2001-2015; see U. Heilemann, D. Müller, op. cit.

  • 9 On this and the following, see U. Heilemann, D. Müller, op. cit., pp. 15 ff. (Table 4).

  • 10 U. Heilemann, S. Schnorr-Bäcker: Could the start of the German recession 2008-2009 have been foreseen? Evidence from real-time data, in: Journal of Economics and Statistics, Vol. 237, No. 1, 2017, pp. 29-62, and L. González Cabanillas et al., op. cit., pp. 10 ff.

  • 11 Formal efficiency tests (Holden/Peel efficiency tests) confirm this finding; see U. Heilemann, D. Müller, op. cit.

  • 12 N. Yamasawa, op. cit., p. 17. It would be tempting to examine the impact of policy activism on forecast accuracy, but this is beyond the scope of the present analysis.

  • 13 Formally expressed, t = 1, ..., T, with pt = forecast, at = actual. If a forecast is perfect, U = 0; if U ≥ 1 the forecast is less accurate than a "no change" forecast would have been.

  • 14 For the G7 countries, see U. Heilemann, H.O. Stekler: Has the accuracy of macroeconomic forecasts for Germany improved?, in: German Economic Review, Vol. 14, No. 2, 2013, pp. 235-253; R. Fildes, H. Stekler: The State of Macroeconomic Forecasting, in: Journal of Macroeconomic Forecasting, Vol. 24, No. 4, 2002, pp. 425-468.

  • 15 In formal terms, RMSE (root mean square error) / σ with

  • 16 R. Döhrn, C.M. Schmidt: Information or Institution – On the Determinants of Forecast Accuracy, in: Journal of Economics and Statistics, Vol. 231, No. 1, 2011, pp. 9-27.

  • 17 See, for example, U. Heilemann, D. Müller, op. cit.


Comments on this article

There are currently no comments on this article.

Your comment

Comments are moderated and will be posted if they are on-topic and not abusive.

Spam guard* Which letter is missing in the following word?