Treatment of Missing Market Data: Case of bond Yield Curve Estimation

Missing observations in market data is a frequent problem in financial studies. The problem of missing data is often overlooked in practice. Missing data is mostly treated using ad hoc methods or just ignored. Our goal is to develop practical recommendations for treatment of missing observations in financial data. We illustrate the issue with an example of yield curve estimation on Russian bond market. We compare three methods of missing data imputation — last observation carried forward, Kalman filtering and EM–algorithm — with a simple strategy of ignoring missing observations. We conclude that the impact of data imputation on the quality of yield curve estimation depends on model sensitivity to the market data. For non-sensitive models, such as Nelson-Siegel yield curve model, final effect is insignificant. For more sensitive models, such as bootstrapping, missing data imputation allows to increase the quality of yield curve estimation. However, the result does not depend on the chosen data imputation method. Both simple last observation carried forward method and more advanced EM–algorithm lead to similar final results. Therefore, when estimating yield curves on the illiquid markets with missing market data, we recommend to use either simple non-sensitive to the data parametric models of yield curve or to impute missing data before using more advanced and sensitive yield curve models.

1. Макушкин М. С., Лапшин В. А. Кривые доходностей на низколиквидных рынках облигаций: особенности оценки. Экономический журнал Высшей школы экономики. 2021;25(2):177–195. DOI: 10.17323/1813–8691–2021–25–2–177–195 Makushkin M., Lapshin V. Yield curve estimation in illiquid bond markets. Ekonomicheskii zhurnal Vysshei shkoly ekonomiki = The HSE Economic Journal. 2021;25(2):177–195. (In Russ.). DOI: 10.17323/1813–8691–2021–25–2–177–195

2. Haitovsky Y. Missing data in regression analysis. Journal of the Royal Statistical Society. Series B (Methodological). 1968;30(1):67–82. DOI: 10.1111/j.2517–6161.1968.tb01507.x

3. Rubin D. B. Inference and missing data. Biometrika. 1976;63(3):581–592. DOI: 10.1093/biomet/63.3.581

4. Little R. J.A. Missing-data adjustments in large surveys. Journal of Business & Economic Statistics. 1988;6(3):287–296. DOI: 10.1080/07350015.1988.10509663

5. Newman D. A. Missing data: Five practical guidelines. Organizational Research Methods. 2014;17(4):372–411. DOI: 10.1177/1094428114548590

6. Cortazar G., Schwartz E. S., Naranjo L. F. Term-structure estimation in markets with infrequent trading. International Journal of Finance & Economics. 2007;12(4):353–369. DOI: 10.1002/ijfe.317

7. Poghosyan K., Poghosyan A. Yield curve estimation and forecasting in Armenia. Armenian Journal of Economics. 2019;(4):1–19. URL: https://www.aea.am/files/papers/a2019_1.pdf

8. Nagy K. Term structure estimation with missing data: Application for emerging markets. The Quarterly Review of Economics and Finance. 2020;75:347–360. DOI: 10.1016/j.qref.2019.04.002

9. Kofman P., Sharpe I. G. Using multiple imputation in the analysis of incomplete observations in finance. Journal of Financial Econometrics. 2003;1(2):216–249. DOI: 10.1093/jjfinec/nbg013

10. Briggs A., Clark T., Wolstenholme J., Clarke P. Missing… presumed at random: Cost-analysis of incomplete data. Health Economics. 2003;12(5):377–392. DOI: 10.1002/hec.766

11. Graham J. W. Missing data analysis: Making it work in the real world. Annual Review of Psychology. 2009;60:549–576. DOI: 10.1146/annurev.psych.58.110405.085530

12. Warga A. Bond returns, liquidity, and missing data. Journal of Financial and Quantitative Analysis. 1992;27(4):605–617. DOI: 10.2307/2331143

13. Zhu H. An empirical comparison of credit spreads between the bond market and the credit default swap market. Journal of Financial Services Research. 2006;29(3):211–235. DOI: 10.1007/s10693–006–7626-x

14. Косьяненко А. В. Опыт восстановления пропущенной рыночной информации на основе Байесовского подхода. Финансовая инженерия, риск-менеджмент и актуарная наука. Препринт WP16/2007/02. М.: ГУ ВШЭ; 2007. 32 с. URL: https://www.hse.ru/data/2010/05/07/1217275563/WP16_2007_02.pdf Kos’yanenko A. V. Missing market data imputation using Bayesian inference. Financial engineering, risk management and actuarial science. Preprint WP16/2007/02. Moscow: SU HSE; 2007. 32 p. (In Russ.).

15. Теплова Т. В., Буданова Д. М. Эффективность ценообразования на российском рынке корпоративных облигаций. Вестник Московского университета. Серия 6: Экономика. 2017;(4):3–28. DOI: 10.38050/01300105201741 Teplova T., Budanova D. Pricing effectiveness at the Russian corporate bonds market. Vestnik Moskovskogo universiteta. Seriya 6: Ekonomika = Moscow University Economics Bulletin. 2017;(4):3–28. (In Russ.). DOI: 10.38050/01300105201741

16. Хакимжанов С., Мустафин Е., Кубенбаев О., Атабек Д. Построение кривой доходности на рынке с низкой ликвидностью. Деньги и Кредит. 2019;78(4):71–98. DOI: 10.31477/rjmf.201904.71 Khakimzhanov S., Mustafin Y., Kubenbayev О., Atabek D. Constructing a yield curve in a market with low liquidity. Russian Journal of Money and Finance. 2019;78(4):71–98. DOI: 10.31477/rjmf.201904.71 (In Russ.: Den’gi i kredit. 2019;78(4):71–98. DOI: 10.31477/rjmf.201904.71).

17. Лапшин В. А., Терещенко М. Ю. Выбор модели срочной структуры процентных ставок на основе ее свойств. Корпоративные Финансы. 2018;12(2):171–187. DOI: 10.17323/j.jcfr.2073–0438.12.2.2018.171–187 Lapshin V., Tereshchenko M. The choice of the model of the term structure of interest rates on the basis of its properties. Korporativnye finansy = Journal of Corporate Finance Research. 2018;12(2):171–187. (In Russ.). DOI: 10.17323/j.jcfr.2073–0438.12.2.2018.171–187

18. Курбангалеев М. З., Лапшин В. А. Анализ требований к оценке срочной структуры безрисковых ставок в финансовых задачах. Управление финансовыми рисками. 2015;(1):50–60. Kurbangaleev M., Lapshin V. An analysis of the risk-free term structure estimation requirements posed by various financial problems. Upravlenie finansovymi riskami = Financial Risk Management Journal. 2015;(1):50–60. (In Russ.).

19. Nelson C. R., Siegel A. F. Parsimonious modeling of yield curves. The Journal of Business. 1987;60(4):473–489. DOI: 10.1086/296409

20. Fama E. F., Bliss R. R. The information in long-maturity forward rates. The American Economic Review. 1987;77(4):680–692.

21. Diebold F. X., Li C. Forecasting the term structure of government bond yields. Journal of Econometrics. 2006;130(2):337–364. DOI: 10.1016/j.jeconom.2005.03.005

22. Zoricic D., Orsag S. Parametric yield curve modeling in an illiquid and undeveloped financial market. UTMS Journal of Economics. 2013;4(3)234–252. URL: https://www.utmsjoe.mk/files/Vol.%204%20No.%203/1–1B-Zoricic-Orsag_-_Croatia.pdf

23. Chou J.-H., Su Y.-S., Tang H.-W., Chen C.-Y. Fitting the term structure of interest rates in illiquid market: Taiwan experience. Investment Management and Financial Innovations. 2009;6(1):101–116. URL: https://www.researchgate.net/publication/288118114_Fitting_the_term_structure_of_interest_rates_in_illiquid_market_Taiwan_experience

24. Wahlstrøm R. R., Paraschiv F., Schürle M. A. Comparative analysis of parsimonious yield curve models with focus on the Nelson-Siegel, Svensson and Bliss versions. Computational Economics. 2022;59(3):967–1004. DOI: 10.1007/s10614–021–10113-w

25. Hagan P. S., West G. Interpolation methods for curve construction. Applied Mathematical Finance. 2006;13(2):89–129. DOI: 10.1080/13504860500396032

26. Авдеева О. А., Цыплаков А. А. Метод адаптивного оценивания срочной структуры процентных ставок. Экономический журнал Высшей школы экономики. 2015;19(4):609–639. Avdeeva О., Tsyplakov А. A method for adaptive estimation of the term structure of interest rates. Ekonomicheskii zhurnal Vysshei shkoly ekonomiki = The HSE Economic Journal. 2015;19(4):609–639. (In Russ.).

27. Kalman R. A new approach to linear filtering and prediction problems. Transactions of the AMSE. Journal of Basic Engineering. 1960;82(1):35–45. DOI: 10.1115/1.3662552

28. Dempster A. P., Larid N. M., Rubin D. B. Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological). 1977;39(1):1–22. DOI: 10.1111/j.2517–6161.1977.tb01600.x

29. Lapshin V., Sohatskaya S. Choosing the weighting coefficients for estimating the term structure from sovereign bonds. International Review of Economics & Finance. 2020;70:635–648. DOI: 10.1016/j.iref.2020.08.011

30. Курбангалеев М. З., Лапшин В. А., Шепелева И. С. Согласованность котировок государственных облигаций России. Управленческий учет и финансы. 2016;(1):40–51. Kurbangaleev M., Lapshin V., Shepeleva I. Consistency of Russian sovereign bond prices. Upravlencheskii uchet i finansy = Management Accounting and Finance Journal. 2016;(1):40–51. (In Russ.).

31. Falkenstein E., Hanweck J. Minimizing basis risk from non-parallel shifts in the yield curve. Part II: Principal components. The Journal of Fixed Income. 1997;7(1):85–90. DOI: 10.3905/jfi.1997.408199