Application of a Model Life Cycle Concept to Investments in Artificial Intelligence Evaluation on the Example of Large Language Models

The life cycle of an artificial intelligence model is the object of research. The purpose of the study is to develop a model life-cycle methodology that describes the economic content of the investment process in artificial intelligence technology. During the study, both general scientific methods such as analysis, synthesis, comparison, abstraction, induction and deduction were used, as well as project methodologies of the life-cycle, employed as the basis for the value creation life-cycle of the model. The analysis was based on identifying the necessary stages of model development in terms of the CRISP-DM methodology and determining the features of each of them in terms of cash flows. Modified versions of the model life-cycle containing risk assessment, including model risk, were also taken into account. In the process of research, the proposed generalized model life-cycle methodology was specified for a specific AI technology — large language models. As a result of the study, the author proposed a three-stage model. The possible optionality between the stages and the characteristics of cash flows are described. It was concluded that an investment project for the development of AI contains several real options — abandonment, reduction, expansion and replacement. For large language models, the life cycle structure and possible optionalities are preserved. The peculiarity is that the value creation process involves cash flows from different areas of application of the model in business processes. The results of the study are of practical importance for medium and large businesses engaged in the independent development of AI models and/or applying them to their business processes. The proposed concept of the model life-cycle can also be used to develop a methodology for evaluating investments in AI using real options.

1. Maslej N., Fattorini L., Brynjolfsson E., et al. Artificial intelligence index report 2023. Stanford, CA: Institute for Human-Centered AI, Stanford University; 2023. 386 p. URL: https://aiindex.stanford.edu/wp-content/uploads/2023/04/HAI_AI–Index-Report_2023.pdf (дата обращения: 01.08.2023).

2. Chen L., Zaharia M., Zou J. How is ChatGPT’s behavior changing over time? Harvard Data Science Review. 2024;(6.2):1–47. DOI: 10.1162/99608f92.5317da47

3. Shearer C. The CRISP-DM model: The new blueprint for data mining. Journal of Data Warehousing. 2000;5(4):13– 22. URL: https://mineracaodedados.wordpress.com/wp-content/uploads/2012/04/the-crisp-dm-model-the-newblueprint-for-data-mining-shearer-colin.pdf

4. Tabladillo M. The Team Data Science Process lifecycle. 2017;552–554. URL: https://learn.microsoft.com/pdf?url=https%3A%2F%2Flearn.microsoft.com%2Fen-us%2Fazure%2Farchitecture%2Fai-ml%2Ftoc.json (дата обращения: 01.08.2023).

5. Haakman M., Cruz L., Huijgens H., van Deursen A. AI lifecycle models need to be revised. Empirical Software Engineering. 2021;26:95. DOI: 10.1007/s10664–021–09993–1

6. De Silva D., Alahakoon D. An artificial intelligence life cycle: From conception to production. Patterns. 2022;3(6):100489. DOI: 10.1016/j.patter.2022.100489

7. Golubev A.A. The life cycle of innovation and resource support innovation. Sovremennye problemy nauki i obrazovaniya = Modern Problems of Science and Education. 2015;(2–2):414. URL: https://science-education.ru/ru/article/view?id=22026 (accessed on 05.08.2023). (In Russ.

8. Gorodnova N.V.Application of artificial intelligence in the business sphere: Current state and prospects Voprosy innovatsionnoi ekonomiki = Russian Journal of Innovation Economics. 2021;11(4):1473–1492. (In Russ.). DOI: 10.18334/vinec.11.4.112249

9. Kashevarova N.A., Panova D.A.Analysis of the current practice of applying artificial intelligence in the financial sector and its impact on the transformation of the financial ecosystem. Kreativnaya ekonomika = Journal of Creative Economy. 2020;14(8):1565–1580. (In Russ.). DOI: 10.18334/ce.14.8.110708

10. Ustinova O.E.Artificial intelligence in company management. Kreativnaya ekonomika = Journal of Creative Economy. 2020;14(5):885–904. (In Russ.). DOI: 10.18334/ce.14.5.102145

11. Åström J., Reim W., Parida V.Value creation and value capture for AI business model innovation: A three-phase process framework. Review of Managerial Science. 2022;16(7):2111–2133. DOI: 10.1007/s11846–022–00521-z

12. Moro-Visconti R. The valuation of artificial intelligence. In: The valuation of digital intangibles: Technology, marketing, and the metaverse. Cham: Palgrave Macmillan; 2022:265–282. DOI: 10.1007/978–3–031–09237–4_8

13. Nikitin N.A. Financial evaluation of projects with artificial intelligence in the banking sector. Finansovyi biznes = Financial Business. 2022;(5):122–125. (In Russ.).

14. Agarwal N., Moehring A., Rajpurkar P., Salz T. Combining human expertise with artificial intelligence: Experimental evidence from radiology. NBER Working Paper. 2023;(31422). URL: https://www.nber.org/system/files/working_papers/w31422/w31422.pdf (дата обращения: 09.08.2023).

15. Pomulev A.A.Artificial intelligence as an object of valuation. Imushchestvennye otnosheniya v Rossiiskoi Federatsii = Property Relations in the Russian Federation. 2022;(6):42–56. (In Russ.). DOI: 10.24412/2072–4098–2022–6249–42–56

16. Nikitin N.A. Probabilistic methods for accounting model risks in assessing investments in artificial intelligence technologies. Innovatsionnoe razvitie ekonomiki = Innovative Development of Economy. 2023;(2):123–134. (In Russ.). DOI: 10.51832/2223798420232123

17. Kiela D., Bartolo M., Yixin Nie Y., et al. Dynabench: Rethinking benchmarking in NLP. In: Proc. 2021 conf. North American Chapter of the Association for Computational Linguistics: Human language technologies. Stroudsburg, PA: Association for Computational Linguistics; 2021:4110–4124. URL: https://aclanthology.org/2021.naaclmain.324.pdf (дата обращения: 11.08.2023).

18. Bubeck S., ChandrasekaranV., Eldan R., et al. Sparks of artificial general intelligence: Early experiments with GPT 4. Cornell University. arXiv:2303.12712 [cs.CL]. 2023. DOI: 10.48550/arXiv.2303.12712

19. Sevilla J., Heim L., Ho A., et al. Compute trends across three eras of machine learning. In: 2022 Int. joint conf. on neural networks (IJCNN). (Padua, July 18–23, 2022). Piscataway, NJ: IEEE; 2022. DOI: 10.1109/IJCNN55064.2022.9891914

20. Waisberg E., Ong J., Masalkhi M., et al. GPT 4: A new era of artificial intelligence in medicine. Irish Journal of Medical Science. 2023;192(6):3197–3200. DOI: 10.1007/s11845–023–03377–8

21. Rivas P., Zhao L. Marketing with ChatGPT: Navigating the ethical terrain of GPT-based chatbot technology. AI. 2023;4(2):375–384. DOI: 10.3390/ai4020019

22. Dohmke T., Iansiti M., Richards G. Sea change in software development: Economic and productivity analysis of the AI-powered developer lifecycle. New Hyde Park, NY: Keystone; 2023. 30 p. URL: https://github.blog/wp-content/uploads/2023/06/Sea-Change-in-Software-Dev.pdf

23. Chung H.W., Hou L., Longpre S., et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research. 2024;25:1–53. URL: https://www.jmlr.org/papers/volume25/23–0870/23–0870.pdf