Journal Article: "Measurement Problem of Enterprise Digital Transformation: New Methods and Findings Based on Large Language Models"
Title: Measurement Problem of Enterprise Digital Transformation: New Methods and Findings Based on Large Language Models
Authors: Jin Xingye, Zuo Congjiang, Fang Mingyue, Li Tao, Nie Huihua
Language: Chinese
Publication: Economic Research Journal, No. 3, March 2024, 0577-9154: 34-53.
English Abstract:
The rapidly growing digital economy in China has become the second-largest digital economy in the world. The importance of digital transformation for enterprises is widely acknowledged, but there are serious disagreements about the effects of enterprise digital transformation. The main reason for this phenomenon is that existing research has problems in measuring enterprise digital transformation, which is reflected in two aspects: Firstly, the measurement ob‐ jects are neither unified nor clearly defined; secondly, the measurement methods are not scientific and accurate enough. This results in many research conclusions being incomparable, difficult to replicate, and often contradictory. Therefore, to ensure that theory effectively informs practice, academia needs to reach a consensus on the measurement of enterprise digital transformation and work hard to alleviate the problem of inaccurate measurement methods. Only in this way can the confusion be clarified, and the way for theoretical insights be paved.
This paper uses an advanced machine learning method, the large language models, to construct a set of digital transformation indicators based on the annual report texts of Chinese listed companies from 2006 to 2020. Specifically, the in‐ dicators are measured in five steps. The first step is to sort out the annual reports of listed companies, and use the two parts of the annual report,“Management Discussion and Analysis”and“Table of Contents, Definitions, and Key Risk In‐dicators”as relevant texts for enterprise digital transformation. The second step is to divide all texts into sentences to form a predictive sentence pool. In the third step, a set of sentences is randomly selected and extracted based on the pres‐ ence of keywords to constitute a pending tagged sentence library. This library is manually annotated to determine whether companies are using digital technologies including big data, artificial intelligence (AI), mobile Internet, Internet of Things, blockchain, and cloud computing. The fourth step uses supervised machine learning methods with models such as ERNIE and BERT for the training of sentence classifiers. The fifth step is to use the trained ERNIE model to predict sen‐ tence by sentence in the predictive sentence pool, to assess whether and which digital technologies are utilized by the listed companies, thereby constructing a new set of digital transformation indicators for enterprises. To verify the effec‐ tiveness of the new indicators, this paper conducts comparisons with patent data, regional data and international literature in six aspects, and finds that the digital transformation indicators constructed are highly consistent with reality. With these indicators, this paper empirically tests the relationship between enterprise digital transformation and corporate financial performance, and obtains some new findings. Digital transformation generally improves financial performance (measured with ROA and ROE). Big data, AI, mobile Internet, cloud computing, and Internet of Things improve ROA and ROE, while blockchain has no such effect. For companies with poor financial performance, digital transformation can signifi‐ cantly improve financial performance, while for companies with good financial performance, digital transformation has no significant effect on financial performance. There are two main channels for enterprise digital transformation to im‐ prove financial performance, namely, improving efficiency and reducing costs.
The contributions of this paper can be summarized in three aspects. Firstly, this paper provides a new method for measuring enterprise digital transformation. It proposes a novel approach to construct digital transformation indicators based on the annual reports of listed companies in China. The new indicators promote in-depth research on enterprise digi‐ tal transformation in terms of research methods, and provide empirical evidence from China for the general digital eco‐ nomics literature. Secondly, this paper reveals impacts of different digital technologies on corporate financial perfor‐ mance and identifies different channels. Thirdly, this paper enriches the application of large language models in economic literature, for there are limited studies using large language models. This article is of great significance for promoting en‐ terprise digital transformation and achieving high-quality economic development.
Keywords: Enterprise Digital Transformation; Digital Economy; Digital Technology; Artificial Intelligence; Large Language Models
Chinese JEL Classification: C89, C43, C45
Full Paper:
