The supply side of open source software (OSS), — that is, the reasons for joining and contributing to open source communities, — has been extensively studied and quantified. For example, the Linux Foundation (LF) has over 1,500 members, and is home to nearly 1,000 open source projects from over 15,000 contributing organizations. The LF estimates that over 50 million lines of code are added weekly to its sponsored projects, which comprise a significant percentage of the world’s mission-critical infrastructures, a figure that’s significantly higher if we include the contributions of other open source communities like the Apache Software Foundation.
However, we know much less about the demand side of OSS. Despite the wide adoption of OSS by firms and industries around the world, we don’t really understand, nor are we able to quantify its overall economic value to the many organizations that rely on open source. What benefits do these organizations derive from OSS? What costs do they incur from using or contributing to OSS? And, if not able to use OSS in a project, what would have been the next-best alternatives, and how much would they have cost?
To get answers to these and other important OSS questions, the LF sponsored a study led by UC Berkeley professor Henry Chesbrough, — whose fourth book on open innovation, Open Innovation Results, was published in 2020. The key findings and conclusions from the study are discussed in a White Paper, “Measuring the Economic Value of Open Source.”
“Most of the research on OSS has taken advantage of the fact that the repositories of OSS are available to the public, including academic scholars,” wrote Chesbrough in the White Paper. “However, if one wishes to measure the economic value of OSS, it is not enough to observe the large and growing number of OSS projects and code commits to those projects. One must also examine how the individual or organization that employed OSS used the software. Yet these actions are not observable to the public. Instead, one must construct ways to probe those actors to uncover the ways in which they use OSS.”
The White Paper cites a 2021 report by the European Commission on the impact of open source on the EU economy, which was based on a cost-benefit analysis of the economic impact of open source investments and a survey of over 900 stakeholders. Most respondents (75%) came from small and medium companies, while large companies, which are major users and contributors to OSS, were underrepresented in the survey results. “Overall, the benefits of Open Source greatly outweigh the costs associated with it,” said the EU report. The contributions of OSS to EU GDP “yield a cost-benefit ratio of slightly above 1:10. After taking into account hardware and other capital costs of the 260,000 EU contributors to OSS, the cost-benefit ratio is still slightly above 1:4.”
To understand the reasons and estimated economic value that have led companies to embrace OSS, Chesbrough and his collaborators devised and conducted a survey. All the survey questions and responses, 28 in total, can be found in the Appendix of the White Paper. “Our sample tilts toward Fortune 500 firms,” noted Chesbrough. 43% of respondents had 2021 revenues exceeding $1 billion, of which 29% had revenues exceeding $10 billion; and 57%, came from organizations with revenues of under $1 billion. A number of the responding organization have been working with OSS for over 20 years, but about half had only started doing so since 2015.
The majority of respondents, 29%, were CEOs, 18% were members of the R&D department, 18% worked in business and marketing, 9% were CIOs/CDOs, 6% were in the IT and software group, and the rest came from a number of smaller groups. Let me summarize the study’s key findings.
The first set of questions aimed to assess the overall economic value of OSS by asking about the general costs and benefits of OSS to the organization.
Based on your experience, rate the degree of benefits derived from using or contributing to OSS for your organization. The highest rated benefit was cost savings, which 67% of respondents rated high or very high and only 10% rated low or none. It was followed by faster development speeds, rated high or very high by 66%, and low or none by 9%; and open standards & interoperability, rated high or very high by 63% and low or none by 13%. Other highly rated benefits were active community for knowledge exchange — 59%; independence from proprietary vendors, — 54%; and attractive IT work environment, — 49%.
Based on your experience, rate the primary costs associated with the use of or contribution to OSS for your organization. The top source of high or very high costs associated with OSS was security gaps, mentioned by 25% of respondents. It was followed by hidden support costs, brought up by 23%; costs related to legal uncertainties with licensing — 17%; and switching costs from proprietary to OSS — 17%.
What is your assessment of the overall benefit-cost ratio of using or contributing to OSS? About two thirds of respondents said that the benefits exceeded the costs, of which 23% said that benefits greatly exceeded costs; 21% responded that the costs exceeded the benefits, of which only 3% said that costs greatly exceeded benefits; and 11% said that costs and benefits were about equal.
How is the overall benefit-cost ratio of using or contributing to OSS trending in your organization over the past 5 years? Almost 50% of respondents said that benefits are rising faster than costs, of which 7% said that they’re rising much faster; 17% said that costs were rising faster than benefits, of which only about 1.5% said that the costs were rising much faster; and 35% said that benefits and costs have been stable over the past 5 years.
The next set of questions asked respondents to consider a major recently completed project that included OSS. “This question demanded a lot of knowledge from our respondents, and we received far fewer responses to this and subsequent questions as a result,” noted Chesbrough.
Approximately how many lines of software code were included in this new offering? The size of the project as reflected by the number of lines of code varied greatly. 18% of responses said that the recently completed project included over 160,000 lines of code (LOC); 8% said 80,000 to 160,000 LOC;13% said 20,000 to 80,000 LOC; 30% —5,000 to 20,000 LOC; 16% — 1,000 to 5,000 LOC ; and 15% said less than 1,000 lines of code.
Approximately what percentage of those lines of code were created from OSS? 22% responded that OSS accounted for more than 80% of the code; 10% responded that its was 60% to 80% of the code; 18% — 40% to 60%; 26% — 20% to 40%; and 24% responded that OSS accounted for less than 20% of the code.
For those OSS lines of code: How much would it have cost you to write the necessary lines of code to achieve that functionality with your own software, rather than with OSS software, including ongoing support and maintenance of the code? 67% of respondents estimated that the software would have cost significantly more without OSS, of which 46% believed it would have cost over twice as much; 21% believed that the software would have cost them less without OSS.
If you had not been able to use OSS in this project: What was your next-best alternative to achieve a similar level of functionality in your release? 50% of respondents would have used a commercial proprietary solution instead of OSS; and 36% said that they would have done the project in-house. In addition, 75% estimated that it would have been more expensive to purchase the software from a commercial vendor, of which 31% felt that it would have been over four times more expensive than OSS; and 13% believed that the commercial product would have been less expensive.
“Considering the various questions and their respective responses, it is quite clear that respondents perceive OSS to have substantial economic value,” wrote Chesbrough in conclusion. “The perceived benefits clearly exceed the perceived costs for a strong majority of respondents — 60% to 75%, depending on the specific question. And the ratio of benefits to costs appears to be rising for nearly half of the respondents, while only 16% felt that the ratio was declining. This strongly suggests that the value of OSS will increase even further in the future for most participating organizations.”
Chesbrough further explained that the survey underestimates the value of OSS to participating organizations because the results don’t take into account the wider societal benefits of OSS. “ Society benefits from the ability of other firms to access the same OSS repositories, something that an individual firm may not value and that our survey did not measure. And the availability of these open repositories may even enable new firms to enter who otherwise might not have, a further societal benefit not captured in this survey. So, from a social perspective, the value of OSS adoption is even greater than the results reported here.”
“A final thought for those organizations who have not yet adopted OSS is to remember one of the insights from the very early days of OSS: It pays to be more open,” he added. “Software is a technology whose importance is steadily increasing over time. … Adopting OSS can allow you to embrace a more vibrant, surprising, and exciting future.”
A small comment - EU report you mention ( https://op.europa.eu/en/publication-detail/-/publication/29effe73-2c2c-11ec-bd8e-01aa75ed71a1/language-en ) includes an alternative to the survey-based methodology, in page 152-153, extending previous work done here ( https://joinup.ec.europa.eu/collection/open-source-observatory-osor/news/contribution-open-source-t ). Despite being based on a purely quantitative measurement of code bases and reuse cost (with the simple, but reasonably effective COCOMO II model) it ends up being very close to the results of the survey - so the underrepresentation of large companies in the Chesbrough work is probably not biig enough to change the soundness of the results.
Posted by: Carlo Daffara | March 16, 2023 at 02:27 PM