GitHub reveals Australian developer trends, impact of ChatGPT

New data on Australian developer trends has been released by GitHub in its updated GitHub Innovation Graph for the first quarter of 2024.

Over the past four years, the Innovation Graph has provided significant insights, helping researchers track and measure trends within the software development landscape. A prominent study leverages this data to highlight how the widespread adoption of ChatGPT has impacted developer engagement across various programming languages. It notes that rather than introducing a substantial number of new developers to the field, ChatGPT is primarily improving the efficiency of existing developers.

The Innovation Graph enables data access by metrics or economies, visualising top programming languages, the total number of users by country, economy collaborators, and more. The latest data for Australia reveals that more than 1,574,000 Australian developers and over 86,000 Australian organisations are actively building on GitHub. Australian developers have uploaded code to GitHub more than 1.6 million times. Additionally, Australian developers or organisations own over 3.4 million repositories on GitHub.

The tool also tracks collaboration between global economies by counting the git pushes sent and pull requests opened between different economies. Australia's top collaborators were identified as the United States, United Kingdom, and Germany. JavaScript remains the highest-ranked programming language among Australian developers based on the number of unique developers who uploaded code, followed by Python and Shell.

An interview featuring Alexander Quispe, a junior researcher at the World Bank, and Rodrigo Grijalba, a data scientist, sheds light on how they have utilised the Innovation Graph data in their work. Speaking about their research at the Munich Summer Institute, Quispe remarked, “Advancements in artificial intelligence (AI) have revolutionised various fields, with software development being one of the most impacted. The rise of large language models (LLMs) and tools like OpenAI’s ChatGPT and GitHub Copilot, has brought about a significant shift in how developers approach coding, debugging, and software architecture.”

Their research analyzed the impact of ChatGPT on the velocity of software development. Quispe noted that ChatGPT significantly increased the number of Git pushes per 100,000 inhabitants of each country, although its correlation with the number of repositories and developers per 100,000 inhabitants was not statistically significant. “The impact of ChatGPT thus far does not lie in the increase of developers or projects, but in an acceleration of the pre-established development process,” Quispe added.

Grijalba highlighted how GitHub’s Innovation Graph data integrated well with their methodologies, particularly with synthetic methods. "Having country- and language-level aggregated data meant that we could define our control and treatment groups, and disaggregate them to find how effects differed by language," he said.

Further elaborating on their research methods, Quispe explained that they used synthetic difference-in-differences (SDID), as the parallel trends assumption required for the traditional difference in differences (DiD) method did not hold for their treatment and control groups. They also faced precision issues with the synthetic control (SC) method due to limited pre-treatment data. SDID offered a more robust analysis by constructing a synthetic control group while acknowledging pre-treatment differences.

Commenting on the limitations of their study, Quispe acknowledged critiques regarding VPNs potentially allowing individuals in restricted countries to access ChatGPT. However, he referenced studies indicating that these barriers still pose significant hurdles, validating their control group assumptions.

Going forward, Quispe expressed a desire to conduct similar analyses using administrative data at the software developer level. Additionally, they aim to compare productivity increases between those with access to GitHub Copilot and those without, while also looking at user experience metrics.

In terms of future predictions, Quispe suggested, "The future likely holds increased integration of AI tools like ChatGPT and GitHub Copilot in software development processes." He recommended policymakers support the integration of AI tools to enhance productivity and economic growth, and advised developers to leverage such tools to boost their efficiency.