This provocative post by Jake VanderPlas argues that “the skills required to be a successful scientific researcher are increasingly indistinguishable from the skills required to be successful in industry” and that important implications follow from this for the future of higher education. There is a “new breed of scientist” that the academic world increasingly struggles to retain:
The increasing data-centeredness of science, however, is already leading to new approaches to problems: in the era of the LHC and LSST, the most exciting research is being driven by those who have the expertise to apply high-performance data-parallel statistical algorithms to ask interesting questions of huge, community-generated datasets. It is driven by the application of new statistical approaches, of new machine learning algorithms, and of new and faster codes to repeat classic analyses at a previously unattainable scale. In short, the new breed of scientist must be a broadly-trained expert in statistics, in computing, in algorithm-building, in software design, and (perhaps as an afterthought) in domain knowledge as well. From particle physics to genomics to biochemistry to neuroscience to oceanography to atmospheric physics and everywhere in-between, research is increasingly data-driven, and the pace of data collection shows no sign of abating.
The development of specialised software is becoming ever more integral to the cumulative progress of scientific inquiry. So too is the openness of that software, with privatised development processes intensifying a broader crisis of reproducibility that afflicts contemporary science. The problem is that not only does higher education not incentivise researchers to spend time and energy on these increasingly important pursuits, it actively disincentivizes them from doing so:
This brings us to Academia’s core problem: despite the centrality of well-documented, well-written software to the current paradigm of scientific research, academia has been singularly successful at discouraging these very practices that would contribute to its success. In the “publish-or-perish” model which dominates most research universities, any time spent building and documenting software tools is time spent not writing research papers, which are the primary currency of the academic reward structure. As a result, except in certain exceptional circumstances, those who focus on reproducible and open software are less likely to build the resume required for promotion within the academic system. And those poor souls whose gifts lie in scientific software development rather than the writing of research papers will mostly find themselves on the margins of the academic community.
To an extent, disconnects like this have always existed. The academic system has always rewarded some skills at the expense of others: teaching is a classic example of an essential skill which is perennially marginalized. But there are two main differences that make the current discussion more worrying:
As I’ve mentioned, the skills now slipping through the cracks of the academic reward structure are the very skills required for the success of modern research.
With virtually the entire world utilizing the tools of data-intensive discovery, the same skills academia now ignores and devalues are precisely the skills which are most valued and rewarded within industry.
The result of this perfect storm is that skilled researchers feel an insidious gradient out of research and into industry jobs. While software-focused jobs do exist within academia, they tend to be lower-paid positions without the prestige and opportunity for advancement found in the tenure track. Industry is highly attractive: it is addressing interesting and pressing problems; it offers good pay and benefits; it offers a path out of the migratory rat-wheel of temporary postdoctoral positions, and often even encourages research and publication in fundamental topics. Most importantly, perhaps, industry offers positions with a real possibility for prestige and career advancement. It’s really a wonder that any of us stay in the academy at all
What do you think of this argument? We’d love to hear responses, particularly from the perspective of the more qualitatively orientated social sciences. Get in touch if you’d be interested in writing something.