I read this article1 and was moved to furious tweetstorm and several respondents asked that I make it into a blog post. This is that post.
Let’s dissect this quote, “They tired of care-taking, even though this involved little more than continuing to host the project files on a server.”
Firstly, it did not involve just continuing to host the files on a server. That server needs to get upgrades to stay secure and to be a solid foundation for future software development. Those upgrades may in turn mean the files must be upgraded to remain usable. Those server upgrades are themselves labor. Ensuring that the web server configuration and file permissions remain in a state where your files can be served is labor.2
If there are enough people who want to host “just a few files”, it’s a lot of labor. And server costs add up. The PHP or ruby or whatever programming language your dynamic files run on, that needs to be upgraded too. And again, this may necessitate maintenance so your files remain usable.
You probably have a database backing the system. Guess what, that needs maintenance too.
Upgrading those files to maintain compatibility is probably less than a week of work, assuming that the files come with documentation about how to deploy them and a test suite to make sure the upgrades don’t break key features.
Guess what?
If the team writing your “project files on a server” didn’t include software engineers, your files bloody well do not come with documentation and a test suite.
Given that the article includes neither the word “software” nor the word “engineering”, I know my hypothesis on this one.
(That AND the fact that when the author talks about tech work, she mostly talks about delegating it to part-time undergrads and the occasional solo grad student. These people may know how to code, but the chance they know engineering is near-zero.3 And even if they DO know engineering, their ability to practice it effectively, and stay up-to-date, as a solo practitioner is deeply limited.4)
OK but let’s back up a bit. I said, best case (remember: this is not the best case), that maintenance is probably under a week of work. Say, yearly.
If you have 30 faculty members who want to host “just a few files”, someone’s doing that full-time for 2-7 months a year.
I actually enjoy doing maintenance work — to a point. Most software engineers don’t. Necessary, but not fun. You ask people to spend too high a percentage of their time on maintenance, they hate their jobs, and they quit.
Also they stop having time for new feature development! Which, in addition to being more fun, is actually the thing that brings grant money in,5 and a thing that stakeholders are constantly clamoring for. Is anyone going to hire another FTE to keep up? Nah.
Alternatively, they look the other way, letting “just a few files” languish unmaintained, and if they are any good they KNOW in their hearts the bitrot, the attack surface. They work in a place that doesn’t value quality and if they’re any good it eats them.
I used to fantasize about being able to sunset things. I inventoried every “just a few files” on a whole set of servers. There were 243 of them. There were, at most, 6 of us. We couldn’t kill them. Because faculty like this might complain.
Some of them were critical services. Most of them were running on obsolete versions of whatever they ran on. We didn’t have business owners, documentation, tests. No principled way to transition or upgrade them. No SLAs giving us permission to kill them in 5 years.
“Just a few files.”
TWO HUNDRED FORTY THREE PROJECTS. SIX OF US.
If you want to talk sustainability for software projects — first, talk to an actual software engineer. Maintainability is part of their expertise.
And second, the labor conditions of your techies’ work lives are an important part of that plan. If your plan rests on making their jobs suck, it is not a sustainable plan. Or one that anyone should feel good about.
Thank you for coming to my TED talk.
Footnotes
[1] “Sustainability and complexity: Knowledge and authority in the digital humanities.”
Johanna Drucker. Digital Scholarship in the Humanities, Volume 36, Issue Supplement_2, October 2021, Pages ii86–ii94, https://doi.org/10.1093/llc/fqab025.
[2] For example, the actual months, on and off, I spent trying to deal with five years’ worth of deferred maintenance coming due at once on one of my personal projects. I was able to wait that long to touch it because my code had good tests and the deployment system had been set up responsibly by the very capable Andy Dorner…but the result was, when I couldn’t avoid doing maintenance any more, 1) everything was broken, and 2) I was dealing with a system I wasn’t super familiar with, because I hadn’t been the one who set it up, and Andy and I were no longer coworkers, so I had limited access to his expertise.
[3] I say this with tremendous respect for graduate students. I teach them, and some of them have been brilliant, hardworking people whose work has already had, and will continue to have, significant impact beyond the classroom. Nonetheless, the difference between writing code and doing engineering is significant. Just as having read Romeo and Juliet (however insightfully!) does not mean you can stage a Broadway production, knowing how to write scripts that achieve particular tasks does not mean you know how to make those scripts usable by others, flexible around changing requirements, easy to debug, or performant under increased load. Nor does it mean that you can document them clearly, figure out where and how to deploy them, automate that deployment system, write effective tests, or work with others. These are skills people generally develop through experience, by working as part of a community of practice on large, shared projects over time. Research code — code written in the course of academic work, for the sake of prototyping something or generating data for a paper — often does not have these characteristics. Indeed, some of the worst code I and my colleagues have encountered has been written by people with Ph.D.s in computer science. They may well be brilliant in their areas of research, but computer science and software engineering are very different. Just as my professional experience has given me little formal background about algorithms and I couldn’t write a proof involving big O notation to save my life, academic training rarely provides the type of experience needed to develop engineering skills.
[4] Truly, I cannot emphasize enough how much the community of practice matters here. The critical habits of mind — things like “writing tests is important” — all come from being embedded in a culture that values them. The techniques and tools for carrying out these habits of mind come from coworkers, conferences, techie Slacks, blog posts, hallway conversations. The ability to keep up with those techniques and tools as they change, doubly so. Asking a software practitioner to work solo inherently undermines their skills. If you support a team that, for whatever reasons of budgetary reality, has only one such person, I implore you to budget for them to attend conferences regularly, and to make time for them to participate in a professional community via conferences, local meetups, and/or open-source participation; this is the most effective professional development you can do, and will also be good for their mental health.
[5] This is a problem that funders are starting to grapple with, typically under the framing of sustainability. I know Mellon has this on its radar, for example, and I want to gratefully recognize this trend. That said, there are also real down sides to funding operations on soft money (see, e.g., “sustainability” here, or the Collective Responsibility white paper on contingent labor in libraries, archives, and museums). Ultimately, good technology work rests in the institution having both a commitment to employ adequate labor to sustain core services, and a funding model to pay that labor.
andromeda yelton 