Bret Davidson and Jason Casden wrote this Code4Lib journal article I adore, “Beyond Open Source: Evaluating the Community Availability of Software”.
From their abstract:
The Code4Lib community has produced an increasingly impressive collection of open source software over the last decade, but much of this creative work remains out of reach for large portions of the library community. Do the relatively privileged institutions represented by a majority of Code4Lib participants have a professional responsibility to support the adoption of their innovations?
(Protip: yes.)
Davidson and Casden then go on to propose some metrics for software availability — that is, how can the developers producing this software estimate how installable and usable it might be for institutions which may not themselves have developers? The first of these is:
Time to pilot on a laptop. Defined as the time needed to install and configure, at minimum, a demonstration instance of the application, particularly for use in organizational evaluation of software.
Well! I now have an alpha version of the Measure the Future mothership. And I want it to be available to the community, and installable by people who aren’t too scared of a command line, but aren’t necessarily developers. So I’m going to measure the present, too: how long does it take me to install a mothership from scratch — in my case, on its deployment hardware, an Intel Edison?
tl;dr 87 minutes.
Is this good enough? No. But I knew it wouldn’t be; the priority for alpha is making it installable at all. And this took about two days; my sysadmin-fu is weak, and the Edison is a weird little platform that doesn’t have as big a software ecosystem as I’m used to, or lots of handy internet documentation (as far as I can tell, I’m the first person to have ever put Django on an Edison).
It’s going to be an additional chunk of work to get that 87 minutes down – it’s hard to make things easy! I need to bundle what I have into a shell script and package up the dependencies for faster installation (luckily fellow MtF developer Clinton Freeman has already done some work on packaging dependencies for the scouts, which will speed things up for me). The goal here is that, after end users have installed the operating system (which will unavoidably take several steps), they’ll be able to run one command, which will then take care of everything else (possibly prompting users for a little information along the way). After that, download and installation will probably still take some time (a lot of bits need to move across the network), but that time should be unattended.
Anyway. That’s the plan! I’m making a public record of my 87 minutes here so you can see how much progress I make in future, and also to provide some insight into the very real work that goes into streamlining build systems.
andromeda yelton 
question: are you planning to test the timing by someone who is NOT a developer? i.e. your target audience? Because I think that will be VERY necessary.
LikeLike
I totally agree that I am not the right person ultimately to be testing this.
That said, you’d have to talk with Griffey about ultimate user testing plans; they’re out of scope for my contract. I hope they’re in the plans 🙂 but I don’t have any end users to test on. The reason I’m doing this test on myself is to throw a gauntlet to myself, basically. The current install process is *really gross* – it’s not the one end users will be using, and it’s unacceptable for anyone who isn’t very comfortable at a command line. I want to measure for myself precisely how bad it is in order to underscore the work that needs to be done in terms of simplifying the install process, and also to appreciate the magnitude of the work after I’ve done it.
At *that* point, user testing will become really important and I hope it ends up in the road map somewhere. But *this* install process…really no one other than me and the MtF team should be subject to it in the first place.
LikeLike