Archive for: February, 2011

Cowboy and centralized research IT

Feb 08 2011 Published by under Research Data

The question of research-IT provisioning came up in my post on data-security horror stories. I saw some confusion from readers about it, and it's worth examining in detail for other reasons, so here goes.

So let's imagine Achaea University for a moment: immense, a diverse research agenda across many disciplines, lots of grants coming in, but some areas (often but hardly exclusively in the humanities) with no grant money incoming at all. How does Achaea U provision researchers with IT tools and services?

Achaea U doubtless has a central IT unit. At a minimum, it handles networking, campuswide administrative IT (payroll, HR, authentication/authorization, likely the course-management system, perhaps calendaring and email if those haven't been outsourced), and a lot of front-line student- and staff-facing IT (computer labs, campus wireless, helpdesk, webspace, basic web-accessible storage, etc). It may or may not have a learning-technology unit.

It almost certainly doesn't have a research-IT-specific unit. Such research computing services as it provides are of two types: repurposed other services (e.g. webspace), or pay-to-play services (e.g. specialized development teams). Big storage, if it exists, is almost certainly pay-to-play; you pay as long as you keep data on central IT's systems, and if you don't pay, central IT blows the data away. Such research-type services also tend to be "enterprisey" in their technical provisioning—which combined with pay-for-play means "serious sticker shock" for the average researcher, even the average well-funded researcher.

Services also tend to be lowest-common-denominator. If you have special needs, such as preservation past grant expiration or diamond-hard security? Tough noogies, chum. Central IT offers what central IT offers; you can take it or leave it. You can yell at central IT all you like that they don't know what the hell they're doing (and they may very well not; insular central IT units can and do gin up services that are convenient for them to provide, while not convenient at all to the intended user). Doesn't matter. Central IT offers what Central IT offers. Take it or leave it.

Most researchers leave it, which means no economy of scale, which means these services cost central IT even more than they need to—and since central IT is pay-to-play, well…

So Achaea U has a lot of other systems running research-related IT. For example, Achaea U does a fair bit of what's called "grid computing" (which has other guises too, but let that go for now). That's not run through central IT, because central IT was too big and ponderous and lowest-common-denominator to jump on that need (it's very hard, organizationally, for central IT to greenlight a service that not everybody on campus will use). Engineering or comp sci owns the grid, or it may have spun off into its own (likely pay-to-play, depending on the status of its internal grant funding) research/service enterprise.

And then we have the other end of the scale: a poorly-funded lone-wolf researcher limping along via a Linux server installed on a dusty beige consumer-grade box under his desk. If it breaks, he's humped, because it was set up years ago by a grad student who has since graduated, leaving no documentation behind, and he doesn't entirely know how it works. It hasn't broken. Yet. Is it backed up? Who the heck knows? Has it been hacked? Who the heck knows? Who the heck knows which networks it's even connected to, for that matter? The researcher sure doesn't. But he knows that his server (plus whatever free-to-him web services he tacks on to his processes) is cheaper by a factor of ten (maybe even a hundred) than equivalent computing provision from central IT! This, folks, is what I mean by "cowboy IT." Yee-ha! And there's a lot of it, scattered all over Achaea U! Yippee-ki-yi-yay!

It is, as I said, a continuum. Based on what's said in the Inside Higher Ed article, Dr. Yankaskas was very close to the cowboy-IT end. Somewhere in the middle, Achaea U has a few research-IT units that work on soft money for small or large groups of researchers. These units are more nimble, discipline-savvy, and responsive than Achaea U's central IT, and they're likely just as competent or more so (especially considering how little central IT knows about research-computing needs); the downside is that they're not as richly-funded and their funding is always in danger, so they probably cut some corners. The worse among them are no better than straight-up cowboy IT; part of the problem is that their staff may be selected by researchers who don't know jack about IT (as clearly happened in Dr. Yankaskas's case).

Plenty of Achaea U researchers, it must be said, can't even muster a cowboy-IT setup, when lack of outside funding combines with lack of skill. They are utterly shut out. Neither central IT nor research-computing units want them because they have no grant money to toss in the pot. The library may do what little it can, particularly for humanities scholars, but it's not enough.

So how do researchers get away with cowboy IT? Well, honestly, nobody's ever looked. It's that simple. And nobody looks because nobody much cares—until there's a huge, embarrassing screwup like the Dr. Yankaskas affair. (If this seems to resemble the laissez-faire IT environment that used to exist for social-security numbers in US universities? Quite right. Same causes.) Classic case of externalities: cowboy IT creates risks, sometimes serious risks to the researcher or even the institution, but mitigating the risks isn't perceived as important (and is known to be expensive) until there's a sudden crisis.

I expect the NSF data-management plan process to expose a shocking amount of cowboy IT in US science research, from the Achaea Universities among us to industry all the way down to the lone-wolves. I also expect the NSF will start to indicate gently that cowboy IT is not acceptable practice… and to become rather less gentle about it over time. This means that researchers will have to internalize risks they hadn't previously worried about, or they'll wind up like Dr. Yankaskas.

I don't entirely know what campus research-IT infrastructures will emerge from this. I wouldn't be celebrating if I worked for central IT; I have serious misgivings that central IT in its ongoing ignorance can even do this right. I'd rather see a mesh of the middles, growing collaboration among research-specific IT units to expand their services, service models, and funding sources to campus cowboys and have-nots. That's a tall order, though; funding models aren't clear, and these units think of themselves as independent fiefdoms, rarely valuing collaboration because of its added process overhead. It doesn't help that central IT will often fight to keep such a mesh from emerging, viewing it as a threat.

So we'll see. The bottom-line truth is that Achaea U will have to do better at research-IT provisioning in the next decade, or it'll start losing grant dollars to universities that work out how to do it right. Yippee-ki-yi-yay.

13 responses so far

Friday foolery: Mitigating repository risks

Feb 04 2011 Published by under Miscellanea

I have some zombie-ish tendencies of my own, so I was most interested to read such a thorough, well-researched investigation of Zombies and Risk to Repositories.

I feel ever so much better now that I know how to protect digital assets from the undead hordes. Don't you?

One response so far

Data-security horror stories

Feb 04 2011 Published by under Research Data

I'm afraid we're going to see more data-security horror stories like this in the next few years. It's truly horrific for everyone involved.

Rather than point fingers, because there are multiple levels of epic fail in this situation and nobody comes out smelling like roses, I'll try to pull out some more-or-less depersonalized morals-of-the-story:

  • Knowing why confidentiality is important is not the same thing as knowing how to ensure it, particularly in a networked computing environment.
  • Cowboy research-IT installations and their staffers must soon expect a fair bit more scrutiny than they're used to with regard to many important data-management questions, data security hardly least. These risks may well swing the pendulum away from cowboy IT (widely perceived as cheaper) back to more centralized, accountable systems and staff.
  • The buck stops at the PI. This means that the practice of leaving computing to the young ’uns and part-timers is not going to cut it any more.
  • If it's this bad in biomedicine, which is well-funded… I'm scared about everything else. Really. I may never fill out a survey again. (Okay, that's just because I hate surveys and believe that much too much lazy survey research is done, not least in librarianship.)
  • Policy, policy, where is the policy around data issues? It's years behind where it needs to be, that's where. And don't talk to me about IRBs (or NSF grant reviewers, for that matter; this is a serious and I hope temporary weakness in the NSF data-management plan model). IRBs are made of PIs, not the necessary gimlet-eyed informaticists and IT-security pros. If you've ever been on an IRB, be honest: would you have thought to ask about IT staff competencies?
  • Anybody who reduces research data management to "storage and backup" needs repeated applications of cold water and horror stories like the above one until they come to their senses. It's more complicated than hardware, people. Much more.
  • Ditto anybody (hello, librarians! hello, OAIS model!) who thinks that data management starts when the data are final.

Data security is serious business, especially now that reidentification risks have entered the picture. If you do human-subjects research, or work with any other sensitive data in digital form, take security seriously before you get caught flatfooted.

7 responses so far


Feb 02 2011 Published by under Miscellanea

So I go to a perfectly nice conference last year and get trapped by a volcano.

I try to get to OLA Superconference, and there's an epic blizzard. (I'm still going to try to make Top Tech Trends, but my Friday talk is cancelled. Pity. I put a lot of work into that talk.)

I dunno if it's safe to go on the conference circuit any more!

One response so far