A closer look at the Air Force’s in-house software development
A few months ago I wrote about Kessel Run, the Air Force effort to both promote agile and bring more Air Force software development in-house. After the blog came out, I was contacted by Lt. Luke Sutherland of the 517th software engineering squadron at Hill Air Force Base in Utah (part of the Ogden Air Logistics Center, recently renamed from the 517th maintenance squadron), who wanted to talk about things they were doing in these areas. I wanted to focus in our discussions on how the Air Force is looking at in-house software development. After some delays on my end and a number of conversations, I am ready to report what I learned.
In-house software development was once notably more common in the federal government than it is now. Agencies such as the Social Security Administration and the IRS employed many developers (then more commonly called “programmers”), as did the Department of Defense. This disappeared in the 80s and 90s as Cobol, which was what the existing programming workforce knew, was replaced by newer languages and the government was unable to recruit new people who had that expertise. (There were no special IT hiring authorities in those days.)
In-house software development in the Air Force is now an obscure corner of the federal IT landscape, and is occurring mostly at three air logistics centers with software engineering units — Warner-Robbins in Georgia, Tinker in Oklahoma, and Ogden. (The perceived success of Kessel Run has led to a number of other Air Force bases exploring in-house development operations.)
The Air Force’s in-house software development has grown out of the sustainment mission of air logistics centers. But they did some development for software embedded in weapons systems from the beginning. Although the history of Hill’s association with software involved embedded software for weapons systems, some of the work the 517 currently does involves web-based applications. The first example involved a system for missing servicemembers. The work was being done by L3, but the Air Force was dissatisfied with the contractor’s performance and put the contract out for bid when it expired. Hill won.
There are five software development squadrons at Hill. The 517th employs 120 software developers and has annual revenues of $32 million. Together, the three bases doing software development employ a total of 4,500 developers and have revenues of $1 billion, about equal to the annual sales of Dyncorp or CGI. Currently, the biggest projects for the 517th are for ground systems software for satellites and next-generation intercontinental ballistic missiles.
What is the case for in-house development, I asked Sutherland? He sees it as quite straightforward — cost and technical data rights. The 517th hasn’t done systematic cost comparisons, but Sutherland tells the story of one situation where the center was working on a project for a civilian agency. The agency initially solicited a big defense contractor to compete for the work, he noted, but “they said it would take 12 months and cost $1.1 million — we bid two weeks and $66,000.”
The other advantage, Sutherland said, is that the government owns data rights for all in-house software. By contrast, Sutherland does not list as an advantage of going in-house gaining IT technical expertise that can help with post-award management of contracts with firms. Hill does not have any such contracts, since it does all development work in-house.
Outsiders sometimes suggest that in-house prices go down because they charge only for direct labor, not overhead. This is not true for Hill. They submit indirect costs as part of the price for work they do, and another squadron at the base calculates indirect costs.
The developers at Hill are all civilians; none is uniformed. How is Hill able to hire developers, given the salary differentials with the private sector? “Our biggest selling point,” Sutherland said, “is work-life balance. We have a healthy work environment — vacation time, fitness leave, holidays. There are lots of activities in the area, such as hiking and skiing. And they are doing something that will really affect people’s lives, they have an impact on the mission of the Air Force.”
Also, he said, “we don’t just develop web apps — that’s another plus working for us. Another selling point is having lots of projects, so if one project doesn’t fit your style, you can transfer over to a different one.”
The local university has a good computer science department, and the base hires undergrads as interns. “We have a set of people who go out to local colleges and to job fairs,” Sutherland said. “We also advertise, and get referrals from people who work out here. The Silicon Slope is an emerging software hub, so the employee pool is larger.” For many of the employees, this is a first job out of college.
Sutherland says that most of the training is on the job, but that the unit also trains people in new languages and new tools, using either Air Force trainers or outside companies. It is not possible for me to know how the intensity and quality of training compares with those for developers at private government contractors or companies that have developers.
One big worry I have is that Hill seldom needs to compete for work. Hanscom Air Force Base, over which the three air logistics centers operate, assigns work to the centers after an internal process. Even the five software development units at Hill don’t compete with each other — work is assigned to them based on a decision of what best fits the squadrons’ capabilities. So competitive pressures are less than would be the case if there were competition, even just inside the government. And it is harder to determine whether Hill’s indirect costs are unrealistically low (or high, for that matter).
What should we conclude from this story? I don’t think any sane person would want to go back to a major role for in-house developers in government, although we should note that government is currently an outlier compared to private companies, which generally do some development in-house. But the right amount of in-house development is probably not zero either.
I think in-house development should be an option in the government mix, depending on factors such as how robust the local IT skills market is, what are the comparative costs, how important data rights are to the government, and how attractive the work is (whether from an intrinsic job attractiveness perspective or a mission perspective) that in-house developers would do.
Shortages of people to do IT contract management should also play a role. In-house labor rates could be a yardstick for contracted rates. Industry believes that it is important to have in-house technical IT skills to evaluate the work product of contractors, and government should believe this too.
We shouldn’t rush to imitate what the Air Force is doing, but we shouldn’t reject it out of hand either.
Published with permission from Fcw.com. This originally appeared in fcw.com: https://fcw.com/blogs/lectern/2019/07/kelman-usaf-software-development-in-house.aspx.
