New CMS BPA showcases non-traditional IT contractors
Recently the Center for Medicare and Medicaid Studies awarded a BPA for work on Medicare Payment Systems Modernization. The MPSM work will replace a system using 10 million lines of Cobol and assembly language with a new cloud-based system, according to a description on the U.S. Digital Service website to “scale and flex rapidly with how CMS processes claims and pays providers, transforming technology from a policy inhibitor to a multiplier.”
I was astonished to learn that all six awards on the BPA went to non-traditional contractors — loosely defined as small new firms, often founded by people who had worked previously in the private-sector tech industry. These firms usually are very oriented to the missions of agencies with whom they work, and have a culture whose core competence is engineering more than business development or the ability to respond to proposals. They also tend to have agile software development in their DNA rather than being a recent add-on. Of the six, I have earlier presented three in various blog posts. And of the six, an astonishing five are members of the Digital Services Coalition, a support group for non-traditional contractors, about which I blogged over a year ago.
The awardees currently have a handful of federal prime contracts, and employ between 30 and 200 people. The subs are even smaller. One was founded a year ago and currently does only commercial work. Traditionally, it wouldn’t have interest in government work (I was told “too much red tape, labor rates don’t support their talent”), but the civic tech movement has attracted that firm to the space, as its leaders very much want to support important missions and agencies, like improving healthcare. Another was started by an Air Force vet who worked for Deloitte prior to starting his company. He wanted to help veterans and felt that he could do it in a more human-centered way that adds more value for users and taxpayers than he was doing a much larger company.
It turns out that this was the second time CMS had awarded a BPA to non-traditional contractors. The first, awarded in late 2016, is Adele, whose scope includes using agile software development for quality improvement work. That BPA was the brainchild of Dan Levenson, a CMS contracting officer who had just recently graduated from the Digital IT Acquisition Program run by USDS and the Office of Federal Procurement Policy.
Adele almost didn’t happen. The CMS Office of Information Technology, which at the time was using a big single-award IDIQ, tried to stop it. However, Levenson got support from senior management at the Center for Clinical Standards and Quality, the intended owner of the BPA; those leaders wanted to try something new after the repeated failures of the agency’s traditional approach. “The memory of Healthcare.gov was pretty fresh at the time,” he notes, “which acted as an incentive to try something different.” USDS provided top cover to experiment. Levinson reached a compromise with the OIT people, whereby he was authorized to compete the BPA for a more limited scope.
Levenson did significant market research before issuing the procurement, mainly involving GSA, to learn about non-traditional contractors in the marketplace. CMS then invited 12 such contractors to bid, but also said those not invited could submit bids as well. Without Levenson, an award to five non-traditional contractors would have been unlikely.
A few years later, Adele had come to be regarded as a big success. Many customers outside the BPA’s home brought in work. The Government Accountability Office was so impressed with the speed and quality of the work under the BPA that it asked CMS to explain the secrets of the success. For the MPSM BPA, OIT moved from critic to champion, and would take the role of sponsoring and funding the BPA. And CMS’s solicitation seeking expressions of interest got 73 contractors to sign up to compete.
CMS used GSA Schedule 70 for the competition, which is the vehicle most open to new, small vendors and doesn’t require the bidder to be an awardee on any existing IDIQ.
The highlight of the procurement was a “tech demo,” where bidders would work on a design challenge in real time, and submit what they had developed. “Do not say” is the watchword compared to traditional written proposals, which center around “say.”
The procurement used an advisory downselect, which was authorized in one of the last regulatory changes I made in 1997 just before returning to Harvard from the Office of Federal Procurement Policy. In stage one, bidders provided references and information about relevant experience, limited to 10 pages, which was then the basis for the downselect. Companies could give commercial references as relevant experience, something that many agency solicitation documents do not allow (it makes it easier for new vendors), though this practice seems to be becoming more common. Of the 73 entering stage one, only eight were invited to participate in stage two (though the rules allow those not downselected to bid anyway in stage two, and apparently two did so). This very aggressive downselect allowed CMS to avoid the near-chaos at DHS when the organization opened its first tech demo to everyone bidding and received 114 bids.
The basic problem presented in the tech demo was as follows: “Imagine that you have come upon the large monolithic legacy system responsible for executing and maintaining the workflow that processes Medicare claims in order to make payments to service providers. The maintenance of the system is proving to be unsustainable as time continues. We want you to modernize this system. In addition to the existing payment processing system, there are other parts of the agency that are involved in experimental payment workflows that traditionally operated outside of the legacy payment system. We want those experimental workflows incorporated into this modernization effort. This will require the integration of two very different work streams, development processes, and development lifecycles.”
The RFQ specified no requirements, but included “criteria” that would be evaluated for companies’ submissions: sustainability (“the system, resources, and processes must maintain their current levels of service during modernization efforts”); agility (“the system needs to evolve, sustainably, at the same speed as the business”) integration (“systems involved in making payments, and other CMS business functions, need the option to use modern technology to interact with the current state of the legacy system”); usability (“users’ interaction with the legacy system need to easily identify and extract the exact data they need, when they need it”); robustness and safety (“the Government will evaluate the robustness and safety of the system, noting how it responds to errors and the safety of modifying the system”); security (“the Government will methodically evaluate the security of the system, as well as looking at ways the system might be extended or upgraded and how that may impact the security of the system”); and code quality and maintainability (“the Government will evaluate the solution on the quality of the code, how easy it is to understand, update, extend and maintain”).
At the end of the tech demos, there would also be a “prototyping exercise to demonstrate how the (bidder) would start to solve a set of system and business owner needs related to users of the system.” The RFQ also stated that “during evaluation, the Government may trigger arbitrary failures to see how the solution responds.”
Bidders also were required to share their prototype as source code for working software.
The tech demo was not the only thing evaluated for award. Evaluation of the quality of the company’s submission from phase one on earlier work was actually presented in the RFQ as being more important than evaluation of the tech demo. Second-phase bidders were also required to submit a document (maximum 20 pages) presenting the company’s “agile technical approach document, and evaluation of this was of equal importance to the tech demo.” I asked one person familiar with the effort if the weight given the technical approach document departed from the “do, don’t say” philosophy of tech demos, and they responded: “I think this is reasonable if the evaluators are experienced enough to differentiate boilerplate text from showing real technical understanding in writing that is centered on the agency and on the scope of the BPA.”
For this tech demo, bidders were given a two-week time period to work developing a prototype, and they could have anyone work on it they chose. The work was done on the company’s site; they didn’t need to come to a government location.
Other tech demos have been 4–8 hour efforts on one day at a government sites. There are upsides and downsides of each approach. The two-week effort can produce something that is real and can be tested, which seldom happens with a one-day effort. It is also more like a real agile sprint, less like a hackathon. The pros of the shorter demo include being able to observe a team in action, which provides the government decision-makers some information about team dynamics they can’t get otherwise. The one-day events on the government’s site give the government greater control over efforts by firms to bring in talent who are unlikely to have anything to do with performing the work after award. One concern about all tech demos is the government’s ability to evaluate them, given the shortage of technical IT talent. In this case, evaluation was done by USDS staff, but this is not a scalable solution. There is obviously less to evaluate in a one-day effort. The cost of a one-day demo is generally less, because although there are airfare/hotel expenses for the event, there is far less need for expensive developer/engineer time.
Tech demos have often been promoted as a way for young firms unfamiliar with the complex world of responding to government RFP’s to bid on work, and it also is sometimes promoted as a way to streamline source selection, because bidders do not need to prepare lengthy documents. Above all, it is promoted as an embodiment of “do, don’t say,” as contrasted to traditional RFP responses. In the case of this competition, the first and third claims are true, but not necessarily the second. If you add on the time doing the design challenge itself to that preparing the written material for this procurement, it does not seem to have been less resource-intensive for bidders. (I asked two of the successful bidders about this, and one said it was more work than a traditional RFP, and the other said it was less work.)
Interestingly, there were no protests. (Adele had one protest, from a more-traditional contractor, but it was resolved before going to adjudication.) One might suggest that non-traditional firms are less protest-prone and that the entry of more non-traditional firms into the marketplace may reduce the government’s exposure to protests.
New firms entering the federal marketplace, including non-traditional firms, often wonder how they can know if an agency might be interested in what they have to offer and thus that it is worth it to bid. So I asked a number of the winners how they decided if the government might be interested in making awards to non-traditional contractors. Two winners I spoke with said their firm looks for a reference in the RFP to the Digital Services Playbook that USDS and the Office of Federal Procurement Policy have developed, and two others looked for references to “user-centered design” or “user research” or “minimum viable product.” Another winner I spoke with said his company looks at “who in government is fun to work with — we work hard to see who’s doing fun things.”
The CMS source selection is an early sign that being a non-traditional contractor is becoming a positive brand name. We may soon get to the point we are already at with agile, where everyone starts claiming to be a non-traditional contractor. As this situation has spread for agile, the Defense Innovation Board actually issued a document called Detecting Agile BS, which I blogged about at the time, to warn against this. This is somewhat easier to guard against with claims to be non-traditional, since this is unlikely to be convincing for a government contractor who’s been around for 30 years. But for newer firms, the government needs to look at evidence of the contractor’s mission orientation and agile cred.
As I mentioned at the beginning of this blog, five of the six BPA winners are members of the Digital Services Coalition support group. In my next blog, I will present an update on what that group is up to, and also present one of the non-traditional winners of the BPA that I haven’t presented before.
Published with permission from FCW.com. This originally appeared in FCW.com: https://fcw.com/blogs/lectern/2019/07/kelman-non-traditional-contractors.aspx
