back
116 comments
A bunch of smart Silicon Valley hackers cannot fix what is wrong with government IT. Government IT problems stem from the fact that government IT contracts are worth Billions, and they are doled out through privileged networks of insiders.

There are plenty of smart companies in America that can fix America's IT problems if they are paid to do so. There is no political will to do so. That is why companies you never heard of come up again and again as subcontractors for things like Healthcare.gov

Believe it or not, there are some very smart people working in government. They actually don't need us in Silicon Valley to tell them how to do stuff. They know what they could do with the latest technology. They are not allowed to do so because of bureaucracy.

"There is no political will to do so."

That was true a couple of years ago, but healthcare.gov was a wakeup call, and everybody from the president on down takes reforming government IT seriously.

I was part of the healthcare.gov "tech surge" and I can tell you firsthand that the government is willing to change. In the last year, the healthcare.gov UI was streamlined from 76 to 16 screens for the average user. Large parts of the site now run on AWS and more is moving. The most problematic individual component, the login system, was rewritten from scratch with an API-compatible, drop-in replacement which now serves 100% of traffic. And this has been done within the same agency responsible for the original site. It is not a story of "smart hackers" parachuting in and wresting control away from government bureaucrats, but rather partnering with the agencies and creating reform from within.

USDS itself, although started by the White House, now has bipartisan support and a real budget to grow. Although you are right that there are powerful entrenched interests in the beltway, the political will to change is there, and now the challenge is to recruit people like Andrew to seize the opportunity.

> I was part of the healthcare.gov "tech surge" and I can tell you firsthand that the government is willing to change.

So, you worked for a short while on an extremely high-profile project with the personal daily attention of the president. Yes, that does tend to smooth things over - but don't mistake that for a fundamental change in government priorities. There are 2.6 million federal employees, you don't re-jig politics and incentive structures that has been entrenched over decades overnight with a single successful project.

Sure, showing that it can be done is always a good first step, and I'm not saying that anyone actively wants to fail - but only just barely. More often than not it feels that a failure you can blame on someone else is better than a mediocre success (good luck running an election campaign on having reduced the number of screens in a system or migrating it to AWS).

I hope you are right, but from my experience with government IT I feel its highly unlikely that a top down approach by a bunch of smart people is going to break through all the horseshit IT fiefdoms in the US government. Billions of dollars are spent every year on government IT stuff, and all those vendors aren't just going to sit quietly while you try to replace their garbage, overpriced BS software that they are charging an arm and a leg for.

Maybe if the President has his personal eye on a project something could get done. But the President isn't going to put his personal eye on the USPTO back-end for example. Or the FAA exemptions submission page. Or any of a thousand other points of contact between the public and government.

> Large parts of the site now run on AWS and more is moving

I'm curious what, if any, security obstacles you faced with this, especially considering HIPAA and all that.

A large part of the current GOP (and to a lesser extent Dem) platform is built on the idea that government cannot under any circumstances (except defense, because magic) do anything successfully. How can there be successful reform when so many political careers are now based on this assumption? Success has become failure, I am not hopeful.
Here's the difference. I've talked to some folks in the digital service. They aren't career government. They have no risk in committing "political suicide" by suggesting something that everyone knows is the right thing to do. Basically, if someone doesn't like what they said, they can respond with, "well I can always got back to my old job". The government workers don't have that freedom.
Government IT contracts are much less insidious than it seems.

The reason you have the same companies has very little to do with "insider" status and much more to do with the fact that these companies have made it their business to be able to operate in the federal environment.

Bureaucracy, far from being a conscious, malevolent tool, is simply the accumulated barriers and obstacles erected over time for every instance there was an anomaly or problem. Accountability was demanded, and new guidelines put in place. These stack up over time, long after the original incidence has ceased to be a major issue. Unlike the private sector where companies and organizations can simply be wiped out and start with a clean slate, government doesn't (and for various reasons, can't) do that.

There's still money to be made however, for those who are familar with the process. Simply being eligable to bid for government work means getting approved on a contract vehicle, complying with various laws on accountability and record keeping, and ticking off various checkboxes for equal representation of genders and minority groups. These regulations are removed and unfeeling of special circumstances or facts on the ground; they are the accumulated "good intentions" of laws passed that now stack up like strata.

People who know this system well are intimidated by all of this, which is why you see many government and defense retirees move on to the private sector to steer these small companies into this kind of work. They know how it works, and just knowing that is a skillset in and of itself.

It's not evil. And there's far less cronyism than one might suspect. (I'm not claiming for a moment that it doesn't happen, but somewhat surprisingly, it's minimal.) Improvements are not consciously forbidden, and making things better is not discouraged. But like an enormous traffic jam, no single person has room to do much of anything. Coordinating a number of people to try and move in a better direction can improve things a little, until they are stopped by another group who has no room to move.

It's a mistake to think of government IT as insidious. It's a problem we have built over time that we have created ourselves. To fix it will require enormous bravery to do away with volumes of regulations that, by themselves, probably did some good. And it will require tolerance of practices done in private that would never have seen the light of day, but in government, has the spotlight of oversight shining on. It will be a challenge to solve.

You did not succeed in convincing me that the system you describe is not insidious. I think you probably describe it accurately, and it sounds quite insidious to me.

Yes, of course the companies that get government contracts need to be experts at getting government contracts -- the problem is that they do not need to be experts at what they actually _do_, what they're getting paid for. There's in fact very little room left to be experts at what they do, after all the expertise they need to develop (and resources spent on same) in getting and keeping contracts.

If you're a company and your motive is making a profit, then of course there's plenty of money to be made, exactly.

The system you describe sounds pretty insidious to me. "Insidious. adj. proceeding in a gradual, subtle way, but with harmful effects."

The harmful effects are exactly the privileging of expertise at government contracting over expertise at doing a good job. That's harmful only if you prioritize doing a good job, if you prioritize making a profit I suppose it's just fine.

> But like an enormous traffic jam, no single person has room to do much of anything. Coordinating a number of people to try and move in a better direction can improve things a little, until they are stopped by another group who has no room to move.

Yes, quite, that's quite insidious, exactly. It has some unique features to government, but also some commonalities with any large bureaucracy.

"Improvements are not consciously forbidden, and making things better is not discouraged. But like an enormous traffic jam, no single person has room to do much of anything. Coordinating a number of people to try and move in a better direction can improve things a little, until they are stopped by another group who has no room to move."

That is a great metaphor. Anybody who's considering joining USDS, I think ckozlowski's comment gets the situation exactly right. You are unblocking a traffic jam.

To be fair, Andrew lives in DC and has for a while afaik. Not sure why this is being posed as Silicon Valley fixing government IT. It seems like a few smart people trying to make government better. Nothing is going to be perfect, but it's at least a step in the right direction. I respect Andrew taking this step, there's a lot of civic hacking in DC, this is just a step further and making it a full time gig from the look of it. At least he's doing something instead of simply whining.
Ehh. The UK's Government Digital Service successfully made http://gov.uk/ which was a massive improvement over what it replaced (DirectGov), making the UK Government's services easy to find and easy to use. Why can't the US Digital Service? It seems to be trying to do the same sort of thing.
"Believe it or not, there are some very smart people working in government. They actually don't need us in Silicon Valley to tell them how to do stuff."

I know -- very condescending, isn't it? Gee, it's a good thing these really smart folks are showing up. What would we have done without them? A broken system must obviously be full of incompetent people.

I wish these guys the best, but this is 90% bullshit and 10% reality.

Just to cite one, small (?) problem: in commercial IT, if you don't like the work somebody is producing? You fire them. In government IT, no such thing exists. Managers spend years shuffling around poor-performing employees in a desperate attempt just to keep the lights on.

I could go on, but it would all just wash over folks. This is a movement, dammit! We all feel so very excited! This time is different!

So I will help do whatever I can -- blog, comment, debug, whatever. But I will not become suddenly blind to the realities of the system that we are attempting to change. That doesn't do anybody any good. That ain't cynicism, that's reality. Hope is not a strategy, and you can't fix a system you can't honestly describe. (or understand how it got that way)

Hi Daniel, thanks for your perspective.

My own perspective: There are amazingly smart people working in government, and even working for many of the contractors commonly cited as examples of The Problem. Many of these individuals are not effective not because they're incompetent (and therefore need to be told what to do), but because they are not empowered, or they lack confidence or a team of people they can use to propel good solutions forward. Bureaucracy and internal politics are rampant. One way to look at what we're trying to do here is to act as a nucleus of tech expertise inside these agencies, to attract and empower those individuals to do the right thing while providing air cover from the White House when they're prevented from doing so.

That being said, there is a vanishingly small number of actual engineers working for the government. Normally this role belongs to the contracting companies. The incentives here frequently encourage overly large, excessively complex designs, and implementations that maximize the number of humans rather than good engineering practices. By improving technical literacy within government we improve the government's ability to identify bad contracts and methods to prevent these situations from happening in the future.

We are not blind to the fact that there have been other attempts before us. The key differences I see are (a) support from the top and (b) funding.

Sadly, part of the cause of this is due to the small government mantra preached by "conservatives". This results in many tasks being offloaded onto contractors who end up bloating the cost and using various other backchannels.

If a lot of these things were done in-house in the government itself, things would be a lot more efficient.

Why would the lack of efficiency suddenly disappear if these tasks were brought in house? Jesus, the VA system in in house and look at that mess.
I am really not clear on the structure of the "US Digital Service."

The phrase "US Digital Service" makes it sound like a "reified" civil service organization in its own right. There is a USDS Headquarters, which seems to support this notion.

In the copy[1], however, it says that the bulk of the work is done by "agency teams" which, if I'm understanding right, kind of "embed" with their assigned gov't agency. The copy also says they're recruiting founding members for new agency teams. The phrase "founding members" makes it sound, almost, like the teams are independent operations but knowing the federal gov't as I do, that can't possibly be right.

Does anyone here have any light they can cast on whether the USDS is an agency unto itself? If not, what cabinet secretary does it fall under? I have a ton of questions about how this works.

edit: My biggest fear with the USDS is that it can be smothered in the cradle by the next administration. There simply doesn't seem to be enough employees to make dissolving it -- or, maybe more likely, starving it to death budget-wise -- a big deal.

[1] https://www.whitehouse.gov/digital/united-states-digital-ser...

Structure wise the Digital Service is made up of a headquarters team within the White House, 18F which is an internal software development organization within the General Services Administration, and then embedded teams growing to be within each cabinet level agency. This creates a mixture of distributed execution plus centralized resources to help each team as needed.
(I work for the Digital Service @ the VA)

This is correct. The HQ team is a part of OMB, the White House's Office of Management and Budgest.

The agency teams can be thought of as a franchise model, we work closely with the HQ team and the other agency teams, but we're also a for real part of the agency we work at.

(Just realized who I was replying to, welp!)

Your profile blurb has a typo in it.
I, too, would happily consume any content about the USDS, how it's structured, what it's plans are, what its focus is, etc. Specifically, I'd like to learn what I can do now to start thinking about the problems of scale and procurement on a government level. It's an area I don't know much about, but the noises made by Mikey Dickerson and his team have gotten me sufficiently inspired to want to learn more and build out my skillset as to eventually help the initiative in any way I can.
I recently went to a recruiting event of theirs in SF, and here is my personal understanding, probably wrong in spots:

The broad notion is that private-sector IT culture has undergone major change in the last couple of decades. The failure of HealthCare.gov under the old model and its subsequent turnaround using the sort of modern methods common in Silicon Valley woke people up to the possibility of doing something new.

The question, then, is how to get government working like we do. (At least where it's beneficial; we solve a lot of our mistakes by shutting down companies, but government departments can't work like that.)

A number of things have been tried, starting with the Presidential Innovation Fellows. [1] The US Digital Service is the latest pass. The USDS HQ is under the US CTO, which I believe makes it part of the Office of Science and Technology Policy [2], which is part of the Executive Office of the President [3]. The agency teams are embedded in the agencies themselves, and so I'd assume those are legally under the appropriate cabinet secretaries. I get all this mainly from the USDS pages. [4]

Personally, I suspect that this is not a lot of money. It's mainly people, and they made clear that they will be paying industry salaries, not government salaries. And given that it is getting government to reduce waste and learn from the private sector, I expect it has bipartisan appeal.

Even if the next administration dissolves it all, though, I expect it will still have a fair bit of effect. As anybody in the industry knows, the switch from 18- to 36-month cycles to 1-4 week cycles is pretty much permanent because the culture has shifted. People just expect stuff sooner now.

But I hope the next administration continues the effort. At the meeting it was clear there were a lot of people in tech who really want to go serve their country by taking a couple years to make the government run better. It'd be foolish not to take advantage of that yearning for public service.

[1] https://www.whitehouse.gov/innovationfellows

[2] http://en.wikipedia.org/wiki/Office_of_Science_and_Technolog...

[3] http://en.wikipedia.org/wiki/Executive_Office_of_the_Preside...

[4] https://www.whitehouse.gov/digital/united-states-digital-ser...

I would be extremely surprised -- and skeptical, actually -- if there is a USDS with its own budget, but the agency teams serving under the secretary. There would be two masters there, and that's just fertile breeding ground for major frustration. (This is also not how inter-unit embeds work in the military, which is what colors my perspective.)

Regarding pay, I dunno. Based on quotes from this[1] article, the pay scale seems to cap out at GS15 Step 10 which is ~132k. Now obviously this is still a lot of money but if I'm reading between the lines of his quotes right, this is the most an engineer can make w/ USDS without bumping up to SES (Senior Executive Service) levels. Not to say that's substantively less than industry salaries, but they definitely are paying government salaries (by definition).

As far as dissolution, I'm talking more from the perspective of attracting & retaining top talent. With a private sector startup, you trade job security for potentially out-sized payoff. But here... Anyway, point being I think the risks should be articulated well.

[1] http://fcw.com/articles/2014/10/27/red-tape-holding-back-hir...

> I recently went to a recruiting event of theirs in SF...

Could you please share how you discovered said event? I'm curious enough to want to attend one.

So, I interviewed with the USDS. Very smart people, seemed like it'd be a fun and challenging gig. Then I was told two weeks later that, oops, they didn't actually have a req for the USDS at all when I had interviewed, would I like to go work for the Department of Veteran Affairs? (No, no I would not like to work for the Department of Veteran Affairs, the list of things I would rather do than that is quite long and includes stapling parts of myself to other parts of myself.)

Felt like a shady as heck bait and switch. Can't speak to the work they do, but I was not thrilled at their hiring practices.

I'm sorry you had a bad experience. I am sure it was not their intention to bait and switch; I've forwarded your comment to somebody at USDS so that they can make sure things like this don't happen anymore.
This isn't a one-time thing, let me tell you about my experience. (Throwaway, for obvious reasons.)

But first a little about me: CS degree, straight to engineering at blue-chip tech companies, did full stack development, then moved to product management, with a bit of project management experience jumbled in. I'm probably not the guy you want banging out 20K LoC/day on the Linux kernel, but I am the guy who can sketch out both the UI and the architecture of a new product, hold the big picture in his head on the one hand and still speak intelligently to specialists about minute details on the other hand. Fixing all the broken systems in government sounded like a dream job at the time. I thought I was a good fit from everything I heard.

I initially reached out to you, Brandon, around the second big round of press that USDS got -- summer/fall 2014ish. You helpfully put me in touch with J.

J rescheduled our calls twice. It's fine, shit happens. When we did get on the phone though, after explaining the above (with more evidence), she asked me if I wanted to do web-dev with 18F. No, I did not want to do web-dev with 18F, and there's very little in my background to suggest I'd be a good fit for web development. She flat out told me she didn't think I could hack it at USDS, but when I asked where she thought I wasn't a good fit, she backed down and said she'd get an engineer to interview me.

Next thing I know, I get an email from H @ 18F, who I understand is a manager there, and says that J said I wanted to speak to her. Okay, fine, I figure what's the harm in a conversation? I respond that I'd be happy to chat.

A month goes by, no response to 2-3 emails to J (or H).

Finally J responds, so sorry, got busy, etc. Alright, fine, shit happens I guess. The call with the engineer is set up. The engineer and I speak, technical stuff, no phone-coding though (yay!). Seems to have gone well. I give it a week, no sign of life, so I email: nothing. I let two more weeks go by, email again: nothing. Two months later (by now I was just doing this for the curiosity): nothing.

So. Maybe there's political will. Maybe they're bipartisan support. But I doubt you're hiring the best and brightest. Because hiring the best people is job #1 of any leadership team, and the best and brightest know that if this is the hiring process, run by deputy directors of these organizations, with the deputy CTO of the US cc'd on all these emails, and it's _this_ bad...well, it can only be even worse on the inside. And all the desire to give back and serve your country falls apart when you know that organization is going to hamstring you when it comes to the "giving back" and "serving your country" parts of the job description.

Oh, but what about H/18F, you're wondering? Over the course of 8 months (half of that time radio silence), I've scheduled 5 phone calls with her, and she's missed every single one, most of them without a courtesy email saying she'd have to cancel before the call. It got to the point where I was scheduling them during times I knew I'd be driving or waiting around at places because I didn't want to waste anymore hours sitting by a phone that was unlikely to ring.

So yeah, there was no "bait and switch" here, just the "bait and". Not that I'm irritated by this or anything...

And now the parent is probably blacklisted for life for speaking out about it.
I'm sorry about this miscommunication and it wasn't intentional--we absolutely consider the Digital Service team at the VA to be part of the USDS; we helped build it. Ellen Ratajak (new eng director at VA) is one of the examples listed on whitehouse.gov/us-digital-service

Hiring bright people isn't enough, we have to empower them by building critical mass on impactful projects with executive support. We were focused on creating a team at the VA when you applied (still are, if you finished stapling yourself), but there are other options now that didn't exist then.

Well, they've created a set of minimal-information web pages with big type and useless clip art, using Wordpress. That's so modern.

More usefully, they're creating a large number of minor software components and putting them on a Git server.

https://www.govcode.org/repos

If this means the government's moving everything to Wordpress...
My thoughts exactly.

"Oh cool, maybe some digital natives are going to beable to bring development practices in government into the 2010's. That would be a cool thing to be a part of! ...Wordpress?"

/closetab

A big chunk of the govt has been moving everything to Drupal for the last 5 years.
On the bright side, it's probably a lateral move as far as government application security goes. And if they're migrating from, e.g. DotNetNuke, probably a net gain.
I'm pretty sure people go into government with the intention of really changing it from the inside out, this time things will be different.

The evaluation period for efforts like these should be a decade or more, that would be a good time to review and see if things have really changed and if the change stuck or if the machine turned out to have too much inertia after all.

Great to see a lot of interest and desire to help from the tech community. Most of what I've heard during the last few years regarding the government was a lot of complaints and a desire to get rid of it. I'm glad that there are people doing something positive instead of whining.
Congrats! I feel like too much of what I read on here is about despair and how TPP is going to eat us and Ted Cruz is going to steal our lunch money. Good for you for taking a positive step.
Sorry, but the lead developer of WordPress belongs in, I don't know, maybe the office of the Press Secretary (since they're so great at marketing), but definitely nowhere near anything anybody depends on.
So the White House got a new intern? How is this news?
Disgusting that anyone would receive praise someone for assisting an entity which tortures and murders.
"an open, transparent, and efficient government" is a blind alley. While the tasks you will be doing are not criminal, you are working for criminals. You might reject my radical statements out of hand but if you are open to why I say this and want to know more please let me know.
Only _some_ aspects and activities of government are criminal.

We still need someone to build roads and make sure the water stays clean. Arguably, we also need an education system, an oversight of healthcare, a social safety net system and many other things. Working in these capacities helps society in general, and everyday citizens to have better lives.

There are way too many ineffective people in bloated bureaucratic stasis managing these things at the moment, and very much poor and overpriced execution which costs all of us in terms of services received and increased tax burden.

Helping to improve the efficiency and effectiveness of these socially important tasks is a very worthwhile use of work time and is not "working for criminals", as the recipients of this work are the citizens of the country.

Even though some of the people operating the government are criminals, the function of government is necessary (as it is for all large human organizations).

"Open, transparent, and efficient" may be a lofty goal, but it's still important for civilians to get involved in moving the government toward that goal.

Nothing is black and white. You seem to be unwilling or unable to see the gray in your chosen area of extremism.

Congrats, now just don't follow the lead of the secret service by drinking on the job and letting people over the fence
I think this is good for Nacin. He's helped lead on of the most popular open source projects on the Internet for years and a well-paid (am I being too optimistic?) job which comes with a bit of autonomy, opportunity for mastery, and a sense of purpose is definitely well deserved.

However, I do empathize a lot of the sentiments in the hidden-by-downvote comments on this thread.

Although Nacin will probably never directly support any of the government agencies responsible for torture, drone-murdering innocent civilians in other countries, or the unprecedented surveillance state that has emerged in the wake of the terrorist attacks 14 years ago, he has still willingly become a cog in the machine of the US government.

Whether he manipulates his environment or gets manipulated by it remains to be seen. Hopefully, he is able to do more than make sure the US government's human-rights-violating machines remain well-greased.

For those who are concerned: From my limited interaction with him, I got more of a Jedi vibe than Sith side, if you catch my drift.

I don't understand this perspective of if you're willing to work for the government you're joining the dark side. If you aren't willing to work freely to make your own government better, what good are you as a citizen? He's willingly started a new job to try to improve people's lives. You want to make the government better? You have two major options, voting and joining the government and actually taking responsibility for some tiny piece of it.