back

by jasonpeacock·13y ago·view on hn ↗
Regardless of any technological or architectural decisions, simple load testing before launch would have prevented this.

1. Load test, discover issues. 2. Fix issues. 3. Repeat steps 1-2 until no more issues.

That is the flabbergasting part. Nobody actually tried applying real load to the system before release? It fell over so easily and quickly the only assumption is either no loadtesting was performed, the results were ignored, or massively incorrect load numbers were used (all of which are signs of incompetent mangement).

2 comments
The official story right now is that they underestimated. Medicaid has had at max 30k users; they doubled the estimate for hc.gov, but instead of ~50k users they're seeing 250k. From what I'd read (in a story linked from this hn page) they did test for load, but nowhere near 200k users.

FWIW, I don't think the issues right now are load so much. I think there's some insanely bad logical pieces that are messing things up, and possibly have corrupted accounts that were created early on. I base this on a few things.

1. The "username" requirements.

The username is case sensitive. Choose a username that is 6-74 characters long and must contain a lowercase or capital letter, a number, or one of these symbols _.@/-

Really? I know some systems use case-sensitive usernames, but given that you're already forcing some odd characters and whatnot in there, why not normalize to lowercase? This just feels like it's going to cause more support problems (mobile safari automatically uppercasing a username that should be lowercased, etc).

Also... the English description of that username is nowhere near intuitive. "A lowercase or capital letter". As opposed to what other kind of letter?

2. Multiple accounts with a single email address.

I've been able to 'successfully' register multiple different username accounts with the same email address. When I do a password reset based on email (the couple times it worked) which username was I resetting the password for? I say 'successfully' because no login attempt has ever worked. And now trying to register yet another username with the same email address doesn't work, but the error message is so vague ("there was a problem" IIRC) that I can't tell if that was a factor or not.

OK... 2 things. That's about as far as I've been able to see in to the system so far, so that's all I can judge it on. But it seems that they've probably allowed some logical inconsistencies in my own signups that may be causing more problems now, and I don't think they are related to load.

Of course, I could be 100% wrong, but it doesn't feel like load is the culprit right now.

I immediately get an error page after logging and and trying to view plans. That could be load but it certainly feels like some kind of logical error to me.
Exactly. They've got the 'waiting room' technique to slow down apps/logins. The login process, once done, is quite fast in getting me to a blank page. (https://www.healthcare.gov/marketplace/auth/userprofile page in question)
They probably load-tested based on the number of sign-ups they expected, and that estimate may even have been accurate. They don't seem to have realized that lots more people would be interested in what's available, even if they aren't ready to enroll. If they had realized that, you wouldn't have to "apply" before seeing the plans, and the plans could have been static content, distributed all over a CDN by now.