▲ 149 points
back
169 comments
We moved back and forth between AWS and GCE (based on who gave us free credits). Once we ran out, we chose GCE and never regretted it.
GCE has many quirks, for instance the inconsistency between API and the UI, it misses the richness of the services offered by AWS but everything GCE does offer is just faster, more stable and much more consistent.
One of the biggest problems with AWS is that once you outgrow the assigned limits, it becomes a hell to get more resources from them. We're running on average around 25k servers a day, majority of them preemptible (spot). AWS requires that you request exact type and location for your instances. GCE only asks for a region and then overall resources (e.g. number of CPUs).
Also the pricing is much less complicated. 1 core costs y, 32 cores cost 32*y.
> AWS requires that you request exact type and location for your instances
Are you running 25k without using Spot Fleets? Spot fleets let you specify value per instance type, and then total value you need. ("Value" could be CPU, memory, network or whatever) AWS will maintain the lowest cost spot instances that fulfill your requirements.
Your post has motivated me to blog some details about the difference between AWS EC2 Spot Instances and Google Cloud's Preemptible VMs:
https://medium.com/@thetinot/google-clouds-spot-instances-wi...
(work at Google Cloud)
> We're running on average around 25k servers a day
25k number of servers or its the cost of the servers ?
Based on which criteria do you preempt your servers? 25K sounds like a lot !
At 25k servers / day one would think pulling this into real hardware and doing capacity planning would be cost effective.
I have serious trust issues with Google. Their history of discontinuing services, dismal support (even for a paid service), and neglect of bugs in SDKs/APIs - all three of which I have experienced first-hand - has left a long-term bitter taste. No doubt some individuals are fantastic, but the organisation as a whole gives me an enduring impression of being systematically arrogant and aloof.
AWS by contrast have demonstrated fanatically helpful support (even on business level, the cheapest), fixing issues within days of reporting, and a willingness to maintain obsolete/deprecated services (like SimpleDB) long after I'm sure they'd wish everyone had migrated away.
It seems to me that Google is highly siloed internally. The GCP team is very visible (they engage on HN and elsewhere), and GCP has excellent support. Very different from other parts of Google.
I have problems with Google as a whole, but I have as much faith in their cloud as I have in AWS or Digitial Ocean (we currently use all three).
The Kubernetes team is also highly engaged on Github (they also occasionally show up on the official Slack channels). The Go community suffers from a depressing abundance of hostility, intransigence and arrogance, and some of Google's projects (e.g. Protobuf, the Go project itself) reflect this, but I was delighted to find that the Kubernetes people are not like this at all. It's a very friendly, quality-focused community. I think they have to be since Kubernetes is still emerging tech that's craving adoption. Same goes for GCP -- Google doesn't hide the fact that they're aggressively courting customers to migrate.
This thread of comments from a year and a half ago, on the article "How Amazon took control of the cloud", is really epic on these points.
One reason why I still can't use GCE in 2016. No PostgreSQL support for CloudSQL.
I can find alternatives for other services, but I don't want to compromise on the choice of relational database.
Note: I understand there are third-party providers for PostgreSQL, but I'd rather have Google's.
Googlers on HN have commented before that they're working on it. No ETA, though.
With AWS now offering both plain-old PostgreSQL and souped-up-PostgreSQL-on-Aurora [1], whatever Google produces needs to be great in order to compete. However, I fear they'll initially come out with something that's on par with the current MySQL support in Cloud SQL, which is just a vanilla MySQL server behind a UI/API. (For example, Cloud SQL's read-replica stuff is reportedly just MySQL binlog replication.) Better than nothing, of course.
Cloud SQL also has some annoyances (such as not supporting private IPs and the need for the Cloud SQL Proxy [2]) that I hope they're working on.
[1] https://aws.amazon.com/blogs/aws/amazon-aurora-update-postgr...
This is on my short wishlist for GCE. It's such obvious feature parity that it's very strange they're not doing it.
Totally agree, Google Container Engine (hosted kubernetes) plus managed Postgresql would be my dream setup.
Have you found any compelling hosted Postgres RDS alternatives?
Two points here really hit home with me about AWS (not in comparison to GCE though since I've never tried it).
1) Reserved Instances: I think the pricing model for this has become very outdated since the beginning of AWS, and it is definitely becoming cumbersome (and therefore scary) to use.
2) ELB + Traffic Spikes: I have tried (unsuccessfully) to pre-scale an ELB to prepare it for the traffic it was about to receive. I tried to pre-scale for this project 3 different times, in coordination with support and without them. I could not do it. Very frustrating.
I think these are all signs of extreme growth, and a strange organization of engineering units inside of AWS. However, as OP descried.. we are much to heavily invested in AWS to consider an infrastructure shift at this point
We also manually sync up with AWS support to "warm up" our ELBs... I guess they don't expose an option to avoid abuse, but it really does seem like an implementation caveat.
Can you talk more about ELB + Traffic Spikes?
Under high traffic, ELB will fail?
How do you pre-scale the ELB?
Many thanks.
God I hate any piece of writing that doesn't define its acronyms at least once. Google Compute Engine isn't popular enough that people should be expected to know immediately what it means.
Also, GCE is a service while AWS is a suite of services. The correct comparison should be Google Cloud Platform (GCP) vs AWS. Guess it's nitpicking.. but still.
This entire blog post is rife with other small issues. AWS support is not a 10% minimum at all! Developer Support is 3%.
> AWS Premium Support is mandatory
Is Google Cloud support even acceptable? Google is known for poor or no support for most services.
Google's known for poor or no support on their free services but their paid services often have decent support. Personally, I've found GCP, GSuite, Project Fi, Google Store and Pixel support to all be pretty great but haven't found any support at all for Gmail, YouTube etc.
Actually it's surprisingly good. We use them a lot and even when we were on the silver package (the cheapest) they were pretty quick (under 1 hour for P1 situations [production is impacted]).
Thought I would chime in here since I didn't see anyone comment here about gold. We have GCP gold support. They answer all the questions / issues quickly and there aren't any limits to ticket counts I am aware of. During both the outages we have been affected by (~1.5 hour google load balancer outage, >3 hour bigquery outage) in the past few months, the support feels pretty bad even though there is nothing they can do I guess. However during the bigquery one, it was "outage will have an update by some time". There proceeds to not be some update for well past that time. Also 3 hours later its still we will have an update by some time but no more info when both of these were past SLA. Overall feels good and they answer any weird / general questions too which is nice (although sometimes on every reply they say I am going to close the ticket).
Whenever I've needed to contact Google support for Gsuite, they've been excellent.
Support for things like the Nexus devices and anything you can buy on their store is very good.
Also, Google My Business support is also pretty good.
AWS premium support is the best support experience I have encountered so far and should absolutely factor into choosing a cloud provider. Reading frequent stories of Google support nightmares across all their services makes me think twice about using GCE. GCE must find a way to counter this.
Also, the AWS premium support fee is negotiable for some customers from what I have heard. They don't like to negotiate down, though!
Strange. I'd qualify them as the most useless support I've ever encountered.
In the year 2016, among maybe a hundred tickets, there was only ONCE where they could change something (an ELB issue).
And well, I'm not sure whether the fix was related to their changes or if it was just an intermittent error that happened once. Thus their implications in the only time something happened has yet to be proven.
My biggest complaint about AWS is still EBS and having to guess about for the right provisioned IOPS. Throw in confusing extras like EBS optimized instances, enhanced networking. GCE just abstracts away all these details.
This was referenced in the previous thread, but here is what amounts to a stl;dr:
> "Unfortunately, our infrastructure on AWS is working " > "I learned recently that we are a profitable company, more so than I thought. Looking at the top 10 companies by revenue per employee, we’d be in the top 10."
I'm a bit interested in what their company's response would be to this article.
previous discussion at
Looking forward to the day Google Cloud provides on-demand GPU-backed machines. Currently AWS is the only game in town for that, as far as I know.
Coming early 2017: https://cloud.google.com/gpu/
What about N-series VMs on Azure? https://azure.microsoft.com/en-us/blog/azure-n-series-genera...
What reason is AWS premium support mandatory? I ask because I'm currently building out SaaS offering on AWS and haven't yet hit any issues requiring support. Can I expect to start seeing issues as traffic scales up to a certain level?
I've used AWS for a bit over 9 months now and it's quite terrible to be honest.
I don't need it for anything professional and it's quite terrible for just some amateur hosting plus the immense fees if you somehow manage to get decent traffic together.
Once my reserved instances run out I'll probably either check out GCE or DO, either seems to be a better option, though GCE seems to be more expensive.
Anyways, the console in AWS is a mess and I'm quite sure that I leaked my entire IAM settings to the internet because some switch somewhere isn't set right.
Since everything recommends to setup IAM users you'll have to setup the permissions, a procedure which I enjoyed about as much as getting my fingernails slowly removed by a glowing red iron.
Calculating any sort of sustained cost is a pain in the backplane if the total doesn't exceed three digits a month.
And lastly the login process is probably the biggest pain I've encountered across many many providers. There are atleast 4 login forms I've discovered, 2 of which I have to use and one of those always asks for a captcha with such low quality that a brain-damaged AI running on my calculator could figure it out, not mentioning never knowing if the 2FA setup was correct or maybe probably blew up somewhere because giving some feedback from the UI is plain impossible.
TL;DR Don't use AWS, anything else is better.
Evaluated GCP, and 2 main issues made it hard to consider moving:
1) quickly bumped into project limits just doing some tests, and the fact that you have to wait until billing cycle to reset the counter was quite jarring (I presume there's a way to increase)
2) Better tooling for S3 than Google Cloud Storage - non-technical members of our team need to work with files, and there's many nice third-party tools for s3.
I really like OVH's SoYouStart in terms of its pricing for CPU/memory intensive computations, their prices just destroy AWS/GCE: https://www.soyoustart.com/ca/en/essential-servers/
Because of speed of light/latency issues, I won't move to GCP until they have an RDS-postgres equivalent
Having used AWS in production for about 2 years, I can say that scaling in the cloud is hard to get right. Spikes of traffic and load are hard to scale with, especially if you're scaling down to just the right amount of instances to manage the current load. We try to keep 50% capacity free for such spikes. Speaking with AWS support, we can fix issues with limits and scaling elbs, in hours not weeks, so I haven't had the same experience as you in that respect. One thing I would suggest, is that you have 2 AWS accounts. One for dev and one for prod. This way you should be able to see the limits that you're hitting before you get to production, and can raise tickets to get them ammended.
GCE has its own issues. The raw price is different than the real/total cost. In a better world you shouldn't have to choose between just a few cloud peoviders.
You should use AWS if you _really_ need something like an infinitely scalable database such as DynamoDB or some other uber-scaling AWS service that you can't replace with some open-source software on a VPS provider. 95-100% of startups don't actually need an infinitely scalable database or whatever. So they shouldn't subject themselves to Amazon's horrible pricing and throttling.
GCE - piece of shit in case of support and solving customers issues. They didn't manage to FIX billing issue for 8!!! months. Support is useless. Missed access rules boundaries - everything you have to share with everyone. It's nightmare for companies.
The cost comparison link is broken.
I'd be interested to know why the OP never considered Heroku