Mysteries of Telegram Data Centers (2022)

This just makes me realize that three letter agencies of basically every country probably have an extremely easy time getting access to things by simply getting people hired as regular employees through mundane and routine means.

Good story, I yet believe the guy is trying to do the right thing. In the lex Friedman podcast he talks about banning extremist channels in both sides always, the story focus more on the Nalvani's block, but accordingly him he also bans other sides depending on the content. I do follow a number of Telegram channels about the Ukraine war, and the pro Ukraine channels are there together with pro Russian channels.

It's a great an telling investigation. Dropped in to share it as well. Telegram deserves no trust from us.

I remember reading about it, but I didn't know they had responded. Thanks.

You may have already read it, but portions of this sprawling (and very very good) essay by Neal Stephenson get into the topic: Mother Earth Mother Board - https://www.wired.com/1996/12/ffglass/

As a tangential niche-tourism shoutout: https://www.telcomhistory.org/ConnectionsSeattle.html

Granted, most of it is a bit newer than the heyday of telegrams, but there's a little bit of overlap.

It makes sense: European users are assigned to EU data center, and Chinese to the one closer to them. The "custom code" should not be complicated, just a map of country to DC.

You are suggesting to develop a compicated solution (spend money) when current simple one is working ok without any elections.

If you've ever actually tried to implement server clustering you quickly find there's no magic cloud, except in specific cases like blockchains. A privately operated cluster system is mostly about directing requests to the correct server out of a finite set of servers.

From what I have read, they only have ~30 employees. They're not exactly onboarding a lot of new people here.

Learn what? How to count to 5?

Most of big tech's major data centers are in Loudoun County, VA on the east coast not the west coast. It's centrally located to be great latency for the east coast and OK latency for west coast and Europe. Plus a friendly regulatory environment and lots of existing DCs (AWS us-east-1, Azure East US 1/2)

If you're feeling any better latency on the west coast it's more likely to be placebo than real.

They do not claim that. They do claim that they store specifically encryption keys in several data centers in different jurisdictions. Here is the exact quote: "All data is stored heavily encrypted and the encryption keys in each case are stored in several other data centers in different jurisdictions". So only keys are distributed.

It's more about the fact that five eyes intelligence services prefer to officially spy on each other's countries so they don't have to answer to their respective bureaucrats. They prefer plausible deniability.

Something like this:

   DC1 politically belongs to UK which "spies" on CA/US but physical servers are located in US so US ultimately retains control.

   DC2 politically belongs to France which "spies" on RUS/UKR/DE but physical servers are located in NL (e.g. in UK because one wouldn't be able to spot difference in ping). Maybe it's politically owned by UK/NZ or UK/AUS because France can't be publicly caught spying on Germany. But France wouldn't risk public arrest of Telegram CEO and the spectacle with russia if there is nothing to gain.

   DC4 politically belongs to USA which "spies" on UK/Israel but physical servers are in NL/UK

   DC5 politically belongs to UK/USA which "spies" on AUS/China/India but physical servers are in Singapore (e.g. former UK colony)

I love mentioning the UK in these kind of discussions because the pushback is biggest every time the Crown is mentioned, and ultimately US/CA/NZ/AUS are all colonies under the King.

Really cool to see realpolitik mapped out like this. It also highlights the problem of metadata with these kind of topics.

idk, they probably tried to get people on DC's as close to their location as possible. Using your phone number's country code might seem like a good way to do this at first, and they probably didn't give it much more thought before building the whole thing on this idea.

Agreed. I don’t see the appeal compared to Signal. Although, Signal is also sketchy with an operating cost in the tens of millions of dollars per year. Where does the money come from?

There's a lot of sketchy things about telegram but I'm confused, what exactly in this article seems sketchy to you?

wait ... when was it not sketchy? Did people not know the intentions behind it? Damn, wild. I get that there are some people that may use it as a communication medium for benign purposes, but it's filth all the way down.

The Lex Fridman podcast episode with Pavel Durov is worth listening to. Their servers are built to be very secure — of course, it would be different for others, and they use some clever tricks

Good ux, pleasant to use. A large community and lots of channels with all kind of content. Api is also great for spinning a bot for whatever purpose. I have for example a critical error bot for a production server running. If a critical error occurs, I get an immediate telegram message.

If you’re ok with abandoning all security and privacy, then Telegram has some nice extra features that can come in handy.

Calling/ringing works reliably. My gf usually calls via Telegram to notify me and then we call via Signal

Signal can't do live location sharing

    Code blocks (let alone syntax highlighting) was the first thing my friend ran into after switching last week

In general, formatting on Telegram is a lot more versatile with >quote blocks that show up formatted as quotes and inline links without needing to have a giant message because of the giant URL

Many more image features, like you can do yellow highlighting and not only ugly shades of olive, you can put a message above an image instead of below, you can mark an image as spoiler, you can send it uncompressed, much simpler UI for selecting multiple images, etc.

Jumping to a date was one of the first things I ran into

Web client lets you temporarily use someone else's computer without needing to install software

Making a Telegram bot takes 10 minutes if you just want a simple http callback and know roughly how it works. Signal has no support at all. Some third parties have made libraries but we keep having issues at work with our internal Signal bot to the point where I think we just gave up altogether now

Many people use Telegram to check if internet works because when opening the app it does a lightning quick check and it's very robust (I think it has no problem if DNS is down, for example) Looking at the title bar confirming that it works is much faster than trying to do a random web search

You can have memorable/pretty group URLs like telegram.me/OpenStreetMapOrg

Large files can be sent (also because of server-side storage not clogging up your phone, which is a double-edged sword of course)

Scheduled messages send at the right time when you're not online & the desktop app supports scheduled messages

As of like next week you can do moderation in groups in Signal. This has of course been a thing in Telegram since 10+ years. I don't know how we'd have run the OpenStreetMap group on Telegram without being able to delete spam messages

Chats take a lot less space. The Signal authors probably don't have many contacts that they use Signal with because you can view the message markers of 4 chats before having to scroll down. In Telegram, 9 fit in view. Similar on desktop but both numbers are bigger. Messages are also more space efficient in Telegram

Telegram so so so much faster than Signal's web, ahem, electron client

The only advantage of Signal that comes to mind is the intangible property of privacy (and maybe local back-ups, so data availability, but then Signal doesn't provide software to view your backup data so is it really a backup if you can't test it and they can lock you out of the app?). I'm nevertheless switching everything over to Signal but it's often very hard to explain to people why they should give all this up for encryption. My family doesn't use all the advanced features but some techy friends do

A good UX and a user base. I had it on my phone, but uninstalled after a while because not a single person I know uses it.

I think it was reposted today because https://core.telegram.org/bots/serverless was just announced, which prompted some curiosity on HN about Telegram's architecture (https://news.ycombinator.com/item?id=48918534). But yes, the title should be updated to indicate this.