▲ 144 points
back
77 comments
I'd like to call out the somewhat related problem of website rot. Meaning, the websites is online, it once worked perfectly, but becomes increasingly dysfunctional due to technical deprecations.
The soft obligation to use HTTPS these days has deranked old HTTP-only websites in search, making them hard to find. These websites are also "defaced" with browser warnings or some subresources may not load at all.
Embedded maps no longer work, since Google regularly breaks their API.
Facebook login or other FB plugins no longer work, since it needs a yearly checkup of your account and there's the new requirement of needing to have a privacy policy.
Those are just some examples of websites partially breaking through no fault of its creator, if you'd agree that the web should be backwards-compatible.
Hi, I happen to work on Google's Maps JavaScript API, and we actually take great pains to avoid breaking our APIs as much as possible. We take very seriously the implications that breaking changes can have for websites all across the internet, and know that many sites are not under active development.
This page covers the breaking changes from the last few years: https://developers.google.com/maps/deprecations#completed_de... (Keep in mind that the top portion of the page covers features still in the deprecation period - meaning they still work. And that this page covers deprecations in other APIs like Android and iOS.)
My favorite: the deprecation period for v2 of the Maps JS API lasted 11 years after the introduction of the v3 Maps JS API.
Happy to hear about experiences to the contrary, but I thought a little insight might be appreciated.
Thanks for responding, you're a good sport.
Indeed I'm looking at this from a long time frame. From recollection the main breaking changes I'm aware of:
1. The change from v2 to v3, like you mentioned. 2. The new requirement to always include an API key. 3. The visual update to touch-sized controls, distorting especially tiny embedded maps.
Y'all have done great work. I have a page that uses the Maps API - haven't touched it in ~7 years. No problems.
also, even if those older http sites get a certificate, any embedded scripts that point at http URLs, even if those URLs are also available on https, will not load and break. Especially hard to fix if you have user generated HTML on there.
Same for http download links from https websites, will not work anymore.
anything that used cookies on embedded resources will also be broken because of the missing samesite header (same for third party cookies obviously)
if google really goes through with removing alert/prompt/etc from their browser like they said they were planning to a while back, so so many things will break
I'm uniquely bothered by this problem as I visit many such dysfunctional sites.
I'm active in the (hobbyist) field of documenting species. There's thousands upon thousands of websites created by amateurs containing unique niche content. For example, somebody might have made it their lifelong hobby to document every species of bee in their territory.
It's a fragmented mess of incompetently produced websites, but I find it incredibly charming and in the spirit of the original web. Above all, it is their content that has lasting value.
The people behind it are good, generous. That's why it makes me so angry when their work is cast aside like this. Things not just technically breaking, in many cases simply disappearing altogether from search results.
Notification prompt is 10x more annoying than alert ever was.
But they are the fault of the creator -- first, by including content entirely reliant on the whims and generosity of 3rd parties wholly outside the author's control. And secondly, using the content and not doing the minimum needed to maintain it. Would you also say it is "no fault of the creator" if s/he did not renew the domain registration?
It's a bit like mowing your lawn once and assuming you're entitled to a perfectly manicured lawn forever.
That is a perspective with zero empathy.
These people are amateurs with near-zero resources generously sharing their content with the world. They may have all kinds of reasons for being unable or unwilling to continuously update their site.
The bigger point is that when you create a website according to web standards, it should keep working indefinitely. As for 3rd party integrations, especially when they're enormous (Google, Facebook) they should do the maximum possible to not break anything. The very point of an API is that it's a contract.
Some of my favorite websites about cycling were created by now deceased webmasters. Luckily they are mirrored, but I’m very thankful that they used regular old HTML and images and aren’t rotting after a mere 11 years.
Wouldn’t all this not be an issue with static websites? We should have more static
No. Static vs. dynamic is a concern regarding how HTML / CSS gets rendered by the backend. HTTPS certificates, embedding of 3rd party scripts, outdated HTML / JS / CSS,... are concerns that live entirely in the browser.
Whether you use a SSG or a CMS, you still need to do upkeep of all of those: make sure your certificates are up to date, make sure you have the latest embed script, make sure you keep your site isn't build on top of brittle API's and frontend frameworks, make sure your site keeps following changing SEO practices, etc.
On another note, if you use a CMS - e.g. WordPress - the availability of your site is directly tied to the operational availability of the CMS and the stack it depends on. So, that means upkeep and maintenance of the CMS become immediate concerns. If you use a SSG that only spits out HTML / CSS, that's far less of a concern: if your SSG breaks, the site doesn't break down.
Of course, YMMV in the age of hybrid solutions where a static layout and a popular JS framework is send to the client which fetches fragments via GraphQL API's from a backend, which implies a coupling between frontend and backend.
> due to technical deprecations
This struck me as a wolf whistle, and the immediate follow-on paragraph confirmed that suspicion. HTTPS enforcement is a major migration, yes, but other than IPv4 (sadly a long way off) it's the only "technical deprecation" of any kind I can think of that websites have dealt with. Implying these kinds of deprecations are commonplace, rather than 2 in 30-40 years, is a little odd.
> Google [...] Facebook
Embedded maps & Like/Share buttons are not "technologies", they're product offerings from service-providers. Businesses change their offerings over time, this is common in all industries & not a problem specific to websites / the internet / tech.
If you want to get that technical, then even https enforcement is the changing of a product offering.
> Implying these kinds of deprecations are commonplace, rather than 2 in 30-40 years, is a little odd.
Large parts of the web are powered by npm.
The vast majority of those sites, if not actively developed, will not be able to build within a year's time.
I've been working on this problem for a while. Website upkeep is hard to quantify, but basically every disk fails and every operating system eventually needs a serious upgrade. The timeframe that a system can run continuously is not that long compared to the timeframe that information is relevant. So the most lightweight way to keep something up and running is to make it trivial to port to many hosting configurations by simplifying the toolchain needed to rehost it. (Note that humans are part of that workflow, if it's a company)
I've written a manifesto about making a commitment to keep websites online and maintained for 10-30 years, for people who are maintaining web content: https://jeffhuang.com/designed_to_last/
And on the flipside (from a user's point of view), I've also been working on a background process that automatically captures full-resolution screenshots of every website you visit, creating your own searchable personal web archive: https://irchiver.com/
I've personally been trying to make a commitment to keep my web projects and writing online for 30 years. My original internal goal when I started thinking about this, was to outlast all the content on Twitter, Google+, and facebook.com. One of those has already been met, kind of sadly.
I think it's less a technological problem and more just that everyone who used to care about that site or its content no longer does. Or the company behind it has gone out of business -- who is going to maintain a website for a defunct organization, and why would anyone want to?
There's no obligation for a person to maintain anything longer than he wants to do it. Putting a blog online is not a lifetime committment. Interests change, or you simply realize nobody much cares about your online musings, and you move on to other interests.
When a company goes out of business you will typically also try to sell of all its assets, and a good domain name might be such an asset.
But what can you do when you do care, to make your website as durable as a printed book?
It’s not uncommon to hear of rack servers with several years of continuous uptime. I wouldn’t be surprised if you could keep a website online for a decade without touching anything by using an LTS distro, enabling unattended upgrades, and running something like nginx.
> system font stack that matches the default font to the operating system of your visitor
If the list just matches the default font of the OS, is there a difference between using such a stack vs. not specifying a font? Wouldn't the browser naturally use the OS's default if font-family is unset?
> I've also been working on a background process that automatically captures full-resolution screenshots of every website you visit, creating your own searchable personal web archive:
How are screenshots searchable? They aren't plain text. You can't grep them.
Why not though. On MacOS you can select text in bitmaps nowadays. So the tech is there to make a grep for pictures.
The tool captures screenshots in addition to text.
Interested personally in the font stack point. The system font stacks listed focus on font styles more than individual typefaces. If I want say, Bookman, is there any harm to long-term use to use a font stack tailored to that font?
There isn't, no, and there's no particular harm in self-hosted web fonts when it comes to long-term stability. Not only is it relatively unlikely that future browsers will deprecate web fonts but not anything else in current CSS (in which case there's a bigger issue than your fonts), it's even less likely that they'll do so in an entirely incompatible fashion. Assuming you've designed your font stack with decent fallbacks -- which you should -- then it's just like any other font stack, e.g., you may not be presenting in the typeface that you really want to, but nothing's going to break.
("Web fonts are okay, actually" is evidently the windmill I will keep tilting at on Hacker News.)
irchiver seems incredible. I hope there's a comparable product for other OSes one day.
One of my ideas is called Wantsfiles at the root of domains. The Wantsfiles contains information about what you want and the price you're willing to pay. An order matching system matches your Wantsfiles.txt to Sellfiles.txt and transacts on your behalf.
So I could host link to my tgz on my Wantsfile and offer $5 to host the tarball a month. There we go a perpetually hosted website.
You could call it autohosting.
disk failure and os upgrades are a non-issue with the right technology stack.... but then you have other problems (more complicated setup)
Hm interesting, I read over your 7 guidelines [1] and I would say I agree 50%.
So the most lightweight way to keep something up and running is to make it trivial to port to many hosting configurations by simplifying the toolchain needed to rehost it -- I agree with this, although I would use the words "standards" and multiple implementations. The linked article doesn't appear to emphasize this.
1. Return to vanilla HTML/CSS
Again I feel the relevant issue is "standards", multiple implementations, and the fallback option of "old code" like GNU coreutils (i.e. taking advantage of the Lindy Effect). Not just just HTML/CSS (which certainly meet all of those criteria.)
I thought about this when designing https://www.oilshell.org/ and the toolchain
- The site started in Markdown but is now standardized on CommonMark [1], which has multiple implementations. So I don't see any reason to stick to HTML/CSS.
- The tools are written in Python and shell [2]. Both languages have multiple implementations. (Ironically, Oil is a second implementation of bash! My bash scripts run under both bash and Oil.)
- Python and shell rely on Unix, which is standardized (e.g. Linux and FreeBSD have a large common subset that can build my site).
This is perhaps unconventional, but I avoided using languages that I don't know how to bootstrap like node.js (has a complex JIT and build process), Go (doesn't use libc) or Rust (complex bootstrap).
On the other hand, C has multiple implementations, and Python and shell are written in C, and easy to compile. I'd say Lua also falls in this category, but node.js doesn't.
I feel like this is at least 60% of the way to making a website that lasts 30 years. The other 40% is the domain name and hosting.
This is a pretty a big site now, you can tar a lot of it up and browse it locally (well it's true you may have to rewrite some self links).
Of course, this is my solution as a technical user. If you're trying to solve this problem for the general public, then that's much harder.
[1] https://www.oilshell.org/blog/2018/02/14.html
[2] https://www.oilshell.org/site.html
2. Don't minimize that HTML
Mildly agree if only because it's useful to be able to read HTML to rewrite self links and so forth. Tools don't need it, but it's nice for humans.
3. Prefer one page over several
If you're allowed to use Python and shell to automate stuff, then this isn't an issue. I suppose another risk is that people besides me might not understand how to maintain my code. But I don't think it needs to be maintained -- I think it will run forever on the basis of Unix, shell, and Python. Those technologies have "crossed the chasm", where again I think the jury is still out on others.
4. End all forms of hotlinking
Yes, my site hosts all its own JS and CSS.
5. Stick with native fonts
Yes, custom fonts have a higher chance of being unreadable in the future. I just use the browser's font preference to avoid this issue. It's more future proof and in keeping with the spirit of the web (semantic, not pixel perfect).
6. Obsessively compress your images
Agree
7. Eliminate the broken URL risk
I think too few people are using commodity shared hosting ... I've been meaning to write some blog posts about that. I use Dreamhost but NearlyFreeSpeech is basically the same idea.
It's a Unix box and a web server that somebody else maintains. I absolutely don't care about CPUs, disk drives, even ipv4 vs. ipv6, and I've never had to.
The key point is writing to an interface and not an implementation. Commodity shared hosting is a de facto standard. The main difference between a tarball of HTML and a shared hosting site is that say "index.html" is always respected as /, and a few other minor things.
So I expect Heroku and similar platforms to come and go, but the de-facto standard of a shared hosting interface will stay forever. It's basically any Unix + any static web server, e.g. I think OpenBSD has a pretty good httpd that's like Apache/Nginx for all the relevant purposes.
Github pages also qualifies.
So I guess this is adding sort of a programmer's slant to it. To be honest it took me a long time to be fluent enough in Python and shell to make a decent website :) Markdown/CommonMark definitely helps too. I had of course made HTML before this, but it was the odd page here and there. Making a whole site does require some automation, and I agree that lots of common/popular tools and hosting platforms will rot very quickly. (and like you I've seen that happen multiple times in my life!)
And I think what your guidelines might be missing is a guideline on how to make "progress". For example CommonMark being standardized is progress that has happened in the last 10 years. You don't want to be tied to the old forever. At some point you have to introduce new technologies, and the way to do that is once there's wide agreement on them and multiple implementations. (just like there's wide agreement on HTML)
I think there is / can be progress in dynamic sites too, so you don't have to stick to static!
It comes down to the person running the website has to care. That's it. It doesn't matter how simple it is if the person doesn't care.
In my own case, I've been running my own website for 24 years now [1]. The URLs I started out with have remained the same (although some have gone, and yes, I return 410 for those) and the technology hasn't changed much either (it was Apache 24 years ago, it's still Apache today; my blog engine [2] was a C-based CGI program, and it's still a C-based CGI program. The rest of the site is static, and there's no Javascript (except for one page). I can see it lasting at least six more years, and probably more. But I care.
[1] Started out on a physical server (an AMD 586) and a few years later on a virtual server.
> It comes down to the person running the website has to care.
Personally I think it is a little more nuanced, in particular I think the relationship between how much someone cares and how much effort is required to keep the website online is what matters.
If your website is super simple you don't need to care about it very much (though you do need to care at least a little bit). On the other hand, if everyone who works at google suddenly quit tomorrow because they didn't care the stack of cards would fall over very quickly because it's a lot of work maintaining millions of servers.
My first domain “expired” in an unexpected way after 18 years. I got a .eu.org because I believed that eu.org would be more stable than a commercial provider. I used the same not-for-profit DNS provider until they were commercially acquired and the parent company shut down the old nameservers.
Now I’m locked out: eu.org does not respond to inquiries for switching nameservers, and my account predates the auth system. While my phone number is the same, auth reset does not work with phone.
It would have been fun to retain the same domain forever, but stuff breaks, people die, and things crumble.
> I’ve often thought about getting together with some friends to pay into a fund to house our websites after we die. I don’t think setting that up would be too hard — the math around insurance policies of this sort is quite simple — I mostly haven’t tried to set something like this up just since it’s a pretty morbid ask. But, if you’d be interested, maybe reach out to me?
> Our ghosts could live forever, if we help each other.
I love this idea and would gladly assist in the effort, let's set it up :)
the year is 2122 and all the "good" (short, memorable) DNS names are owned by ghosts. in the grim dark future, there is only joe_smith_from_minneapolis_born_2055.com
Previously <https://news.ycombinator.com/item?id=31047818>
-----
From the NearlyFreeSpeech.Net FAQ <https://www.nearlyfreespeech.net/about/faq#Interest>:
> Q: Do I get interest on my deposit? A: No[, but...] We periodically reevaluate this situation, because we think a web account that runs forever purely off of its own interest is a pretty cool idea.
NFSNet also has an interesting part in their FAQ in response to the question If I think services you host are currently unavailable due to lack of funds; is there anything I can do?, they outline a process whereby third-parties can fund a hosted service by creating an account themselves, depositing funds into their own account (NFSNet services are prepaid instead of billed after the fact), and then submitting a manual (but free) request to transfer those funds to the original accountholder based on the service's domain name. I've always thought this was interesting because in theory someone could set up a community, disappear, and then the community could step up to keep it funded long enough for the person to get out of the hospital/be rescued at sea/etc, so long as the infrastructure is solid enough to remain operational without being attended to (not vulnerable to exploits, etc.)
They've also got a policy where if the member who operates the service is a willing participant, they can publish their NearlyFreeSpeech.Net account ID and have donors add funds to cover 100% of service costs via automated transfers. <https://www.nearlyfreespeech.net/about/faq#Lifeboat>
> you may attempt to archive it, but should you wish to avoid sadness down the line, you should accept now in your heart that all archives will eventually succumb to the sands of time.
Enjoyed this.
You could etch your data and website onto copper plates and launch them into the depths of space; this will likely last near until the heat-death of the universe!
Sad and depressing when things die, especially without notice, but that's life.
I have a blog, https://langsoul.com, that could die anytime. After all, sites are a lot of work and only a few make anything worthwhile to continue.
Not to mention, if I were to suddenly get ill, what then? Do I write into my will to launch a farewell blog post?
I think there was this internet cult that maintained their site for over 20 years from the leaders death. It required alive cult members to keep it alive.
Unfortunately I don't have cult members, nor funds to start a trust to keep the site going.
I think the cult you are referring to is Heaven's Gate: https://www.heavensgate.com/
Unstoppable domains solve this problem by making domain purchases permanent and stored on a decentralized blockchain - although that brings some problems of it's own - and IPFS solves that by not requiring that a central server stays online to serve that content, although this does require that _someone_ be insterested in serving the website at all.
This article isn't about websites going offline. It's about websites dying of 'natural causes', meaning the author or site manager lost interest in updating and maintaining the site.
There's much more to the soul of a website than the datacenter where it's hosted. I doubt if the handful of mirror.xyz 'decentralized' sites I see popping up will still exist in 5 years' time.
>blockchain
>solves this problem
Sure. Absolutely. Probably cures cancer too.
This really makes me want to redesign my site with minimal external dependencies and clear documentation.
There have been some attempts to capture sites that became the ghosts or disappeared completely. During the dotcom bubble through 2008, Steve Baldwin(co-author of Netslaves) had that as a side project.
https://www.disobey.com/ghostsites/
Our Incredible Journey still does this, with more snark and humor.
A very nice read with some poignant thoughts and quotes on digital
impermanence. Life on the Internet is nasty, brutish and short.
The automated url # fragments on highlights for this website is a cool but weird feature
I personally sunset a handful of websites this year. only kept a handful to try to dedicate time to.
writing open source the new site is called Make Post Sell (https://www.makepostsell.com)
hoping to restore a bunch of content as shops in this multitenant SaaS.