▲ 461 points
back
81 comments
In case you're similarly misled by this headline as I was, this is about making the archive specifically of "Editor & Publisher" available, not about some larger archive of the contents of American newspapers.
I'm disappointed.
Google actually scanned a huge number of newspaper microfilm and microfiche back in 2010-11: https://news.google.com/newspapers
It's an amazing resource, but it's hidden away and researchers within Google have trouble accessing it even if they knew it existed. I spent months with lawyers to get access to the original files for research and at one point they told me they were going to delete it! Whaaaaa!?!
It was something like 6 PB, which seriously, to Google isn't much, but the team that "owned" the data wasn't using it and to them it was just an expense. Ugh. People don't care about history.
Google doesn't want to make the web and the world searchable any longer like their original set values set, now they just want to put up just good enough service to get all your private information and then aim ads at you. Gone are those heady days of some streak of altruism in their mission as a corporation.
Which is infuriating because people would happily pay even a fairly premium price I suspect for good access to that archive, but of course Google can't just sell a good product for a reasonable price, it has to be FREE******.
(Each * here representing some unknown third party getting access to your email address, phone number, blood type and sexual preferences.)
It is very frustrating.
New York State funded an effort to scan, but not digitize historical newspapers, and while the microfilm is stored in the state archives, the online versions are hosted by an eccentric guy who digitizes the microfilm as a hobby. The guy puts everything online, but makes it difficult to work with in a variety of ways.
https://chroniclingamerica.loc.gov/ -- "Search America's historic newspaper pages from 1777-1963"
That's fantastic, though this too appears to have quite limited scope, going by the about page. Certainly a great resource though, and since I'm currently doing some genealogy research on two separate branches of my family that emigrated to the US I'll definitively use it.
Ok, we'll switch to the subtitle above, which makes that clearer.
Almost every major American periodical in the public domain is available here:
https://www.unz.com/print/All/
Almost every major American periodical in the public domain is available here
According to the site, it specifically avoids "major" periodicals:
"A Collection of Interesting, Important, and Controversial Perspectives Largely Excluded from the American Mainstream Media."
etrabroline did say "in the public domain." I consider Isaac Asimov's inclusion major, for example. I've even heard of a lot of the periodicals he's listed in.
I highly recommend this book: https://www.amazon.com/New-York-Times-Page-One/dp/1578660882
It's a century's worth of NYTimes front pages. An amazing long term dataset that is fun to flip through to answer infinite questions you might have about both how the last 100 years really went down and also about the evolution of presenting information to readers.
You can also fetch:
https://static01.nyt.com/images/<year>/<month>/<day>/nytfrontpage/scan.pdf
I have a giant coffee table version of this, and its awesome. I should find more like it. Talk about a conversation starter.
Me too! I got it used for $17 iirc. One of my best purchases.
If you do find more, please let me know.
I was hoping scientific American or wsj etc would have them but couldn’t find any.
I highly recommend everyone go back and read the headlines (and articles) from a year ago from multiple news sources frequently. It gives great perspective on the narratives at the time with hindsight as to how those narratives changed and if the predictions made were substantiated.
Beyond headlines, news outlets also silently edit old stories, sometimes years after the fact.
The Washington Post was recently caught scrubbing an unflattering quote from a 2019 profile of Kamala Harris:
https://reason.com/2021/01/22/the-washington-post-memory-hol...
Do you have any interesting examples?
Almost anything in politics is viewed completely differently after a year. Just look at any news story you followed last year. Around the election. Impeachment. EU/Brexit, etc.
Not exactly an article, but John Oliver's concerns over the security of voting machines in the US (November, 2019): https://www.youtube.com/watch?v=svEuG_ekNT0
Personally I had not seen anything from him stating he believed the security holes were fixed on a broad scale, or on any scale. But his recent video in November 2020 suggests he no longer had any substantial concerns regarding voting machine security: https://www.youtube.com/watch?v=cMz_sTgoydQ&t=521s
Two easy ones: COVID coverage from January to March, and Kamala Harris coverage from Democratic Debates to VP nomination.
For COVID, back then, it was "tech bros are afraid of the flu (and maybe also racist)" and then "go hug a Chinese person", and then "closing flights from Wuhan is racist (even though China itself was doing it domestically)". It was funny to see the escalating tsunami of wrongness coming at the media, but of course, they would never admit wrong-doing.
For Kamala Harris, well she was (is?) hugely unpopular. She has terrible political baggage, has been personally responsible for terrible systemic racism, and the news coverage reflected that quite accurately up to a very specific point. Tulsi torpedoed her early on in the debates by bringing all this up.
However, when Biden said his VP choice would be a woman of color, there was an immediately 180 in coverage and retroactive editing to make it look like she was an exceptional candidate who has always championed racial issues and is a regular everyday person just like all of us.
Link to the tool:
Wasn't a huge quantity of images of newspaper articles accessible via google news search, by setting a pre-Internet date range, at one time?
Is that material still available anywhere? It was really extensive IIRC.
Are you thinking of this?
https://news.google.com/newspapers
It's fun to browse but I'm sad that the project seemed to just sputter out. At one time I thought it would grow to make historical newspapers searchable with coverage comparable to the books searchable through Google Books.
Yeah, that looks like the same content, but it used to appear in "news" search results if you set the date range accordingly.
Sad they haven't made more of it really. Thanks for the pointer!
There was a recent news story about a 150 year old town hall in New Hampshire burning down. As of the writing of the article they weren't sure if any of the old town records were destroyed but it seemed likely. Towns should probably be digitizing all of these things now, and archive.org seems like a great place for them.
History isn't what it used to be. Recent events suggest that Internet Archive too will cease to be as soon as it's content becomes politically or ideologically unpalatable. Enjoy it while you have it, don't count on it being around forever (in any useful state anyway).
An interesting fact about archive.org and the Wayback Machine especially is Amazon's involvement in initially donating (and continuing to?) Alexa crawl data to it: https://www.forbes.com/sites/kalevleetaru/2016/01/18/the-int...
Alexa APIs, incidentally, were one of the first AWS products along with ECS (e-commerce service) and SQS in 2004.
Does anyone know of a similar resource for UK newspapers?
Interesting question. I wonder if history will view the years prior to 2000 as mostly American since we have such a passion for digitizing everything and got on the internet early.
This is amazing. I wonder how log it will take until it turns up as a cleaned text dataset.
As a side note, I'm loving the 90s tabloid layouts.
Internet Archive forever!
Wikipedia and Archive.org are doing what the internet was made for at least on the history side of things, but also culture, information and education. If you can, donate.
Archive.org and Eff.org are my two do not question donations.
Well said, agreed! I should add Wikipedia.
Wikimedia's financial reports are at [1]
At a first glance, they seem reasonable.
- 43% Direct support to websites. Keeping the Wikimedia websites online is about more than just servers. It also includes ongoing engineering improvements, product development, design and research, and legal support.
- 32% Direct support to communities. The Wikimedia projects exist thanks to the communities that create and maintain them. We strengthen these communities through grants, projects, trainings, tools to augment contributor capacity, and support for the legal defense of editors.
- 32% Direct support to communities The Wikimedia projects exist thanks to the communities that create and maintain them. We strengthen these communities through grants, projects, trainings, tools to augment contributor capacity, and support for the legal defense of editors.
- 13% Administration and governance. We manage funds and resources responsibly to recruit and support skilled, passionate staff who advance our communities and values.
- 12% Fundraising. Wikimedia is sustained by donations. Millions of remarkable individuals and institutions ensure that we have the necessary resources to continue our global mission.
[1] https://wikimediafoundation.org/support/where-your-money-goe...
Agreed. These are probably my two most regular donations.
I would suggest against donating to them. See https://news.ycombinator.com/item?id=26028644 https://news.ycombinator.com/item?id=26028531 https://news.ycombinator.com/item?id=26029978 for wikipedia.
As for archive.org they offer basically no transparency as to which sites are excluded from their archiving (and as far as I know they will remove all the content of a site once the owner asks them to).
Those threads don't show anything interesting, some just bash the org for spending any of the money they receive in a way someone doesn't like from years and years ago. This is silly
I don't get why people are so upset that Wikipedia spends more money on personnel then on hosting.
Most internet organizations do that (free, charity or commercial). Good people are expensive. You need sysadmins, programmers, some graphics artist, probably more than a few lawyers etc.
And I wouldn't want them to use the cheapest, possible people that they can find for those roles. And asking good people to work for free or cheap, is just as shitty.
And a lot of free software and opensource foundation spend most of the money organizing conferences, so that people can meet in person and have presentations and working groups etc.. But when Wikipedia does the same its somehow wrong ?
This is not helpful. Wikipedia is a community project, and managing that community takes a lot of the resources, obviously.
The Internet Archive has proved itself so many times over, and is so underfunded, that it deserves all the money it can possibly raise - which will allow them to solve more of the problems they face.
I wonder if the Wikipedia community generally supports the Wikimedia foundation (if WMF does a generally good job), or if they'd like it replaced.
Why, so I can give a few more dollars to book publishers when they take Archive for all of its assets after its reckless "library" tactics last year?
I like them, but they seriously need a change in management.
It's unpopular here, but that stunt really made me wonder about their management.
They opened themselves up to millions of dollars (if not billions of dollars) of liability for no good reason. (Covid doesn't give you a pass to give away someone else's property without permission.)
Similar to you, I'm not going to donate to an org that is so reckless with donations. If they would admit it was an error in judgement and come to a quick settlement I might become a donor again.
Even still, the damage has been done -- it'll take a long time for publishers to trust the Internet Archive again.