Item — HN Reader

Hi, I'm the author of this project. The code will be available soon on github: at https://github.com/valentinvieriu/visual-hacker-news - right now that's the old version using Angularjs

The project uses as a starting point the amazing Vuejs Implementation from https://github.com/vuejs/vue-hackernews-2.0 . It has super fast server side rendering, that's why the feedback is imediate as you load the page.

The thumbnails are generated using phantomjs server. I'm using https://github.com/blockai/phantom-pool to make sure the phantomjs instance is reused. This improves the speed of the screenshots quite well. Also I'm using a scheduling node package (https://github.com/Automattic/kue) to make sure that images are pre cached as the link are submited to hackernews.

I'll post the code on https://github.com/valentinvieriu/vue-hackernews-2.0, but need to take care of some hardcoded redis credentials first. I'll give a reply when it's up

Thank you for the interest! I'll do my best to reply to all questions if there is interest on this project.

6 comments

May be you should consider rejecting random urls from your thumbnail generation endpoint. Now I could generate thumbnail of any website e.g. my blog https://hnews.xyz/thumbnail/?url=http://rajeeshcv.com

It's a hard one to do.I agree you can do that now. I'm thinking on a solution, but it's not really easy. Right now I have no solution.

It shouldn't be too difficult - surely right now you have some sort of task that scrapes the Hacker News API to present the stories? Generate the thumbnail then, store it on S3.

Setting up a same-origin policy (https://developer.mozilla.org/en-US/docs/Web/Security/Same-o...) would prevent people from embedding the image in other websites.

You could try generating a unique token for each page-view, and use this as an access-token for all other resources pulled by the page.

The reality is that it's not too many people who will misuse your API... so don't pick a mechanism that will place undue load on your server until you really have to.

You could your scraper service a unique token that random people won't have so they get 401ed.

Do what https://hackernewsfilter.com/ does, use a Bloom filter. It's simple enough.

Thank you for this idea. This seems like a quite friendly one. So you are using the Firebase API, and each time a website is added you add i to the Bloom filter? This seems it will work for my website, but then the screenshot api can be use only if you previously pushed that url in the bloom filter

Another solution I can think of is - encrypt the URL with a symmetric key. Decrypt it to generate the thumbnail

Thank you for the suggestion. I'm not so familiar with this. Will this work on frontend too? If I expose the secrets on the fronted, then it's pointless. Do you have some suggestions on how to do this? Appreciate the help! It's an important aspect and a very resource intensive process that I need to protect.

To the extent that you are pulling new stories directly from Firebase on the client side, encrypting (or signing) urls with a symmetric key will not work. This is because you would need to embed the secret in the front-end code which means the secret is no longer secret ;-)

You could also tail the Firebase feed from a server process, generate the relevant images and only serve images for pre-existing urls.

Your decryption/encryption secret (key) is stored on the server side only and if a client requests a thumbnail, you can validate the URL value by successfully decrypting it with this secret and probably do a simple HTTP URL validation check of the decrypted value. And you have to encrypt all thumbnail URLs with this secret before passing them to the client.

My suggestion would be to wait a second or two for the pages to load before taking a screenshot of them. It seems that for some websites, there is dynamic loading or an animation first, and this shows up on your board as uninformative pictures.

Great job on the site though! It's very smooth.

Take this as a personal opinion, but I felt like I was having to work to find the titles...maybe they'd do better on top? Is that something that could be easily A/B tested?

I had the same thought.

What do you mean by - It has super fast server side rendering, that's why the feedback is imediate as you load the page?

How is this different from a normal VueJS app where I would fetch data with ajax or vue-resource and then fill the elements with content?

Found it: https://vuejs.org/v2/guide/ssr.html

Hello - excellent work. I'm working on a Vue-based UI myself and really like your use of infinite scrolling + performant rendering. Great Job!

I'm finding it hard to read the title of the tiles because of the opacity of the background. I don't need such a large thumbnail, nor one that is so legible.

How did you like working with Vue? Did you have javascript framework experience prior to using it?

Design can be changed. Can you tell me what would be a right proportion for you? Wider screenshot ( so higher resolution so you get more context of the page ) but less height? Actually on a retina ipad, the text is quite readable

You already designed it to your preference. I'm sharing mine with you. Reduce the tile size by 50%, turn opacity to 100% on the title bar and increase opacity of the tile thumbnail by 25%?

What I like about it is actually being able to read content in the tiles. For me, reducing their size would just turn it into a gimmick - cute thumbnails but why bother?