Firefox is one of those projects that just keeps on giving, and it's all thanks to the hard work put in by all the contributors. It's easy to forget that sometimes. We use Firefox headlessly to render videos at https://www.musicvideodispenser.com , a task that certainly wasn't the initial intended use-case for that browser, but it still works, and we have the Firefox team to thank for that!
back
2 comments
You may be interested in the work currently being done to add a native headless mode and browser control API into Firefox.
Can you elaborate on the "using Firefox headlessly" ?
I recently had to deal with WebRTC on a headless machine and managed to hack up something with some Python scripting and GStreamer. I didn't know it was possible to run Firefox headlessly and now you got me wondering whether that would have been preferable to my stitched up solution.
On a unix system you can run any browser headlessly by using a headless X11 display such as Xvfb.
Instead of having a real screen, Xvfb uses an in-memory framebuffer for screen output that you can screenshot etc if needed.
It's as easy as:
Xvfb :123 &
DISPLAY=:123 firefox
### firefox is now running and outputting
### to the framebuffer display on :123, and
### can be driven by WebDriver etc.For practical use you want the xvfb-run wrapper script, and use a saner mode than the default 256-color one:
xvfb-run --auto-servernum --server-args="-screen 0, 1024x868x24" firefoxI also tried to hack together WebRTC headlessly a while ago actually! I looked at GStreamer and Kurento, but ended up using Janus instead. Anyway, turns out you can use Xvfb, essentially a "fake screen", to headlessly run GUI programs: https://en.wikipedia.org/wiki/Xvfb , which is what we're doing with Firefox.
Thanks, I will look into it. I started off with Janus too but I had some odd bugs and segfaults. Since time was short I decided to give Kurento a try and since it worked, I stuck with that. There are still some warts but it does the job so far.
Xvfb looks to be pretty straightforward about what it does; now I just need to figure out how to control firefox after it was started.
Firefox is basically the first port of call for running web tests on buildservers. Run Firefox headlessly in xvfb, then throw Selenium or similar at it. It's a pretty mature setup.
Even better, selenium has preconfigured containers for Firefox and Chrome and that use Xvfb. At the start of a web automation project I worked on, I spent a lot of time trying to get various versions of browser, selenium and pyvirtualbuffer installed and running with mixed results. These containers are well tested and work very well.
I would think PhantomJS, being a headless, scriptable browser (plus built on WebKit so closer rendering to the most popular browser), would be the "first port of call."
I have used phantomjs (with casperjs) for scrapping a website, it was complete nightmare.
1. It use javascript to script so writting sequential code wasn't possible so the code ended up using continuation passing style.
2. The very first example snippet on the front page don't work in the repl (https://github.com/ariya/phantomjs/issues/11180). So the repl is useless.
3. When an error occur in your script (not the one on the webpage) there is no message anywhere no matter what you do.
My mistake was to continue with phantomjs the hype around made me think others options would have been worst.
I have used selenium+firefox+php to do a quick test it was wayyyy better, just for n°1 using sequential php remove a lot headaches.
Why use a fake browser when you can use a real one? It's also harder to inspect the actual rendering of an invisible browser.
Near the top of a google search for 'phantomjs vs selenium', we have folks on HN complaining about phantomjs https://news.ycombinator.com/item?id=8418071