▲ 162 points
back
52 comments
Hi, this is Matthew Butterick. If you have specific questions about Pollen's capabilities (e.g., "how hard would it be to ...") I encourage you to post them to the GitHub repo at http://github.com/mbutterick/pollen. I'd love to make it more useful for other authors, especially those in technical fields, so I welcome all suggestions.
TeX has been mentioned several times — I don't mind that comparison at all, since my thinly-veiled ambition is to create a contemporary successor to TeX. Contemporary = optimized for web publishing + better "macro" language (Racket) + shallower learning curve.
That said, TeX got a lot of things right. Especially the basic notion that a book (or other publication) should be represented as a program, and that authors should be able to freely intermix text and code. With digital books, I think it's essential.
(PS on Racket: HN was built with Arc, which was also built with Racket.)
Matthew: interesting project, thank you for releasing it.
I've always thought lisps were a natural fit for this sort of thing; the cl-typesetting project (http://www.fractalconcept.com/asp/cl-typesetting) didn't get a lot of traction but was pretty effective, are/were you aware of that one?
Any intention to tackle higher quality typesetting for hardcopy/pdf ?
"(...) my thinly-veiled ambition is to create a contemporary successor to TeX. Contemporary = optimized for web publishing + better "macro" language (Racket) + shallower learning curve.
That said, TeX got a lot of things right."
So why not resist the start-from-scratch instinct and build a framework on top of TeX instead? In that way, you would not only inherit all that TeX gets right already; but most importantly, you gain an existing huge userbase practically by proxy by making it easy for many TeX-using authors to use your framework without having to change their existing workflows or rework their source files (including, as it happens, many in this very thread, as you can see).
I understand starting a new publishing framework from scratch is much more appealing than the daunting prospect of building a higher-level lispy macro language on top of TeX that achieves your goals of readability, power and ease of use; but there are reasons why sometimes improving existing frameworks is often preferable than tossing everything away and starting over from scratch[1].
I didn't start from scratch. I built the project on top of Racket & Scribble (which itself has similarities to TeX).
Bear in mind that creating Pollen was a side mission of the main project: making the website http://practicaltypography.com. The existing tools weren't good enough, so I ended up making my own. I looked at using TeX. No one had anything nice to say about its HTML capabilities. So I moved on.
Pollen does less than TeX. But it also demands less (in terms of setup & learning curve). Authors who need everything TeX can do aren't going to be interested in Pollen. But that's OK — they already have TeX.
True, true. "By scratch" I erroneously meant "on top of TeX", my bad. I take that back.
On the other hand, I can't help but remark on the vague sense of missed opportunity in your project. I know for a fact that "authors who need everything TeX can do" are indeed interested in a streamlined TeX to web-book authoring library[1], precisely because of the lack of good HTML packages that stopped you from developing on it in the first place. An easy-to-use package like beamer but for outputting nice, responsive webbooks or articles on a whim would be a gigantic boon to the sciences.
[1] There are several TeX to HTML tools out there already, but the output quality isn't that great and in any case the well-known ones are not intended specifically for webbook authoring. There's also PubMed Central's PubReader, but for specially-tagged XML articles, not TeX. See: http://tex.stackexchange.com/questions/43847/why-havent-any-...
Well, you can make dvi and convert it into svg with dvisvgm; taking the inconsistent, usually terrible browser rendering and absolute lack of a proper typesetting in HTML into account this is probably the only way to achieve a nice web-published text (;
The bit in the book / manual about typing a "lozenge" character that shows how with an Apple then has two blank lines for windows and linux is a little disconcerting.
As with any unusual character in Windows, you seem to be pretty much stuck with "build and install a new keyboard layout" as the general solution, though there may be editor-specific alternatives.
Indeed. Fixed!
I've been looking at this recently. It looks like an extension of what made TeX so successful: your typesetter is programmable -- and macros can live in your documents, defined at any time -- so any common tedious task can be automated.
Sure, Markdown and the rest have a simple syntax for everything, but what if you're writing a book and want cross-references? Or you commonly use side-by-side examples (good grammar vs. bad grammar, say) and want a macro that generates the appropriate table layout. Or you want some macros which typeset entries in your CV. And instead of crufty TeX macros you can use Racket.
If I weren't already writing a book in LaTeX I'd try writing one in Pollen.
I've been contemplating using my literate programming project https://github.com/jostylr/literate-programming to write a book. It would allow one to do all sorts of things in terms of reordering text, doing macros, calling out to stuff, etc. as well as having multiple compile-to targets.
It would also allow one to write why one is writing what one is writing without actually putting it in the book. Kind of like writing a "behind-the-scenes" while writing the book.
And the macro language is actually sane!
Beautiful, and I'll use it for my first book.
But I have to call out the claim:
> I think Pollen just won the Hello World Tournament.
That tournament was won permanently by the M4 language in 1977. http://en.wikipedia.org/wiki/List_of_Hello_world_program_exa...
I was considering an esoteric (joke) language that would compile a blank file to print "Hello, world!", q to a quine, g to guess, and similar. I never got around to creating it though.
HQ9+ already fills that niche well enough.
Haha. Touché.
Awesome!
I love Butterick’s Practical Typography. It taught me a lot about typography, and started my love for typefaces.
The fact that Matthew Butterick is, a lawyer, a typographer and a programmer just increase my respect for him.
Thank you for your work!
"The ◊ character is called a lozenge. In Pollen, the lozenge is a special character that marks anything Pollen should interpret as a command (rather than plain text).
How to type a lozenge: Mac: option + shift + V Windows: holding down alt, type 9674 on the num pad Ubuntu: ctrl + shift + U, then 25CA"
That's an unfortunate choice, imo. Pollen looks great but I wouldn't want to have to type that character all the time.
Sorry to nitpick, I think everything else I've seen (so far) is very encouraging. Looks like something halfway between Markdown and LaTeX.
If our languages never stray outside of ASCII, then our input methods will never improve. The Emacs version is not much better:
C-x 8 RET loz[enge]
or
C-x 8 RET 25CA
Which suck, but can at least be rebound to a useful key. Unfortunately, the \lozenge command in TeX mode produces U+2727 “white four pointed star”. Sigh.
If you're interested in something that actually is halfway between Markdown and LaTeX, take a look at Softcover: http://softcover.io/.
This looks very interesting. Books really are code: I now have 10+ conditional flags in my book: ifPG13, ifPRINT, ifEBOOK, ifEREADER, etc. I'm looking to add more, but the basic \ifthenelse macro in LaTeX won't be enough...
How hard would it be to implement things like \ref and \pageref and support more of the LaTeX environments supported by MathJax?
I think a triple output format HTML + ePub + LaTeX|PDF|print would be awesome --- I'd switch right away!
Thanks for releasing this as LGPL.
>I now have 10+ conditional flags in my book: ifPG13, ifPRINT, ifEBOOK, ifEREADER, etc. I'm looking to add more, but the basic \ifthenelse macro in LaTeX won't be enough...
Makefiles can be your friend... in the file `Makefile` put
.PHONY: all pg13 print ebook ereader
all: pg13 print ebook ereader
pg13: book-pg13.pdf
book-pg13.pdf: mainfile.tex
pdflatex "\setstuffforethisversion\input{mainfile}"
print: book-print.pdf
[etc...]
Hm... reading-in the src from the command looks a little brittle, but it's a must try. The makefile will also help with the need to compile 3 times with latex.
It's definitely brittle, but it's brittle along a dimension that usually doesn't change much over the course of the project (paper, book, etc). If you have automatically generated images or tables the benefits can be huge.
`latexmk` solves the "compile many times" problem, and can monitor for changes to the source file in the background.
Interesting software. Actually, I wish there was a way to use some other character (or string) instead of the lozenge character (◊) for commands.
I mean, I see the reason for using a weird symbol, but I'm not even sure how to enter it from the keyboard other than by copy-pasting it. Is it some sort of standard symbol in DrRacket environment?
Maybe, a directive for changing this symbol can be added: #commands_start_with my-char
Apparently you can do this via setting Pollen’s 'world:command-marker' value; though I don't know how to do that myself :-/
Ref for others, on OSX, its: option + shift + V
See http://mbutterick.github.io/pollen/doc/reader.html
edit: key commands now fixed for other platforms
If you are frequently creating text using the special character, a good solution is to add some hotkey for it to your editor.
It already exists: world:command-marker
This looks great! I like the documentation itself even more, is that generated with pollen as well?
Edit: I answered my own question, the documentation is generated using Scribble: http://docs.racket-lang.org/scribble/
http://www.cs.cmu.edu/~spot/markup/markup.html
also used scheme for writing books, provided for embedding code execution, tex for math, targeting html and paper, etc. i made it in 1996 to write my phd dissertation, based on markup by jar@mit.
I have been looking forward to trying this out ever since I heard about pollen in one of MB's talks on typography. Unfortunately it requires racket 6.0 and Debian only has 5.3.6 in unstable and nothing in experimental.
According to the "package 6.0" bugreport it looks like things are not going to change very soon:
As far as I understand, the build system and general organization of
racket has been changed quite a lot for 6.0. I'm not sure how soon I'll
find time and motivation to work on this, especially since I'm committed
to using 5.3.6 at $WORK through the end of April.
You can install Racket 6 directly from the Racket web-page: http://download.racket-lang.org/
You can see the SHA1 hashes of the distribution here: http://www.cs.utah.edu/plt/snapshots/
This looks cool and I'll take some time to read the docs in depth.
I wonder when Atwood's Law will kick in and we'll see more ambitious JavaScript-based tools like this (maybe with sweet.js macros to provide higher level constructs). JS already turns a document format into a program, but plain HTML+JS is not the best authoring format.
Can I export it to PDF like in TeX?
Just saw this too: https://github.com/mbutterick/hyphenate
"Racket implementation of the Knuth–Liang hyphenation algorithm."
I am a bit flummoxed about the lozenge character being used to signify markup. Is it convenient to input by keyboard? Seems like that would be necessary.
Does it have mathematical typesetting like Tex?
This is the real kicker for some applications.
After all, Knuth didn't write TeX because he needed a typography solution, he wrote it because he needed a typography solution for mathematics.
All these years later, and TeX (with or without LaTeX) is far and away the best solution I know of.
As for the mass market: Word's typesetting of mathematics is terrible, as is the interface (for anything but the simplest expressions). MathML is an interesting idea, but has a long way to go on the actual typesetting, as well as being much more painful to use than LaTeX...
Mathematica has a pretty decent typesetting system, but it's not open source or free...
I've read about "experimental partial MathJax support" or something like that in the changelog, but I haven't looked into it, yet.
It's actually one of the big questions I'm having, as well.
You can use MathJax inside Pollen today. As far as building more integrated support for math typesetting, I'd like to do that, but what I'm not sure about yet are the ergonomics. Ideally, you'd just be able to drop TeX-style math expressions into the markup, indicating whether they're inline or block, and Pollen would do the right thing in terms of handling the rendering (based on whether the output is a web page vs. something else).
How do you think about MathJax vs. pre-rendering formulae to images and using those, with the TeX input as the alt attribute? Oh, and using STIX or whatever as web font.
knitr does this reasonably well for letting you drop LaTeX directly into R reports with a delimiter... if you haven't seen that might be worth a look at the interface in some examples (e.g. http://yihui.name/knitr/demo/showcase/)
[How] Can we add keyboard navigation?
Wonderful!
there's also literate haskell.
http://www.haskell.org/haskellwiki/Literate_programming#Hask...