If I had to reckon, it's because the web comes in very many shapes, and outsourcing that work to a generalist LLM/SLM like GPT Nano is expensive, and doing it deterministically will never catch all the edge cases as well as a purpose-built encoder when run at webscale.