True though this assumes these direct words are mentioned in the text (assuming know synonyms). Algolia has basic synonym functionality but ES gives you a lot more power here.
Further the more words you remove, the less constrained the results get possibly creating a lot of noise. Though I assume you remove based on document frequency, which helps.
This would be a problem with this strategy regardless of search engine (ES or Algolia)