This isn't indexing by search engine spiders, which are usually fairly benign and easy to identify with user agent etc. This is searches for "scraping footprints" executed en mass by "SEO proxy farms", which are designed to be very difficult to detect (e.g. originating from globally distributed residential IPs, quite possibly ordinary home user's machines which have been compromised). The main giveaway that something is a "scraping footprint" is the long search query which includes text that would appear on a template, e.g. ""This website is proudly using the open source classifieds software OSClass" rega turntables", for someone looking for OSClass-powered pages they could "search engine optimise" for the query "rega turntables".