For example, a common training data storage format looks like
grade(0-4) queryId list of feature values...
Such as:
4 qid:1 1:0.5 2:0.24 # a doc with these features is 'good' for this query
0 qid:1 1:-0.5 2:0.24 # a bad doc for query id 1 is described by these features
Features in an LTR context are some relationship between the query and doc (such as scores computed from Solr/Elasticsearch queries)
Of course turning clicks/session info into grades isn't easy. One way to go about it is with click models (https://pdfs.semanticscholar.org/0b19/b37da5e438e6355418c726...). Another way is to equate conversions in a session with one of the grades (a purchase is a 4).
Needless to say, such usage data is sparse for long tail queries (queries that rarely happen - 'fuscha shoes'). Luckily it's not sparse for head queries (common queries - 'shoes'). So those head queries can provide some benefit, helping to generalize to the rare long tail queries.
You'll see a lot of patterns that are modifiers on head queries. Like 'shoes' is a head query. But 'blue shoes' and 'purple shoes' and other 'color shoes' are tail queries.
You can do a lot with that insight
- Supervised grouping of queries into one 'query id' by watching how users refine queries. For example users type "shoes" then in the same session add one color adjective "red shoes". Using that to group '<color> shoes' queries
- Unsupervised clustering of similar sessions or searches where users seem to expect ranking to perform similarly based on user behavior
The downside/tradeoff is you lose a lot of nuance that you're getting with per-query training data. But it's a technique people use.
It also substitutes one ML problem for another, which may or may not be harder than the original one