back
2 comments
Original Quepid creator here:

Quepid has been really powerful for helping teams bootstsrap on relevance problems. Often we start with a dozen or so queries, get some ratings, create a bit of success criteria and tune in Quepid. You can do this in a 'sprint' like structure, where the next 'sprint' you tackle the next dozen or so queries, ensure the stuff you fixed before is still working, and make progress on the next set of use cases.

The sweet spot here is teams just getting started on the problem. When a team's maturity increases, they start to (rightly!) ask questions on whether the success criteria initially captured in Quepid is correct, needs to be measured even more scientifically (combining engagement, business rules, human judges, etc).

But the takeaway here is before getting a headache about how hard it is to measure relevance, get started on some intuition, and begin iterating and gaining confidence you CAN make progress. Teams ask us all the time how to start with the training wheels before getting to cosmic-brain level status, and Quepid has been a big part of that!

Looking forward to an avalanche of PR's! It's hard to open source something that was internal for so many years, but hopefully we've done it successfully!