That‘ll be super interesting!
Local context is definitely the key factor for small models achieving better quality than copilot (related: [1], [2]) .
One things I’d really wanna have in Sourcegraph: A Search API supports custom retrieval / ranking. Research works (e.g [2]) show simple BoW fetched context is more efficient for code completion tasks.
Disclaimer: I’m building https://github.com/TabbyML/tabby an open source alternative of copilot.