Try the following tool description: "Calculate the sum of two numbers. Use for any arithmetic or math question." instead of "Add two numbers." Let me know how it goes, thanks!
back
2 comments
It does better with clearer tool description, but we are taking note of these complaints for future improvements.
works better that way, thanks :)
Thanks, we shall improve this for the next release.
but still struggle when changing the quesion:
import needle
@needle.tool
def add(a: int, b: int):
"Calculate the sum of two numbers. Use for any arithmetic or math question."
return a + b
agent = needle.Needle(tools=[add])
print(agent.run("what is 5 + 7?")["reasoning"])
Ah, another failure point on our end! So a simple "5 + 7" and "add 5 and 7" works. But to handle ambiguity, the python package ships pipelines to synthesize augmentations and fine-tune on your samples for robustness. Just run "needle playground" and use the UI. Apologies.