The benefit is that you can decouple the implementation of the operation from the implementation of the sum type. For example, a library can provide a sum type, and users of the library can implement their own operations on it. This is often necessary for proper separation of concerns. For example, the sum type may be a domain object, and the operations may be database operations, UI operations, or other I/O or conversion operations. You don’t want the domain layer to have dependencies on the database, UI, I/O, etc., which it would have if those operations were implemented in the sum type itself.
It’s exactly the same benefits provided by language-native sum types with ADT pattern matching (aside from the visitor pattern requiring more boilerplate).
The operation can itself be a polymorphic operation on a different type hierarchy or sum type, which is the scenario you are thinking of, but that’s not necessary for being able to benefit from the visitor pattern.
Read the original description of the visitor pattern in the GoF book. It’s about implementing operations on an existing type hierarchy without having to modify the code of that type hierarchy.