Most of the collection of the British Museum is not on display at any given moment (if ever). They could lose 90% of their inventory and the display would be exactly the same.
But that's beside the point. Museum entry may be free, but London is pretty expensive to go to, especially if you are from a place where the items in question were plundered (ie poor third world countries). In some cases it may even be illegal. Most of the people whose cultures those items belong to cannot afford to go visit the museum.