I am passionate about performance in web, native, and mobile applications. It irks me tremendously when web applications are slow to respond to my actions. There are some otherwise very-useful sites that routinely frustrate me with their high latency.
But I think my greatest performance peeve is embedded devices such as commercial DVRs that are not simply slow but agonizingly slow. I've seen some DVR UIs that frequently consume several seconds to react to user input. I consider it remarkable that engineers can be so beaten by their budget-tightening peers to allow for these abysmal user experiences.
Machines need to act like machines: fast if not clever. They process input literally, so they had better do so quickly. That's their lot in life, so to speak. When I speak with a human, pauses—a few seconds of thinking—are expected because we are thoughtful creatures. My DVR should not need to think about what it means that I pressed the "Guide" button. The channel guide should just plain appear. In fact, a DVR does not think about anything. As the user, I perceive the DVR software squandering hundreds of millions, perhaps billions of processor cycles doing unimaginably wasteful operations instead of simply displaying the channel guide.
Bravo to the OP for looking well beyond his own use-case and considering what his users may be doing with his application!
<http://askubuntu.com/questions/88908/many-many-bookmarks-in-...
In his beginner lesson on programming a Poker game with Python, Peter Norvig ask the student to write a test for a 10000 cards deal.
That's the same. "It just work" is the user's resulting feeling for a product that is working well under any conditions, known and unkown, aka a well-engineered product. (Which often means getting over the project manager's head)
This means to have few to no implicit assumptions in the code. Vim is a great example: developped when a 10kb file was a huge file, it will kindly open and edit a 2Gb log file for you (if you don't have all these silly 3rd party plugins). Vim may be burping a bit on very very long lines, but it still shows that the running code behind is not assuming on your behalf that you are editing a couple of haikus.
Being conscious of the structural decisions you're making at a low level, and abstracting them out so that they can be refactored goes a long way to dealing with performance issues along the way.
As a self-taught programmer, I take speed and big-O considerations very seriously, but I've gotten the impression that my opinions on this topic are too extreme. I hate nothing more than a slow program and website, and I especially hate it when the site runs so slow it causes my fan to whir. It is nice to see that there is at least one other person that probably wouldn't think I am insane.
And it must apply in many other areas. How many sites have you visited that takes more than 10 seconds to load lately? How many "cloud-based" programs have you used that takes far longer than it should to do basic operations? I know I can name quite a few.
Putting effort into performance is probably something you'll want to do at some stage if your app gets big enough. But a lot of the time you have more important things to do right now. Users will put up with an awful lot if you've got something they really want; get that part down first.
Finding the bottlenecks should always be a priority. Fixing them can be done at a later point.
I encountered a database once where there was a hidden limit on the number of categories for customers. Specifically, you could add more categories over some very low number (16 or 20) the program would fail to open - it would hang on a loading screen.
We didn't have source, so we ended up having to find and pull the developer out of retirement (it was a 16-bit windows crystal reports program), who found the error after looking at it for a few weeks. It was a one line change that bumped up that limit to something like 1000.
Most of the time the limit you may have aren't realized, and you assume that the programmer is smart enough to note and communicate with the user about all possible limits.
Don't cripple your software. It's better if it runs poorly than it not running at all.
Taking that into account, your proposed approach would be: don't worry about performance on what you consider to be abnormally large data sets, and instead just wait for people to e-mail you about it, and you can decide if it's worth fixing then.
And there's nothing wrong with that approach, I think! Although I think a lot of developers rely on it too much. But it's better than putting in an artificial roadblock.
Maybe on a touchscreen keyboard, or in an app limited to that; it would behoove the OP to consider cross-platform possibilities and very long timeframes, or even look to use cases in similar systems. In org-mode (which I've got on my phone and desktop), I've got tens of thousands of lines, and could easily see keeping 30k notes. Granted, it's different, there's more than purely human entered data, but as the OP sagely observes, there may be automated systems inputting to Vesper someday.
I wish you all good luck with over-engineering your software!