Profiling your code early when it is still small and simple gives you the feedback on whether the code does what you think it does. I usually start the profiler on my AllTests.
To get around the most common misconception first: EarlyProfiling does not mean that you do any perfomance optimization to the code. (You may though, if the profiling shows that you really have to do it, or if you find a way to do it via "usual" XP refactoring, thus catching two flies at once.)
I would suggest early and regularly, e.g. every Friday afternoon.
In a desktop application, EarlyProfiling can help you make good low-level design decisions (like, does this behavior need to be run in a background thread) by keeping your familiar with the performance characteristics of your code - especially in the presence of 3rd party API's you may know very little about.
SteveWilson? writes that any response slower than about 50ms is going to feel "sluggish" to the end user.
This page mirrored in ExtremeProgrammingRoadmap as of April 29, 2006