Early on, I asked most YC founders I met whether they did testing in the early days, and almost all of them said "no". I've also not written tests in the past simply because it's a time investment--why test if you could be working on something entirely different in a few weeks? Code can be very volatile in an early stage startup.
Think it makes more sense the later stage your startup is where you're more certain of what exactly it is you're building.
Even when you're prototyping, I find it useful to write one test. The gains from the first test are the biggest - pretty low investment, with reasonable returns.
It won't be great, but it will provide some form of sanity checking when you work on other stuff. Of course, it informs the design, which is a very overlooked feature of testing.
Lastly, it provides a foothold for more tests. When you're working on something hairy, there won't be any obstacle to "well, maybe I'll just add this one more test to save me some time".
If you are throwing your code away every few weeks, it is probably wasted time. If your codebase is in a lot of flux, it will save you a ton of time, since a good test suite tells you what breaks every time you change something.
If you are trying to really quickly get code in front of users, and are working through a lot of ideas that don't end up going anywhere (code that is eventually thrown away), then heavily tested code is probably not the best use of your time.
Once you get a product with some traction, and are going to be working with a codebase for some time (especially a code base that will be growing), heavily tested code is invaluable.
Example: Upgrading a large Rails app (~250k lines) from rails 2.3 to 3.0 in eight weeks. Having roughly a 1:1 code:test ratio allows us to be extremely nimble. It also allows developers to work in almost any area of the codebase with confidence.
For apps that will be around for awhile and will be growing, a large test suite is indispensable.
Think it makes more sense the later stage your startup is where you're more certain of what exactly it is you're building.