Just had this experience. Took an existing code base and worked about four weeks to try and modify it to support new functionality. Had an epiphany that 'this shouldn't take this long', started from scratch and duplicated all existing functionality as well as the new code within two days.
Sometimes tossing badly designed code seems like two steps backward, but that isn't always the case.
The big trick of course is that you were in a good position to actually make that call and it worked. The difference is that there are people that will shout 'rewrite it' without being in a good position to make the call and as some kind of personal 'NIH' syndrome.
I've done what you just did to a package that I maintained literally for years for a company that I contracted for and at some point the same realization hit me, it shouldn't be this hard to just toss this and do it again. But by then I had a pretty thorough understanding of the problem and of the flow of the code that solved the problem (even if it was weighed down with a lot of junk). Rewriting it made life much better. But typically if someone is new to a project and needs to fix a minor bug and starts to say we should re-write all of this they're just plain wrong.
I don't think it is just NIH, the new people just have a very superficial view of the amount of issues, choices and decisions that were faced in building the system. That means they have a tendency to underestimate the costs that go into building the system.
Exactly, that's the core of the issue: it is very easy to overestimate the advantages of the new design/rewrite. But even worse very easy to underestimate the advantages of the old design. Because in a bad system (the ones you want to rewrite), things are generally not that well specified.
That's also defines what may work as a rewrite and may not work: if your application has a lot of external dependencies, and is used by customers in a very tightly way, rewriting it will take forever unless you don't care about losing your existing customers (because you lose what works and what does not). What makes matter worse is that you are more likely to make those mistakes early in the business.
If your application does not have tight integration with the customer, then it becomes much easier to replace it, one part at a time. Otherwise, you are likely to just recreate the same monstruosity anyway once you managed to support half of the features from the old version.
I think rewrite make sense in some cases, but the natural reaction should be don't, especially when you don't have that much experience. Successful rewrites are the exception, not the rules. To make a dubious analogy, that's like the junior programmer would think that his bugs are actually in the libraries/compiler he is using. The senior programmer knows it is almost always his own fault, and the very senior one has a few stories about long night debugging caused by compiler bugs in the old times.
I'd think that that requires that you really understand the code top to bottom. I know I've looked at code other people have written and gone, "that doesn't look right" and redone it 'right', only to discover the hard way that there was a good but subtle reason the code looked the way it did and my fix introduced a couple of very subtle bugs.
Sometimes tossing badly designed code seems like two steps backward, but that isn't always the case.