Most podcasts are a hobby endeavor which usually means budgets for gear, websites, software and everything else involved in making a podcast is limited. When deciding where to put your money first, focus on the elements of your podcast that are most likely to affect the overall quality of your show.
The best place to start is your microphone. This is likely the single largest factor in getting a good sounding podcast. The best budget microphone on the market, as of the date of this post (02/15/13), is the ATR2100 from Audio Technica.
For less than $50 U.S., you get a dynamic mic (best for the typical podcast studio) that produces excellent audio quality (on par with the infamous HEiL PR40) and plugs directly into your computer via USB.
If your budget was only $50 (the minimum I would recommend to start a podcast) you could add free editing software (Audacity on Windows/OSX) and you’d be ready to record a quality sounding podcast.
When you’re ready to upgrade your studio (mixer, audio interface, etc) the ATR2100 can make the transition with you employing an XLR connection alongside the USB. Your original investment is not wasted because this mic can grow with your studio.
So when it comes time to decide where to put your money, start with the mic (where the audio gets in) and upgrade from there. Next, if you decide to integrate a mixer into your production, there are affordable, quality options. Lastly you can think about upgrading your software to paid apps like Adobe Audition.
If your starting budget is larger than $50 and you want to get more gear, you’ll still want to start with the micrphone. The particular model you purchase might depend on what other gear you want to get with it. If you had $200 to spend you could still get the ATR2100 and have $150 to spend on a mixer and required cables.
Shift your budget around but prioritize the most critical elements so that your podcast will be a quality production from day one.
*consider all links affiliates.