There are a few major things that need to be dealt with in setting up your private consulting business:
Register a Business
You have to register your business to get any of the balls rolling. I'm thinking that this has to happen first because you won't know if your business name is available until a name search is provided. And having a name is important because you'll need it to get...
In my city, I have to get a business license as well as a permit to run a home-based businesses. This is so that it stays fair between those that actually pay business tax because they have a dedicated business storefront/office and those of us that want to relax at home while writing everything off. Part of the application asks for your business name though, which is why I'm assuming you need it. But oddly enough, that means that really your first stop happens after you...
Hire a Lawyer
Lawyers are necessary for setting up a business, and don't try and talk your cheap little ass out of that statement. Lawyers can provide services and knowledge that we as code-monkeys just don't know, and they can probably expedite the process faster than we would on our own. The lawyer I'm going to be going with does a full incorporation service, including the business lookup. Once you have all that then you still need to ...
Hire an Accountant
I'm not even going to think about Quickbooks...I have no clue how to pay myself, how the tax structures are set up, etc. That's where these guys come in.
Other things to think about for developers are startup costs. "Startup costs, how can there be startup costs? It's just me coding!" Yes, but have you considered...
Because your work will probably want theirs back when you leave.
Software (i.e. MSDN Subscription)
Because your work wants those disks back with their laptop.
Error and Ommission Insurance
Some head-hunting agencies will cover this, or will offer a program where you can opt in...but its always good to ask and its always good to CYA no matter what.
Might not be as big a deal for some, but at the same time remember that you're doing this on your own...you have to create your own safety nets. So if you aren't married to someone that has fantastic benefits, you need to consider getting some for yourself...even if you are an army of one.
Now I'm sure that most of these things can be written off as a business expense on behalf of your company...but have you thought about how you'll fund your company as well? Where will the initial money come from? Will you get a company credit card, and if so where will the initial payments come from if you don't get a contract right away? All things to consider...