Choosing a Host
So, you have your domain and want to host it somewhere. The options available are rather dizzying.
- Shared Hosting is where you rent space on someone else's server. These plans range from fragile, insecure, and wildly-oversold shops to 'semi-dedicated' ranges that offer you massive parts of a server, dedicated IP addresses for SSL, and so on. Most website operations start here.
- A Virtual Private Server, often shortened to VPS, is a virtual server that runs on a larger dedicated machine. Note that, despite what you might think, a VPS is not necessarily an 'upgrade' from shared hosting, though it may seem like it. You can find many shared hosting offers that are far more powerful than most company's VPS offerings. Instead, what a VPS gives you is flexibility - you install your own software and manage your own needs. VPS hosting plans tend to come in managed (someone who knows their way around cPanel or whatever manages your software stack for you) and unmanaged packages.
- Dedicated Hosting means you are renting a server, though you may sometimes own some of the hardware or otherwise buy it down for them. As with VPSes, it comes in managed and unmanaged varieties, but unlike VPSes, it is definitely an upgrade from a shared plan - your machine is not host to anyone else.
- Colocation is where you own the physical machine, but you hand it over to someone who has floor space in a datacenter (or owns the datacenter itself) to host it for you. While always 'self managed', you can purchase management from various companies, though real administration ability will cost you.
In general, VPS hosting best for fulfilling needs that shared hosting cannot handle. If you want training wheels while setting up your first server, or if you need to run some odd piece of software, or if you want to run your own mailserver because your host's mail gets spambinned, then a VPS may be for you. If you just need more power, more space, or something similar, you can find good, powerful plans. Searching for "Semi-dedicated" and/or "Not oversold" / "No overselling" may help.
For the love of whatever god(dess) you may believe in, if you have a community with any meaningful level of activity, do not subject your members to a VPS with platter drives. VPS I/O management is terrible in general, and platter disks can render a large forum nearly unusable.
Eventually, you may end up outgrowing any possible VPS/Shared solution, and it is time to go dedicated.
I recommend starting off with dedicated rather than going straight for colocation, because if something goes wrong with your hardware, and it is not a piece that your host's team can easily swap (hard drives and RAM are pretty common), and you don't have a machine there to do your work while things get fixed... you are in trouble.
Location
When I first looked into hosting providers, I was hoping for one in my home state (Minnesota). I found some; insanely expensive, and all traffic going to them was first routed through Chicago.
Telegeography has a map that shows how well-connected cities are, though you may want to take the numbered rankings with a grain of salt; I doubt Montreal services more traffic than Atlanta or Denver, for example. Target your hosting to your audience and purpose. In the US, hosting in Chicago is pretty fast for all of North America - but if most of your audience is in Europe, you may want to look at London or Amsterdam.
Purpose matters as well, of course. Fractions of a second may be important for human audiences, but your mail and backup servers do not need to be so responsive. Less expensive hosts that are somewhat off the beaten path can do well in these cases.
Many shared and virtual hosts do not bother advertising their location or worse, actively try to conceal it. While location is probably not your main concern if you are just setting up your first website, this type of sketchiness should put you off.
Range of Services
So, imagine you have your website, and something goes wrong. When you call them, do you know how many people are between them and those actually working on the hardware involved?
If a company only offers shared hosting, this is a very bad sign. Even if you have no use for anything more, reseller plans go for $20 a month or less. This is generally done by people running their own web design business - they host their customers, and likely do not have much web presence. This is fine if you know the person. Otherwise, no.
A company that offers reseller hosting probably owns their own machine, at least, and a company offering virtual servers certainly does. These people should have current clients who will vouch for them, at a minimum.