If you want the ultimate all-in-one device, then iPhone is really the only way to go. If there's anything you've ever thought you might like to be able to do with your phone, iPhone probably does it.
However, as others have said, AT&T's network is overloaded (because millions of people love the iPhone). They simply can't keep up with the demand for network bandwidth. Part of that is AT&T's fault for being too slow in beefing up their infrastructure, but part of it is also Apple's fault (tongue in cheek) for building such a useful device. So, if you want "blazing" fast speeds on your mobile device that rival the speeds of your home broadband connection, then the iPhone will probably frustrate you. For me, I've never expected my mobile device to perform as quickly as my computer at home.
That said, the other problem with the overloaded AT&T network is dropped calls. I don't spend a ton of time actually talking on the phone, but dropped calls and/or bad connections really bite. So again, if you talk a lot on your mobile and really rely on it (e.g., if you have no land line phone), then the iPhone might be quite frustrating.
Personally, I'm optimistic that despite AT&T dropping the ball, they'll eventually catch up on their infrastructure and then everything will be fine. And if they don't, Apple will be forced to drop them as the exclusive U.S. carrier and then Verizon or someone else can hopefully do better at managing bandwidth issues.