Chew's right. Since the iPhone 4 has a smaller sim, you'll need to get AT&T to issue you another card for the Inspire, essentially crippling your swapping.
Or just get a micro-sim adaptor. That's what I'm using when I switch from iPhone 4 to samsung focus