There are some preachers who determine to make their sermon last 20 minutes and no longer. There are others who never fall short of an hour. The ones who go long tend to argue that the text demands it, and that spending time on exposition is a reflection of how important we think the Bible is. The ones who go short tend to talk about the attention span of their listeners, and usually focus on a single point which is hammered home over the 20 minutes.
There are always exceptions, and that is a massive oversimplification, but I have a few thoughts on the matter.
You are the preacher!
Remember that you are preaching, not someone else. Some people are quite capable of talking for an hour on the same subject without becoming stale. Its a real blessing to hear people like that preach, but you are not them. If you cant hold peoples attention for more than 30 minutes without being boring, then for their sake dont go over the 30 minute mark! Just because the sermons on your iPod last an hour doesnt mean yours need to.
The Bible is deep.
So deep in fact that you can take a passage and spend 2 hours plumbing its depths, and still have a long way to go. You could quite comfortably spend an hour on most passages without exhausting them completely. This means a couple of things.
First of all you havent failed to do the passage justice just because you didnt say everything. You should be inviting your listeners to look at the Bible for themselves, it would be a tragedy if they thought you had told them everything so they didnt need to bother looking.
Secondly you will almost certainly have to cut out a lot of the material you covered during your sermon preparation. Cutting material which you have worked hard on during the week is difficult, but if you dont then I can guarantee that people will get bored, and your sermon will be too long.
Open your eyes
When youre preaching, open your eyes and look at the congregation. If half of them are nodding off, then you have probably been going on too long. There are lots of ways of making a sermon more engaging, but no matter what tactics you employ there will be a limit. If youre paying attention to the listeners (and you really should be), then you should be able to spot when you have reached that limit for most of them.
Theres no right answer
I would be a little worried if you never spoke for more than 5 minutes, because you cant really say much in that short a time. However, so long as you are actually talking about what the passage says, and have covered it to the extent that your listeners have understood and can do something with it, then youre probably doing fine. 20 minutes is a fine time for a sermon, and so is an hour, as long as the length is appropriate to you as a preacher, and does justice to the passage.