Trouble starting, after sitting in car for 20 mins

Bit of a strange one, My car normally starts first time. only time i have a problem is if I park up somewhere, turn the engine off but keep the radio on. then after 20 mins the engine wont start. All lights and radio work so doesnt seam like flat battery, unless it's almost flat? would sitting with radio on cause a battery to drain that quickly. Once it's started by a jump i have no problems. Any ideas?
It does sound as if the battery is weak and the radio is pulling the volts down enough to still power the lights but not the starter.
If you are in the UK I'd take it to your local Halfords - they seem to be very happy to do free tests with a sophisticated machine that gives a printout of the battery and alternator health. Or any other similar place of course.