Thanks for the input guys. I've gone ahead and ordered a Medium. I've also ordered a pair of A* tech series gloves to replace my JR Speed Master gloves which had an off bike excursion a while back. Man those A* tech series gloves sure did cost a lot of money, but man they look like heavy duty protection for me the next time I decide to depart from my bike while it is in motion. Hope it doesn't come to that, but if it does I'll be a little better protected.
Personally I would stick with the 48. Leather stretches and I'm pretty sure with a back protector, you would be right in the ball park of a snug fit. However you are in the best position to make that decision.