Accidentally offended someone?

He's a guy. And we were talking online. I think I mightve had offended him by saying 'you like them young'. I knew he has a thing for girls slightly younger than him but I dint mean to offend him. Twas a bad joke and I dint realize what I said after sending it. His ex (who hes crazy about) is younger than him and a couple of days ago he just told me some of the things that happened between them, which was unusual for him to do cause he kinda has that i-dont-give-a-f attitude. Now I feel really bad cause he opened up to me and I offended him. Even though twas in-deliberate.