You sound exactly like those people who say
Soccer is a girls sport, all you do is kick a ball and run after it.
I don't see how rugby could take a lot more "skill". Why don't you point out why because you sure don't have me convinced.
Both sports are similar in concept, but strategy comes into play hardcore with football...you have passing, running, and creative plays that you can use. It's definitely hard work to know and keep your composure to pull them off correctly.
Now rugby doesn't have pads yes, but football is definitely more intense for a short time. Rugby has the overall tough factor, where you have to be tough for minutes compared to seconds.
I'm not taking anything away from rugby, it's a very intense sport, but you have it more spread out over time so you aren't forced to make a play by a clock and such. Rugby on the other hand takes a lot more endurance, you see a lot more fat people in football and a lot more fit people in rugby. Just because of the longer endurance needed.
Both sports can hurt.