The reason why men’s shirts have their buttons on the right side while women’s garments have theirs on the left stems from opulence and convenience. In the 19th century, only affluent women can afford clothes with buttons. And if you are a well-off lady during those days, you don't dress up by yourself, you have chambermaids to dress you up so their buttons were placed on the side most convenient for right-handed servants. The idea must have came from dressmakers who assist clients during fittings.
In the case of men's buttons, even the wealthiest bloke of the olden times generally dressed themselves up, even if they have servants who can very well assist them. Thus, the buttons are on the right side to make fastening or unfastening a piece of garment quicker and easier for the right-handed man. This proved to be useful during unexpected duels where gentlemen can easily unbutton their coats with their left hand while drawing their swords with their right.