What kind of olives are you eating? Olives are like cheese in that there is a range. You have mild Cerignola olives and some rather funky Sicilian (or green) olives. You also need to consider how the olives are cured (oil, salt, water, dry, etc.). I recommend starting with oil cured .
Most olives you find in the super market are pretty funky. If you find yourself at an olive bar try a few different olives. The problem isn't your taste buds, its how you were introduced to olives.