i wouldn't say they taste like olives per se, but the comparison is worth keeping in mind. the reason behind this is that capers can be brined the same way green olives are, so, naturally, if you know (and like) the taste of brined green olives, you won't be caught off-guard when you try them for the first time.