Shift will select all the tags between the previously selected tag and the one you're clicking. (Also, you need to hold it down as you click.)

Rather, holding down Control, rather than Shift, as you click, should do what you want!