In order for your voice to really connect with people you need for it to be consistent. You can’t have an anti-establishment outlook on the world in one bit and then suddenly be pro-establishment in the next or else your voice will seem disjointed and nobody will be quite sure what to think of you. For example, if your comedic voice is that of a “nerd,” then it’s unlikely that bits about how you were a cool kid in high school are really going to fit your voice. This is another reason why it’s important when considering what your comedic voice is to choose carefully because it will ultimately impact your material and every other decision you make about your career.
I've learned a similar lesson when doing characters onstage. Play a certain role and it becomes much easier for the audience to know exactly who you are, where you're coming from, and the "filter" that all your jokes go through. The bad part: Being a fully, fleshed out human being with nuanced views doesn't fit this mode that well.
You may be tempted to give an audience all your different sides, but a caricature is easier for 'em to figure out. Especially in a short set.