Your look, your delivery, your rhythm, your attitude, and a bunch of other stuff are what make you funny. Think Groucho or Emo Phillips or Rodney Dangerfield. You want to laugh from the second you hear them talk.
I first felt this onstage while doing my French character, Simon Beauregard. People would laugh at the character. Just the accent and phrasing of what I said would get laughs. A light bulb went off. I was like, "Wow, this is so much easier than relying exclusively on setup/punchline to get laughs."
From Woody Allen On Comedy:
It's all about being a funny person, not about having funny material. Give yourself not your material...
Don't make the mistake of falling into the material trap...You should think of it in terms of you as a funny person. To the degree that you're a funny person, that's how much you'll succeed, not to the degree of the funny material that you have...
The audience wants an intimacy with the person. They want to like the person and find the person funny as a human being.
The biggest trap that new comedians fall into is trying to get by on the basis of their material. Trying to buy or write material and hiding behind it and not getting out there and just opening themselves up."