Give it a try!

"It" needs an antecedent.   That is a noun or noun phrase that it refers back to.   The key word here is "back".   In your examples #1 and #3 you are trying to refer forward to "doing something" and that won't work.   If you have a previous sentence defining "it", you can just say, "I'll give it a try."   For example: "They say bicycling is easy.   I'll give it a try."

Your example #2 could work but it is a little awkward. "I'll give a try bicycling" should be rearranged to "I'll give bicycling a try" and it flows a lot smoother.

There are areas like this that are not addressed by the cold mechanics of grammar and syntex, some place for a 'living sense' of a language. Is there an English word for that? --something that is not grammar but still implies more of 'rule' than, say, 'common usage' would.

I agree with Dick's judgement:   "I'll give it a try", or "I'll give bicycling a try".   You can also say "I'll try bicycling", but of course Rafee was asking specifically about "give it a try".