it's used as a number to simplify conversations. instead of saying the limit of the expression 1/x as it tends to zero is infinity, we simplify that statement and say 1/0 = inf. it's nothing to get mad and write a scathing dissertation over. if you want to delve into it you can if necessary.
there are infinite stars in the heavens. decidedly untrue if you actually were able to make a count of them. but by the limits of our ability, it is understandably true.
I can only recommend reading his book 'On Numbers and Games':
"On Numbers and Games is a mathematics book by John Horton Conway. The book is a serious mathematics book, written by a preeminent mathematician, and is directed at other mathematicians. The material is, however, developed in a most playful and unpretentious manner and many chapters are accessible to non-mathematicians." (http://en.wikipedia.org/wiki/On_Numbers_and_Games)
There just isn't one official list of "what is a number". Real mathematicians will consider anything a number temporarily if it's convenient for them.
For anyone who cares, I recommend David Foster Wallace's book "Everything and More" for a great history of mathematicians' thoughts about infinity.