When shouldn't I implement IDisposable?

When shouldn't I implement IDisposable?

Firstly, sorry if this is considered a duplicate - I know it is a common
topic, but I have looked and not found a satisfactory answer.
There are a lot of questions asking when to use IDisposable, but from
everything I've read, I just can't see why you wouldn't implement it in
every class you make. What is there to lose? Does it have a big
performance hit?
Part of my understanding of IDisposable is that one of the things it does
is: Dispose()ing other IDisposables owned by the object. Forgive my
ignorance, but does this apply only to fields/properties of the given
object, or does it also extend to objects created within its methods too?
For instance, if a Font was created inside a method within a class that
implements IDisposable, but that Font wasn't initialised with a using
block, or .Dispose()d explicitly at the end of the method; would it be
disposed when its IDisposable parent/class was GCd/disposed? Or otherwise,
would the Font never be disposed?
I don't mean to digress, but if it is true that it would act as a 'catch
all' like this (effectively disposing of any erroneous child IDisposable
objects that would otherwise be left undisposed), isn't that reason alone
enough to justify always implementing IDisposable whenever possible?