The fear is that the bad thing you just said it was lucky it didn't happen is just about to happen because you just said it hadn't happened. Example:
"I'm surprised no one's eaten those big, plump Pekin ducks we have out there."
"I know. The fence really isn't that good."
And now the fear is that, tonight, the quackers shall be eaten. I guess we should have knocked on wood?
Not my picture.