We can always tell if we forgot to feed the fish, because when they see us walk by they will all group up at the top of the tank, to hungrily wait for food. But even if they've just eaten, some of them get overly hopeful and line up to wait for food when they see us walk by, even though they must be full.
I don't know if it's peer pressure, or just a couple of our fish randomly have a worse memory than the others.