Interesting observation! My hypothesis is that babies love outlets for the same reason they love little ants crawling across the floor, cracks in the wall, etc. They are an intriguing -- even distracting -- interruption of the normal.

As to outlets being installed upside-down, I believe that it is normal to install an outlet upside-down only if the outlet is partially switched. E.g., if a switch in the room controls the bottom but not the top socket, then typically the outlet is installed upside-down as an indication of this fact. I don't know if this is a matter of building codes, but it does seem to be a very common practice.

The outlets don't have to be installed face-side-up. It's perfectly acceptable (although less common) to install them with the "mouth" at the top.

Actually, outlets remind me more of Much's scream than a cute face :O

An interesting thought. I personally believe it is because they resemble the "put a block into a slot" style games - and babies seem preoccupied with inserting things into slots, especially their own mouths.

Of course, your hypothesis could be tested. It would make a nice experiment.

Looks like a frowny face to me... :)