Today after a power problem a production server wouldn't start correctly. We found a rogue root cron entry installed by a senior team member that deleted all files starting from root that hadn't been accessed in 4 days. Needless to say, that cron entry did a lot of damage.
I made a mistake like this once. There was a zero length file in / called ‘????’, so I, being the smartest unix guy on earth, did a rm ????. Little did I know that that would remove all files with a filename of length 4. Back then, on AIX the kernel was kept in a file called /unix, so on the next reboot you can guess what happened. That’s right….nothing!
That is how you learn the hard way. I spent most of that night restoring the system….it sucked!
