The most obvious refactoring analogy I can think of is communal areas such as the kitchen of a shared flat. It’s everyone’s responsibility to keep it clean but often it quickly gets in a mess because people don’t bother to clean up after themselves. Sure, the cycle time to getting a meal may be quick, but after a while the kitchen becomes unusable. Finally a huge amount of effort has to be put in to cleaning it as some of the dirt such as on the cooker is really caked in by then. Other things are beyond cleaning have to be thrown away altogether.
Yesterday I spent a few minutes tidying the bookshelf at work. There was stuff on the shelves which shouldn’t have been there such as screws and mobile phone chargers ( commented out/redundant code), planning stuff spread across multiple shelves and mixed in with books (poor cohesion) and various colours and sizes of index cards in big unsorted piles (obfuscated unreadable code).
The same principles apply – leave it in a better condition than you found it. Be considerate of your colleagues and everyone benefits.