One of the toughest parts about being busy and working with lots of people is the, "Oh, one more thing..." when you're on your way out the door. Usually, whatever it is turns out to be relevant and important, yet you can get "one more thing'ed" to death if you always take these.
There's no great solution. Sometimes saying, "Gotta head out now. Urgent?" is the right way, other times listening is correct. Maybe best of all is to stop it from happening by addressing key stuff earlier in conversations.
And also -- though it's probably underrated as advice because it doesn't sound glamorous -- making a clean exit seems to hold this down. Once you say you're going, just head out immediately, or potentially face a bunch of one more things.