Yoshi got dragged into Peachgate entirely by accident. If the poison princess hadn't chosen his native habitat for a sordid rendezvous with Bowser in 1990 he'd still be living a perfectly innocent and happy life. But we all know what happened. Unaware of the backstory to the whole depressing affair, Yoshi agreed to help Mario on his never-ending quest for emotional buggeration and forged a strong friendship with him in the process, despite the obviously crippling back pain that comes from having a fat plumber on your back for hours at a time. You wouldn't believe how sore that gets. (We've been told)
Later on however, presuambly through conversations with Luigi during the first Super Mario Kart, Yoshi found out what was going on. He continued to hang out with Mario for a while, still feeling the bond of their shared journey and sympathising with the hard start in life the plumber had had on his very own island as a baby. Having a man's arse imprinted upon your person kind of makes you feel pretty close to him too. (We've been told)
But eventually Mario's lack of self-esteem got too much and Yoshi followed Luigi's example. Thus he went straight to the roof when Peach engineered a disappearance in Super Mario 64, and resolutely stayed there until Mario had finished humiliating himself. And he didn't even bother turning up for Super Mario Galaxy. What does that tell you?
Once the loyal servants of the princess, the Toads caught onto her endearingly unstable nature pretty early. They happily spent the entirely of Super Mario Bros. waiting around for Mario and Luigi in order to point them in the right direction, but after doing that over two different games and being forced to run item houses to help the brothers on another rescue in SMB3, enough was enough. They'd seen Peach handle herself in the US SMB2, so they knew that she was getting kidnapped suspiciously easily. By the time word got back to them about the events of the ill-fated Dinosaur Land trip, they knew beyond all doubt that they were being used.
"F*ck that shit!", the Toads thought. "We're not hanging around out in those cold-ass castles any more for no-one". And they didn't.
Not only that, but they now actually look forward to Peach's little vanity trips. They know she's going to be okay (And frankly, after all these years of dealing with her crap they couldn't care less if Bowser ate her whole. They inherit the castle anyway), so they treat the kidnappings as free holidays. After really coming into their own in both SMB2 and Super Mario Kart, the Toads have now developed a healthy taste for life and they want to live it to the max. Screw what the crazy girl in the pink dress is up to.
Ever wondered why they were remarkably sparse in number in Super Mario 64? They were all down at the pub. Super Mario Sunshine? They turned up, but only to take advantage of the resort while Peach was busy with Bowser. And Super Mario Galaxy? Peach got kidnapped and their immediate reaction was to build a spaceship and disappear off around the galaxy exploring. Was it a coincidence that we found them on a beach with a barbeque? As soon as Mario's back was turned, they were pulling out the beers and skinning up. But don't do drugs kids. The Toads are going to die.