You won't have to break through the crust of Victorian London's Thames to wallow around in the water looking for music boxes. Happily, they're all on dry land for the most part.
Secret #14 - Thames
This Secret is between Cannon St. Station and the easternmost viewpoint
If you look toward Cannon St. Station from the viewpoint, you'll see a pier. Make your way over there.
Climb up to the roof of the hut at the end.
The music box is up here.
Secret #03 - Thames
This music box is right next to the southern viewpoint.
From the viewpoint on the boat, look to the port side (left as you're facing the pointy end) and you'll be able to see the music box on the roof of the deckhouse on the tugboat below.
Pop over and pick it up.
Secret #20 - Thames
You can find the music box on a strut of the bridge to the west of Waterloo Station.
Start off at the viewpoint in Southwark to the northwest of Waterloo Station. The music box is on the first strut of the bridge that you can see from here.
Make your way along the outer edge of the bridge and drop down to the ledge on the strut.
Grab the music box and head off to pick up the last Secret in the Thames.
Secret #21 - Thames
The music box is sitting smack dab in the centre of the easternmost bridge over the river.
It's on a bench on the western side.