Google Street view shows the Swan Lane bridge was still there as recently as three years ago, and the pedway seems to have followed parallel to Arthur Street to emerge opposite the Monument. Arthur Street itself is closed at present, I think for work on the Bank station enlargement. We may see the pedway return?
By the way, the thoroughfare crossed by the Pudding lLne bridge is Lower Thames Street - the name changes from Upper to Lower as it passes under London Bridge.