Pedestrians have the right of way if the are walking in the direction of traffic even when the red hand is up and they are in a crosswalk. This would also be true for bikers. The hands are a general indicator of safety, not at all similar to a stoplight to cars.

Is it smarter and usually safer to abide by the walk signs... yes.

There are a lot of sections of my ride where its simply unsafe to ride on the road, its too narrow for me and cars. So I have two choices, be a dick and stay in a lane and act like a vehicle forcing everyone into another lane to pass me, or ride the sidewalk. I take the sidewalk most of the time but if the lights green as I come up to a crossing I'm not going to fucking stop, press the button, then wait for a whole cycle to go though just so the little white man can say I can go now. Fuck that, I ride like that crosswalk is mine, while still being mindful to see if any cars on the road are expecting to turn, actually using their turn signal which they are supposed to do by law.