I’d say this is plausible, but not a huge risk. A healthy individual won’t have trouble either breaking it down or expelling the foreign matter when the flesh heals (like wooden splinters often do).

I’m not an expert, though, and I’d love for one to chime in on this subject!

Slip socks over the end against the wall to protect the surface. Angle the ladder so the steps are parallel with the floor and go for it! Have someone hold the base of the ladder for additional security.