Polymer Clay + Swarovski crystals?

Depending on whether you mold around the "pushed" hole, it will probably fall out. You have to provide edges to hold the glass. On the other hand, having made the hole, you can glue the glass back in place.