I think the on/off button/switch only controls the LEDs. I would put the gargoyle light in the sun for 1/2 hr or so to charge some, then bring it inside and cover the solar panel. If the lamp doesn't light, press the button or toggle the switch to see if the light was turned off. I think that should be more than sufficient time to give it enough charge to see if it works or not. No light either way, I'd exchange it.
BTW I always cover the solar panel when in a store and try the button to see if the item--solar lamp/solar rock/solar Tiki god--works. Generally if it's been sitting on a shelf where the overhead lights reach the solar panel there will be some charge. Sometimes people turn these things on and leave them on a shelf that doesn't get enough light to recharge and because it is "dark" and left on it will deplete the charge without being able to recharge, so it's not always a reliable test if the item will work.