Are you sure your ankle is broken?
What is the actual diagnosis given by your doctor?
Many people think they "broke" or "dislocate" your ankle, when in fact it's a sprain. There's different severity of sprains. Grade 1, 2 or 3.
http://en.wikipedia.org/wiki/Sprained_ankle
Anyway, you can and should still workout around your injury without aggravating it.
You could isolate muscle groups and train every muscles in your body apart from your lower leg.
As your ankle gets better, slowly introduce light compound exercises for your leg and increase intensity.
Don't forget to do your ankle rehab while you cross workout/train.
Keep your body moving without aggravating the injury, and you will heal faster.
By leaving it in a cast immobile for weeks and more, hoping it will recover faster, will only lead to prolong recovery.
The sooner you get back on your feet, the faster it heals. Of course as I mentioned, you do not aggravate the injury in the process.
Good luck!
Yours in Fitness,
Glen Poh
www.evori.net