The Dollar Store sometimes has plastic or silk vines, otherwise I purchase stuff on eBay. The plastic aquarium plants are perfect, and easy to clean. The taller ones lean over and create ground cover, the little guys love it.
I picked up some river rocks from our yard, the really colorful ones that don't have sharp edges, and some driftwood. I have some seashells too. All of these I boiled in water with a capful of bleach, rinsed really well and then baked at 250° for about 20 minutes.
Other than that, get creative! Look at garage sales, or around your house for something that would look cool. Just make sure it is sealed or well cleaned if it is porous.