http://packages.python.org/py2app/tweaking.html
This might shed some light.
You'll need to specify what platforms the application will run on otherwise it will default to the runtime libraries for your current OSX release (might be 64 bit for instance and old time macs with 32bit can't run it (as a rough example))
The same principle goes for Py2exe which does the same thing but for Windows binaries.
If you compile it on Win7, and try to run the binary on a XP machine, you'll get missing runtime dll libraries. Then you have to choices, either compile it within the app/exe or ship the runtimes with the application alongside the binary.
What you're mainly looking for in Py2app is the --resource parameter.
http://svn.pythonmac.org/py2app/py2app/trunk/doc/index.html#option-reference
--resource <comma separated list of resources> which will include not source code but libaries, images and whatever it is you need to ship to older platforms.