This is pytz's workaround for the fact that datetime.tzinfo, the abstract class that represents the interface between time zone objects and datetime.datetime objects, is expected to be able to discover the offset of a time zone given only the local time, which is not in general possible because some local times are ambiguous on account of offset changes caused by Daylight Saving Time or other governmental action.
The purpose of localize is to take a local time and an additional is_dst parameter and return an unambiguous datetime.datetime with a time zone object that is tailored to give the correct offset for that time. But a pytz time zone that isn't the result of localizing a time knows it can't always give the correct offset, so it doesn't try very hard-- instead, it just defaults to the first entry in the Zoneinfo database for that time. In the case of US/Eastern, that's just local mean time in New York (hence those crazy four minutes). You can get that same offset by localizing an early enough time:
In [28]: pytz.timezone('US/Eastern').localize(datetime.datetime(1901, 1, 1))
Out[28]: datetime.datetime(1901, 1, 1, 0, 0, tzinfo=<DstTzInfo 'US/Eastern' LMT-1 day, 19:04:00 STD>)
I do not know why pytz version 2012j exhibits different behavior, but I would guess that either the historical entries were added to Zoneinfo sometime in the past two years, or that at some point in that period unlocalized pytz time zones switched from a (sometimes subtly wrong) guess at the correct offset in Zoneinfo to the (obviously wrong) oldest offset in Zoneinfo.
Once PEP 431 is finished, datetime.tzinfo methods will take is_dst parameters where appropriate and pytz will be able to implement time zones that do the right thing without having the user jump though localize and normalize hoops.