Basic Ultralights can typically avoid flying in areas that require a transponder as they are generally restricted to flying in Class G & E airspace.
Transponders are not required to fly in Class G (uncontrolled airspace) but are required in some Class E (controlled airspace).
Advanced ultralight pilots that intend to fly in to airspace identified as requiring a transponder Including all Class C and some Class D & E airspace will need to be appropriately equipped.
All Class A & B airspace also requires a transponder. Class A airspace starts at an elevation of 18,000ft ASL and VFR flight is not allowed. Class B airspace starts at an elevation of 12,500ft ASL or above. Most Ultra light pilots fly well below Class A & B airspace.