cellio | daf bit: Kiddushin 69

According to the mishna, ten genealogical classes of people returned to Israel from Babylon after the first exile. These are: kohanim (priests), Levites, Israelites ("regular" Jews), halalim (disqualified priests), proselytes, freedmen, mamzerim (products of a forbidden union), netinim (temple assistants, possibly Cana'anites), shetuki (to be defined), and foundlings. On these last two, the mishna says that the shetuki knows who his mother is but not his father (so he knows he's Jewish but doesn't know his tribe), while a foundling was gathered in from the street and knows neither his father nor his mother (but if he's circumcised the g'mara assumes he's Jewish).

The mishna goes on to list who may marry whom (which may be why it bothered to enumerate these ten groups). Kohanim, Levites, and Israelites may all intermarry. (These are, by the way, the three main groups; if you hear people talk about three categories of Jews, this is what they mean.) Levites, Israelites, halalim, proselytes, and freedmen (but not kohanim) may all intermarry. And prosselytes, freedmen, mamzerim, netinim, shetuki, and foundlings may all intermarry. (69a)

If you map this out, you'll see that these last few categories, who mostly have uncertain or non-Jewish parents, are pretty limited in whom they can marry. One reason for the restriction is to prevent accidental forbidden marriages -- for example, we don't want a foundling who doesn't know who his parents are to accidentally marry his sister. I guess the rabbis see the odds of this being very low if they stick within these groups; in principle two foundlings could have the same parents, but the only alternative would be to forbid them from marrying at all. (Skimming ahead a bit, it looks like they do in fact make the rarity argument on 73a.)