מקפים לוהטים

[This may be my longest-sitting draft that I finally made into a post. I started it in June 2017, shortly after this graphic made the rounds.]

After studying phonetics in my first year at Tel-Aviv University, I developed a pet theory. See, every phenomenon we came across that was "unique" (or rare, or marked) seemed to have somehow skipped over the Hebrew language. No crazy nasals, no retroflexes, no gutturals (in the standard Israeli dialect), no clicks, no ingressives, no voiced alveolar lateral fricative, just the five canonical vowels, et cetera. My thoughts were, since Modern Hebrew is a revived language co-learned by people from very distinct linguistic backgrounds in a relatively messy (high-entropy) distribution, the phonetic portion of it evolved to a low-common-denominator, most phonetically boring language out there.

But where there's data, there's a chance to test out pet theories. So as soon as I got word of the phoible dataset I immediately jumped and put my theory to the test (then waited 15 months for absolutely no reason to actually publish my findings).

Phoible is an open, simple-format data source for phonetic inventories of language. A few clicks, and you have a table of all phonemes across all languages. Thus my definition for boringness of language converged to:

A language is boring if it chooses boring sounds for its inventory.

Now I need to figure out which sounds (phonemes) are boring, but that I just define based on their frequency across languages, which gives me a clear algorithm:

1. Calculate each phoneme's frequency in the database
2. For each language, calculate average phoneme frequency from its inventory
3. Rank languages from high to low average (high = boring)

That's it! For my first finding, Modern Hebrew got a score of 0.415, which ranked it 1,426th most boring in a field of 2,155 languages, an utter refutation of my hypothesis. The most boring language according to this metric is Southern Nuautl with a score of 0.764; the most interesting is !xóõ (yes, that's a click sound it's starting with) with an astounding 0.103. The average score was 0.464 and the score progression is a normal-distributed thing of this:

Here's a taste of some languages I thought could be of interest. Check out that lovely long tail of Igbo, which has may phonemes but a lot of the frequent ones; or how fast Quechua plummets from frequent to semi-frequent to rare phonemes; or how boring Swahili, a high-contact language dominating the 20's of the x-axis, is (it's what I expected the Hebrew situation to be, and even so its score is a very unboring 0.292); or how English falls so quickly in the beginning, with all its weird vowels and labiodentals and taps and flaps.

This could mean a few things about why I didn't get what I expected, as well as some other unrelated reasons:

1. Like always, data is dirty (or at least, this data, for my analysis needs). In this case, more well-documented languages may have more phonemes in their dataset, probably leading to some that are rare, than languages with less extensive research done upon.
2. My metric must suck. See how Hebrew has the fewest phonemes in the selected sample? That's gotta account for boringness and yet, with my mean it doesn't. Look how many frequent phonemes Swahili has, and yet its average is very low. Let's consider some other metrics ("Future work". Remind me to upload the data if I don't do so soon):
   1. Number of phonemes (a boring metric for a boringness question)
   2. % of phonemes above a boringness threshold p.
   3. Area under the boringness curve (this is just the sum of boringnesses again – but maybe cut it off at some point?)
   4. Deep neural net trained on all these features with the single data point <Hebrew, TRUE>.
3. A bug in my code. As soon as I find it I see if I can look deeper. I mean it's kind of a miracle that I have the post-processed TSVs around, to be honest.

Here's the path I took, as a non-US-citizen, before moving to the US and starting a CS PhD program. I took the GRE and TOEFL around summer of 2015 and moved in August 2016.

I provide it as a reference for similar-minded folk, but keep in mind times change, circumstances vary, and I may have forgotten crucial steps. In any case, enjoy.

(Created using GraphViz)

Since Tuesday's elections, I've been hearing a lot about the alleged irresponsibility of the American voter, not turning out for the elections. At the same time, there has been the usual fuss over the electoral college system and how some states are meaningless to bother going out to vote.

I have yet to see a piece trying to tie the two together (please correct me if I'm wrong).

My claim is simple: citing the nationwide 56.5% figure as a strong indicator for voter apathy is somewhat misleading. If a Californian feels they don't see the point in voting (and registering beforehand), it's different than a Pennsylvania voter (in this elections cycle at least, but pretty much usually). It's unfair, but understandable, if there's a (say) 15 point difference in their turnout rates.

Let's look at the numbers then, shall we? On the x axis, we'll place the ultimate victory margin (collected Thursday from Wikipedia) as a proxy for how inclined an average voter was to believe that his vote would be crucial. It's not a perfect proxy of course, as there were some state-level surprises. Maybe poll margins prior to registration deadlines would have been a better one. The y axis will denote the voting turnout (collected from electproject.org).

Before the chart, observational data: of the 11 states with highest voter turnout rate, 10 ended up with a margin under 5%. Of the 10 states with margin under 4%, only one had a turnout of less than 60%. Now you can look at the chart, including a simple linear trend line.

As you can see, the results are pretty straightforward. With a not-bad correlation of 0.21, it seems voters chose to turn out based on how close they anticipated the race to be in their state. I didn't leave out the outliers but they're not shown in this chart (DC is always ridiculous, this time with an 86% victory margin. Hawaii significantly undervoted with a 34% turnout, way under the next, California at 45.5%). It was cool to see Utah as a special case here with its 3-way race – a 19% D-R margin brought significantly less people to the polls than Montana or Washington state who ended up with about the same margin.

All in all, the voters who mattered in this Presidential election (tough phrasing but that's the way it is) came in at about 65%, much higher than the national average.

It's worth noting that the numbers, even for the swing states, are still low compared to most of the democratic world. But I'll also note that the US has other factors going for it, such as the no-day-off thing, or the huge amount of expats allowed to vote, which is a unique characteristic. According to electproject, these compose roughly 2% of the eligible electorate.