Wednesday, December 30, 2015

NPR Sexism Analysis #2

This is a followup to a prior blog post of mine, Sexism at NPR?, which garnered a mostly favorable response. Several of the NPR forum comments posed a few excellent questions and suggestions. My analysis covered years 2011 through 2015 and the graph below presents the same data broken down by year, demonstrating a clear trend.

My prior analysis relied solely on NPR’s own tagging system of applying text labels to indicate story subjects. One of my readers posted on the NPR Ombudsman’s Facebook page and the response was “Well, I've written several times about how NPR's tagging is faulty--unfortunately it's just not a reliable way to analyze coverage”.

I’d generally agree that the tags aren’t perfect, but measuring their accuracy is a difficult task. Nevertheless, I took a few approaches and thought they might be interesting to present.

The program used for this analysis can be found at github.com/cmumford/news

“Gendered” Words

All of my analysis so far (and here too) relies on my list of a few gendered words. These were not meant to be all inclusive. I tried to be balanced, choosing equivalent words for each gender. These words (patterns, really) were used in several different analytical tests.

Male Words:

Adult: 'mens?', "men's", "man's", "father'?s?", "grandfather'?s?", 'grandpa', 'males?', 'masculism', "men's rights"

Youth: 'sons?', 'boys?', 'grandpa'

Cancer: 'prostate cancer’

Female Words:

Adult: ‘womens?', "women's", "woman's", "mother'?s?", "grandmother'?s?", 'grandma', 'females?', 'feminism', "women's rights?", 'ovarian transplant'

Youth: 'girls?', 'daughters?', '15girls'

Cancer: 'breast cancer'

Fuzzy words (patterns)

Note that some words, like ‘girls?’ have a question mark. The question mark means that the letter just before the question mark is optional. So, in this example, both ‘girl’, and ‘girls’ will match the ‘girls?’ pattern.

“Gendered” Word Counts

The first test was simply to look at every single word in every single story; to count the number of occurrences of each word, and then tally them up.

Female:Male	Counts	Ratio
girls?:boys?	13511:10227	1.32
women's rights?:men's rights	296:18	16.44
grandmother'?s?:grandfather'?s?	1800:1474	1.22
woman's:man's	946:1291	0.73
women's:men's	3432:1286	2.67
womens?:mens?	30810:19432	1.59
daughters?:sons?	7673:11691	0.66
ovarian transplant:<nothing>	4:0
females?:males?	5732:4133	1.39
grandma:grandpa	432:159	2.72
mother'?s?:father'?s?	15102:13627	1.11
breast cancer:prostate cancer	1034:324	3.19
feminism:masculism	311:0
Overall	81083:63662	1.27

Overall there were 81,083 “female” words and 63,662 “male” words in all stories equating to 27% more female words than male.

It’s difficult to learn much from this metric. A sentence like “my mother was a great cook” and “my mother was a horrible person” would each increment the female gender count by one, but the former is a positive statement about the mother, and the later is negative. This approach ignores any context, merely counting the number of gendered words.

Sentiment Analysis

Sentiment analysis is a fascinating topic. It’s an approach which, in it’s simplest form, uses lists of positive and negative words to analyze sentiment to determine whether a statement is positive or negative (and possibly how much) about a given subject.

Twitter and Facebook feeds are analyzed in order to determine public opinion using these statistical methods. This is one tool used by politicians to learn reactions to their speeches, policy decisions, etc.

Emotional Sentiment Counts

My first approach was to generate my own list of positive and negative (mostly) emotional words. I analyzed every sentence of every story. If the sentence contained a gendered word (see above), and contained a positive or negative word then I incremented the positive or negative count.

Positive words:

accepted
accepting
approval
cheer
cheerful
delight
delightful
devoted?
funny
generous
good
gregarious
happy
kind
kindness
liked?
loved?
romantic
sacrificed?
warm

Negative words:

awful
bad
coward
cruel
deadbeat
disapproval
disapproving
dislike
hate
hit
hurt
jail
sad
violent

The results were:

	Female	Male
Positive	9,247	7,182
Negative	1,643	1,645

This shows 29% more sentences with positive emotional female sentiment than men. Better systems (than mine) measure the weight of words; for example, “love” is more positive than “like”.

Overall Sentiment Counts

I then used a larger list of words called the Opinion Lexicon obtained from the University of Illinois at Chicago. The lists were much longer and contained much more than “emotional” words.

# positive words: 2014

# negative words: 4818

Doing the same analysis as above we get:

	Female	Male
Positive	43,213	33,225
Negative	42,492	36,048

If we calculate (net positive for women) / (net positive for men) then we get 55% more sentences with positive female sentiment than male.

Story Classification (i.e. tag generation)

NPR’s tags are a form of story classification. For example, NPR’s story 'Things Have Changed,' U.S. Judge Says Of Case Over Men-Only Military Draft has two tags: “women in combat” and “Pentagon”.

My classification approach used a natural language classifier which takes a statistical approach to classifying story text. This is the same, but simpler, approach used by SPAM filters to classify your email. Classifiers must be “trained” with example stories that are already tagged. My approach was to use stories with existing tags to train the classifier, which can then be used to tag (classify) other similar stories.

Generally, the more stories the better, but I was limited by the number of currently tagged stories. I only used stories with 17 or more common tags. These were the tags:

Women:

Tag Title	Matching Stories
Women's Health	348
breast cancer	87
women	52
women's rights	20
Total	507

Men:

Tag Title	Matching Stories
Men's Health	50
prostate cancer	28
Total	78

Result

I was surprised at the accuracy of the classifier given the limited number of stories used for training. I only applied a “gender tag” to stories that previously had no gender tag at all. These were the newly tagged stories:

	New Tags
Female	97
Male	18
Total	115

So there were (at least) 97 missing “female” tags and 18 missing “male” tags. I reviewed about ⅓ of them and did not find a single misclassified story. Generally the missing tags were in the same proportion (5:1) as existing tags.

Final Thought

Analyzing Twitter sentiments is a significantly easier task than analyzing news sentiment. A paper titled Large-Scale Sentiment Analysis for News and Blogs does a good job of explaining the difficulty of this task. I would say that the quick analysis presented here supports my hypothesis of a bias in favor of women at NPR — and that bias seems to be growing.

I must confess that I also became increasingly aware of my own bias when writing this story. I constantly had to check myself when selecting various words, or approaches, to this analysis to guard against collecting, even searching for evidence that would prove my hypothesis.

It’s only a matter of time until computer systems exist that will more accurately parse the meaning of sentences. I will be very interested to see how our perspective on a variety of subjects change once we get this magical power to analyze every word/picture/video ever written. I can’t wait!

Sunday, December 13, 2015

Sexism at NPR?

I've been asking myself the following question, "is it my imagination, or does NPR have significantly more stories on female topics than male"? Seemingly every single day I see nominally 1-3 stories on women's issues, and maybe one on men's issues.

So I decided to sit down and see if I could answer this question with data. Fortunately NPR has a web API allowing me to write a program to extract not only the stories, but the story metadata. Additionally NPR tags each story to identify their subject.

Edit 1: The program used as a basis for this post.

Stories

I first downloaded 154,554 stories covering 2010 through December 2015. I then wrote a program to analyze all story tags from the two lists above.

Overall there are

5.38 times more female stories than male.
2.26 times the number of stories about girls than boys.
2.84 times the number of stories about female cancer than male.
7.8 times the number of general female stories than male. By general I mean once you've removed cancer, and youth totals.

Summary

So I guess my real question is, "is NPR biased against men"? One can, and many do, argue that the world has been a man's world for a very long time now, and now that women are making inroads to male spaces (business, government, military, etc.), these are newsworthy, and explain the greater attention to "female" stories than "male".

I don't know - I'm not a statistician or scientist - so can't say for sure. This was a very unscientific one day analysis, so I hesitate to draw more than a general impression from what I've seen so far. However, my general impression is that these totals skew very strongly toward women. Given that breast cancer kills about 50% more woman than prostate cancer kills men (according to the CDC) why are nearly three times the number of stories about breast cancer than prostate cancer? Are girls worthy of well over twice our attention than boys? The largest disparity are stories about healthy adult men and women - why are there nearly eight times the stories about women's issues?

Does NPR care about men?

Tuesday, November 24, 2015

How Masculism Hurts Women

I was recently directed to Micah Murray’s blog post titled How Feminism Hurts Men. The title is intentionally misleading and is really just a “stop whining men, if you were women you’d have to put up with” list.

That kind of post isn’t really my style - mostly because I think it tends to be divisive changing nobody’s mind. It blatantly mocks those with opposing views, while giving those within the echo chamber that little bit more reinforcement that their view is correct and needs no reflection. It breeds animus.

Something I’ve seen done often is to swap genders to see how different something reads. Often you will read a news story where if you put a man in place of a woman and read it again you’d say, “holy crap - that guy would be in jail instead of walking the streets”. So, as a thought experiment, I’d like to see what the exact same thing would look like - but in reverse. In my case I’m keeping the genders, but swapping the complaints. Micah’s patterns are basically:

If I (a man) were a women then <this is the downside of being a woman that I, a man, would now experience>

- or -

If I (a man) were a women then <this is the upside of being a man that I, a women, no longer get>

So I’ll reverse that to be:

If I (a woman) were a man then <this is the downside of being a man that I, a woman, would now experience>

- or

If I (a woman) were a man then <this is the upside of being a woman that I, a man, no longer get>

So here goes…

Because of masculism

<somewhere> <somebody> wrote that the men’s rights movement hurts women. That merely discussing issues disproportionately affecting men oppresses women.

She was right.

Because of masculism female suicide has gone up by 12.8x (yes 1,280%!) it is now four times greater than men’s.

Because of masculism female war deaths have increased by 1,014x (yes 101,408%). Women now account for 97% of war deaths

Because of masculism female work related deaths have increased by 164x (yes 16,415%). Women now account for 93% of work related fatalities.

Because of masculism homelessness in women has gone up nearly thirtyfold and makes up 85% of the homeless population.

Because of masculism a woman can be involuntarily conscripted into the armed forces and sent off to war to die. The men stay home with the children.

Because of masculism a woman now lives 2.5 years less, and a man now 2.5 years longer.

Because of masculism women now make up 90% of the prison population. Men, merely their gender are presumed innocent, and often avoid prison so that they can care for their children.

Because of masculism women now dominate the ten most dangerous jobs.

Because of masculism woman’s sole custody of children after divorce has gone down thirteen fold and is only awarded in 7% of cases.

Because of masculism it doesn’t matter how good, hard, or long she works. A man with the same job title, working less hours or less well, will get the same salary.

Because of masculism an intoxicated woman is now considered a rapist for having consensual sex with her partner, but her intoxicated male partner is considered an innocent victim.

Because of masculism it is now acceptable for TV/movies/people to joke about cutting of a woman’s breast and putting it in the garbage disposal - men even laugh with glee at that joke.

Because of masculism women are now the butt of all jokes. Men are considered intelligent and compassionate while women always the buffoon lucky to be married to the wonderful men in their lives.

Because of masculism it is now legal for automobile insurance companies to charge a woman more money based solely on her gender even though she is less likely to get in an accident.

Because of masculism women must now pay significantly more into social security for significantly less benefits. At the same time men pay in much less, and draw benefits for much longer.

Because of masculism research on cancer's specific to women has dropped hugely, spending in research for male cancer’s is fifteen times greater than woman’s.

Because of masculism young girls in school are graded down 15% based solely on their gender. Girls are seen as “defective boys”.

- and finally -

Because of masculism society no longer cares about women. Men’s issues get special laws and consideration, but nobody cares about women - not at all. We too often hear the phrase “there were 200 dead, including 60 men and children” - what about the women!

So stay strong sista’s

One day we’ll all be equal.

That list was just off the top of my head - it could be much longer.

I like Micah’s site. He seems to be strong, intelligent, thoughtful, spiritual, and empathetic - certainly empathetic towards women. I challenge him to consider his, and other men’s, vulnerability - to step back and widen his views.

Sunday, October 20, 2013

GPSBabel

So I've been using MapMyRide to record my bicycle rides using my iPhone. One day I took a long ride and my phone battery ran out. After this I decided to pickup a GPS - I settled on the Garmin eTrex 30. I did a ride and manually uploaded my track to MapMyRide and noticed that it's calculated elevation gain just didn't add up.

After researching a bit I came across Garmin Connect. I honestly didn't expect much from Garmin's site, but that is really a great site. After uploading my workout to Garmin Connect I was hooked. I had a few workouts that I wanted to import from MapMyRide, so started that process. Unfortunately, even though MapMyRide will export workouts in the GPX format, they couldn't be imported by Garmin Connect.

After some more digging around it looks like MapMyRide exports the GPX files w/o timestamps, which Garmin Connect needs. I found that these could be "faked" using GPSBabel like so:

gpsbabel -i gpx -f FromMapMyRide.gpx \
    -x track,faketime=f20131020084500+10 \
    -o gpx -F ForGarminConnect.gpx

The workout you import has meaningless speed data, but the position and Garmin's calculated elevation can be trusted.

Friday, December 31, 2010

Will electric vehicles be better for the environment.

Lately I've been wondering, just in the back of my mind, if electric vehicles will really be better for the environment. Here's my (albeit synical) rationale. They say that crude oil (petroleum) will run out before coal. Apparently we have hundreds of years of coal reserves in this country, but very little oil. So will electric vehicles allow us to run on a carbon emitting fuel longer? I hope not - but I still want a Nissal Leaf.

Wednesday, December 29, 2010

Kindle for PC on Linux

I just got Kindle for PC running under Wine (on Linux). It works great! - nearly as good as my Windows partition. I'm running Ubuntu 10.04, Kindle for PC 1.3.0 (30884), and Wine 1.3.9. My trick was to do a complete Wine removal, rm -rf ~/.wine, install Wine 1.3.9, and then the KindleForPC-installer.exe. I did not need to specify that the Kindle app be run as a Windows 98 program. The only thing I see wrong is that the blue underline under hyperlinks in the book have a fixed width (longer than the word). My guess is this is a font issue.

Many thanks to the Wine team. I'm off to read now...