Jumping (into) the Turnstile (Data)

-
This blogpost marks the completion of my first week at data science bootcamp. In our first week, we were assigned a group project to use NYC MTA turnstile data to approximate the best times of day and locations for an engagement street team to be placed.

The approach to the project was very open-ended, which meant I made a lot of useful mistakes.


EDUCATIONAL MISTAKE #1 - I tried to transform data before really looking at it

When I first started the project, I downloaded the data and immediately started trying to do operations on it. I knew that cleaning data and exploring it beforehand are important - but surely the MTA cleaned their data and made it ready for easy public use before publishing it?

I spent four hours figuring out how to filter data, making subsets, aggregating, and converting datatypes before I ever did a single .describe( ). The second I did, I saw that column names had whitespace, counts were randomly negative every once in a while, and there were a ton of strange outlier data points. 

Not only did I jump into working with unexplored data, I spent barely any time reading the documentation accompanying the data and explaining the different columns. Sure, it seemed a bit strange that there seemed to be billions of people walking through a single turnstile in a week, but New York has a lot of people, right? (oh, those numbers are cumulative? Ohhhhhh..... oh.)

So, after an especially conscientious classmate came over to ask me how I was dealing with the cumulative data, I actually looked at descriptions of the data. And then I remembered the lecture we'd had about exploratory data analysis, and the notes I'd taken about the kinds of things you do before working with data. And I felt very embarrassed. But research has indicated that memories associated with pain and negativity can be more easily accessible than positive memories, so I'm going to assume this means great things for my future explorations.


EDUCATIONAL MISTAKE #2 - Stopping as soon as it works: aesthetics gtfo

The graph above was the first graph featured in my group's eventual presentation, made by me. Too many stations, too many unnecessary colors, station names left in their column heading shorthand.... it's a mess.

Look, I am not unfamiliar with visual communication. I'd argue I have 8 years of experience as a science teacher trying to distill complex topics into the cleanest, clearest presentation possible. BUT - when I'm coding - I spend so much time wrestling with errors and typos and problems I don't understand.... and so when I finally get some decent graphical output, my reaction is "IT WORKED! I'm a genius!!!! I deserve a snack. What's next?" and it doesn't really occur to me that a few quick changes will make my hard work significantly more legible.



One week down, many more (great) mistakes to come!




Comments

Post a Comment

Popular posts from this blog

Predicting Ambitious Instruction at CPS

-
Image
(Github Project Repo) What predicts where you will see 'good' teaching? It's easy to imagine that this question has a simple answer - wherever you have good teachers, you see good teaching. All we have to do is identify good teachers and get rid of bad ones, and education will be fixed! Teachers will cringe at this type of thinking, because we see our own effectiveness fluctuate constantly,  from year to year, from day to day, based on a huge number of contextual factors - some within our control, some without, some predictable, some not. I personally experienced a huge change in my teaching effectiveness a little over a year ago, when I changed schools. At my previous school, I had found it relatively easy to use 'best teaching practices' - pushing my students to take on more of the cognitive load, to explain themselves in detail throughout class, and to persist through struggle and failure as we engaged in difficult projects. At the new school, I encountere...
Read more

Mini-Project: Finding Allies Based on Mission Statements

-
Image
An important aspect of developing a campaign for social change is finding your allies. A broad approach to this problem might simply be to identify organizations that share your aims. But building a coalition around similar aims is not sufficient, as conflict may arise around strategy if your organizations do not share similar values . Two organizations may share aims, but if there is a large difference in values, they will not agree on strategy. Thus, in order to find new organizations to ally with as part of an organizing campaign, it would be useful if there were a way to draw out the values  of an organization. This can be hard to do even as a human, since many organizations will not explicitly shout their values for fear of alienating donors and funders that are not ideologically aligned. I decided to look at mission statements of several nonprofits in Chicago that are related to community development and housing, to see if I could use natural language processing and ...
Read more

Impact of Reform on Chicago Police Accountability

-
Image
(Github Project Repo) In the past 20 years in Chicago, there have been a few attempts at reforming how complaints and allegations against Chicago police are handled, usually in response to major scandals.  Up until 2007, police misconduct complaints were handled by the Office of Professional Standards (OPS), a branch of the Chicago Police Department. In  response to several scandals with regard to the 'code of silence'  that suppressed police officers from holding each other accountable, the Chicago city council voted to dissolve OPS, and establish two separate entities to review police complaints. The Independent Police Review Authority (IPRA) would be outside CPD and have leadership appointed by the mayor, and it would review cases related to  “excessive force, domestic violence, coercion through violence, or verbal bias-based abuse.” The CPD-internal Bureau of Internal Affairs  (BIA) would review all other complaints. In the aftermath of several s...
Read more