Data Analysis Flyby: Selected Highlights from the Past Week

Last week, the spacecraft New Horizons flew by Pluto and took high resolution photos of the dwarf planet and its moons, giving a more detailed view of Pluto’s landscape, and revealing the presence of icy mountains as tall as the Rockies and a distinctive shape on the surface of the planet that is being called “the heart.”

Pluto is dominated by the feature informally called “the heart.”  Photo credit: NASA

In that spirit, I’d like to make today’s post a quick flyby of some of the interesting and useful articles and blog entries about data analysis that I’ve encountered over the past week.

From the perspective of better understanding the landscape of data analysis, I particularly enjoyed a post at the Abbot Analytics blog entitled “Data Mining’s Forgotten Step-Children.”  The post points out that two types of data mining get most of the attention.  The most talked-about by far is predictive modeling, also known as “supervised learning.”  The second most popular is clustering: “Despite being second banana to prediction, clustering enjoys widespread application and is well understood even in non-technical circles. What marketer doesn’t like a good segmentation?”

So, then, what are the “forgotten step-children”?  As the post explains, they include anomaly detection, association rule discovery, and data visualization.  From my perspective, as someone who has long been interested in data analysis but is still a relative novice, it is good to keep all of these in mind, as I learn more about different ways of working with and asking questions of data.

On that note, there is a helpful article at the Harvard Business Review called “Dispel Your Team’s Fear of Data”  by Thomas C. Redman.  He makes the point that many people have had bad experiences with data in the past, and that, partly because of those experiences, it was easy for many people to ignore data and analytics, but that has been changing.  Now more and more managers and their teams need to engage with data and to understand it, but before that can happen, they need to overcome their fear of data.

Redman suggests a few recent books that he finds useful and informative which might help people gain a greater appreciation for and understanding of working with data.  But what I really liked about his article was that it overlapped with some of the lessons from the “data storyteller” that I wrote about a week ago.  Redman suggests that people should practice finding more ways to work with data that interests them:

Then, find ways to practice using data. Pick something that interests you, such as whether meetings start on time, your commute time, or your fitness regimen, and gather some data, recording it on paper or electronically. Create some simple plots (such as a time-series plot) and compute some statistics (such as the average and the range). Ask yourself what the data means and explore its implications.

Finally, a third post guest-written by Matthew Scharpnick at Beth Kanter’s blog was noteworthy for relating the idea of “data storytelling” with one of the “step-children,” data visualization, and then tying both back to the nonprofit sector.  It is entitled “Five Tips for Nonprofit Data Storytelling.”  I recommend reading the post for all of the details, but the five tips boil down to: 1). Context is king, 2). Avoid unnecessary distractions, 3). Labels matter, 4). Strive for surprises, and 5). Be honest.

Did you come across any particularly interesting or noteworthy articles or blog entries about data analysis during the past week?  If so, please share them in the comments below.

And if you haven’t already done so, look for me on Twitter for more items of interest throughout the week.

Saturday Matinee: The Lunchbox

As I was writing yesterday’s post, I was reminded of a film I saw on DVD a few months ago.  The movie was called The Lunchbox and it told the story of an unhappy housewife, hoping to revive her marriage, and a widower nearing retirement, who form an unexpected connection when Mumbai’s lunchbox delivery service mistakenly starts delivering the lunches meant for her husband to the older man’s office, instead.

I thought of the movie because essentially the lunchbox delivery service–known as the Dabbawala–poses a massive logistics and operations problem, delivering 400,000 lunchboxes each day from homes to offices all across the massive and crowded city.  The Dabbawala service has been held up as an example by the Harvard Business Review as a model of service excellence and NBC News has described it as the envy of Federal Express.  In the movie, when the housewife, Ila, contacts the service to complain that her husband’s lunchbox is being delivered to the wrong person, the person with whom she speaks insists that she must be mistaken, that the service is highly efficient, and was even the subject of a case study at Harvard Business School.

Given the high accuracy and success of the Dabbawala system, such a mistake happening over and over again seems very unlikely, and yet it makes for a good story.   Ila is a very talented cook, and over the course of the story, the man, whose name is Saajan Fernandes, looks forward to the amazing meals he will find in the lunchbox.  The two begin exchanging letters where they talk about their lives.

As they form an unlikely connection, Ila muses about the happenstance that has led them to communicate with each other in this way: “Sometimes the wrong train will get you to the right station” (a bit of folk wisdom which is repeated by Fernandes’ young colleague later in the film).

At another point, Saajan Fernandes makes an even more poignant observation when he notes that “I think we forget things if we have no one to tell them to.”

And that brings me to my questions of the day.  Have you ever found that the “wrong train” got you to the right destination?  Maybe you were doing something wrong for a long time and despite that you found what you were looking for?  We talk often about learning from mistakes, but I wonder what we learn when we don’t realize we are making a mistake and yet we still, somehow, manage to find the right answers?

Or, on the other hand, are there things you have learned that you worry about forgetting if you have no one to tell them to?

Learning from Business: Lean Production

In a previous post about the business model of prospect research offices, I mentioned that, as a result of efforts to increase productivity several years ago, my office started validating many more prospects than the development directors could meet with in a reasonable period of time.  We eventually developed such an oversupply of prospects and potential prospects that we scaled back on validating many more names for a while; in the meantime, we tried to figure out the best way of identifying and validating prospects to ensure that they ended up moving into the prospect pipeline.

As it turned out, concurrent changes in our division brought along changes in the way prospects were assigned, and that, in turn, gave us a new opportunity to reconsider the issue.  One of the changes we came up with was to start handling some elements of the prospect identification and validation process more like the reactive research request process.  While we continued doing proactive research and recording recommendations for assignment in our database, we started worrying less about pushing out names to development directors until they said they needed more names to work with.

Where in the past, we would have set a goal to identify or validate a large group of names and then push them out for assignment in batches as we finished them, now we identify and validate some of them as we go about our daily work, and we keep development directors apprised of the most interesting ones through e-mail alerts and our department newsletter, but we don’t focus on working through specific batches of validations until those batches are needed.

“Toyota Manufacturing UK – Assembly” by Toyota UK licensed under CC By-NC-ND 2.0

About two months after this change in our process last fall, while taking a course in business operations, I was introduced to the concept of lean production and “just-in-time” manufacturing.  Toyota engineer Taiichi Ohno is generally given credit for having initially developed the principles of lean production in the years after World War II, though others have further refined them since then.  There are several definitions of these concepts available online, as well as many websites devoted to teaching the principles, but for the sake of simplicity, I’ll just use a definition of “just-in-time” copied from Investopedia:

An inventory strategy companies employ to increase efficiency and decrease waste by receiving goods only as they are needed in the production process, thereby reducing inventory costs.

This method requires that producers are able to accurately forecast demand.

A good example would be a car manufacturer that operates with very low inventory levels, relying on their supply chain to deliver the parts they need to build cars. The parts needed to manufacture the cars do not arrive before nor after they are needed, rather they arrive just as they are needed.

This inventory supply system represents a shift away from the older “just in case” strategy where producers carried large inventories in case higher demand had to be met.

While the shift in my department’s work flow patterns hadn’t been inspired by the principles of either lean production or just-in-time, I couldn’t help but notice the similarities with the more efficient process we had recently adopted.  That, in turn, has gotten me thinking much more lately about how I can adopt more of the principles to effectively guide the work of my department.

Below is a list of ten basic principles of lean production that can be applied to any sort of work process:

1. Eliminate waste
2. Minimize inventory
3. Maximize flow
4. Pull production from customer demand
5. Meet customer requirements
6. Do it right the first time
7. Empower workers
8. Design for rapid changeover
9. Partner with suppliers
10. Create a culture of continuous improvement

Looking over that list now, I can identify areas for potential improvement, but more importantly, I see a list of topics and questions that can be asked regularly as I examine the flow of work through my department, or on a more general level, as I plan or structure days or weeks, both at work and at home.

The concept of lean production has a great power to transform the way we work and the way we use our time.  Writing about it again now, I think it is something I definitely need to keep exploring, and, where possible, applying to my work.

Have you applied any of the concepts of lean production in your workplace?  If not, what principle(s) do you think you would choose to focus on first?  And if you have applied the concepts, what changed as a result?

Sunday fun: The Data Storyteller

This weekend, while researching something else, I came across an enjoyable talk by Ben Wellington which was recorded at a TEDx event on Broadway this spring.  In the talk, he explains that over a year ago, he wouldn’t have have known what a data storyteller was, nor would he have imagined that he would have become one, but that all changed after he used some data about biking accidents from the New York City Open Data Portal to create the heat map shown below and it became the subject of articles on a few New York-area websites and blogs.

The Terrifying Cycling Injury Map of NYC, 2013 Edition by  Ben Wellington

The talk is entertaining and informative, and I recommend watching it to learn more about what he does with his blog and how he has turned into a data storyteller.  As he explains, though, he realized that one reason his blog had caught on was that it provided a way of combining his interests in data science, urban planning, and, of all things, improvisational comedy.

From my perspective, both the talk and his approach encapsulate the “use everything” philosophy that I wrote about in an earlier post.  As he explains, he’s not using complicated math here, but he’s asking a variety of creative and simple questions, and then trying to see what answers the data provides to those questions.

In my last post, I wrote about some ways to use some of the features of Excel and Access to get started with data analysis.  Now while that may seem like a very basic way to start, as this talk illustrates, basic analysis can tell us many kinds of things if we use our imaginations and ask questions related to our interests and experiences.  Wellington even provides examples at the end of his talk of how students who were new to statistical analysis were able to use the New York City data to begin asking and answering their own questions.

Beyond those points, though, both the talk and the approach that it outlines illustrate that for people who are interested in learning to work with data, there are lots of free and accessible ways to get started.  The New York City Open Data Portal is a great resource that contains a lot of information, but there are many other large, open data sets out there if one knows where to look.   Here, for instance, is but one listing of the many sorts of public data sets that are available.  And although I plan to talk more about other kinds of data analysis software in future posts, here is a link to the open-source geographic information system software that Ben Wellington used to create his initial map of cycling accidents.

For me, though, one of the key points to take away from this talk is that if you are interested in working with data, you no longer need to make the mistake that I made for many years in thinking that you are seriously limited by the data that is readily available or even by the software that you do or don’t possess.  Even if your long-term goal is building predictive models, you can gain a lot of knowledge and expertise by practicing with other sorts of data first, and when you do, you might find that your interests take you in different directions than you originally imagined.

Have you used data to do any interesting storytelling of your own?  If your city or state has any open data portals, what sorts of questions would you be interested in finding the answers for?  Please share your thoughts below.

Getting started with data analysis

I first became interested in the idea of applying data analytics to development work 14 years ago, when I was invited to interview for a job as director of prospect research at a university that had just completed a major fundraising campaign and was looking to do a lot of analysis of its campaign data to figure out the best path forward.  I didn’t have a great amount of experience in the field at the time, and I hadn’t actually applied for that particular job, but the hiring manager had come across my resume and thought I might be worth interviewing for the opening.  Even though I wasn’t hired for the position, it was a great learning experience for me in many ways, not the least of which is that it inspired a lot of curiosity in me for the whats, whys, and hows of data analysis.

It remained an interest of mine, as gradually more presentations began to appear on the APRA conference schedule about analytics.  At the time, though, I was working at an institution with an old database that was not user-friendly and which I couldn’t query on my own.  It was frustrating knowing that there was a world of possibilities which I felt incapable of tapping into.

Eventually I got a job at the University of Nevada, Reno, where I could query Raiser’s Edge, and also run my own reports and exports.  Suddenly, the possibilities began to open up again, but I still didn’t have access to statistical analysis software yet, and even if I had, I wouldn’t have known right away where to begin.

As it turned out, though, I came to recognize that there was a good amount of basic data analysis I was able to do without it, just using exports, Excel and Access.  If you haven’t undertaken much in the way of data analysis yet, I’d recommend becoming familiar with what can be done with those programs.

Most of the attention in discussions of analytics has to do with predictive models, and while they are definitely valuable, they aren’t the only game in town when it comes to data analysis.  One way to think about it is to think about the difference between running and walking.  One should learn to walk before learning to run, but even once one learns to run, one doesn’t need to run everywhere all the time. Walking is still a great way to get around most of the time.

Although there are many faster and more sophisticated statistical packages out there, if you don’t have access to them, or don’t want to take the time to learn them, many insights can be generated simply by learning to make use of the features provided by Excel or Access to get a picture of your data.  What are some of the things you can do?

1).  Filtering: One of the first tips for data analysis I learned about was to use the filters in Excel or Access as a way of querying and analyzing your data.  You might start by exporting a very broad query or data set and then you can refine it to the parameters you want to examine using filters, or you can combine several different filters to see what patterns emerge about the records in that particular data set.

2).  Tracking with additional columns: Once you have selected certain combinations of variables with filters, you can add another column and enter a tracking variable or a set of variables.  (You can also code such variables automatically using formulas in Excel or in Access once you know what you’re looking for.)

3).  Joins and queries in Access:  You can use Access to clean and modify a data set, or to join several different tables into one.

“It’s PIVOT-TABLE MAN!” by caseorganic (Amber Case) licensed under CC BY-NC 2.0.

4).  Pivot Tables: Once you determine the variables you want to examine, you can use pivot tables to evaluate the relationships between them and to look for patterns in your data.

5).  Data Analysis Module: In Excel, you can add-in the “Data Analysis ToolPak” and use that to conduct more sophisticated forms of analysis, and you can make more detailed charts and tables and to practice looking for relationships between variables.

Once you’ve mastered these basic five steps and are ready for more, fortunately there are a lot of great options, both through books and through numerous online resources.  I am planning to write more future posts describing other simple and economical ways to get started.

What are some of the basic tools and techniques you have used to begin learning about data analysis?  Where and how did you get started?  Or, if you are just getting started, are there tools or techniques I have described above would you like to learn more about in future posts?  Please comment below.


Prospect Research Questions: What is your business model?

When I started in prospect research many years ago, the idea of a business model or even the applicability of business concepts to the work I was doing was foreign to me.  After attending my first conference and hearing from experienced and successful researchers, my view of the matter began to change.  I took what I learned at that conference (and subsequent ones) and applied it, and eventually became skilled at tracking all sorts of metrics for the work of my office.

Tracking productivity is one of the first steps towards analyzing your department’s business model.  Most researchers already track things like the number of requests they receive, the number of profiles they write, or the number of prospects they validate.  It takes more of an effort to track things like capacity confirmed or increased capacity identified, but it is not difficult to do.  Beyond that, there are things like prospects identified, or prospects identified by different researchers, as part of different projects, or through the use of particular resources.


The important thing is that once you start tracking these numbers, the way you think about them changes.  Many years ago, it was very easy to justify the change in the work produced by my office from a profile-centered model to one that focused more on validating screening data and producing fewer profiles, simply because it was easy to point to the differences in productivity judged by the number of prospects and potential prospects that could be identified, researched quickly, and assigned out to fundraisers for initial contact.

It was easy for us to point to and justify our productivity based on the metrics we were tracking.  But as is often the case, the numbers we were tracking, only told one part of the story.  In reality, my office had became so efficient at validating and identifying new prospects we eventually discovered that we had an oversupply of prospects and potential prospects who weren’t being contacted.  And that, in turn, has required further adjustments in our business model.

I have written in several posts already about the shift in the profession to a focus on prospect development, but one of the crucial reasons for the shift is–again–a change in the business model to one that is more focused on maximizing the efficiency and impact of our work.  Analytics offers a way to help do that by making sure we are directing our efforts on identifying and assigning the right prospects; likewise, taking a more active role in relationship management helps to insure that the work we are doing is having an impact.

In my most recent post, I wrote about customer service and smartphones; it may have seemed like a shift in focus for my blog away from topics such as research, data analysis, and prospect development; in some ways it was, but in other respects, the idea of customer service is something that prospect development offices always want to keep in mind when going about their work. In future posts, I plan to say more about how thinking about customer service can further transform the prospect development business model.

If you work in fundraising, how would you describe the business model of your prospect research or prospect development office?  How has it changed over time, and what drove those changes?  Please share your thoughts and experiences in the comments below.

Do Smartphones Undermine Customer Service?

As we head into the Independence Day holiday weekend, many people will be going away on trips, and when they do, they will be bringing their smartphones with them so they can keep up with e-mails and business contacts.  It sounds like a great plan, right?  What could go wrong?

Lately I’ve had some frustrating experiences trying to take care of some personal business with people who were pursuing that very strategy.  It has led me to wonder if smartphones and other technology have caused too many people to disregard long-standing customer service practices.

“Love (of technology)” by Streetmatt licensed under CC by 2.0.

What do I mean?  Well, instead of people making arrangements to deal with their business clients while they are away or out of town, they just don’t respond or they take a long time to respond.  Instead of assigning a back-up who can handle things while they are away, or instead of making it clear to their clients when they will return and what they will or won’t be able to do in the interim, they just figure they can be reached on their smartphones and let things slide for a while.

So why am I blaming it on smartphones?  I think that for some people, the fact that they have a smartphone makes them believe that they can always be connected and so they don’t have to figure out a backup plan for the times when they are away or busy.  And, yes, some people really do stay connected like that all of the time, which is admirable (if not necessarily always wise), but few are that devoted.

Likewise, I’m sure many of us have had the experience of going into some business where service staff were preoccupied with their smartphones instead of being focused on or attentive to customers.  Now while that can be blamed on poor management–if they were not instructed to do otherwise–the reality is that such behavior even occurs in commission-driven businesses.  I’m not suggesting smartphones are the only reason for a decline in customer service, but they can be one of many contributing factors.

So what is to be done?  Those who work in a field where customer service is a key part of their responsibilities need to regard their smartphones as a useful tool, but one that can help them do their jobs better,  not one that alleviates them of responsibilities that they would otherwise have to worry about.

What are your thoughts?  Do smartphones contribute to a decline in customer service?  What have your experiences been?