Stats and Personal Finance: Oh My!


Lions and Tigers and Bears… Oh my! Wikipedia user: Wulfstan; Wikipedia user: Dibyendu Ash

Last time, I wrote about volatility in one’s expenses.  I kept it pretty generic.  On purpose.

Today, I’m going full nerd.  We’re going to look at expenses and how much volatility there really is.  And, we’ll quantify that volatility.  I’ll do my best to keep it accessible, but it will involve a smidge of math.

If you’re still trying to get started tracking your expenses, is a great way to start.  You can probably get everything set up in about an hour.  After that, it’s almost entirely automatic.  You get all the charts and graphs that most normal humans would ever want to look at.  And, for us nerds, everything is downloadable so you can do extra analysis when no one else is looking.  OK, back to the task at hand.

Before we get too deep in the proverbial muck, a question: why would you want to look at expense (or income) volatility!?

Answer: so you can plan better for long term things like financial freedom, retirement, house purchases or pay-offs, and budget risk.

In my last post, I showed a time series plot of our monthly expenses:

And because I'm a nerd, I also track spending within each pay period to ensure I don't go over budget.

And because I’m a nerd, I also track spending within each pay period to ensure I don’t go over budget.

So that I can work with our actual data in the public domain of the internet, I’m going to convert all of the expenses to a standard normal distribution so I’m not sharing the actual values.  You’ll still be able to see each monthly value expressed as a z-score relative to the average of all the others.  (That translates into the following: expensive months will have big numbers.  Cheap months will have small, possibly negative numbers.)   If that leaves you confused or angry, you can watch Psy dancing Gangnam Style again and feel a little bit better.   Or, you can just skip the conversion when (if!) you try this at home with your real data (and not post it on the internet).

std_expense-varianceA couple of things jump out at me based on these two views.  Lets start with the time series plot:

  • There’s a whole lot more volatility in the past 3 years than in the first 3 years.  Lifestyle inflation, buying and renovating a rental property, and having a little one will do that.
  • Baseline expenses are going up.  And, we probably have more increases to look forward to in the future.  That’s even after accounting for the “unusual” expenses of starting a rental property
  • The histogram shows two humps: the main one centered on 0, and a second one centered on 3 sigma.  Given the relative sizes, that says to me that we’re pretty good about sticking to our budget.  But, when we overspend, things really come off the rails.
  • As with so many things in life, “Average” doesn’t tell the whole story.  That’s the crux of this post, so I’m going to break out of bullets and expound.

A lot of conventional wisdom is centered around the idea of working to an average.  Average home prices, sales figures, time to commute to work, heights, salaries, etc.  The list goes on and on.  We’re taught from a very young age that in order to summarize a data set, we should add up the numbers and divide by the number of numbers.  That’s the average.  It’s a useful statistic to start with, but it doesn’t tell the whole story.  Check it out.  Each of these lines (H, S, and P) all have the same average.  What’s different?  The spread.


There’s more to data than just the average. If your head is in the freezer but your feet are in the fire, on average you’re fine! Right?



Enough stats for a minute.

Back to personal finance.  If I were to plan my 2017 monthly budget around last month’s expenses, I might pick up an outlier where we paid all the fees for an au-pair (and ate nothing but peanut butter and jelly to compensate.)  If I take a 6 year average, I might overlook the fact that my recent baseline expenses seem to be increasing.  So, let’s only look at the past 3 years.  That might have a more representative sample.

The past 3 years of standardized expenses

The past 3 years of our standardized expenses

Now I have a range of expenses to plan for in the future.  Picking the average and planning for that level of spending would result in having enough money allocated to cover 54% of months!  I’m dipping into my savings or emergency funds almost half the time.  If I want to be more confident that I’ll be allocating my budget appropriately, I should plan for a higher level of spending: enough to cover 80% of months.  That means I need to be budgeting at the 1 z-score level and not the average  (Or, we could slash our expenses by eating nothing but Ramen noodles).

Even if I upped our budget to cover an 80% level of spending, we’ll still be short in approximately 2 months of each year!  I have more confidence that the financial plan I’m building will get us where we want to go.  Is it relevant for 20 years from now? Probably not.  But, for the next couple of years, yes.  This model is probably relevant and can give me a good idea of what life will cost, including some pretty major surprises.  With this new information, I can set new thresholds for my emergency funds to ensure that we’re as prepared as we can be if/when disaster strikes.

Has anyone else looked at their spending (or income) with an eye towards variation?  What insight did you draw from the analysis?

Even When You Track Every Penny, You Still Have Uncertainty!

Source: US Mint. Do you know where all of your Lincolns are?

My friend and I were discussing, Your Money Or Your Life the other day.

There’s this cool graph where they extrapolate current earned income, current expenses, and investment income.  At some point, investment income surpasses (hopefully!) expenses.  At that instant, you’re financially free.  It’s a profound idea.

Now, don’t get me wrong.  I think tracking your expenses is a crucial start to to developing financial freedom.  But, I also don’t think it is a silver bullet.

As someone who has tracked expenses for almost 6 years, I’ve really struggled to apply the Your Money or Your Life view for myself. Maybe it’s because we’re 35 and have a new daughter.  I can tell you firsthand, she has injected more financial uncertainty into our lives than I thought possible.  Maybe it’s simply because financial freedom seems so far away.  Or, maybe it’s because there is lots of volatility in day to day, week to week, month to month, or even year to year financial ins and outs.  Let me show you what I mean.

Here’s our tracked expenses from since 2010:

Mint's auto expense tracking is super convenient. Here's our chart of monthly expenses since 2010

Mint’s auto expense tracking is super convenient. Here’s our chart of monthly expenses since 2010

And here’s, my spreadsheet version…yes, I track things in 2 spots.  Ugh, and I’m admitting that publicly.

And because I'm a nerd, I also track spending within each pay period to ensure I don't go over budget.

And because I’m a nerd, I also track spending within each pay period to ensure I don’t go over budget.

First, they provide different results. automatically bucketizes things.  It’s super convenient, and I use it less for monthly tracking than for long term trends.  I also use a modified envelope method to keep savings funds for things like home improvement/repairs, rental property expenses, insurance premiums, etc.  My goal is to take big periodic cash outflows and bake small amounts of savings into my budget so that when the expense hits, I’ve already saved for it.  Unfortunately, Mint doesn’t do such a good job of managing these type of periodic expenses.

My Excel version is completely manual and is really used tactically.  How much do I have left in my budget for this pay period?  Can I go out to lunch with the work crew tomorrow, or am I brown-bagging it?  So, the bucket granularity is much different.

The first thing that jumps out at me is the upward slope of the trend-lines.  Yes, our real expenses are increasing.  That much is clear from both charts.  So, is it time to finally cut the cable TV cord!?  Should we start eating Ramen?  Probably not.  In this case, we’ve purchased our first rental property.  That’s the big set of spikes at the end of 2015.  And, our little girl is costs us way more than our two dogs do!  So, while the upward trend is disconcerting, it’s also understandable given everything that’s been going on in our lives.  To me, this is the power of tracking every penny you spend (twice!).

The real reason that I think tracking every penny still leaves you with uncertainty is the volatility.  I put trend-lines into each plot to also illustrate how far from “average” our expenses are in any given month.  Some months, they’re high.  Some months, they’re low.  This volatility also makes me pause when I try to extrapolate a trend-line 20 – 30 years from now.

I would like to explore this topic further in future posts.  I think there is great wisdom in tracking one’s spending.  I also think with thoughtful analysis, using income and expense trend-line extrapolation can reveal deep insight about one’s financial freedom trajectory.

Has anyone else tracked their expenses manually vs. using an automated tool like  Have you tried to forecast your financial freedom point? What did you learn?