Aaron Hipple
The recent news about Visa's Acquisition of Plaid got me thinking about privacy and the use of tools like Mint, Personal Capital, and so on. For those not familiar, these tools aim to provide a one-stop view of your personal finances, with features like budget reports and investment tracking.
Personal Capital's Dashboard
These are really useful features to be sure, but as I've grown as a software engineer and observed more about how companies in the industry use the data we grant them, I've gotten less comfortable with the idea of data aggregators having free rein in my bank account. To that end, I'm undertaking a project to take more control of my financial data by using self-managed, self-hosted tools.
Establishing Some Goals
Everybody needs different things from their finance workflow. Some people want to get control of their expenses. Others are planning saving for the future; maybe they're trying to buy a house, or are planning for retirement. My use cases are roughly these, and in this order:
- Monthly Budget Tracking
-
An overview of our monthly and yearly expenses and forecasting/automating our cash flow.
- Budgeting Medium Term Savings
-
Projecting when we'll be able to do things like take more substantial vacations, buy a house, etc.
- Monitoring Long Term Savings
-
Keeping an eye on how we're tracking toward goals like retirement, college savings for nieces, covering elder care expenses for parents, etc.
I've been using Personal Capital so far, which weights more heavily to the long-term use cases than the short-term. Their budget forecasting tools are rudimentary (though they do a good job of providing a retrospective view of your spending), and their retirement planning tools are fantastic. In our case, with a retirement plan that's fairly fuzzy (it'll probably be a gradual shift from career-focused jobs to personal-fulfillment-focused as we get closer to our "we can afford to retire on this amount" savings number), I'm more interesting in optimizing the short and medium term forecasting tools.
Finally, I'm a software engineer in life as well as in profession, so any solution I use should be automatable, auditable, and most of all fun to use. It should be hackable, in short.
One Option: Spreadsheets
I've long kept a spreadsheet with some financial data in it for projection purposes, so I considered leaning more heavily on that for budgeting and tracking. Spreadsheets are a great paradigm for this sort of thing that millions of people use successfully every day.
In my years of software engineering I've had enough experience hacking on spreadsheet software to know that it pretty quickly devolves into a mess. VBA scripts in Excel can do just about anything, but the development and maintenance cycle with those scripts is unpleasant at best. Manipulating Excel sheets from outside is also quite doable, but in most languages this work gets repetitive and boilerplate-y in a hurry, and again it's not a ton of fun.
Finally, there's the issue of version control. My personal workflow for just
about everything runs on git
these days for many reasons (this is worth
another essay, honestly) and version controlling non-text files like
spreadsheets is an exercise in frustration.
All these issues left me looking for another choice.
Plain Text Accounting & The Unix Philosophy
A little googling led me to the idea of plain text accounting (PTA). In short, PTA is a workflow for accounting that relies in plain text files containing records of transactions. Like other plain text formats, these can be easily and usefully version-controlled and manipulated with the rich ecosystem of tools for working with plain text. It's the Unix philosophy applied to personal finance.
ledger
ledger
appears to be the first substantial entry on the scene. It's based on
the centuries-old innovation that is double-entry bookkeeping (there was a
great Planet Money episode about the fellow who invented that back in
Renaissance Italy). It reads transactions from a text file format that simply
encodes the date, payee, source and destination account(s) and amount(s) for
each one.
2020/01/01 Credit Card Payment
Assets:Checking $100.00 = $100.00
Liabilities:MasterCard $0.00 = $0.00
Equity:Starting Balance $-100.00
2020/01/18 Amazon.com
Expenses:Books $10.00
Liabilities:MasterCard $-10.00
2020/01/20 Credit Card Payment
Liabilities:MasterCard $10.00
Assets:Checking $-10.00
We can then do things like check our balances by dispatching commands with
ledger
. Here's ledger balance
:
$90.00 Assets:Checking $-100.00 Equity:Starting Balance $10.00 Expenses:Books -------------------- 0
ledger
-likes: hledger
ledger
is great, but it's got a few limitations and quirks that have inspired
people to make similar, enhanced tools like hledger
and beancount
. I've
chosen hledger
in particular for its enhanced budgeting and forecasting tools.
We can easily establish a budget using scheduled, recurring transactions.
2020/01/01 Starting Balance
Assets $1000
Equity:Starting Balance -$1000
~ Daily ; Food goes on the credit card
Expenses:Food $8.00
Liabilities -$8.00
~ Monthly
Expenses:Rent $1000.00
Assets -$1000.00
~ Every 23rd day ; credit card payment
Liabilities = $0.00
Assets
~ Every 2 weeks
Assets $1000.00
Income:Salary -$1000.00
hledger
can produce rich forecasts. Here's a cashflow forecast for 2020 using
the above budget (hledger cashflow --forecast --monthly -p 2020
):
Cashflow Statement 2020 || Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec ============++=============================================================================================================== Cash flows || ------------++--------------------------------------------------------------------------------------------------------------- Assets || $2824.00 $752.00 $768.00 $752.00 $760.00 $1752.00 $760.00 $752.00 $752.00 $760.00 $1752.00 $760.00 ------------++--------------------------------------------------------------------------------------------------------------- || $2824.00 $752.00 $768.00 $752.00 $760.00 $1752.00 $760.00 $752.00 $752.00 $760.00 $1752.00 $760.00
And here's the forecasted balances for each month (hledger balancesheet
--forecast --monthly -p 2020
):
Balance Sheet 2020/01/31,,2020/12/31 || 2020/01/31 2020/02/29 2020/03/31 2020/04/30 2020/05/31 2020/06/30 2020/07/31 2020/08/31 2020/09/30 2020/10/31 2020/11/30 2020/12/31 =============++================================================================================================================================================ Assets || -------------++------------------------------------------------------------------------------------------------------------------------------------------------ Assets || $2824.00 $3576.00 $4344.00 $5096.00 $5856.00 $7608.00 $8368.00 $9120.00 $9872.00 $10632.00 $12384.00 $13144.00 -------------++------------------------------------------------------------------------------------------------------------------------------------------------ || $2824.00 $3576.00 $4344.00 $5096.00 $5856.00 $7608.00 $8368.00 $9120.00 $9872.00 $10632.00 $12384.00 $13144.00 =============++================================================================================================================================================ Liabilities || -------------++------------------------------------------------------------------------------------------------------------------------------------------------ Liabilities || $64.00 $48.00 $64.00 $56.00 $64.00 $56.00 $64.00 $64.00 $56.00 $64.00 $56.00 $64.00 -------------++------------------------------------------------------------------------------------------------------------------------------------------------ || $64.00 $48.00 $64.00 $56.00 $64.00 $56.00 $64.00 $64.00 $56.00 $64.00 $56.00 $64.00 =============++================================================================================================================================================ Net: || $2760.00 $3528.00 $4280.00 $5040.00 $5792.00 $7552.00 $8304.00 $9056.00 $9816.00 $10568.00 $12328.00 $13080.00
Getting data into a ledger
One thing that ledger
-likes don't solve for is retrieval of data. Many people
choose to enter the data by hand and reconcile against their banks' transaction
logs periodically. One great advantage of a simple plain-text format for
transactions, though, is that it's easier to automate the retrieval of this data
from (some) banks so manual entry isn't required.
-
ledger-autosync
-
Many banks provide Quicken-formatted (OFX) exports of transaction data.
ledger-autosync
is a tool that allows you to either import downloaded OFX files into your text ledger, or will retrieve them automatically (if your bank supports this).ledger-autosync
supportshledger
-style syntax as well as the classic syntax, so it works great for my use case. - CSV import
-
hledger
has rich support for importing from CSV using.rules
files, an instruction format forhledger
's own CSV import tooling. These allow you to export CSVs from your online banking portal(s) if automatic import through OFX isn't available. Unfortunately this is still a slightly manual process, but I've actually found it pretty low-friction to log into a couple of bank accounts once a week to pull transactions. -
Investment data with
yfinance
-
Because the text ledger format is so simple, it's easy to generate entries from almost any other tool. I wrote a little Python script that uses the
yfinance
library to pull ticker data from Yahoo Finance's API.
Evaluating & Looking forward
In general I'm pretty happy with the setup I've built. I control more of my data
now, and hledger
allows me to do granular forecasting of our budget, which has
made it possible to automate even more of our transactions confidently.
There are a few more projects I'd like to undertake though:
- Scraping Transactions
-
Unfortunately, while I'd prefer to simply vote with my dollars and bank only at institutions that provide good APIs, it's not always possible. I have had some success in scraping transaction data using Selenium and other tools, though, so I plan to continue automating even more so it's less necessary to download transactions by hand.
- Investment Forecasting
-
One thing I traded off in the switch from Personal Capital was the use of their excellent retirement planning tool. With good data available from
yfinance
, I'd like to explore the use of common stock market modeling techniques to simulate possible scenarios and help us keep an eye on our retirement plans.