Aaron Hipple

The recent news about Visa's Acquisition of Plaid got me thinking about privacy and the use of tools like Mint, Personal Capital, and so on. For those not familiar, these tools aim to provide a one-stop view of your personal finances, with features like budget reports and investment tracking.

These are really useful features to be sure, but as I've grown as a software engineer and observed more about how companies in the industry use the data we grant them, I've gotten less comfortable with the idea of data aggregators having free rein in my bank account. To that end, I'm undertaking a project to take more control of my financial data by using self-managed, self-hosted tools.

Establishing Some Goals

Everybody needs different things from their finance workflow. Some people want to get control of their expenses. Others are planning saving for the future; maybe they're trying to buy a house, or are planning for retirement. My use cases are roughly these, and in this order:

Monthly Budget Tracking

An overview of our monthly and yearly expenses and forecasting/automating our cash flow.

Budgeting Medium Term Savings

Projecting when we'll be able to do things like take more substantial vacations, buy a house, etc.

Monitoring Long Term Savings

Keeping an eye on how we're tracking toward goals like retirement, college savings for nieces, covering elder care expenses for parents, etc.

I've been using Personal Capital so far, which weights more heavily to the long-term use cases than the short-term. Their budget forecasting tools are rudimentary (though they do a good job of providing a retrospective view of your spending), and their retirement planning tools are fantastic. In our case, with a retirement plan that's fairly fuzzy (it'll probably be a gradual shift from career-focused jobs to personal-fulfillment-focused as we get closer to our "we can afford to retire on this amount" savings number), I'm more interesting in optimizing the short and medium term forecasting tools.

Finally, I'm a software engineer in life as well as in profession, so any solution I use should be automatable, auditable, and most of all fun to use. It should be hackable, in short.

One Option: Spreadsheets

I've long kept a spreadsheet with some financial data in it for projection purposes, so I considered leaning more heavily on that for budgeting and tracking. Spreadsheets are a great paradigm for this sort of thing that millions of people use successfully every day.

In my years of software engineering I've had enough experience hacking on spreadsheet software to know that it pretty quickly devolves into a mess. VBA scripts in Excel can do just about anything, but the development and maintenance cycle with those scripts is unpleasant at best. Manipulating Excel sheets from outside is also quite doable, but in most languages this work gets repetitive and boilerplate-y in a hurry, and again it's not a ton of fun.

Finally, there's the issue of version control. My personal workflow for just about everything runs on git these days for many reasons (this is worth another essay, honestly) and version controlling non-text files like spreadsheets is an exercise in frustration.

All these issues left me looking for another choice.

Plain Text Accounting & The Unix Philosophy

A little googling led me to the idea of plain text accounting (PTA). In short, PTA is a workflow for accounting that relies in plain text files containing records of transactions. Like other plain text formats, these can be easily and usefully version-controlled and manipulated with the rich ecosystem of tools for working with plain text. It's the Unix philosophy applied to personal finance.

ledger

ledger appears to be the first substantial entry on the scene. It's based on the centuries-old innovation that is double-entry bookkeeping (there was a great Planet Money episode about the fellow who invented that back in Renaissance Italy). It reads transactions from a text file format that simply encodes the date, payee, source and destination account(s) and amount(s) for each one.

2020/01/01 Credit Card Payment
    Assets:Checking                     $100.00 = $100.00
    Liabilities:MasterCard                $0.00 = $0.00
    Equity:Starting Balance            $-100.00

2020/01/18 Amazon.com
    Expenses:Books                       $10.00
    Liabilities:MasterCard              $-10.00

2020/01/20 Credit Card Payment
    Liabilities:MasterCard               $10.00
    Assets:Checking                     $-10.00

We can then do things like check our balances by dispatching commands with ledger. Here's ledger balance:

              $90.00  Assets:Checking
            $-100.00  Equity:Starting Balance
              $10.00  Expenses:Books
--------------------
                   0

ledger-likes: hledger

ledger is great, but it's got a few limitations and quirks that have inspired people to make similar, enhanced tools like hledger and beancount. I've chosen hledger in particular for its enhanced budgeting and forecasting tools. We can easily establish a budget using scheduled, recurring transactions.

2020/01/01 Starting Balance
    Assets                        $1000
    Equity:Starting Balance      -$1000

~ Daily  ; Food goes on the credit card
    Expenses:Food                 $8.00
    Liabilities                  -$8.00

~ Monthly
    Expenses:Rent              $1000.00
    Assets                    -$1000.00

~ Every 23rd day  ; credit card payment
    Liabilities                 = $0.00
    Assets

~ Every 2 weeks
    Assets                     $1000.00
    Income:Salary             -$1000.00

hledger can produce rich forecasts. Here's a cashflow forecast for 2020 using the above budget (hledger cashflow --forecast --monthly -p 2020):

Cashflow Statement 2020

            ||      Jan      Feb      Mar      Apr      May       Jun      Jul      Aug      Sep      Oct       Nov      Dec
============++===============================================================================================================
 Cash flows ||
------------++---------------------------------------------------------------------------------------------------------------
 Assets     || $2824.00  $752.00  $768.00  $752.00  $760.00  $1752.00  $760.00  $752.00  $752.00  $760.00  $1752.00  $760.00
------------++---------------------------------------------------------------------------------------------------------------
            || $2824.00  $752.00  $768.00  $752.00  $760.00  $1752.00  $760.00  $752.00  $752.00  $760.00  $1752.00  $760.00

And here's the forecasted balances for each month (hledger balancesheet --forecast --monthly -p 2020):

Balance Sheet 2020/01/31,,2020/12/31

             || 2020/01/31  2020/02/29  2020/03/31  2020/04/30  2020/05/31  2020/06/30  2020/07/31  2020/08/31  2020/09/30  2020/10/31  2020/11/30  2020/12/31
=============++================================================================================================================================================
 Assets      ||
-------------++------------------------------------------------------------------------------------------------------------------------------------------------
 Assets      ||   $2824.00    $3576.00    $4344.00    $5096.00    $5856.00    $7608.00    $8368.00    $9120.00    $9872.00   $10632.00   $12384.00   $13144.00
-------------++------------------------------------------------------------------------------------------------------------------------------------------------
             ||   $2824.00    $3576.00    $4344.00    $5096.00    $5856.00    $7608.00    $8368.00    $9120.00    $9872.00   $10632.00   $12384.00   $13144.00
=============++================================================================================================================================================
 Liabilities ||
-------------++------------------------------------------------------------------------------------------------------------------------------------------------
 Liabilities ||     $64.00      $48.00      $64.00      $56.00      $64.00      $56.00      $64.00      $64.00      $56.00      $64.00      $56.00      $64.00
-------------++------------------------------------------------------------------------------------------------------------------------------------------------
             ||     $64.00      $48.00      $64.00      $56.00      $64.00      $56.00      $64.00      $64.00      $56.00      $64.00      $56.00      $64.00
=============++================================================================================================================================================
 Net:        ||   $2760.00    $3528.00    $4280.00    $5040.00    $5792.00    $7552.00    $8304.00    $9056.00    $9816.00   $10568.00   $12328.00   $13080.00

Getting data into a ledger

One thing that ledger-likes don't solve for is retrieval of data. Many people choose to enter the data by hand and reconcile against their banks' transaction logs periodically. One great advantage of a simple plain-text format for transactions, though, is that it's easier to automate the retrieval of this data from (some) banks so manual entry isn't required.

ledger-autosync

Many banks provide Quicken-formatted (OFX) exports of transaction data. ledger-autosync is a tool that allows you to either import downloaded OFX files into your text ledger, or will retrieve them automatically (if your bank supports this). ledger-autosync supports hledger-style syntax as well as the classic syntax, so it works great for my use case.

CSV import

hledger has rich support for importing from CSV using .rules files, an instruction format for hledger's own CSV import tooling. These allow you to export CSVs from your online banking portal(s) if automatic import through OFX isn't available. Unfortunately this is still a slightly manual process, but I've actually found it pretty low-friction to log into a couple of bank accounts once a week to pull transactions.

Investment data with yfinance

Because the text ledger format is so simple, it's easy to generate entries from almost any other tool. I wrote a little Python script that uses the yfinance library to pull ticker data from Yahoo Finance's API.

Evaluating & Looking forward

In general I'm pretty happy with the setup I've built. I control more of my data now, and hledger allows me to do granular forecasting of our budget, which has made it possible to automate even more of our transactions confidently.

There are a few more projects I'd like to undertake though:

Scraping Transactions

Unfortunately, while I'd prefer to simply vote with my dollars and bank only at institutions that provide good APIs, it's not always possible. I have had some success in scraping transaction data using Selenium and other tools, though, so I plan to continue automating even more so it's less necessary to download transactions by hand.

Investment Forecasting

One thing I traded off in the switch from Personal Capital was the use of their excellent retirement planning tool. With good data available from yfinance, I'd like to explore the use of common stock market modeling techniques to simulate possible scenarios and help us keep an eye on our retirement plans.