How to Think About Automation and the Minimum Viable Product

Virtually every time I get started on a new project, I find myself going down the following trail of thoughts:

-         This would be awesome (or just needs to be) in the cloud

-         Cloud configuration is kind of a pain and I really just want to be coding the actual thing

-         sigh I will configure the cloud part of this

-         Wait, cloud configuration is actually kind of fun!

I then proceed to compose an elaborate symphony of cloud infrastructure only to realize that it’s now 1am, that I can’t remember the last time I ate, and that my kids will shatter my precious sleep well before the sun comes up.

So, in the spirit of self-awareness and growing in the new year, here are some questions I (and any developer reading this) should ask themselves before delving into automation, much in the spirit of Joe Pera’s three questions at the supermarket. As it turns out, knowing when to start automating (and when to stop) is also key to dialing in exactly what is needed for a minimum viable product. For data scientists, these two problems often have the same solution.

First, let me share a fun and illustrative example for this – one in which I am both the client and the contractor. We recently hired a nanny for our kids, and one of my least-favorite chores is tallying up her hours and overtime in compliance with local labor laws here in Senegal. It takes maybe a half-hour per month to do it by hand, but I hate it. The current solution? A handwritten notebook of in and out times, which I do my best to keep on top of, along with a million unrelated things. My proposed solution? A QR code-based clock-in, clock-out system for her to scan when she arrives and when she leaves that writes timestamps to a database, which I can then query to make a monthly pay stub. Let’s ask some hard questions to see how much should be automated, and thereby what the MVP will look like.

Question 1: what are the risks of not automating?

It’s going to cost me that half-hour a month, but more importantly, it is simply easier for me to make a mistake in calculating the nanny’s pay doing it by hand. This is not acceptable! She works hard and deserves every penny. So, easy yes to automation – particularly because I already have the skills to make it happen fairly quickly. More generally, it is important to see manual processes – whether calculating a pay stub or deploying a machine learning pipeline – as inherently error prone. Plus, life is too short to be doing most anything by hand when you have the skills to automate.

This is not to say that all manual processes are bad. In fact, they are incredibly important for (human) learning. One of the first times I deployed an EC2 instance was to create a deployment pipeline server (itself an exercise in over-automation), and the first several times I created it, I ran through all of the configuration manually so I would know what was going on and could walk away with a new skill.

Finally, it’s not quite enough to say that “not all manual processes are bad.” Sometimes it is absolutely necessary to keep a human in the loop. Examples of this include human review of generative AI output for public consumption, medical diagnostics, and nuclear weapon strategy. (We’ve all seen The Terminator.) In this example, my human review element is looking over the timestamps and the paystub to make sure nothing looks off.

So, the balance here is really:

-         How much time will you save?

-         How much do mistakes matter?

-         How much does your current setup allow you to learn?

-         Would automation eliminate a critical element of human review?

Question 2: how hard will this be to fix if I do automate it and it breaks?

Okay, so we’ve decided to automate – now what? Now we’re deep in the process of debugging AWS security groups and editing configuration files and…wait a second. If I ever have to go back into the weeds on this, how difficult is it going to be to figure out how everything is set up? Three options exist for me here:

1)      Ignore it and hope for the best. This is possibly acceptable here, since the system failing means a little bit of manual database reconfiguration. It also could mean data loss – really unacceptable – but this can be mitigated by sending a clock-in/clock-out receipt to the nanny by text or email along with every database write. (As mentioned before, thinking about automation is also informing the details of our MVP, so we will add a notification service to the MVP.)

2)      Write better documentation. At least a sticky note to say how to fix it! Writing the documentation itself can be time-consuming, and saves only the time of figuring out how to redo the database configuration, not actually doing it.

3)      Just automate it. What would this entail? I’d have to open up my EC2 instance to talk to S3, store the edited configuration files in S3, and make sure my EC2 pulls the configuration files on first boot. Seems like this is the way to go.

But hang on, did we just solve the problem of brittle automation with more automation? We sure did. The automation was brittle because there was a step that involved manual processes. But still, I’ll be sure to document what’s going on here so if something does break in 17 months, I’m not trying to reverse engineer my own code.

Question 3: does it need to scale?

Scaling and automation go hand in hand. I do not want to configure the same web server manually over and over behind a load balancer. But, who says I’ll ever have enough traffic for a load balancer? This is about when it’s time to think about expectations.

Let’s say this system is so awesome that all my friends with nannies want in. They’ll pay $15 a month for access. And I’ve got a lot of friends, and so do they, and all their friends have nannies, too. I’m retiring early. Yes, in this case, I will want an amazing amount of automation to spin up zillions of EC2 instances behind a load balancer.

But it is important to recognize that this is not the case. I have a handful of friends with nannies. They probably are fine pulling out a pocket calculator once a month and just doing the math. This is a pet project, because I like to build stuff, and it’s more fun for me to build and learn something for a couple of days than it is to do elementary school arithmetic once a month until my kids are old enough to stop climbing up the bookshelves.

Conclusion

In thinking about how to automate a timekeeping system for a nanny, we developed some questions to guide general-purpose decision-making around automation. Automation can be a gray area involving some code and some manual processes, and Question #2 tells us that thinking a little more black and white is often helpful, while Question #1 reminds us that sometimes we can literally save lives, or at least learn a fun new trick, by keeping humans involved. Question #3 tilts the balance toward automation again if the system is going to be large, but still isn’t enough to counteract the considerations brought by Question #1.

And typically, once you’ve answered those questions, you have a better sense of what your MVP is going to look like. For today, my MVP is just an EC2 instance that writes timestamps to a database and texts my nanny, and that can be started and stopped just by running Terraform.