Dan Goldin

Yahoo fantasy football stats: 2021-2022 edition

Sun, 29 Aug 2021 15:53:06 -0500

I had a bit of a blogging break this year but the pull of tradition is too strong. For the sixth year in a row I updated my Yahoo fantasy football stats scraper to pull Yahoo’s projected fantasy stats for the upcoming season. The only change this year was the addition of a “O Rank” column which was simple to implement so I have no excuse for pushing this off till the last minute. Hopefully this shames me into writing more.

Instacart price markup

Mon, 15 Feb 2021 15:53:06 -0500

Last year I looked at the markup of a few different food delivery services by comparing their prices against the ones on the restaurant’s menu. I got the chance to do the same thing for Instacart a few days ago when the receipt was included alongside the delivery.

The markup wasn’t as significant or variable as the food delivery apps and was just over 9% for the majority of items. I imagine the intention was to keep it single digit in case anyone did the math and 10% does seem like a reasonable markup - even with the additional tip.

Slack Idea: Hierarchical Channels

Sun, 14 Feb 2021 15:32:14 -0500

While there are a lot of strong opinions about Slack on both sides, it clearly changed the way work gets done. I’m mixed. I appreciate the ability to quickly drop a question or response asynchronous yet acknowledge how disruptive it is and prevents deep work.

The one thing that I did notice in our usage of Slack is that quite often the same message ends up being posted across multiple channels due to relevance for multiple groups. Yet many members exist in both groups and the repetition is redundant and inefficient. What if Slack had a way to have hierarchical channels?

Clubhouse as a solution to Deepfakes

Sun, 31 Jan 2021 23:37:18 -0500

My mind started wandering while half-listening to a Clubhouse discussion and I had the thought that Clubhouse - and generally any real time communication app - can end up being a solution to Deepfakes. At some point Deepfakes will be good enough to render in real time but until then having synchronous communication is a way to confirm the authenticity of the participants. On top of that I’m sure you can find some blockchain way to authenticate the session to prove that it was done live and was unaltered.

Pre-internet businesses were tough

Sat, 23 Jan 2021 00:00:00 +0000

One of my favorite podcasts is NPR’s How I Built This that conducts interviews with entrepreneurs across a variety of industries, not just tech. A recent episode was an interview with Judi Sheppard Misset, the creator of Jazzercise - a dance fitness class that she launched in 1969. We all realize that there’s much less friction to launch a business these days but the interview makes you see how significant the changes have been.

Billion dollar features

Fri, 22 Jan 2021 00:00:00 +0000

There’s an old story of Steve Jobs meeting Drew Houston in order to acquire Dropbox but after being refused calling Dropbox a feature. Dropbox kept innovating, building a great product, went public and is currently a $9 billion company. Clearly Dropbox proved Steve Jobs wrong. But did it really? Dropbox recently announced a round of layoffs and their stock has dropped 20% since they went public. Even on a personal note I plan on moving my files from Dropbox to Google Drive.

From AMP on Jekyll to Hugo and GitHub Actions

Fri, 08 Jan 2021 00:00:00 +0000

This the first post written on my new blogging setup so what better topic than the new blog setup? Years ago I redesigned my entire site to use AMP across all devices, powered by Jekyll, and hosted on GitHub pages. At the end of 2020 I decided I was ready to abandon AMP and move toward a more traditional and straightforward setup while leveraging a more modern toolchain.

Due to my adoption of AMP my Jekyll/GitHub Pages setup was highly custom. I had to build it locally and push the rendered version to GitHub since I had custom ruby scripts handling a lot of the AMP conversions. In addition, I had to resort to a variety of workarounds to get Disqus working within AMP. Also, since AMP inlines all the CSS the build took almost 8 seconds despite being highly optimized and running a pretty powerful computer.

Everyone's a hustler

Wed, 30 Dec 2020 00:00:00 +0000

It’s the holiday season and mine has been filled with a lot of Bing Crosby. I’m one of those people that will Wikipedia anything new and came across this sentence that describes his popularity:

“Also in 1948, Music Digest estimated that his recordings filled more than half of the 80,000 weekly hours allocated to recorded radio music”

This reminded me of another celebrity from that era, Charles Lindbergh, and his popularity:

Science vs policy

Wed, 30 Dec 2020 00:00:00 +0000

In what seems to be a part of the modern American experience there’s another case of a scientific group proposing some changes and a regulatory agency ignoring the suggestions. In this case, a group of scientists proposed further limiting sugar and alcohol but the USDA didn’t include these in their official guidance. There may be some truth to the USDA’s claim that there wasn’t enough evidence to justify a change but I can’t help but think this is another example of strong private interest groups sacrificing the public good for themselves.

2021 goals

Tue, 29 Dec 2020 00:00:00 +0000

Despite 2020 showing us how unpredictable the world can be, it’s that wonderful time of the year when everyone comes up with goals for the upcoming year. On one hand it’s impossible to predict what will happen but on the other hand it is useful to give some sense of purpose and direction. Being thoughtful and public around our goals is also a way to encourage us to commit and see them through completion. I didn’t do so well with my 2020 goals but even the process of generating them was useful to help me understand where I am and what I’m interested in. For 2021 I’m adopting a similar approach but rather than get too specific I want to think more thematically and use 2021 to recover from the bad habits introduced in 2020.

Challenges in identifying a spawned OS X process

Tue, 29 Dec 2020 00:00:00 +0000

As part of my project to come up with a utility to deduplicate images on S3 I started working on a proof of concept to validate the flow and functionality. I know that the AWS S3 API has everything I need but I wanted to see whether I could use the built in OS tools to handle the image comparison without resorting to a webapp. The steps I had in mind were pretty simple:

Improving the accuracy of evergreen content

Tue, 29 Dec 2020 00:00:00 +0000

While looking through my blog’s analytics I noticed that I have a few posts that have views despite being written years ago. Some of these are still relevant while others are woefully out of date. For example, the difference between yyyy and YYYY in Java’s SimpleDateFormat is still relevant but how to scrape web pages has changed significantly since 2013 and anything that has pricing or performance information is likely wrong.

Time to remove AMP from my blog

Tue, 29 Dec 2020 00:00:00 +0000

A few years ago I redid this entire blog to use AMP - not just for mobile but also for desktop. The motivation was to improve the performance while scratching my curiosity itch. It was a fun experiment but these days I’m more averse to Google’s dominance and think I can do even better by taking what I liked from AMP but going even simpler. It took quite a bit of work to get everything working the way I wanted to in AMP - for example converting AMP markup into a valid RSS feed and getting Disqus working - and while it’ll be a shame to undo all that work it feels like the right time. The current deployment is custom Jekyll on top of GitHub Pages but I’m hoping to simplify the entire workflow and take advantage of some of the modern tools - for example GitHub Actions.

Blogging: 2020 and beyond

Mon, 28 Dec 2020 00:00:00 +0000

I have a script I use to analyze and visualize my blogging. One script generates a simple CSV file containing a variety of metadata about my blogging that I then feed into a Grafana dashboard to see how I’m doing. Another script does some simple visualizations. In the past years I had an annual tradition of sharing these visualizations and describing each one but I took a look at this year’s and for the most part there’s nothing that interesting.

Increase reliability through loose coupling: An adtech example

Mon, 28 Dec 2020 00:00:00 +0000

A few weeks ago I evangelized degrading functionality in the case of system outages as an option versus building out full regional resiliency. Someone asked me to dig into my example more so I’ll share a bit of how one of our core systems at TripleLift is set up.

We run an advertising exchange handing tens of billions of ad requests each day. AdTech has a ton of different companies with each doing a bit of everything but a simple way to think of us is an “exchange” that sits in between buyers and sellers of ads and ad space. In this case, a seller would be a publisher, such as the NY Times that has ad opportunities available. The buy side would be a Demand Side Platform (DSP) that has an API integration and submits bids on behalf of advertisers.

SpaceX Starship vs Saturn V

Sun, 27 Dec 2020 00:00:00 +0000

As a kid I used to be obsessed with all things space. A big part was having the same name as the head of NASA, Daniel Goldin, where I harbored illusions that they wouldn’t even have to change the nameplate on the door. My parents supported this interest and got me subscriptions to a variety of space and science magazines. I did not go the space route but despite that I’m sure that passion and curiosity contributed to my growth and I still have a soft spot for space.

2020 donations

Sat, 26 Dec 2020 00:00:00 +0000

Last year I wrote a short post highlighting my donations with the hope that they encourage others to donate. Given this year’s trials and tribulations it’s even more important to donate and help others if you have the means.

I continued my donations from last year and donated to the EFF, ACLU, Wikipedia, and Planned Parenthood for the same reasons as last year. The additions this year have been a function of everything that’s happened. Earlier in the I made multiple donations to various BLM charities and most recently contributed to Fair Fight to help with the Georgia election.

Tools of 2020

Sat, 26 Dec 2020 00:00:00 +0000

Despite not being in the code too much these days I still spend a significant amount of time on the computer so thought it would be fun to share my commonly used tools. Maybe there’s something here you’re not familiar with and it might be enough to encourage you to give it a shot.

Hardware

For the first half of the year I used Ubuntu on a ThinkPad X1 but switched back to a Macbook (Early 2019 Pro) due to a variety of stability issues. The keyboard isn’t as nice but I’m overall more productive and glad to have the Microsoft Office suite back.

My Tesla stock trade

Fri, 25 Dec 2020 00:00:00 +0000

The above is Tesla’s stock price since its IPO in 2010. On its first day it closed at $3.84 and was relatively flat until earlier this year at which point it shot up closing at $661.77 on December 24. If you invested $1,000 at the IPO that would be worth over $172,000. I’m one of those unlucky fools that purchased Tesla early - in 2011 - but then sold it in 2015. My $1,148.20 investment resulted in a pre-tax profit of $7,691.26. A nice gain until you consider what could have been.

Incident response

Thu, 24 Dec 2020 00:00:00 +0000

In honor of everyone on-call during the holidays I thought I’d share my evolving thoughts around incident response. Every company has something here and as much as we can strive for perfection it’s impossible and we’ll inevitably have incidents that need handling.

I’m still thinking through the exact way to approach and summarize my thoughts but I find it useful to start with goals and first principles. The ultimate goals are to resolve the issue as quickly as possible and prevent them and other similar issues from recurring. In terms of axioms, we want to align incentives and make those able to solve the problem own the problem. In addition, we need to embrace that we work on highly dynamic systems - both the systems and people change. And finally, we need to consider incident response as one of the many responsibilities engineers have.

An ounce of prevention is worth a pound of cure

Wed, 23 Dec 2020 00:00:00 +0000

As we get into the last week of the year many software teams are in a code freeze - avoiding any deployments that may cause an incident and cause a dreaded holiday page. We have a tendency to reward the firefighter for pulling an all-nighter fighting an outage but don’t value those that avoid the fire in the first place. It’s natural to throw accolades at the firefighter since it’s very easy to see the impact in their work; it’s more difficult to see the value in the person that prevents issues from even happening. The counterfactual is difficult to prove so we diminish its significance but that’s where long term value is actually built.

Browser zero

Tue, 22 Dec 2020 00:00:00 +0000

Inbox Zero is a productivity philosophy that’s centered around getting to an empty email inbox. That means you treat your inbox as a todo list and once you’ve handled every email you’re good to go. I’m somewhat successful dealing with my inbox but my browser tabs are another story. Browser Zero is a corollary to Inbox Zero. What good is getting to an empty inbox if you still have dozens of browser tabs open awaiting your attention?

2020 Goal review

Mon, 21 Dec 2020 00:00:00 +0000

At the beginning of the year I gave myself 4 goals for 2020 and now that the year is wrapping up it’s time to see how I did: not well. The year was a tough one for most people but all things considered I’m in a privileged position and can’t blame COVID too much. For sure there were periods of ennui but the root was that I just lost interest and motivation in some of these. So how did I do?

Stop charging for support plans

Mon, 21 Dec 2020 00:00:00 +0000

My latest rant is against support plans for enterprise software. It’s frustrating to be spending hundreds of thousands of dollars on a contract and yet be forced to pay an additional fee for support. Even worse, this additional fee is usually a function of your existing spend.

I understand the rationale - there needs to be a cost to support otherwise everyone has an incentive to escalate and spend is likely correlated with complexity and volume of services used but it just feels as if I’m being nickel and dimed.

Blame the algorithm

Sat, 19 Dec 2020 00:00:00 +0000

Stanford Medicine is in the news for using an “algorithm” and allocating minimal COVID vaccines to those actually on the frontlines. Instead it supposedly prioritized senior administrators who are working from home. I’m sure there’s some truth to the algorithm biasing allocation to those who are older but whenever I see someone blame an algorithm my spidey-sense goes off. The algorithm in this case was likely some extremely simple calculation in Excel that could have been easily verified by a human - both before it was run as well as a quick analysis of the results.

Removing duplicate files in S3

Fri, 18 Dec 2020 00:00:00 +0000

I’m a digital hoarder and whenever I had to switch computers, I was always worried about losing files. These days it’s both lower risk since so much is scattered across the cloud but with the ascent of AWS I’ve resorted to just backing up my computers onto S3.

I simply do a recursive copy of my home folder to S3 and call it a day. One problem this exposes is that there are duplicate files scattered all over the place. For example I’d have something both in my Downloads folder as well as in a Photos and maybe even a Dropbox folder. Or I would just have the same file duplicated in the same directory. At the end of the day it’s not a huge deal but at the same time it feels dirty so I started working on a script to identify these duplicates.

What do adtech and BGP have in common?

Thu, 17 Dec 2020 00:00:00 +0000

Despite writing web applications many engineers are not familiar with the infrastructure side of the internet - DNS, BGP, TCP/IP - and yet depend on it all working. Over the years I’ve gotten a better understanding of the guts of the internet but only in the past few weeks have I discovered Resource Public Key Infrastructure (RPKI) on top of BGP. RPKI is meant to add a layer of trust on top of the existing infrastructure which was designed many decades ago without worrying about malicious actors.

Substack can evolve into a marketplace

Tue, 15 Dec 2020 00:00:00 +0000

When I first heard of Substack I wasn’t too impressed. I fell into the usual engineering trick of thinking it’s something that could be “built in a weekend” but there’s a lot going on behind the scenes. Last week I listened to an interview with Substack’s CEO, Chris Best, which made me realize the scale and scope of Substack’s ambitions.

At the most fundamental level it’s a product that allows writers to publish a newsletter - taking care of all the mechanics to let writers write. Yet I naively assumed this was just a text editor (think Medium), coupled with payments (Stripe), all hooked up to email (Sendgrid).

Do your users delete or archive your email?

Mon, 14 Dec 2020 00:00:00 +0000

If you’ve been following this blog you likely know that I subscribe to the Inbox Zero approach to email management. I treat my inbox as a todo list and strive to get it down to zero with varying success. In any case, I spend a lot of time in my email and I noticed an odd pattern to how I deal with handled messages. Some I will delete while I others I’ll archive but it happens naturally. It’s not entirely subconscious but it’s a very quick decision I make as I run through my messages. The obvious rule is that anything I expect to search or reference later on I’ll archive while the truly useless emails I’ll simply delete.

Impact of getting to the top of Hacker News in 2020

Sun, 13 Dec 2020 00:00:00 +0000

Two days ago I wrote up a quick post highlighting how Amazon owns more than $2B domains and it struck a nerve. It’s been shared quite a bit on Twitter and I posted it to Hacker News on Friday night before going to bed to wake up to it being near the top which drove a ton of views. It’s always a surprise seeing what takes off but there’s something about Amazon owning billions of dollars in domains that’s shocking and gets people riled up. My goal is not to write in order to get my posts to go viral but it’s empowering knowing that thousands of people are reading something you’ve written.

Evolution of open source licenses

Sat, 12 Dec 2020 00:00:00 +0000

Open source changed the way we write code. It’s given many of us a starting set of tools that allowed us to quickly put together new products and allowed us to specialize. These days open source is as strong as ever but there are a lot of changes happening on the licensing side. Under the older open source licenses, any company could take your code and then launch it as their own service. This wasn’t a problem in the past due to a highly fragmented market and trusting the developer to provide the gold-standard enterprise service but the rise of the cloud providers changed the dynamic. They have the scale to both build the expertise but also to lock-in customers.

Amazon owns more than $2B worth of IPV4 addresses

Fri, 11 Dec 2020 00:00:00 +0000

While listening to a podcast discussing BGP I heard the fact that AWS owns more than $2B worth of IP addresses. I knew AWS was massive but this came as a big shock so I decided to do some digging around. I came across a site that listed the market prices of IP addresses and the range looks to be anywhere from $20 to $30 per IP depending on the block size. Now it was time to figure out the IP addresses owned by Amazon. I figured this would be difficult but lucky for us AWS actually publishes their entire set of IP addresses as JSON.

Pay to enjoy video games

Tue, 08 Dec 2020 00:00:00 +0000

As a kid I was incredibly addicted to video games. I had a whole cat-and-mouse routine going on with my parents. They would put a BIOS boot password on the computer and I would open up the case, remove the battery, and reset the BIOS settings. They would lock the basement, where our one and only computer was located, and I would climb in through the window. They would lock the door to the computer room and I would undo the door hinges. It was difficult stopping me but remarkably I outgrew them and am now a somewhat well-adjusted member of society.

Degrade functionality instead of building cross region availablity

Wed, 02 Dec 2020 00:00:00 +0000

AWS had a large multi-hour Kinesis outage last Wednesday that affected a variety of dependent services, including Cloudwatch, Lambda, ECS, and EKS. These systems are complicated and highlight the scale and complexity of modern cloud computing.

It’s impossible to be perfect but it’s a reminder for all of us to think through our applications and identify their failure cases. Just because there was a major AWS failure that affected your service does not mean you should drop everything and convert your application to be highly-available with regional failover. Instead try to model the probability of a particular failure, its impact, and the effort of a fix. With the above you should be able to identify some easy wins that get you fairly far.

Static code analysis with DeepSource

Thu, 26 Nov 2020 00:00:00 +0000

While listening to the Software Engineering Daily podcast I came across an interview with Jai Pradeesh and Sanket Saurav who are the founders of DeepSource, a modern code analysis tool.

I’m a sucker for these types of tools and willing to try anything that’s low friction and promises to me more productive so I gave it a shot on two of my open source repos - health-stats and blog-analytics.

Bringing a user centric approach to the command line

Wed, 25 Nov 2020 00:00:00 +0000

I have a small script that takes an export of Apple Health data and then dumps it into a MySQL database that I then use to visualize my health metrics over time. My prior workflow was to open the app and then Airdrop the file over to my computer at which point I’d unzip it, copy the relevant file over into the script directory, and simply run the script.

Subconciously learning to type Russian

Sun, 22 Nov 2020 00:00:00 +0000

I was born in the Soviet Union, came over as a 5 year old, and have seen my Russian getting worse and worse due to a lack of exposure. I’ve been using quarantine as a way to practice and improve my Russian through daily lessons in the Duolingo app. It’s a well designed app and while it has some flaws I’m on a 191 day streak and am loath to give it up. One thing that’s surprised me was how I’ve subconsciously picked up the Russian keyboard layout. I started having to hunt for each letter but these days have a subconscious feel for where the letters are. I actually do a better job typing without any thought than trying to figure out what letter comes next and where it is.

Cursed by GDPR

Fri, 20 Nov 2020 00:00:00 +0000

I had a fun little experience over the past few days. Someone had ordered Domino’s in the UK and seemingly typed in my email address by accident. This gave me the privilege of receiving daily emails from Domino’s promoting their latest deal. The first few days it was novel and interesting seeing what was popular across the pond but this morning I decided I might as well unsubscribe.

Lo and behold the unsubscribe link didn’t work. Rather than a web page I got a friendly server error indicating “Access Denied” and that I did not have permission to access the unsubscribe page. My suspicion that this was a result of Domino’s blocking access to the site turned out to be true since I was able to unsubscribe using a VPN.

Medium is a shopping mall

Sat, 14 Nov 2020 00:00:00 +0000

I empathize with Medium’s need to make money while avoiding ads but as an author it’s a bad deal. Medium has great editing and writing tools and a built in audience but if your goal is to share your articles with as broad of an audience as possible the fact that Medium throws up a paywall is a non-starter. Sure you can avoid it by going incognito but how many readers know that?

Poor Intel

Wed, 11 Nov 2020 00:00:00 +0000

Earlier this week Apple announced the new MacBook Air, featuring the Apple-built M1 chip, and heralding the return to their own chips. I wrote the first post on this blog in 2008 covering Apple acquiring PA Semi and claimed that it made sense to design their own chips for phones but not PCs. Well it’s been 12 years and Apple has figured out how to make incredible chips for their phones so might as well expand to laptops as well.

Google and the Apple Watch

Tue, 27 Oct 2020 00:00:00 +0000

If you were to summarize my personal tech stack it would be Google software and Apple hardware. There are a few exceptions but those two companies likely cover the bulk of my technology usage. They’re great at what they do within their domains yet the intersection is lacking, especially on the Apple Watch. I suspect Apple is to blame for some of this but it doesn’t seem as if Google doesn’t even try to build for the Apple Watch.

Gasoline

Wed, 21 Oct 2020 00:00:00 +0000

I’m a believer in climate change and know we need to change our habits and yet every time I drive I’m reminded of how amazing gasoline is. A gallon of a liquid is not much - we can all drink a gallon of water a day and yet a gallon of gasoline is enough to move a mutli-ton car more than 20 miles. It’s so normal that we don’t appreciate how amazing that actually is.

Substack and paid newsletters

Wed, 21 Oct 2020 00:00:00 +0000

It’s starting to feel as if everyone I follow on Twitter is launching a Substack newsletter. I don’t know how this plays out but my inbox is already overwhelmed and having more longish-form content to go through just won’t happen. It’s an interesting space that’s changing quickly and will look very different in a few years. Many seem to think that there’s a huge opportunity here for authors to build a niche audience that’s able to sustain their writing but i’m not wholly convinced. If you’re able to get the best of the best does it make sense to go for the second tier? Some people may want that additional depth but the majority will be happy with the highly skilled writer covering a broad topic. These will be the small number of authors who amass huge audiences and capture the majority of the revenue while the rest will be focused on niche topics with enough of an audience to keep them going. Inevitably, these “tail” writers will band together and offer their newsletters as a bundle. What’s old is new and we will see the rebirth of newspapers and magazines.

Cold turkey and fighting temptations

Tue, 22 Sep 2020 00:00:00 +0000

Some people are able to control urges but I’m much better off just removing the objects of desire entirely. In high school this manifested in me spending way too much time on video games. These days it’s buying a bag of potato chips and then being unable to stop once I crack it up. I tried giving TikTok a go to see what all the fuss was about but had to drop it cold turkey after realizing that I was spending too much time on it.

Get Google Group membership using Calendar

Sat, 05 Sep 2020 00:00:00 +0000

I’m not sure where I discovered this tip but I’ll often want to know the members of a Google group. The ways to get this are constantly changing - from double clicking the group within the Gmail compose window for a summary modal to going deep into the Groups UI. Somewhere in between is the group view provided by Google Calendar. I’m often in there and it turns out to be a pretty quick way of getting the list of all the users in a group without having to have anything open in another tab. One of the best parts is that it gives you the ability to expand and dedup group memberships which is great for dealing with nested groups - such as sub teams.

iOS 14 and local network access

Thu, 03 Sep 2020 00:00:00 +0000

Last week I installed the iOS 14 beta since I’m curious about what’s coming - even if it ends up bricking my phone. I expected it to come with more privacy features but one that surprised me was an alert that apps wanted to “find and connect to devices on your local network.” This is an obvious one Apple covered this functionality during WWDC. It’s interesting seeing which apps request this permission and speculate why.

Working your way up the engineering management stack

Fri, 28 Aug 2020 00:00:00 +0000

During a engineering management peer group discussion this morning we were talking about how involved each of us are in our team’s activities - for example how often do we look at code? Do we leave comments on pull requests? How involved are we in spec reviews? And a variety of questions of that ilk.

At that point in time I wasn’t clever enough to come up with a framework but it’s very much about working your way up further from the code as you work your way up the management track.

Back to Mac

Mon, 24 Aug 2020 00:00:00 +0000

After almost 2 years of Ubuntu I’ve moved to a Mac. There were a variety of reasons for switching back but the ultimate reason was simply that I had stability issues on Ubuntu and it was rare for it to go a week without freezing up. The overall lack of polish in Ubuntu didn’t help but I could have dealt with it had it not be for the frequent freezes and crashes.

An era of productivity

Fri, 21 Aug 2020 00:00:00 +0000

A trap software engineers usually fall into is underestimating how long some things will take and I’m no different. Yet lately I’ve somehow been on the other end. Nearly every task I’ve had to do the past few weeks - either technical, managerial, or personal - has ended up taking a shorter amount of time than I expected. The blocker has been getting into the mindset and finding the time to do it but once I start I crank it out.

Apple and Mr. Burns: Indestructible

Thu, 20 Aug 2020 00:00:00 +0000

Apple hit a $2T market cap and yet I can’t help but think they’re in an incredibly precarious situation due to the relationship between the US and China. I find the Apple/US/China dynamic much more interesting than what’s going on with TikTok. Right now it feels as if every side wants Apple to succeed. For the US it’s a success story and a cultural export to the world. To China it’s an indicator that they can power the manufacturing, at scale, of one of the most sophisticated technical products in the world and a massive boon to their economy. Yet it only takes a bit of politics to completely disrupt that balance.

Yards, balls, fences, and software engineering

Thu, 20 Aug 2020 00:00:00 +0000

One of my strongest beliefs is that the best engineers are able to find ways to unblock themselves. Similarly, a pet peeve is when people say “that’s not my job” and expect a problem to be fixed by someone else. Sure this may occasionally happen but it’s important to have the attitude that you’re willing to get your hands dirty in order to solve your problem and move on.

An analogy I used to represent this concept is to imagine having a house with a yard that’s next to a neighbor. Let’s say you’re throwing a ball around and it lands in your neighbor’s yard. Do you quickly hop over, grab it, and then come back? Or do you go to their front door, ring the bell, and then have them retrieve it for you? If you have a good relationship with your neighbor you’d clearly just go and get the ball.

The sandpaper learning method

Tue, 18 Aug 2020 00:00:00 +0000

When gluing two pieces of wood it’s useful to use sandpaper to improve the effectiveness of the glue. Learning follows the same pattern: you want to struggle a bit and get yourself ready so the knowledge sticks.

Imagine you have two people who each encounter a problem they don’t know how to solve. The first immediately goes and asks for help from a peer and quickly get it resolved. The second person tries to solve it themselves, does research, tries a few different approaches, struggles throughout and then ends up getting help from a peer. The first person gets to the solution quicker but it’s the second person who actually learns. The struggle for the second person made it easier for them to both understand and retain the resolution. It’s the second person that will have that faster rate of learning and will be able to apply their knowledge more broadly than the first.

Yahoo fantasy football stats: 2020-2021 edition

Tue, 18 Aug 2020 00:00:00 +0000

For the fifth year in a row I’ve updated my script to fetch the projected stats for the upcoming fantasy football season. These days I’m torn on football as a whole - both due to its politics and dangers - and don’t plan on watching too many games. Yet I enjoy the competition with my friends and the rote work of updating my scraping script to work every year.

This time around there haven’t been too many changes: Yahoo changed the order of a few columns and introduced some minor stylistic changes but the code only needed a few changes to work - much simpler than last year which required running with an adblocking extension to bypass a script blocker.

Coupling in action: protobuf enum fields

Mon, 17 Aug 2020 00:00:00 +0000

The next few blog posts are all topics that have come up recently in my 1-1s and serve as good examples for how I think about code and software engineering. The first post is a discussion around coupling and the tradeoffs we considered.

Our pipeline was covered in depth on the high scalability blog but the important part for this post is our event collection piece which consists of the following components:

Big money, not big data

Sun, 16 Aug 2020 00:00:00 +0000

It’s common for companies to complain about the challenges of running big data and how difficult it is. The reality is that unless you’re running at massive scale (Google or Facebook) your problems are more to do with big money rather than big data. It’s expensive to store, process, and expose terabytes and the difficulty is in doing it cost effectively, not in simply doing it. There are enough modern tools out there across all the cloud providers (AWS, GCP, Azure) and vendors (Snowflake, Databricks) that it’s possible to do nearly everything you want but you’ll just have to pay for it.

Traveling at the speed of light through life

Sun, 16 Aug 2020 00:00:00 +0000

Years ago I was reading The Elegant Universe and came across a mind-blowing idea - both then and every time I remember it: we’re always traveling at the speed of light through four dimensions - the three spatial dimensions as well as time. It just happens that almost all of that speed is through time. But that does mean that as you speed up in the spatial dimensions, time slows down - which has been shown to be true by measuring time drift in airplanes.

Three types of advice from a manager

Sun, 09 Aug 2020 00:00:00 +0000

During one of my first meetings with a former manager he gave me an introduction to his working style. One of these was how he gave three types of advice:

A passing thought. This is not something that was thought of too deeply and just popped into my mind. I’m sharing it in case it’s helpful but it’s just as likely to be useful.
A suggestion but trust you. I’ve run into similar scenarios in the past and learned from those experiences. This is one of those. At the same time I trust you to make the right decision since you understand the current problem and constraints better and know the modern tools. And if it turns out that the approach doesn’t work it would act as a learning opportunity.
Just do it. This is an order. While I will do my best to explain my reasoning I am your manager and don’t need to justify it.

The expectation is that the first two should make up at least 90% of the advice (and ideally more) since they’re the ones that actually instill ownership and empower the individual to make decisions. It’s also a failure of a manager if you end up in a situation where you have to override your team that often. It may be an indicator that you did not provide enough context and that your team is not seeing the full picture. It may also be that the team just doesn’t have the skill or desire to operate at the level you need them to. In any case the fact that you need to resort to this last level often is a symptom of a problem that you need to root out.

TikTok: A Prisoner's Dilemma

Sun, 09 Aug 2020 00:00:00 +0000

Everyone has an opinion here and I’m late but joining the fray. I fall into the ban TikTok app. I would not be doing it the way it’s currently being done but there’s a legitimate case to make that the US should adopt a tit for tat strategy with China. China prevents many US companies from competing there so why should we allow it?

There are of course arguments around us being the United States and needing to take the high road. This was a worthwhile belief decades ago when we had the belief that a richer China would be a more democratic and fair China but in fact the opposite has happened - China has gotten more and more emboldened and now has leverage and power. Why let it get any stronger?

Products change, humans stay the same

Thu, 30 Jul 2020 00:00:00 +0000

While on a quick driving errand a few days ago I decided to listen to the radio for the first time in years. I was quickly hit with a bit of nostalgia - the radio show that was on had one of those “First caller to answer this correctly gets Y” sessions. I’ve heard this thousands of times growing up but this was the first time in a very long time and for whatever reason I immediately thought of both how modern music streaming services no longer have these contests and that a similar need is being met by HQ Trivia.

Analyzing the AWS EC2 reservation options

Mon, 27 Jul 2020 00:00:00 +0000

While writing the post on AWS reservations I started thinking if there’s any arbitrage opportunity in the reservations. For example - does it make sense to do a 1 year or 3 year reservation for some instance types of upgrade an instance class to get a better reservation value?

You can do this manually using ec2instances.info which provides a quick way to look at EC2 pricing info although forces you to pick the type of reservations you’re interested in. To speed things up I just ran the ec2instances.info script to scrape the data into a JSON with a quick script to extract the relevant details into a CSV file. After that, it was as simple as loading into Google sheets for some analysis and visualization.

Amusing Russian translations in Duolingo

Fri, 24 Jul 2020 00:00:00 +0000

Many are using the lockdown as a way to improve themselves. I grew up a native Russian speaker which has unfortunately eroded over the years and I’ve spent the past few months improving my Russian through Duolingo. It’s a surprisingly effective app and has enough gamification to keep me going. My approach has been to just take the tests to skip out of lessons until I master every lesson. It started off easy but I’m now failing to advance about 20% of the time. Some of these are very much due to Russian specific issues - namely the fact that very often you can change the order of words in Russian without affecting the translation which Duolingo doesn’t seem to be able to handle well. Yet to keep things fun I’ve been screenshotting amusing translations and thought I’d share the best ones here.

Quibi's problem

Sat, 11 Jul 2020 00:00:00 +0000

Everyone loves to bash on Quibi, including me, and yet I feel guilty. We should applaud those who try something new rather than mock it in hindsight. Sure, Quibi raised a ton of money and has an illustrious team but they still deserve respect for attempting to build something from scratch.

Earlier this week I received a Quibi survey and decided to do my part to give them honest feedback. As I was going through the survey that was attempting to understand my usage (or lack of) I developed a personal theory of why it’s failing.

Optimizing AWS reservations

Wed, 08 Jul 2020 00:00:00 +0000

The Information has an article making the case that Uber was better off having their own data centers versus relying on the cloud given the impact of COVID but that would depend on their reservation strategy. Sure if they reserved AWS capacity then they would be on the hook. Alternatively, if they had no reservations and were running everything on demand they would have incurred higher previous costs but would immediately be able to shut things down. This got me thinking about the optimal reservation strategy - the reality is that this depends very much on your business and the ability to forecast but this is an interesting exercise in speculation.

Overthinking how to load data into MySQL

Tue, 30 Jun 2020 00:00:00 +0000

I have two projects that generate data to dump into my personal dashboard. One loads the health export from my Apple watch into MySQL and the other analyzes my blog posts and generates a CSV file of statistics that I then load into MySQL. The input to both is basically the same - either a file or a directory - and yet two different approaches to the processing.

Specify versions in technical blogging

Sun, 28 Jun 2020 00:00:00 +0000

I’ve been getting back into coding by hacking around on some side projects and am definitely rusty. Lately it’s been learning the modern Django stack and discovering it’s multiple major versions ahead of the last time I used it. But thanks to the wonders of the modern internet there’s a ton of content online that explains how to do certain tasks. Stack Overflow is one but there are also a ton of individual bloggers and tutorials that make it easier than ever to answer questions.

AWS 101

Fri, 19 Jun 2020 00:00:00 +0000

Earlier this week I hosted an “AWS 101” session at work. The goal was to give a small group an interactive introduction to the primary AWS services over 90 minutes. I figure I’d share our agenda and exercises in case they’re helpful to anyone else. We had trouble sticking to the times but the fact that everyone was remote did make it easier to do group-debugging with those stuck sharing their screens.

AWS Snowcone and my first job

Fri, 19 Jun 2020 00:00:00 +0000

AIT Tapes on Wikipedia

AWS recently launched Snowcone and it immediately reminded me of my first post-college job. I was a “Business Information Specialist” at ZS Associates, a consulting company with a strong emphasis on leveraging data to support pharmaceutical companies. The industry was setup in such a way that there were these massive vendors that would aggregate pharmaceutical prescription data from pharmacies, trace them to the prescriber (ie doctor), and then sell these back to the pharmaceutical companies. In turn, the pharmaceutical companies would give us this data to run a variety of analyses - ranging from calculating sales rep commissions, to setting their quotas, to identifying off-label prescriptions.

Software defined photography

Mon, 15 Jun 2020 00:00:00 +0000

I was talking to a coworker earlier today about the ridiculous quality of today’s smartphone cameras. He pointed out how it’s becoming more and more difficult for professional photographers to charge what they have been. The parallel here is how the invention of photography. That completely destroyed the value of artists and yet led to the launch of the modern art movements - Impressionism, Surrealism, Data, and countless others. Without photography would these have happened?

Dark Mode is for when you run out of real features

Thu, 11 Jun 2020 00:00:00 +0000

I’ve seen a lot of apps launch dark mode and I just don’t get it. I’m sure part of it is that I mostly stick to the default settings but I can’t help but think that if someone is prioritizing dark mode then they must have run out of actually useful features to build. I’m being a bit unfair since dark mode can also be a fun side project that someone picks up and may only require changing a few colors around but if it’s anything more than that it’s an indicator that something is off with their prioritization. Unless a product has reached the mature stage there are incredibly valuable things that the team can be investing in that actually improve the workflow and experience for their customers - dark mode is rarely that.

Catch up with your former coworkers

Wed, 10 Jun 2020 00:00:00 +0000

It used to be the case that people used to work at a company for decades, some even retired at the same company they started working at. We’re no longer in that world. These days it’s rare to find someone who’s worked at a company for longer than 5 years.

There is a benefit to this rotation though: when people change jobs they also bring their ideas and experiences, mix them with those at their new company, and converge to better and better approaching. A great way to accelerate this is by reaching out to all of your coworkers a few months after they left and asking how they’re doing. It’s a great way to stay in touch and maintain that relationship but it’s also a good opportunity to compare and contrast how they do things. They would have left recently enough where they understand your processes and embedded enough in their new roles to know how things work there. Those differences give you a lot of valuable advice on what other companies are doing and whether there’s anything you should change on your end.

Black Lives Matter

Sat, 06 Jun 2020 00:00:00 +0000

It’s difficult to think of or write about anything other than the protests going on right now. I hope that everything going on is a necessary step to get the United States to accept its sordid treatment of Black people and into a more fair and equal world. The past few weeks have made race come to the foreground as a topic that must be discussed. In the past I shied away from discussing anything race related; especially coming from my privileged background and despite my intent I was worried about the language I used and concerned about being misinterpreted. Given how pervasive and obvious Black mistreatment is it’s not something that anyone can be silent about. It’s critical for those of us that don’t know about what’s going on to educate ourselves and have the real and honest and often painful conversations. Those in turn will spur action which will lead to us finally tackling the nearly 500 year history of Black oppression.

Giving Obsidian a shot

Sun, 31 May 2020 00:00:00 +0000

I tried using Notion a while back but rebelled against having my data stored in the cloud in a proprietary format. For a variety of reasons I want my notes to be available locally in a text based format. At the same time there’s a lot to like about the modern tools and I feel myself being worn down. A few weeks ago I discovered Roam and am sold on its networked thought model. I’ve been using it for some of my note taking but was hesitant to commit given my concerns. Earlier this week I came across Obsidian - a Roam-like product that runs locally and stores everything in Markdown. This was opportune timing since I’m able to lean in on the networked thought model while still staying true to local files in an open format. I’m still getting used to Obsidian’s semantics - tags, links, and Markdown - but overall I’m bullish and glad to give it a shot.

Three categories of software engineering work

Sat, 30 May 2020 00:00:00 +0000

Managers love frameworks to help think through problems and I’m no different. While trying to think of a way to reason through our engineering work I started grouping it into three major categories:

Should have been done yesterday. Self explanatory but these are the items that as soon as you discover them you wish they had already been done. Critical bugs and issues fall into this but also product oversights that you wish you caught earlier.

Anatomy of a crypto mining hack

Fri, 29 May 2020 00:00:00 +0000

A few months ago I set up a simple ftp server to help a friend. I took a few security shortcuts which came to bite me this week when I received an alert from DigitalOcean that an instance was running hot.

I dug into it and noticed a series of processes being run by the ftp_user - the most impactful was a command called rsync. I’m familiar with rsync which syncs files across devices - and this was nothing like that. At this point I realized that I got hacked and quickly disabled the user and killed all its processes. The CPU usage dropped back to normal and now it was time to dig into the damage.

Metrics, logging, and error reporting

Wed, 27 May 2020 00:00:00 +0000

As software engineers it’s vital to understand how our applications are performing. The more information we have the better we can address problems, improve performance, and generally better solve problems for our customers. Imagine releasing a product with nothing being collected - you’d be flying blind.

I’m a huge advocate for measuring application performance in terms of business metrics and aligning it as much with the customer experience as possible. This post does not go into that. Instead, I want to share my beliefs around measuring the technical application performance. In my mind, they’re grouped into three major categories: metrics, logging, and error reporting. There is some overlap but I find it a helpful way to think about them.

Stock portfolio hedging during the coronavirus pandemic

Sat, 23 May 2020 00:00:00 +0000

I have had the majority of my liquid wealth invested in the stock market and I wanted to share what I’m doing these days. As a bit of a background, at the beginning of the year my portfolio primarily consisted of the tech companies I’ve held for years - Netflix, Amazon, Equinix, Google, and Facebook. After the first decline I liquidated about half of my portfolio - my thinking was that I’m still long term bullish about these companies and they will likely emerge stronger but I should hold on to some of my gains and have a bit of cash.

Prefer verbs to adverbs

Thu, 21 May 2020 00:00:00 +0000

Years ago I read a book that I unfortunately can’t recall that had a very insightful bit of writing advice - prefer verbs to adverbs. The premise was that many of us will spice up our writing by using adverbs to decorate our verbs but instead we should use strong and unique verbs. Your writing gains pithiness and leads to a more memorable message.

As an example, rather than prepending “said” with yet another adverb instead think of verbs that get your point across better - maybe it’s shouted, or exclaimed, or charged, or yelled, or accused, or hundreds others. Each of them carry more weight than just decorating said. Not to mention this is a great way to expand your vocabulary!

Shopify for food delivery

Thu, 21 May 2020 00:00:00 +0000

I wrote about the idea of a service that aggregates delivery drivers back in 2018 but given the latest wave of food delivery service bashing I couldn’t help but think of it again. This time around my thoughts are a bit more refined about how to actually build a business but do think there’s a real opportunity here, especially after seeing the challenges with the current model.

The existing food delivery apps charge high fees because food delivery is one of the highest costs. That’s the part that needs to be improved in order for the companies to make sense. As many people note restaurants have been delivering food on their own for decades while maintaining profitability so it’s clearly possible. Yet these online services struggle. Instead of focusing on the delivery side they’re in a race to capture as much of the demand as possible.

Limits of our imagination

Sat, 16 May 2020 00:00:00 +0000

While listening to a podcast describing the CSO role there was an interesting anecdote mentioned (~14:19 mark) that I couldn’t find anywhere else: in the 1900s the chief electrician on wall street was one of the highest paid positions. Back then, the electricity infrastructure was not as stable as it is now and it was not a certainty that it would stay up so those dedicated to keeping the systems running were appropriately rewarded. Over time, the electrical infrastructure became stable enough that it’s become expected and very few worry about it.

The curse of mastery

Sat, 16 May 2020 00:00:00 +0000

One of the most valuable skills a software engineer can have is a “tabula rasa” mindset. Tabula rasa, Latin for blank slate, is a philosophical concept dating back to Aristotle to explain how human minds are blank from the beginning and acquire knowledge through the human experience.

I think of it as being able to approach new challenges and experiences with no prior bias. Done well, this is an incredible power since you’re able to cut through the complexity and focus on what actually matters. Imagine you’re a software engineer who joins a new company - there’s so much business context you need to ramp up in addition to the code that’s grown in complexity over the years. By the time you’ve mastered everything it all makes sense and you’ve embraced the complexity.

Sometimes it really is just the dirty, hard work

Sat, 02 May 2020 00:00:00 +0000

It’s easy to fall into the trap of only wanting to do the new and sexy thing and giving up as soon as something requires the dirty work. That’s rarely a strategy for success and jumping from one idea to another idea and only focusing on the interesting parts is a sure fire way to not get anything done.

Last week I had a brief conversation with Twitter that got me looking for Apple’s S-1. Apple filed this in 1980 as part of their IPO process and on one hand it’s remarkable that you can find a 40 year old document online and yet on the other hand it’s disappointing that it came as a PDF. I imagine every year there are thousands (tens of thousands?) of people that want to take a look at the IPO docs of one of the most impressive tech companies and yet end up with a poorly rendered version. What if someone took the time to modernize it and convert it to a modern S-1? A massive amount of people would benefit and they would get the credit.

Data analysis and visualization

Wed, 29 Apr 2020 00:00:00 +0000

Yesterday I had the rare chance to actually do some coding and realized how rusty I am at numerical analysis in Python. The task was simple - ingest a CSV that had a date column, two categorical columns, and a numerical column - and then generate a grid containing a series of line plots, each of which would be a combination of the two categorical columns.

I did a ton of this work years ago so knew what was possible. That’s half the battle and after a bit of searches I got a working solution. At the same time I’m disappointed it took me that long, especially after seeing the brevity of the end result. Software ate the world and now every business is generating tons of data. Being able to make sense of it is an increasingly important skill set especially if you’re a leader. It’s unfortunately an area I haven’t kept up with over the years and something I plan on remedying after this experience.

Engineering management resources

Sat, 25 Apr 2020 00:00:00 +0000

At the beginning of the year I gave myself a goal to write more management posts. We’re nearly 5 months into the year and I’ve written only two posts even relating to management. A big reason is impostor syndrome - while I have only been managing a few years there are so many others that have more experience, are better managers, and are more effective writers. At the same time, the lack of experience hasn’t really stopped me from writing on other topics though so why should this be any different? This coupled with the fact that I write to organize my thoughts and clarify my thinking is all the more reason to commit to writing about new topics. As a start, I thought it would be helpful to share some resources that have helped me grow as a manager.

Margin of victory

Fri, 24 Apr 2020 00:00:00 +0000

During the AlphaGo documentary there’s a scene where the AI makes a very conservative move that the commentary notes is very different from what a human would do. The explanation is that AlphaGo doesn’t care about the margin of victory, just the victory itself. It’s trying to maximize the likelihood of winning, not the score differential - and that leads to a unique playing style.

I found this part of the documentary fascinating as it’s one of those things that’s obvious when explained but you’d never stop to think about. Humans rarely think about margin of victory. Most of our decisions and competitions aren’t quantifiable so we always try to do the best we can. In sports, on the other hand, where there are actual scores, teams will play for the victory rather than the score. It’s interesting to think of decisions in our daily lives that would benefit from shifting from margin-of-victory thinking to the victory itself. It’s not natural and being able to challenge human nature is a valuable skill.

AlphaGo documentary

Sat, 18 Apr 2020 00:00:00 +0000

Spoiler: If you haven’t seen the documentary and want to avoid any spoilers don’t read the post. At the same time you should watch it - it’s an incredible film.

I watched the AlphaGo documentary and it left a powerful impression on me.

The accomplishment by the AlphaGo team is incredible. At the same time I felt incredibly sad watching it and became slightly depressed. It felt very much like a man vs machine battle yet the machine was a huge team of engineers with nearly unlimited resources. You know that it’s inevitable that the machine wins and yet the way the documentary was done made it seem less of a highlight of the great work by the team and instead a defeat of a single person.

Google and Apple's COVID tracking project

Sat, 18 Apr 2020 00:00:00 +0000

Google and Apple recently announced a partnership to help people anonymously see if they’ve been exposed to someone with COVID. The idea itself is elegant and does a nice job balancing the functionality with the need for privacy. One of the most common takes I’ve seen in tech circles is that no one will use it due to a concern for privacy.

I think the opposite. I believe that with enough marketing this will be a massive success. Over and over again we’ve seen that most people do not value privacy and this will be no different. In fact, for this case it may just be the opposite - most people would love to know the answer to “is there a chance I have Coronavirus.” Curiosity, one of our greatest traits, will be enough to overcome whatever privacy concerns people have.

Optimizing code? Think theoretical limits

Sat, 18 Apr 2020 00:00:00 +0000

This post was inspired from a conversation with an engineer who was tasked with optimizing the performance of a heavily used static JavaScript script. This code gets loaded billions of times a day across a variety of devices and small improvements to its load time and performance can drive significant value to our customers and us.

When you start it’s easy to find the low hanging fruit and get the simple wins. But as you keep working on the same problem these wins become rarer and rarer and oftentimes a win in one area may be a loss in another. Using our static JavaScript code as an example we can argue that we should split it into multiple, smaller files and load them on demand. That would make the total amount of code loaded and executed lower but, on the other hand, would increase the number of network requests. And if you then layer in caching and the variety of devices it will run on it’s not at all obvious what the impact of a change will be.

Dumping Apple health data into MySQL

Sat, 11 Apr 2020 00:00:00 +0000

I apparently can’t get enough of Grafana and the latest quantified self push was to visualize the data from Apple health. Apple makes it pretty simple to export the data but it’s in XML so there’s a small bit of processing to turn into something that can be visualized. For my personal stats I’m dumping the data to MySQL and writing fairly simple queries to visualize them. Since I already did a similar export in my email-stats code I was able to reuse a fair amount. The major difference was that the Apple health export is fairly large (my export for 2020 was an 80 MB file) and it would be a shame to not apply a few optimizations.

Gamification works

Tue, 07 Apr 2020 00:00:00 +0000

Gamification works. Last week I hacked together a Grafana dashboard to measure the number of emails in my inbox and sure enough this gave me enough motivation to actually go through them. Earlier this week I added another metric to track the average age of an email and sure enough that caused me to go through the 4 and 5 year old emails.

Start open sourcing government software

Sun, 05 Apr 2020 00:00:00 +0000

Two years ago I wrote a post describing an open source model for developing government software. The driver then was improving the user experience of government services and making them more approachable but the news this week makes this much more than a nicer UX. The state of NJ is apparently looking for COBOL programmers in order to update the unemployment system to handle the unprecedented volume. It’s easy to make a government joke but there are real people that are unable to get the services they need. This particular problem will be solved but another one will pop up in its place.

Finding the truth in a sea of misinformation

Sat, 04 Apr 2020 00:00:00 +0000

Two days ago I read an interview between Ben Thompson and Zeynep Tufekci which highlighted the current state of information (and misinformation). That same day I came across an article by Will Remus on the cause of the toilet paper shortage (spoiler: it has to do with differences between the corporate and personal toilet paper supply chains). Both of these highlight how little I actually know and how valuable skeptical mindset is these days. I still don’t know the truth and it’s likely something that changes constantly but being able to separate fact from fiction is a massive advantage. The obvious example is knowing how serious COVID-19 would be and preparing for it in January. The challenges and opportunities will only get larger as the amount of information (majority of which will be misinformation) being produced increases. The ability to be objective, dig into the sources, and understand biases is going to be a critical skill in the upcoming century.

Visualizing my journey to Inbox Zero

Tue, 31 Mar 2020 00:00:00 +0000

I subscribe to the “Inbox Zero” philosophy and treat my email inbox as a todo list that I slowly work through. As part of the desire to get more and more quantitative I wrote a quick script to pull the number of emails from my Inbox and then insert the data as a row into a new table in my personal stats database. As usual, most of the work was in deciding to do it and once I got to coding the hacky solution was done within 20 minutes. The script uses Python’s built-in imaplib library to log in to an email provider and then a simple MySQL query to insert the resulting data. I hooked this up to run every 15 minutes via cron and put together a Grafana dashboard to plot the count over time. I’m currently not actually going through the content of the email messages themselves but there are tons of directions I can take this - for example slicing the data by sender or examining the age of the messages. For now I’m just hopeful this motivates me to keep going through that email.

A TikTok experience for streaming services

Sun, 29 Mar 2020 00:00:00 +0000

During an interview with Ben Thomson, Eugene Wei has the following response to a question digging into Netflix solving the “I just want something on in the background” use case:

Yeah, that’s a great point. I also do hope that this macro shock forces the streaming companies to think a little bit harder about innovating on their user interfaces. I still think it’s a little bit of a shame that most of them still present the grid of icons. Everybody has probably had that experience of just paralysis where you’re just scrolling around, you’re like, “What do I watch? I don’t know what to choose.”

Disabling TabNine to improve Ubuntu performance

Sun, 29 Mar 2020 00:00:00 +0000

At the end of 2018 I switched from using a Mac to using Ubuntu on a ThinkPad. While much of the development experience is better the one thing that’s noticeably worse is performance (usually first noticed in browsers) coupled with the occasional computer freeze. Back in the day when nearly everyone used Windows I remember the blue screens and the need to reboot frequently and I’m finding Ubuntu has developed the same pattern. I rarely had to reboot by Mac but find myself rebooting my current laptop at least once a week.

COVID-19

Sat, 28 Mar 2020 00:00:00 +0000

It’s taken me a few weeks to write this post. Partly due to lack of motivation, partly lack of time, and partly due to my fluctuating feelings towards COVID-19.

Some days I was completely disillusioned and sad about the modern state of the world. It seems to be spreading unconstrained despite the more than one hundred years of scientific progress we’ve had since 1918. Politicians seem to care about their own ego more than they do about the people they purport to represent. If we didn’t have the resources or knowledge to fight back I’d be more accepting of our fate but it’s the fact that we can fight and defeat this thing if we just channel our energies the right way that’s frustrating.

Zoom's blow up and their financials

Sun, 15 Mar 2020 00:00:00 +0000

Zoom has had an incredible rise in the past few weeks as the tech world has shifted to remote work with nearly all meetings occurring over video conferencing. The stock is up 30% over the past month and it’s the top app in Apple’s app store - all great things! What I’m curious about is their current financial performance. Zoom makes money by charging a flat fee per user per month. And that fee was set based on both the value delivered but also some assumptions around usage. What happens to their margin if customers go from using Zoom for a few hours a week to multiple hours in day?

Paying for improved productivity

Sun, 01 Mar 2020 00:00:00 +0000

Years ago I used to prioritize cost when it came to developer tools. I’d resist paying for anything if I could find an open source alternative that worked reasonably well. These days I prioritize my effectiveness and am much more likely to pay for the right tool. A great example of this is my search for a database IDE. I started using Sequel Pro (when I was on a mac) for MySQL and SQuirreL for everything else. Sequel Pro was great - it was smooth, responsive, and felt optimized for MySQL. SQuirreL, on the other hand, was designed to support nearly all databases which made it worse across all of them.

My personal Grafana dashboard

Thu, 27 Feb 2020 00:00:00 +0000

Last year I wrote about the idea of a personal dashboard and earlier this year I described my 2020 goals and how I’d go about measuring my progress. The past two days I was able to combine the two concepts and created a simple Grafana dashboard to measure my progress against the blogging goal. As with most tasks, the most difficult part was getting started and the actual exercise took a few hours. While it’s still fresh in my mind I want to document the step by step process in order to both provide a perspective into how I work while also giving others a guide to setting up their own.

Embracing smartphone productivity

Tue, 11 Feb 2020 00:00:00 +0000

Unlike a modern teenager, I tend to avoid using my smartphone for anything except consumption. I’d read emails but not respond, check up on Twitter but not tweet, look at my calendar but not schedule any events, and so forth. The excuse was that I’d be able to do these much more effectively in front of a real computer. Lately, I’ve realized that while the above is true I can still knock out a ton of small and simple tasks on my phone and save the meaty stuff for the computer. I’m not much more willing to respond to an email or send an invite on my phone rather than waiting to get on a computer. It also has the side benefit of reducing the amount of items on my mental todo list which will inevitably have loss.

Repository of configurations

Fri, 07 Feb 2020 00:00:00 +0000

While tuning Kafka yet another time I started thinking of how useful a repository of configuration files would be. Very often we install an open-source project and default to the standard options. This works great when you’re starting out but as you grow you realize the deployment is no longer cutting it. So you go back to the documentation and do a few searches to see how others have it tuned. This gives you a bit of breathing and lets you keep going until the next scaling bottleneck. Throughout you’re likely scaling up, out, or both and yet very few of us actually know whether we’re properly configured.

Micromanaging is underrated

Tue, 04 Feb 2020 00:00:00 +0000

A common belief is that micromanaging is a poor way to manage. I subscribe to this view - it’s demotivating to have someone second-guessing every step and eliminates agency and autonomy. I’d rather have someone make a few mistakes and learn along the way rather than impose a top-down view that disincentives growth.

While reading Bob Iger’s biography, I came across a simple statement from his former boss and Disney CEO, Michael Eisner: “Micromanaging is underrated.” That gave me pause. So often we hear that micromanaging is terrible and no effective manager does yet here’s someone incredibly successful leaning into it. Bob Iger distances himself away from this black and white view by discussing finding the balance while admitting that in many cases the details do, in fact, matter and it’s up to the CEO to make sure the output of the team is as polished and great as possible. The passage reminded me of Steve Jobs and his reputation for being incredibly detail-oriented and obsessive. In many ways, his approach was all about micromanaging and yet Apple made some of the most incredible and breathtaking products. Clearly there’s some value in micromanaging.

Finding parking spots using YOLO

Fri, 31 Jan 2020 00:00:00 +0000

I finally managed to make a bit more progress on the Parking Spot Finder project. Last time I tested the computer vision products offered by Google and AWS to see how well they were able to detect cars. This time around I decided to actually start working on the computer vision side and found a nice tutorial that allowed me to quickly try out the YOLO computer vision library. Surprisingly, it did significantly better than AWS but was more mixed against Google. On the original image it did better than both AWS and Google at detecting the cars but it did worse than Google on the cropped images. The next step is to actually try training a model and see how well it can do. Depending on how that goes I may end up changing my approach up and see how far I can get using heuristics.

2020 Goals

Tue, 28 Jan 2020 00:00:00 +0000

What better way of improving your chances of completing a task than publicly proclaiming you’re going to do it? That’s the idea for publicly posting my goals for 2020. I unfortunately skipped this exercise in 2019 but hope to correct it for 2020. It’s the start of a new decade after all and a good opportunity to get into better habits. Another lesson I’ve learned is that to hold myself truly accountable they should be measurable so I came up with a quantitative way of measuring each of them over the course of 2020.

Software comparison done right

Fri, 24 Jan 2020 00:00:00 +0000

There are a variety of sites out there that exist solely to do comparisons of enterprise software - just type “product A vs product B” in Google and the first few pages will be littered with results. They all generally follow the same format: a side by side table comparing them feature by feature. Some of the values are factual - for example pricing - while others are based on user reviews.

Instagram targeting ads based on likes

Tue, 21 Jan 2020 00:00:00 +0000

I noticed a new type of ad targeting behavior on Instagram I haven’t seen before - liking someone’s shot containing a product leads to a targeted ad containing that product. In my case, a friend posted a picture of an order from the Milk Bar and I gave it a friendly like. A few days later I started seeing ads for the Milk Bar Store. I know it may seem like a manifestation of the Baader-Meinhof phenomenon but I tend to notice ads and am pretty certain I have not seen any Milk Bar ad until I liked that image.

Visualizing my 2019

Fri, 17 Jan 2020 00:00:00 +0000

In order to better understand myself, I’ve been collecting daily stats over the past few years with the idea that tracking various metrics would show me ways to improve. This happened to some degree - seeing many of the numbers leads to a sense of shame - but there hasn’t been a huge insight that looks at the interaction of the various items I track. To do that, I suspect I need to dive deeper into the quantified self movement and start continually tracking my physiological metrics rather than the current approach of a daily check-in describing my mood and what I consumed.

Whatever happened to fast software?

Tue, 14 Jan 2020 00:00:00 +0000

Maybe it’s just me but it feels as if modern software has been getting slower and slower. My computer is the fastest it has ever been and yet I’m more frustrated by the actual performance more than ever. Websites seem slower than they’ve ever been and I often find myself typing faster than the letters appear on the screen. Part of the problem is that everything seems to be web-based; in fact, many “desktop” apps are powered by Electron which is a wrapper around Chromium that makes it easy to write desktop apps. Unfortunately, easy doesn’t mean fast and we get the performance of a browser tab in standalone tab.

Quick math calculations

Fri, 10 Jan 2020 00:00:00 +0000

I came across a tweet earlier today that received a surprisingly large reaction for a relatively simple math shortcut: “Percentages are reversible. 8% of 25 is the as 25% of 8 and one of them is much easier to do in your head.” I followed it up with one of my own: “Similar one is use factors of 10. Want to divide by 5? Multiply by 2 and shift decimal point. 24/5=24*2/10=4.8 same with 4 and 25., 8 and 125, etc”

Social network usage is exclusive

Tue, 07 Jan 2020 00:00:00 +0000

I remember the day when there were dozens of social networks with people hopping from one to another. These days it feels that there are only four: Facebook, Instagram (I’m explicitly separating it out from Facebook), Twitter, and Snapchat. This is a very US view and I realize there are tons of niche ones but at this point it feels as if those constitute the bulk of usage. More importantly, and this is pure speculation, it seems as if you pick a random person and look at their social network usage it’s very likely they’re spending the majority, if not all of their time, in only one of those networks.

Build for optionality

Fri, 03 Jan 2020 00:00:00 +0000

I wrote a company wide email back in September titled “Building for optionality” that describes how our engineering team maintains its impact despite the growing complexity of the business. There are a few TripleLift-specific concepts that I stripped away to make it more approachable but the theme is the same.

For a company that’s growing and changing as quickly as TripleLift, it’s critical to make sure our code is as impactful as possible. One way we do this is by building for optionality.

Advertising an Airbnb on Instagram