{"id":5252,"date":"2026-03-13T14:25:20","date_gmt":"2026-03-13T14:25:20","guid":{"rendered":"https:\/\/influencerswiki.org\/blog\/5-years-of-forgotten-jobs-what-we-learned\/"},"modified":"2026-03-13T14:25:20","modified_gmt":"2026-03-13T14:25:20","slug":"5-years-of-forgotten-jobs-what-we-learned","status":"publish","type":"post","link":"https:\/\/influencerswiki.org\/blog\/5-years-of-forgotten-jobs-what-we-learned\/","title":{"rendered":"5 Years of Forgotten Jobs: What We Learned"},"content":{"rendered":"

At Buffer, we recently embarked on a small project to clean up how parts of our systems communicate behind the scenes.<\/p>\n

Some quick context: we use Amazon SQS (Simple Queue Service), which acts like a waiting room for tasks. One part of our system drops off a message, and another picks it up later. Think of it like leaving a note for a coworker: \u201cHey, when you get a chance, process this data.\u201d The system that sends the note doesn\u2019t have to wait around for a response.<\/p>\n

Our project was to perform routine maintenance: update the tools we use to test SQS locally and clean up their configuration.<\/p>\n

While we were mapping out what queues we actually use, we found something we didn\u2019t expect: seven different background processes (or cron jobs, which are scheduled tasks that run automatically) and workers that had been running silently for up to five years. All of them doing absolutely nothing useful.<\/p>\n

Here\u2019s why that matters, how we found them, and what we did about it.<\/p>\n

Why this matters more than you\u2019d think<\/h2>\n

Yes, running unnecessary infrastructure costs money. I did a quick calculation and for one of those workers, we would have paid ~$360-600 over 5 years. This is a modest amount in the grand scheme of our finances, but definitely pure waste for a process that does nothing.<\/p>\n

However, after going through this cleanup, I\u2019d argue the financial cost is actually the smallest part of the problem.<\/p>\n

Every time a new engineer joins the team and explores our systems, they encounter these mysterious processes. \u201cWhat does this worker do?\u201d becomes a question that eats up onboarding time and creates uncertainty. We\u2019ve all been there \u2013 staring at a piece of code, afraid to touch it because maybe it\u2019s doing something important.<\/p>\n

Even \u201cforgotten\u201d infrastructure occasionally needs attention. Security updates, deprecation notices, and other maintenance tasks can be a challenge when the processes are no longer used.<\/p>\n

How we found them<\/h2>\n

The process of finding these workers was actually quite straightforward. We used AWS CloudFormation to create a template that would automatically create the workers and their SQS queues. We then used AWS CloudWatch to monitor the workers and their queues.<\/p>\n

When we noticed that some of the workers weren\u2019t responding, we used AWS CloudWatch to detect the issue. We then used AWS CloudFormation to delete the workers and their queues.<\/p>\n

What we did about it<\/h2>\n

Once we had identified the workers and their queues, we decided to take a more proactive approach. We created a new AWS CloudFormation template that would automatically create the workers and their queues, but also include a small script that would periodically check the status of the workers and their queues.<\/p>\n

If the workers were not responding, the script would automatically delete them and their queues. We also created a new AWS Lambda function that would periodically check the status of the workers and their queues and update the CloudFormation template accordingly.<\/p>\n

We also created a new AWS CloudWatch metric that would track the number of workers and their queues. This metric would alert us if the number of workers or their queues was increasing or decreasing, and would help us to identify any potential issues.<\/p>\n

The result<\/h2>\n

After deleting the workers and their queues, we noticed a significant reduction in the number of workers and their queues. We also noticed a reduction in the number of onboarding time for new engineers, and a reduction in the uncertainty and confusion that came with encountering these mysterious processes.<\/p>\n

We also noticed a significant reduction in the amount of money that we were paying for these unnecessary infrastructure costs.<\/p>\n

The conclusion<\/h2>\n

In conclusion, finding and deleting these unnecessary workers and their queues was a great success. It helped us to reduce our unnecessary infrastructure costs, improve the onboarding process for new engineers, and reduce the uncertainty and confusion that comes with encountering mysterious processes.<\/p>\n

We hope that our experience can help other teams to identify and delete unnecessary infrastructure, and to improve the overall efficiency and effectiveness of their systems.<\/p>\n

FAQ<\/h2>\n

Here are some common questions that we received during the cleanup process:<\/p>\n