Imagine the scene: it’s just an ordinary day, until you’re called into the office of the CEO (!!!). There, you learn that your organization’s email no longer works. At all. No sending, no receiving. This is already a chilling thought, but once you start investigating, you discover that your company has landed up on not just one, but several, email blacklists.
For any sysadmin, this is a nightmarish scenario. But this was exactly what Greg, a young sysadmin working for a large charity organization, was facing. How did they land up on an email blacklist? To answer this question, we need to rewind a bit to another problem that Greg faced a few weeks earlier…
The first incident also began with a trip to the CEO’s office. The CEO informed Greg that the charity company had a received a “cease and desist” order from a movie company. The reason? The movie company had noticed the organization’s IP address was illegally downloading movies.
Greg hadn’t realized this was going on, but he vowed to put an end to it; he felt personally responsible for what was happening on the network.
His first intuition was, of course, BitTorrent.
“I figured it out pretty quickly, being somewhat familiar with the BitTorrent client,” he explains in the video below. He knew exactly which ports to look for, and when he looked at the network traffic for those ports, things became instantly clear. He was then also able to figure out which of their branch offices the traffic was coming from.
So Greg closed the ports to stop the traffic, and sent out a message to all employees that using the work network to download movies and other media was not permitted. Eventually he was able to put an end to the illegal activities.
He then also put network monitoring in place to watch for exactly this kind of activity in the future. Greg explains the story in this video:
Mission accomplished, right? Well, kind of. Greg had solved the illegal downloading issue, but unbeknownst to him at the time, a bigger issue was about to develop out of this initial problem.
Now we get back to Greg’s second visit to the CEO's office. When he discovered that email was no longer working for his organization, Greg started investigating. It soon became evident that the organization’s IP address had been placed on a blacklist. Antivirus software and mail filters use these blacklists to block mails, and so all mails being sent from Greg’s organization were either being blocked or landing up in the spam folder.
Greg contacted the relevant email blacklist, and they removed them from the blacklist. But after a few days, they were blacklisted once more – and this time they’d ended up on several other blacklists, too. Greg and the organization he worked for faced the reality that if this problem persisted, they might need to buy a new IP address…
After blocking Port 25 for SMTP to stop the problem getting worse, Greg could see all the outgoing requests from computers. And he realized several were attempting to send “nefarious” emails (as Greg called it). It was then apparent what the problem was: malware had entered the network and infected several workstations, which were constantly sending problem emails, which in turn landed the IP address they were being sent from on multiple blacklists.
And how did the malware get into the network in the first place? The answer: a BitTorrent download.
Once they understood the problem and could eradicate the malware, Greg was then able to get the IP address removed from the blacklists.
To prevent the issue in the future, Greg used Paessler PRTG to monitor the Exchange queues so that they would immediately be notified if mails were queuing up. He also used the DNS Blacklist sensor of PRTG, which would notify him immediately if their IP address landed up on a blacklist.
Watch Greg tell the story:
Have you had similar situations? We need your stories for future videos! Get in touch with us in the comments below, or email at us at prtg@email.paessler.com and we’ll get back to you.