Are You Protecting Your Data From Bots

Published: June 19, 2023

In today’s world, bots are becoming increasingly common.  In fact, in 2022 bots made up more than 40% of the internet’s traffic and digital ad fraud, of which bots are a key component, is projected to increase in cost from $35 billion USD in 2018 to $100 billion in 2023.

Bots are software applications programmed to perform certain tasks autonomously.  They perform a wide variety of tasks.  Some even have good intentions such as giving visitors of websites a more personalized experience or monitoring websites to make sure they are working as expected.

However, not all bots are benevolent, some can cause significant damage to an organization’s online infrastructure by performing tasks such as overwhelming servers with requests, scraping sensitive data, or hijacking user accounts.  They can negatively impact an organization’s reputation and trustworthiness among customers and partners.  If, for example, a bot spreads spam or malware on a website, users may lose confidence in the website’s security and stop using it.  Bots can also harm an organization’s bottom line by stealing intellectual property, disrupting business operations, or manipulating advertising campaigns.

In the real world, the impact of malicious bots can be significant.  In 2016 for example, the Mirai botnet launched massive DDoS attacks that disrupted major websites and Internet services across most of the east coast of the US and much of the software in use at that time is still in use today.  In 2020, a bot network was discovered by Comparitech researchers which was using Facebook to run and post from accounts posing as legitimate users with the intent to manipulate political opinions.  Just from these two incidents alone we can see how bots can cause serious harm and highlight the importance of protecting against them.

There are many ways to protect against bot attacks.  One technique is to use machine learning to detect unusual patterns of user activity.  For example, if your users primarily interact with your service during the week but there’s a sudden spike in activity at midnight on a Saturday, that could be a sign of a bot DDoS attack. To protect your data, one option may be to add a rate limiter in front of your services as discussed in one of our previous articles.

Another layer of protection could be the use of CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) to prevent bots from performing certain actions, such as creating accounts or sending messages. Additionally, CAPTCHA can be used to help train machine learning models to better detect bot activity.

Bot detection software can also help prevent attacks by identifying and blocking suspicious traffic from known bad sources. One such example is the bot detection plugin available for the Kong Gateway. The rules.lua file (which is part of the plugin) defines a list of multiple known bots. While this is not an exhaustive list and new ones may emerge that are not caught by this plugin, the code is open-sourced and customizable, which means that it is possible to pick and choose which bots you permit to access your environment.

It is clear that bots are an ever-present threat to an organization’s online infrastructure. While some bots may have a useful purpose, malicious bots can cause significant damage to an organization. Protecting against bot attacks is essential, and there are many techniques available to help do so. Machine learning, rate limiting, CAPTCHA and bot detection software are just a few examples. By being proactive in taking steps such as the ones discussed here, you can help safeguard your organization from a number of potential incidents.