Good Bots, Bad Bots, and Humans. Can You Tell Who Is Who?

By Sean Martin

A bot is a software application that automates repetitive tasks via scripts over the Internet at a higher rate than would be possible for a human to perform manually. (Wikipedia)

 
  Nearly half of all traffic is driven by bots   Image source:  Distil Networks 2016 Bad Bot Report

Nearly half of all traffic is driven by bots
Image source: Distil Networks 2016 Bad Bot Report

 

Bots can do pretty much anything that can be scripted; some good for the business, others not so much. Keep in mind, like most tools we have available to us as humans, their use for good vs. bad is up to the user. Nonetheless, here are a few examples:

Good bots

1.     Chat bots for automated customer service

2.     Vulnerability assessment scanners to find exploitable weaknesses when used by internally-known penetration testers or consultants

3.     Marketing search engine optimization and performance analysis

 

Malicious, or otherwise dangerous, bots can

1.     Alter site statistics

2.     Exfiltrate sensitive business or customer information

3.     Automate the theft of funds from bank accounts

4.   Vulnerability assessment scanners to find exploitable weaknesses when used by malicious outsiders

 

Some bots are placed there by the admins of the sites and applications, knowing that they are in fact “good” bots that will help them automate some of their business processes.

 

  Bad bots get a foothold in a variety of industries   Image source:  Distil Networks 2016 Bad Bot Report

Bad bots get a foothold in a variety of industries
Image source: Distil Networks 2016 Bad Bot Report

In some instances, organizations may find bots are introduced to their environment unknowingly via the installation of applications that contain scripts or script-like functionality. These bots may not be intentionally ‘bad’ – but their impact on the network or the risk they bring to the business may not be completely understood if not deliberately assessed and deployed.

More troubling, however, are that some bots are built to be malicious and are installed unknowingly by external sources through a variety of means: malware, phishing, or even embedded in commercial software.

Having looked at these statistics for quite some time, a post from Distil Networks outlined an initial list of bad things bad bots can do to an organization. Below you will find an updated (partial) new list. As part of this list, you will also find the tell-tale signs that these types of bots may be on your network.


1. Scrape

 

What it does: Steals proprietary content and performs eCommerce price automation

How to spot it: You won’t know unless you look. Once you start investigating, tell-tale signs include:

  • Seeing your content elsewhere on the web; oftentimes on competitors’ sites

  • Noticing that your competition changes their prices based on your price changes


2. Skew

 

What it does: Potentially changes the way an organization plans and executes against their business plans (sales, marketing, and investments) by establishing a bogus reality based on a false bot-driven perception.

How to spot it: There are a few means with which to spot this bot on your network. For starters, your analytics reports could show unexplained spikes, signaling a bot attack like a bunch of credentials being attempted, or  even a vulnerability scan on your site. Another sign could be a sniping attack buying tickets from a bot from a country in Europe; so now your traffic looks like it originates from that country. Do you spend more money marketing and selling in that European country based on this information or is it skewed by the bots? Can you trust your business decision-making from the data you have?


3. Denial of Service

 

What it does: Brings systems and applications to a halt

Before you go to spot this one, take note of an important distinction: DDoS and Application DDoS are different beasts. With a DDoS attack, the website is flooded, preventing access to the services provided by the site. These are fairly easy to spot as the layer 3 attack sends packets upstream with such volume that they never arrive at the web server. In contrast, bot attacks against applications oftentimes fly under the radar and aren’t limited to volumetric-based attacks. Instead, bots programmatically abuse and misuse the website in ways that intensifies the applications’ services, thereby slowing down the application to a point where it becomes unusable. It won’t be noticeable via your firewall monitoring and the load balancer will look okay; the web app and the associated back end services keel over instead. Unlike a traditional DDoS attack that could require significant resources to employ, it doesn’t take much for an application DDoS attack to take hold.

Image Source: Distil Networks

How to spot it: The first two examples may seem obvious, but deeper investigations related to the applications and services running on the website could be a sign of a bot-driven denial of service attack targeting the applications:

  • Increased dissatisfied customer calls because the site is not working, or is too slow to use

  • Slowdowns of response times for the site and the applications/services

  • Excessive spikes on certain pages and increased service requests to access the shopping cart database, payment processing system, and fraud detection tools


4. Crack

 

What it does: Brute force guesses passwords by mapping known lists of usernames against (oftentimes stolen) password databases

How to spot it: An abnormal rate of failed login attempts could indicate stolen credentials are being tested on your site. Check your user directory and authentication logs.


5. Stuff

 

What it does: Uses known good pairs of usernames and passwords against a wide set of systems and applications

How to spot it: Similar to cracking, check your logs showing an abnormal number of attempted log-ins. Only, this time, look for a successful log-in that follows the series of failed log-ins. Then, look for fraud associated with those accounts.


6. Defraud

 

What it does: Digital ad fraud alters the statistics (and therefore the spend) associated with marketing ads; the number of impressions and the number of clicks. A few samples of digital ad fraud include:

  • Impression fraud: fake ad websites are used by bots to repeatedly load the pages which generates fake ad impressions

  • Click fraud: fake ad search websites are used by bots to get paid on expensive search terms

  • Retargeting: sends bots to legitimate ad websites for the purpose of creating a valuable cookie profile which is used to earn premium ad revenue

How to spot it: When a site is being defrauded, conversion rates typically go down. Google ad buys could hit the maximum each day, but earlier in the day than you would usually experience or desire. This is likely because a bot has clicked through the ads instead of real humans.


7. Assess

 

What it does: Runs a vulnerability assessment against the systems and applications to identify weaknesses that can be exploited (when used by the company, this could be a good bot – when used by a cybercriminal, it is obviously bad).

  Whitelisting your own vulnerability scanners is good practice.   Image Source: Distil Networks

Whitelisting your own vulnerability scanners is good practice.
Image Source: Distil Networks

How to spot it: If you see a vulnerability scanner on your site that you didn't authorize, that is usually an indication that a bot is collecting assessment data from your environment.


8. Footprint

 

What it does: Inventories and analyzes business applications to understand their logic, structure, algorithms, functions, methods, configurations, and other attributes that can be exploited.

  Image Source: Distil Networks

Image Source: Distil Networks

How to spot it: Similar to bot-driven assessments, bot-driven footprinting can be identified by spotting vulnerability assessment and network inventory scanning tools running on your network. It is difficult to differentiate Footprinting from Assessments (and Fingerprinting as well) except when you look at specifically what they are scanning. More traffic on services that don't normally see much traffic because they are being scanned could signal a bot. A spike in requests for pages or services that don't exist could also trigger a non-human interaction with your network and applications. Significant (abnormal) amounts of 'page not found’ deliveries could also signal that a bot is preparing a footprint of your environment. Look at your logs to identify what is not normal human traffic so you can make an informed decision to block it.


9. Fingerprint

 

What it does: Probes the network communications associated with systems and applications to identify traffic patterns, attributes and weaknesses that could be exploited.

  Image Source: Distil Networks

Image Source: Distil Networks

How to spot it: These bots typically access resources than are not visible on the site and that a normal user shouldn’t access - seeing this indicates something has gone awry. If the bot requests pages that don't exist it could generate a spike in ‘page not found’ deliveries. Differentiating Fingerprinting from Footprinting and Assessments can be difficult when looking within the traffic. For example, some scanners use automation to scan for particular things. If you can identify that it is not a browser doing these scans, you could deny it access to anything through the website.


10. Spam

 

What it does: Like email spam, but executes via the web to target your site’s visitors – bots essentially stuff web forms with garbage data, oftentimes malicious in nature (which could distribute malware/ransomware).

How to spot it: The most obvious signs that there is a spam bot running on your network include:

  • Spam will begin to appear in the reviews sections of your site

  • Links to other sites that take the visitor from your site to another site


Now That You Know: Take a Moment to Investigate

Based on research and other data, chances are you have a significant number of bots taking up bandwidth on your network. There’s also a good chance they are doing some not-so-good-things on your network. Since the probably is high, it’s worth taking a look at your logs and real-time network traffic to see what’s going on. You might be surprised with what you find.


About Sean Martin

Sean Martin is an information security veteran of nearly 25 years and a four-term CISSP. Sean is the co-founder and editor-in-chief at @ITSPmagazine and the president of imsmartin, an international business advisory firm.

More About Sean