Shape’s VP of Intelligence Center to Speak at Retail Cyber Intelligence Summit

Shape’s Vice President of Intelligence Center, Dan Woods, will present at the upcoming Retail Cyber Intelligence Summit on September 24-25, 2019, at the Four Seasons Hotel in Denver, Colorado.

2018 saw a significant increase in user credential spills from retailers. And as the retail industry continues to increase its digitization, it creates more incentives for attackers, as well as increases retailers’ potential attack surfaces. In fact, more than 50 percent of all e-commerce fraud losses were from cyber-attacks such as ATO, gift card cracking, and scalping. In addition, up to 99% of traffic on retail and e-commerce login forms was due to account takeover attempts! 

Dan’s session, titled “The Anatomy of Web and Mobile Application’s Costliest Attacks,” will discuss actual attacks launched against retail and hospitality organizations and explain attackers’ motivations and monetization schemes. Dan will also share the latest threat intelligence on effective attack tools and techniques that cybercriminals are using to circumvent traditional countermeasures with devastating effectiveness. 

“We’re looking forward to continuing our partnership with Shape Security and are pleased to have them as a presenting sponsor at our upcoming Retail Cyber Intelligence Summit in Denver,” said Suzie Squier, president of RH-ISAC.

The Retail Cyber Intelligence Summit is tailored for strategic leaders and cybersecurity practitioners from both physical and online retailers, gaming properties, grocers, hotels, restaurants, consumer product manufacturers and cybersecurity industry partners. The full conference agenda and information on how to register is available here.

The War No One is Talking About

There is a war brewing in cyberspace. The general public is blissfully unaware, and very likely will remain so. The media, when it talks about cybersecurity, tends to focus on the breach of the week, even though there cannot possibly be any lessons left to learn in that parade of spectacle and shame.

The war we speak of is against malicious automation (bots), and it’s being fought largely outside the gaze of journalism. On one side are the organizations putting their stores, intellectual property, processes, and businesses online in their journey toward digital transformation: the “good guys.” On the other side are malicious actors armed with nearly undetectable automation, intent on theft, political influence, fake news, and fake transactions: the “bad guys.”

Asymmetric Conflict

The comedy of this “automation war” is how lopsided it is, technologically. The bad guys have accumulated an impressive arsenal of tools from Sentry MBA, PhantomJS, and simple proxies, to browser extensions (Antidetect), human click farms, behavior collection farms, global proxy networks and, finally, to headless chrome steered with a real orchestration framework like Puppeteer.

Meanwhile, the good guys have only ancient traps like a CAPTCHA or a web application firewall (WAF), both of which are trivially easy for bad guys to bypass. Organizations aren’t thrilled about annoying their customers with friction (like making them click on blurry pictures of buses for 20 minutes) and endlessly rewriting WAF rules when attackers retool every week. It’s an unfair fight, and who has time for that, honestly.

The Silent War of Automation

The primary tactic of an automation attacker is to imitate a legitimate transaction. It doesn’t matter if the transaction has a very low probability of gain for the attackers, because they can multiply their gains by scaling the transactions into the millions at nearly no cost. Because they are blending in so perfectly, many victim organizations have no idea that it’s happening until they see an effect like fully booked inventory, credit card chargebacks, or a competitor who seems to know the price of every single munition with all possible discounts.

The media won’t write a story about how a competitor reverse-engineered an insurer’s policy premiums through the creation of a million slightly different fake profiles, or how an actor deluged a work-for-hire site with a million fake low-wage contractor profiles that represented their tiny firm in the Philippines, because it’s too complicated and there’s no one to shame. There’s no spectacle there.

So, the silent war goes on, with the bad guys getting better and better at imitation, and organizations in nearly every vertical experiencing bizarre side effects (“All our free passport interview slots have been booked and are being sold!”).

What Won’t Save The Day

Everyone’s been hoping that the silver bullet for the good guys was going to be AI. Surely the incredible volume of modern transactions can be used to train machine learning engines to differentiate real traffic from fake, right? The answer is no, it can’t. At best, today’s ML engines can spot not individual anomalies but patterns of suspicious activity. 

When a campaign is identified as being underway, human operators must step in and determine the intent of the campaign, because understanding is crucial in determining next steps. The mitigation can’t just be simple blocking, because that’s a signal which helps the attacker retool. 

Sometimes, the info-war tactics of misinformation and redirection are the solution for the day. Or evidence collection. You need tacticians. You need real people using automation to fight real people using automation.

CyberHub Summit

The war in cyberspace will be a main topic of discussion next week in Atlanta at the CyberHub Summit. Classy people there will be talking about meta issues like defending the region’s online financial services and de-risking the supply chain. A few of us from Shape Security will be there, and over some pints of the venue’s product, we can show you how we’re fighting the war against malicious automation.

If you can’t make it to the CyberHub Summit, please don’t hesitate to contact us at any of the channels listed under our logo, but otherwise we hope to see you in Atlanta next week!

Healthcare CAPTCHA: The Cure that’s Worse than the Disease

A healthcare insurer was forced to use a CAPTCHA. 70% of their aged patients could no longer refill their prescriptions. It was a complete disaster.

“This is not who we are,” muttered the CIO of one of the largest health insurance companies in the world as he looked over the report. 

The digital team had been forced to put up a CAPTCHA on the site’s login page, and this had driven a full 70 percent of the company’s older patients off of the website. Pharmacy orders were also down a shocking 70 percent, and the call center was swamped at 130 percent of call volume with site users unable to pass the difficult visual puzzles. It was a complete disaster.

Photo by Joshua Ness

The seeds of this catastrophe were planted quite innocently.

The global healthcare insurer had introduced an innovative Health Rewards program that was hailed as a bold gamification of wellness. The program rewarded patients with points for achieving preventive medical milestones, such as scheduling wellness checkups, screening for bone density, and getting flu shots. Patients even got points for volunteering or participating in nutrition classes, activities that were good for their social and mental health and community bonding. It was beautiful; this is how markets are supposed to work —personal rewards for conscientious behavior.

The reward points themselves had no cash value, but could be redeemed in the insurer’s online mall for gift cards from retailers like Amazon and Walmart—and those gift cards definitely did have cash value. These rewards proved a juicy target for gift-card crackers.

Credential Stuffing and Gift Card Cracking

Almost immediately, automation attackers began credential-stuffing the login page of the insurance company’s rewards program. Credential stuffing is the act of testing millions of previously breached username and password combinations against a website with the knowledge that some of the credentials will work there. Success rates for an individual credential-stuffing login are low; they vary between 0.1 percent and 2 percent depending on the client population. 


Attackers counter the low probability of any individual login succeeding by scaling their attempts into the millions via automation—scripted programs called “bots.” Modern bots look very much like human users to a target computer—telling them apart is one of the most difficult problems in modern computer science. A 1 percent success rate in a credential-stuffing attack is a reasonable statistical estimate; one million leaked credentials will yield 10,000 successful logins against a third party, leading to account takeovers by the attacker. Today there are over 5 billion leaked credentials on the market.

The attackers breached thousands of accounts at the healthcare insurer’s rewards program. They consolidated reward points and converted them into gift cards, from which they exfiltrated the real cash value. The insurer’s CIO and IT security team were actually not that worried about the losses incurred through gift-card fraud.

“We were much more anxious about the PII exposure than the fraud.”

Global Health Insurer CIO

The attackers appeared to be ignoring the Personally Identifiable Information (PII) associated with the cracked accounts in favor of getting the rewards points, but the exposure was alarming.

The security team turned to their Content Delivery Network (CDN) vendor for help. The CDN’s “bot management” solution put a CAPTCHA into the user login process in an attempt to stop the automation. 

And that’s when the wheels came off.

Human success rates for CAPTCHAs are already distressingly low—as low as 15 percent completion rates for some populations. Because computers have gotten so good at solving CAPTCHAs, the tests have gotten more and more difficult.

For elderly users, who are visually impaired more often than not, CAPTCHA success rates are even lower. In fact, one would be hard-pressed to devise a worse user experience than CAPTCHA for an aging population.

Immediately after the CDN put their CAPTCHA in place, login success rates plummeted. Seven out of ten elderly users could no longer log in to their accounts, access the rewards program, or renew their prescriptions online. 

Online pharmacy orders plunged by 70 percent.

Frustrated patients had to phone the health insurer’s call center to renew prescriptions.

Meanwhile the attackers easily bypassed the “bot management” solution through one of the many underground services that offer 1,000 solved CAPTCHAs for $1. Now they were the only ones earning rewards.

“This is not what we do.”

Global Insurer CIO

The CAPTCHA was far more damaging than the fraud it was supposed to stop. The cure was worse than the disease.

Can you make an introduction?

The CIO reached out to a C-level colleague of his at a top-3 North American bank. He explained the situation and said, “Hey, you guys are a bank, and you don’t use CAPTCHAs. How do you get away with that?”

His peer said, “We use Shape Security,” and he made an introduction.

Shape worked with the healthcare insurer’s CIO and his team to get our technology deployed. We went into monitoring mode first, to study attack traffic patterns. Because Shape came in behind their CDN solution, the monitoring period became an informal bake-off between the CDN’s bot management service and Shape’s.

Understanding Users and Risk

Web and Mobile Visitor BehaviorsRisk
Legitimate users with good behaviorStrong passwords
No password re-use
Low
Legitimate users with bad behaviorWeak passwords
Password re-use
Prey to phishing
High
Illegitimate users with ill intentAccount Takeover
Phishing
IP Theft
High

Even behind the CDN’s CAPTCHA, Shape was detecting large amounts of credential stuffing and gift-card cracking—sometimes up to two million attempts per day. While the attackers had been smart enough to “hide” their traffic spikes within the diurnal patterns associated with human logins, they were not otherwise trying to disguise their traffic.  Sometimes they connected through proxies, sometimes through a partner healthcare insurer, and even once through a financial aggregator.

Shape fought the attackers as they retooled, attempting to get around the Shape defenses. Within weeks, most of the attackers gave up, resulting in a 90% decrease in overall traffic.

The CIO was sufficiently impressed by Shape to completely displace the CDN for bot management at the healthcare insurer’s web property, and the CAPTCHAs were removed from two dozen entry points.

Shape then began working with the team to monitor the mobile property, because that is where attackers always retarget to after we block them on the web. After another month of monitoring the mobile traffic, Shape was able to show that the healthcare insurer’s mobile property could be further improved to remember legitimate users, and we cut their legitimate “forgot password” transactions in half. Shape also provided the insurer with a customized list of recommendations for information access and password protections policies.

Steady State Unlocked

Today the healthcare insurer’s website has zero CAPTCHAs in front of their pharmacy, the account profile, and their rewards program. The Shape mobile SDK is integrated with nearly all the mobile platforms that the insurer reports.

Attackers and aggregators continue to probe the insurer’s web and mobile properties. Shape sees them, and foils the attackers. The health insurer is notified of the aggregators, who are encouraged to use authorized API gateways.

Attackers continue to probe (unsuccessfully) today

The online pharmacy is accessible to all customers again. Call volumes have dropped to levels not seen since before the CAPTCHA crisis. Attackers and aggregators continue to probe the insurer’s web and mobile properties. Shape sees them, and foils the attackers. The health insurer is notified of the aggregators, who are encouraged to use authorized API gateways.

And, perhaps most importantly, the healthcare insurer is again free to focus on innovating new programs and rewarding customers for taking preventive steps for their medical and social wellness.

10 Questions to Ask a Bot-Mitigation Vendor

You figured out that you have a bot problem. Maybe you have a high account takeover (ATO) rate, or someone’s cracking all your gift cards, or scraping your site. You tried to handle it yourself with IP blacklists, geo-fencing, and dreaded CAPTCHAs, but it became an endless battle as the attacker retooled and retooled and you’re sick of it.

So now you’ve decided that you’re going to call in professionals to stop the problem, and get some of your time back. You’ve narrowed it down to a couple or three, and you’re going to get them in and ask them some questions. But what questions? Here are some good ones that can give you an idea if the vendor’s solution is a fit for your environment.

1. How does the vendor handle attacker retooling?

This is your most important question. When a countermeasure is put in place, persistent attackers will retool to get around it. Victims of credential stuffing say that fighting bot automation by themselves is like playing whack-a-mole. You are paying a service to play this game for you, so ask how they handle it, because attackers always retool.

Attackers always retool. How does the vendor respond?

2. Does the vendor dramatically increase user friction?

CAPTCHAs and 2FA dramatically increase user friction. Human failure rates for the former range from 15% to 50% (depending on the CAPTCHA), and lead to high cart-abandonment and decreased user satisfaction. Honestly, think carefully about vendors who rely on these countermeasures. Your goal should be to keep CAPTCHA off your site, not pay someone to annoy your users.

3. How does the service deal with false positives and false negatives?

A false positive for an anti-automation vendor is when they mark a real human as a bot. A false negative is when they mark a bot as human and let it through (this is by far the most common case, but sometimes the less important one). Bot mitigation will have some of both; be suspicious of any vendor who claims otherwise. But a vendor should be very responsive to the problem of false positives; that is, you should be able to contact them, complain, and have the false positive determination addressed.

Ideally, games of whack-a-mole should be 100% outsourced

4. When an attacker bypasses detection, how does the service adapt?

There will be advanced attackers who manage to bypass detection, becoming a false negative. When it happens, you may not know about it until you see the side effect (fraud, account takeovers, etc.). Then you’ll need to contact your vendor and work with them on how to remediate. How do they handle this process?

5. How does the vendor handle manual fraud (actual human farms)?

If your vendor is particularly adept at keeping out automation (bots), a very, very determined attacker will hire a manual fraud team to input credentials by hand in real browsers. Many services do not detect this (since technically, human farms are not bots) Can their service detect malicious intent from even real humans? Shape can.

How does the service handle manual fraud farms?

6. If one customer gets bypassed, how does the vendor protect that bypass from affecting all other customers?

Ideally, the vendor should have custom detection and mitigation policies for every customer. That way, if an attacker retools enough to get around the countermeasures at one site, they can’t automatically use that config to get into your site. Each customer should be insulated from a retool against a different customer.

7. If an attacker bypasses countermeasures, does the service still have visibility on attacks?

It is very common for a service to be blind after an attacker bypasses defenses. If the vendor mitigates on the data they use to detect, then when an attacker bypasses mitigation, you lose the ability to detect. For example, if they block on the IP, when the attacker bypasses the block (distributes globally) the vendor may lose visibility and doesn’t know how bad you are getting hammered.

An example of a system that is working correctly is when 10,000 logins come through and they all look okay initially because they have behavioral analytics within the proper range for humans. But later it is determined that all 10,000 had identical behaviors, which means the logins were automated. A good vendor will be able to detect this for you, even after the fact.

8. Is there a client-side or browser agent?

If yes, how large is the integration and how expensive is the execution? Does the user or administrator have to install custom endpoint software, or is it automatic? If there is no endpoint presence how does the vendor detect rooted devices on mobile and how does it detect attacks using latest web browsers on residential IPs?

For example, one of our competitors takes pride in having no endpoint presence – not even a browser-agent. A common customer of ours used both their solution and ours simultaneously and found that the competitor missed 95% more automation (ask for details and we can provide them).

9. Does the vendor rely on IP-Blacklisting or IP-Reputation?

Our own research shows that automation attackers re-use an IP address an average of only 2.2 times. Often they are only used once per day or per week! This makes IP-Blacklisting useless. There are over a hundred client signals besides the IP address; a good service will make better use of dozens of those rather than relying on crude IP blacklisting.

10. How quickly can the vendor make a change?

When the attacker retools to get around current countermeasures, how quickly will the vendor retool? Is it hours, or is it days? Does the vendor charge extra if there is a sophisticated persistent attacker?


There are other questions that are table stakes for any SaaS vendor. Things like deployment models (is there a cloud option) and cost model (clean traffic or charge by hour). And, of course, you should compare the service level agreement (SLA) of each vendor. But you were probably going to ask those questions anyway (right?).

Yes, this article is slightly biased, as Shape Security is the premiere automation mitigation service. But consider the hundreds of customers we’ve talked to who chose us; these are the questions they asked, and we hope that they help you, even if you end up choosing a different bot-mitigation vendor.

On The Launch of Shape Connect

The war against “fake” begins today, with the launch of Shape Connect.

Shape spent the last eight years building a machine-learning engine that has a single focus: to distinguish humans from robots on the Internet. The engine is constantly learning as it processes over a billion transactions every day from 25 percent of the consumer brands in the Fortune 500. It’s actually a billion-and-a-half on payday and National Donut Day (June 7, thank you, Dunkin’ Donuts).

We’ve made this incredible engine available to everyone and we call it Shape Connect. Connect is self-serve, takes minutes to set up, and is free for two fortnights (yes, GenZ, that’s the correct spelling).

Why is Connect so revolutionary? Distinguishing automation (bots) from humans is the most difficult, and most pressing, challenge on the Internet. Stopping fake traffic should be job #1 for any website that has value—yet, Facebook, Twitter, and Google all struggle with fake traffic. Shape Security can, and we’re practically giving the service away.

Solving Modern Problems

Okay, okay, so we built a computer that can identify other computers. How does this help you? Many businesses are being defrauded by bots and don’t even know it. They might know they have a problem of some kind but not understand that automation is the real threat vector.

Credential Stuffing Causes HUGE Business Losses  

Credential Stuffing: Shape didn’t invent it, but we DID name it. It’s where malicious actors  acquire login credentials belonging to blithely unaware Internet users, employ bots to pour billions of username/password combinations into millions of websites, then drain users’ accounts of money, credit-card numbers, email addresses, and other valuable stuff.

Website breaches resulting in gargantuan credential spills are common occurrences these days despite mighty efforts to boost privacy and security measures. A sophisticated criminal industry has sprung up that uses automation to access online accounts across the board, including social media, retail, banking, travel, and healthcare.

What credential stuffing looks like before Shape Connect stops it

Believe it or not, credential stuffing-related activity can make up more than half of a website’s traffic. It’s estimated that this kind of nefarious pursuit results in business losses of over $5 billion annually in North America alone.

Gift Card Cracking

Another super-annoying problem is the cracking of online gift-card programs. Most gift-card programs allow recipients to check the card balance online. Attackers create bot armies to check the balance of every possible gift-card number! When they find a gift-card number that has a positive balance, they use it to purchase re-sellable goods before the recipient can use the card. Isn’t that horrible? It costs retailers millions of dollars per year.

Business Logic Mischief

But it gets worse. Almost any site that has significant intellectual property in its business logic is either being attacked or is at risk. Consider the stalwart health-insurance company. Insurance websites allow you to get premium estimates based on your profile. Their rates are based on diligent research and proprietary actuarial tables accumulated over decades of experience. One of our customers found that a competitor was creating millions of fake profiles, each with a slight tweak to its age, income, and pre-existing condition to map out the insurer’s quote-rate tables. What took decades to create was being stolen by a competitor using bots. That’s not fair, is it?

Are You Dating a Robot?

One of the curious facts that emerged from the aftermath of the Ashley Madison breach in 2015 was that a significant number of the female profiles on the affair dating site were fake. They’d been created by bots to yield vehicles by which swindlers around the world could establish online relationships with men whom they would then defraud through a money transfer. While Ashley Madison is no longer with us, there are other, less controversial dating sites that still have the same problem. Shape helped one of them deal with fake-account creation, leading to a much lower probability of robot dating. (Sorry, robots, true love is for humans.)

Hotels and Airlines: Point Theft

Hotels and airlines have their own currencies in the form of loyalty program “points” or “miles.” These have long been a target for fraudsters who can take over thousands of accounts, merge all their points, and convert them into re-sellable goods. In many cases, attackers prefer going after points. Your average consumer will notice immediately if their bank account is drained, but may not quickly (or ever) notice that their points are gone. They might just assume the points had expired. Room rates and flight fares are another form of intellectual property, and aggregators scrape the sites constantly, pulling rate information for competitors, leading to overly low “look-to-book” rates.

Fight The War Against Fake

Those are just a few examples of automation as a threat vector for business. We could tell you about a million cases of sophisticated bots threatening every different type of business, but we hope you get the picture already.

So let’s get back to Shape Connect, what it is, and how it works.

How Shape Connect Works

Our fully cloud-based service stands staunchly between your site and the Internet, deflecting bots and protecting you credential stuffing, DDoS, account takeovers, gift card cracking, and all other malicious activity done at scale.

We’ve put together a couple of videos showing how Shape Connect works to protect your site. For those of you blessed with short attention spans, we have a 90-second, visually stimulating cartoony video (above).

If that piques your interest and you want the whole story, here’s a six-minute video that goes deeper into the workings of Shape Connect.

And if you’re a reader, we’ll break it down for you right here.

Without Shape Connect, there’s nothing between your website and the user’s browser. But what if it’s not a browser or a real user? Both real users and bots follow the same steps to get to your site.

  1. The client (user or bot) queries DNS.
  2. DNS returns the IP address of your website (or load balancer or cluster, or whatever).
  3. The browser or bot sends a request directly to your website.
  4. Your website returns the response.

With Shape Connect, there’s a layer of protection between your site and the user or bot.

  1. DNS returns a dedicated Shape Connect IP to the user or bot.
  2. All client requests are routed through Shape’s Secure CDN for fastest response.
  3. Shape Connect absorbs any DDoS attacks that the client might have sent.
  4. Shape Connect’s artificial intelligence determines if the request came from a real human using a real browser or from an automated bot. It passes only human requests through to your website.
  5. Your website responds only to legitimate requests, sending the data back through Shape Connect and to the human at the other side.

Of course, if you have “trusted bots” that you want to allow, you can manage your own whitelists.  

With the Shape Connect Dashboard, you can see all the requests that have come through, and marvel at all the automated malicious requests that Shape blocked!

Your Honor, I Object!

The rest of the industry is catching on to the bot problem, and some are pushing approaches that differ from Shape Connect.

What about WAF?

One of those alternative solutions is so-called “bot management” integrated into a Web Application Firewall (WAF). We’re seeing many WAF vendors trying this, but failing. Here’s a long treatise that explains why we think WAF is a suboptimal approach.

What about PCI?

With Shape Connect, you can drive away all unwanted automation and still be PCI compliant. We’ve got more details for you in this informative and colorful brochure.

Connect with Shape Connect

To celebrate the official launch of Shape Connect, we were going to throw ourselves a gigantic poolside party, with mumble rappers from LA and rivers of Henny.  But we decided, instead, that it would be more fun to watch all the new customers come in and bask in the delight they experience as they get connected.

Shape Connect is live right now, and if you’re comfortable and confident, you can sign up for a free trial. But we’re also here if you want to chat first about how Shape Connect can secure your business, reduce your latency, keep your servers afloat, and improve your customer experience journey. Talk with you soon!

Do You Need a WAF, or Something Better than a WAF?

The King is Dead, Long Live the King, by Bart Hoffstein (cayusa), license Creative Commons 2

“The king is dead! Long live the king!” The jarring conflict embodied in this timeless hoorah is about to apply to the application security space. Subjects are giving up on the old king—the web application firewall (WAF) technology—as their primary appsec tool, for several reasons. First, because WAFs are too complicated. Second, because attackers have changed their attack vector to target credentials at scale (credential stuffing) before hacking. Third, and most important, because the market has evolved to offer an approach superior to WAF in efficacy, value, and worker hours invested.

While we at Shape Security have been predicting the shift away from WAF for years, others have been taking note. The PCI DSS specification had previously mandated a WAF, and that drove WAF sales for a decade. However, the language of PCI DSS has changed in 6.6, and other solutions can be used to fulfill the requirement.

The new approach is a distributed, cloud-based, machine-learning Turing service backed by anti-automation specialist operators. Let’s call it “anti-automation” for short until a clever analyst comes up with a better name.

2018 had 1500+ critical CVEs

WAFs are Too Complicated

Consider the statement that WAFs are too complicated. In our experience working with customers over the last decade, we’ve rarely, if ever, seen complete WAF protection cover even a tenth of critical applications. Frequently, the WAF has just a single dedicated (and expensive) administrator, and the ruleset for the WAF must be updated under the following conditions:

  1. When attackers evolve an attack to get around existing signatures.
  2. When content has been added (which is constantly in today’s agile web paradigm).
  3. When a web vulnerability is detected in the application or any supporting infrastructure (2018 had over 1500 critical CVEs — six for every working day).

These factors, which are all external to the WAF, quickly overwhelm the administrator and end up protecting only a handful of applications (or a single application). And usually not well.

Credential Stuffing and Retooling are the New Threat Vectors

Even if WAFs had done their job properly, it wouldn’t really matter because attackers have radically changed their approach. Gone are the days of attackers manually hacking websites. Today, they focus first on taking over the accounts of legitimate users. From there, they perpetrate their blight or escalate their privilege.

Today it’s all about credential stuffing. Attackers test millions of breached credentials using automated tools like Sentry MBA, PhantomJS, or automated headless browsers to gain their initial beachhead. Between 0.2% and 3% of credential-stuffing attempts are successful—a piteously low rate, which is why attackers try millions of credentials at a time. Even a 0.5% success rate using one million breached credentials will yield 5,000 accounts.

WAF technology was designed to stop SQL injections, not credential stuffing. An on-premises WAF managed by a single or part-time resource has no hope of defeating sophisticated credential-stuffing campaigns.

When a defender concocts a rule to stop a credential-stuffing campaign, the attacker pauses, retools to get around it, and then resumes the campaign. We at Shape see this all day, every day, with up to ten different levels of retooling. No single resource can keep up with that degree of sophistication, and the world is coming around to admit the problem.

The New Paradigm for Application Protection

If we all admit that the WAF is too long in the tooth, and that attackers have changed their approach anyway, the obvious question is: What is the right approach?

There are only a handful of highly skilled specialists with the right combination of technologies to consistently defeat and deter attacker automation. The key technologies of the best approach are:

  1. Artificial Intelligence (AI). Each attacker is launching millions of login tests, from millions of different IP addresses around the world. Only an AI-assisted SOC can see through the tidal wave and pick out real users.
  2. Expert-Assisted Mitigation. As useful as AI is, no vendor has machine-learning models that can detect and block all automation without also blocking real users (false positives). AI must be used to detect and flag campaigns to real human operators who make the final determination and remediation.
  3. Collective Defense. Most attackers launch credential-stuffing campaigns against multiple defenders in a serial fashion. The right approach must include defending a plurality of targets in each vertical market, so attacks seen against one company can be used to inoculate all the other companies before the attack can get to them.

Shape Security pioneered all these technologies for the Fortune 500 and Global 2000, and we’re now bringing them to everyone else to take the burden off WAF admins.

Looking Beyond the WAF

The OWASP Top Ten is the Open Web Application Security Project’s top-ten application security risk list. The legacy WAF technology was the only tool specifically designed to speak to the OWASP Top Ten, but at the end of the day, it was poorly suited to solve the list’s issues. Table 1 shows a breakdown of how well a WAF executes against an anti-automation service like Shape for each entry of the Top Ten.

RankOWASP RiskWAF AbilityAnti-Automation
1Injection******
2Broken Authentication****
3Sensitive Data Exposure***
4XML External EntitiesN/AN/A
5Broken Access Control***
6Security Misconfiguration****
7Cross-site Scripting***
8Insecure Deserialization*N/A
9Known Vulnerabilities*****
10Logging and Monitoring****

Let’s dive a little deeper into some of the Top Ten.

#1: Injection, #3: Sensitive Data Exposure

One could argue that the number-one job of a WAF is to prevent SQL injection. Modern organizations have learned to use identity as perimeter to keep unauthenticated users from causing any kind of SQL query, and that in itself is a commendable first line of defense. To get around the perimeter, attackers must gain control of an account. To do that, they use credential stuffing or brute force, both techniques that are much better blocked by an anti-automation service than a WAF.

#2: Broken Authentication, #5: Broken Access Control

Authentication systems are difficult to perfect. When they fail, they increase risk disproportionately to other systems, which is why OWASP keeps them high on their list. With sufficient tweaking, a properly configured WAF can assist broken authentication or access control system. But wouldn’t the knowledge to create the necessary defensive WAF configs be better utilized fixing the original misconfigurations? The anti-automation service simply detects that systems probing for these vulnerabilities are not human, and blocks them—which is a much simpler and broader approach than trying to make sure every knob is at the right level.

#10: Logging and Monitoring

Insufficient logging and monitoring of the application weaken   incident response. WAFs can help by flagging attacks before other systems do, but an anti-automation service comes with its own highly trained, specialized SOC. There is no contest here.

Conclusion

The final defense for WAF apologists used to lie in the PCI DSS WAF requirement, but even those have been relaxed to allow for a more flexible solution, and that’s a good thing. Shape Security has additional documentation on how cloud-based services can meet the requirement here.

Given all these factors—the deprecation of PCI DSS, the decreasing emphasis on WAF (and its magic quadrant), the evolution of credential stuffing, and the strategy of identity as perimeter—the market has been casting about for a new solution. Shape’s distributed anti-automation service, fronted by machine learning and backed by specialist operators, is rising to meet the challenge.

What Your Login Success Rate Says About Your Credential Stuffing Threat

One of the problems with imitation attacks such as sophisticated credential stuffing is that they are designed to blend in with legitimate traffic. How can you measure something that you can’t detect? Fear-mongering marketing compounds this problem and makes everything sound like a snake-oil solution for a problem people don’t think they have.

Imitation attacks against your services and APIs leverage inherent functionality in your system. In other words, they can be successful even when you have patched, firewalled, and done everything perfectly from an application security standpoint. Blocking basic credential stuffing attacks generated from naive bots is straightforward but, as attackers evolve, they develop more sophisticated tools to launch attacks that blend in with legitimate traffic better. They use machine learning to emulate user behavior like mouse movements and keystrokes. They generate or harvest digital fingerprints to distribute across a botnet to make each node appear more “real.” They proxy requests through residential IP addresses to give the traffic the appearance of originating from known-good home networks. Googles themselves make tools like Puppeteer & headless Chrome to automate and script the world’s most common browser which is exactly what you would use if you were trying to blend in with legitimate users. This is a problem that is getting harder, not easier.

Imitation attacks like advanced credential stuffing do have one thing in common, though – they send millions of requests hoping that a fraction of a percentage end up successful and result in an account takeover, a valid credit card, an account number with a loyalty balance, anything. This success/failure ratio is observable with data you have now. What we’ve found at Shape, is that similar companies have similar success ratios for similar user experience flows.

If you’re debating if you have a credential stuffing problem, then take a long look at your login success ratio.

What is your average login success ratio?

The average login success ratio drops dramatically during periods of credential stuffing attacks. These attacks use combolists with millions of usernames and passwords and of course the majority of these credentials aren’t valid on your site. Shape sees credential stuffing success rates between .2 and 2%, typically – attackers don’t need a very high success rate as long as the attack is cheap to perform. These attacks push the login success rate for your site down well below normal numbers. Some Shape customers have seen login success ratios lower than 5% before enabling countermeasures. Success ratios that low are abnormal and should be immediately investigated. Below are average login success ratios for one month of traffic across three major industries:

  • Financial institutions: 79%
  • Travel industry: 73%
  • Retailers: 62%

Individual companies deviate from this average as much as 10% – the sites where customers log in more frequently tend to have a higher login success ratio. The users at these sites are more likely to remember their passwords and are also more likely to have stored their credentials in their devices or web browsers. Banks and financial institutions only keep users logged in for 15 minutes leading to more successful logins than retailers or social media sites that keep users logged in for longer periods of time. This results in much higher login success rates for banks than for retailers.

Users also have access to few bank accounts and do not change them often, as a result they are more likely to remember their login credentials. Users however regularly shop at multiple retailers and it is easy to create a retail account. This results in lower login success rates for such sites, reflecting a higher rate of users who may be visiting for the first time in months or even years. Infrequent visitors naturally forget their passwords more regularly.

Companies should expect to see 60-85% login success rates. Anything higher or lower is suspect.

No matter the industry, companies should expect to see 60-85% login success rates. Anything higher or lower is suspect. Spikes in traffic can temporarily affect the login success ratio but those should be explainable by commonly understood events like promotions or viral marketing. If there are spikes that have nothing in common then you should look deeper, that traffic is probably a credential stuffing attack that you need to stop as soon as possible.

Graph of a customer who experienced a credential stuffing attack during a steady state of normal login successes.

One caveat

Some industries like banks and other financial institutions are frequently targets for aggregators, services like Mint and Plaid that act as delegates with user permission to log in and gather data across many companies and present it in one unified interface. Aggregators use legitimate credentials and log in multiple times a day, unnaturally inflating the login success rate. You can look for evidence of aggregators by querying for successful logins across multiple users from the same IP addresses, especially if the IP addresses are from cloud or hosting providers. This is not a foolproof method of detection but you will see traces that will help you get a better understanding of your true login success ratio. If you see login success rates in the high 80s or 90s, that is abnormally high and indicative of low credential stuffing threat but high aggregator traffic. Whether or not to consider aggregators a threat is different for every business.

Where to go from here?

What do you do if you find a login success ratio that is concerning? Like with any threat, you need visibility into the attack before you can think about mitigation. Start with free options before committing to a vendor. Tying yourself up with a vendor too early can spin you in the wrong direction and end up wasting months of time. I’ve written an article on 10 things you can do to stop credential stuffing attacks which goes over some free detection methods as well as some mitigation strategies. This should be enough to get you started understanding your problem and, once you understand the scope of your issue, then you can have better conversations with security vendors. Of course we at Shape Security are available to answer questions any time of day and you can feel free to reach out to me personally on twitter.