Rules-based systems use ‘if-then’ rules to derive actions. For example, a fact that ‘Sally is 22 and unemployed’ is matched to a rule that ‘If a person is between 18 and 65 and is unemployed, then that person can claim unemployment,’ and concludes that therefore ‘Sally can claim unemployment.’

One advantage of these systems is that they are relatively easy to understand and can be built to represent expert judgment on simple or complicated subjects. Another is that cause-and-effect is transparent. Even though the ‘if-then’ reasoning can become complex, a domain expert can verify the rule base and make adjustments if necessary.

That said, there are three critical weaknesses to rules-based systems — and these overwhelm the advantages. First, the rules engines do not scale. They must logically become nearly as complicated as the problem the system is trying to solve. Rules must be added; they are not learned (as they are in machine learning). In practice, a ‘rules explosion’ occurs where a series of seemingly simple rules becomes a complex net of conflicting rules, overlapping rules, etc. They thus become very hard to understand in the aggregate. The more knowledge a user adds, by adding more rules, the more complex and opaque the system becomes.

Second, these systems don’t handle incomplete or incorrect information very well — data that does not have an associated rule will be ignored. This means that rules-based systems are particularly bad at detecting ‘unknown unknowns’ like new derivations of malware or new disease epidemics.

Third, rules-based systems don’t know what to do with variables that have an infinite number of possible values. There are many of these variables — for example a person’s weight, or the time it takes to complete a task, or the price of a gallon of gas. Arbitrarily making these continuous variables into discrete variables may mean missing patterns or deriving false patterns. For example, a rule flagging someone who is more than 90 days’ delinquent on debt payments may miss someone else who is chronically delinquent on debt in the last five years, but never more than 89 days late for any one account. Who is riskier?

Haystax does not use a rules-based system as the foundation of its risk analytics models. Nonetheless, we use rules in two ways:

• First, in all models there are hard rules (e.g., ‘you must be a US citizen if you want a clearance’), which can be implemented as policy or compliance restrictions regardless of the amount of risk they may indicate.
• Second, we use rules after the analytics have occurred to sort or highlight information. For example, we use rules to dictate what we’d like to display on our dashboards or in our reports.

In general, the Haystax approach is to use rules to reason on processed data, not as a way to process data.

Note: This is the fourth article of a five-part series. Previous articles were:

• Part 1: Three Security Analytics Approaches That (Mostly) Don’t Work
  
• Part 2: Making Bayesian Networks Accessible
  
• Part 3: Machine Learning vs. Model-First Approaches to Analytics

Tom Read is Director of Insider Threat Programs at Haystax Technology.