*Cheap*/lightweight IDS algorithms/strategies/policies

Hi,

I have a modest sized distributed system (~100 processors) that
I am trying to protect against intentional acts of sabotage, theft,
etc.

[The current implementation is *wired* though that will probably
change in the future to a hybrid wired/wireless approach.  I'll
deal with those issues, later]

I am not concerned with intrusion *prevention*.  The stacks and
protocols are hardened and I am pretty confident an attacker would
just waste MIPS and bandwidth (mine and his) trying any sort of attack.
Even the "public" interface operates in stealth mode -- if you don't
*know* it's there, you'll never stumble across it!  (i.e., the i/f
is very difficult to fingerprint)

Instead, I'm interested in *cheap* algorithms to detect intrusion
*attempts*.  E.g., port scans, etc.

I've been working with a couple of "security types" and am pretty well
versed in how they (large corporations) try to detect and fend off
intrusion attempts.  But, they have a lot larger attack surface to
defend (and, lots more resources with which to defend it!)

Users, here, will range from "John Q Public" (i.e., tech-illiterate)
to small corporations with "traditional" IT departments (i.e., they
may be aware of the sorts of attacks possible but probably not
skilled in handling them first-hand).

As such, it is imperative not to worry the user with false positives
as that increases their level of anxiety -- needlessly so.

[Recall, the system is hardened so there is no risk of these attacks
getting through.]

I can easily and reliably identify obvious unauthorized attempts at
accessing services or protocol spoofing.  Since these can only be
caused by *bugs* (i.e., a legitimate host accidentally mangling it's
MAC/source-IP/dest-ip/etc. in a transaction) or *attacks* (i.e.,
deliberately masquerading as someone you are not), any such event
is grounds for an alarm (bugs don't exist; attacks SHOULDN'T exist!)

The system is physically secured -- though there are ports by which
an "invited intruder" could attempt to gain physical access to the
network (e.g., sit down at an unused desk in a back office and
plug in a rogue laptop, etc.).  I've taken steps to prevent such
access...

The biggest problem is handling "sporadic" bogus contacts that may or
may not be hostile (keeping in mind the "false positive" constraint
above).

E.g., an incoming *legitimate* ping(1) shouldn't ring alarm bells.
Nor should a blind attempt to contact an HTTPd(8) which may or may not
be running at a particular IP.  (e.g., google routinely tries to
do this when you make public posts)

OTOH, a ping flood or sequential port scan should probably raise
some eyebrows.  And, attempts at exploiting "common vulnerabilities"
(even if I am immune to them) would be indicative of hostile intent!

I concede that there is nothing I can do to prevent "bandwidth theft"
and the DoS that accompanies that.  But, services and functionality that
do not rely on that "external" bandwidth should remain unaffected
even in the presence of a full-scale attack.

Lastly, what do I *tell* the user when I am convinced that an
attack *or* a potentially hostile probe is underway?  What remedies
would a user actually have available to himself other than to
simply "ride it out"?  (remember:  system is hardened so no damage
comes from the probe/attack -- though connectivity may be compromised)
Given the classes of users I've described, about all they *might*
be able to do is alert their upstream provider and hope that provider
can accommodate them by blocking the attacker(s) at it's upstream
router(s).

As usual (for me :> ) lots of words but, hopefully, a clear message.
I figure this sort of problem will only become more and more common
in the systems that we deploy so getting ahead of the curve might be
prudent!

Apologies if I don't reply promptly.  I'm not "on-line" often so
don't get around to checking email, USENET/forum posts, etc.  :-/

Thx,
--don

On Thu, 27 Jun 2013 15:14:35 -0700, Don Y wrote:

> Hi,
> 
> I have a modest sized distributed system (~100 processors) that I am
> trying to protect against intentional acts of sabotage, theft, etc.
> 
> [The current implementation is *wired* though that will probably change
> in the future to a hybrid wired/wireless approach.  I'll deal with those
> issues, later]
> 
> I am not concerned with intrusion *prevention*.  The stacks and
> protocols are hardened and I am pretty confident an attacker would just
> waste MIPS and bandwidth (mine and his) trying any sort of attack. Even
> the "public" interface operates in stealth mode -- if you don't *know*
> it's there, you'll never stumble across it!  (i.e., the i/f is very
> difficult to fingerprint)

Depends on the value of the target.  If the attacker thinks (or
knows) there could be something valuable there, they will spend
more time than you think.  Even if they cannot get into the box
if they can prevent meaningful work being done by the system (the
~100 processors) then I smell ransom ware.  Do you trust that
cleaning person not to tell one of his friends what he as seen
on some display.

> 
> Instead, I'm interested in *cheap* algorithms to detect intrusion
> *attempts*.  E.g., port scans, etc.
> 
> I've been working with a couple of "security types" and am pretty well
> versed in how they (large corporations) try to detect and fend off
> intrusion attempts.  But, they have a lot larger attack surface to
> defend (and, lots more resources with which to defend it!)
> 
> Users, here, will range from "John Q Public" (i.e., tech-illiterate) to
> small corporations with "traditional" IT departments (i.e., they may be
> aware of the sorts of attacks possible but probably not skilled in
> handling them first-hand).
> 
> As such, it is imperative not to worry the user with false positives as
> that increases their level of anxiety -- needlessly so.
> 
> [Recall, the system is hardened so there is no risk of these attacks
> getting through.]

Famous last words.  Hopefully you have gone through a complete security
audit on the source code with "outside" eyes.

> 
> I can easily and reliably identify obvious unauthorized attempts at
> accessing services or protocol spoofing.  Since these can only be caused
> by *bugs* (i.e., a legitimate host accidentally mangling it's
> MAC/source-IP/dest-ip/etc. in a transaction) or *attacks* (i.e.,
> deliberately masquerading as someone you are not), any such event is
> grounds for an alarm (bugs don't exist; attacks SHOULDN'T exist!)
> 
> The system is physically secured -- though there are ports by which an
> "invited intruder" could attempt to gain physical access to the network
> (e.g., sit down at an unused desk in a back office and plug in a rogue
> laptop, etc.).  I've taken steps to prevent such access...

This may or many not be true depending on the value of the target. A
recent study put a usb drive in the company parking lot.  60% of
the people that picked it up plugged it into their work computer.  If
I put a company logo on it that number will jump up to 90%.  If you 
have a valuable target someone will look for a way to get at it
from the inside.

> 
> The biggest problem is handling "sporadic" bogus contacts that may or
> may not be hostile (keeping in mind the "false positive" constraint
> above).
> 
> E.g., an incoming *legitimate* ping(1) shouldn't ring alarm bells. Nor
> should a blind attempt to contact an HTTPd(8) which may or may not be
> running at a particular IP.  (e.g., google routinely tries to do this
> when you make public posts)
> 
> OTOH, a ping flood or sequential port scan should probably raise some
> eyebrows.  And, attempts at exploiting "common vulnerabilities" (even if
> I am immune to them) would be indicative of hostile intent!

If the target seems valuable the bad guy wont do a sequential scan.
They can take several days probing it.  Unless you are capturing
the events and correlating you probably wont see this.

> 
> I concede that there is nothing I can do to prevent "bandwidth theft"
> and the DoS that accompanies that.  But, services and functionality that
> do not rely on that "external" bandwidth should remain unaffected even
> in the presence of a full-scale attack.

If this is an embedded low performance device you need a way to shed
the processing time spend dealing with the attack packets.  This would
mean 2 different network interfaces if the "private" side must stay
responsive while the public side is under attack.  With cloud
computing I can get a lot of horsepower and a lot of bandwidth
for not much money.

> 
> Lastly, what do I *tell* the user when I am convinced that an attack
> *or* a potentially hostile probe is underway?  What remedies would a
> user actually have available to himself other than to simply "ride it
> out"?  (remember:  system is hardened so no damage comes from the
> probe/attack -- though connectivity may be compromised) Given the
> classes of users I've described, about all they *might* be able to do is
> alert their upstream provider and hope that provider can accommodate
> them by blocking the attacker(s) at it's upstream router(s).
> 
> As usual (for me :> ) lots of words but, hopefully, a clear message. I
> figure this sort of problem will only become more and more common in the
> systems that we deploy so getting ahead of the curve might be prudent!
> 
> Apologies if I don't reply promptly.  I'm not "on-line" often so don't
> get around to checking email, USENET/forum posts, etc.  :-/
> 
> Thx,
> --don

Noticed you have been a little quiet lately....
Hope all is going well and you have more work that you can handle.

-- 
Chisolm
Republic of Texas

Hi Joe,

[Check your mail]

On 6/27/2013 8:05 PM, Joe Chisolm wrote:
> On Thu, 27 Jun 2013 15:14:35 -0700, Don Y wrote:
>
>> I have a modest sized distributed system (~100 processors) that I am
>> trying to protect against intentional acts of sabotage, theft, etc.

>> I am not concerned with intrusion *prevention*.  The stacks and
>> protocols are hardened and I am pretty confident an attacker would just
>> waste MIPS and bandwidth (mine and his) trying any sort of attack. Even
>> the "public" interface operates in stealth mode -- if you don't *know*
>> it's there, you'll never stumble across it!  (i.e., the i/f is very
>> difficult to fingerprint)
>
> Depends on the value of the target.  If the attacker thinks (or
> knows) there could be something valuable there, they will spend
> more time than you think.

Yes -- especially as much of the initial probe can be automated, etc.
A big reason for making the interface hard to fingerprint is to
frustrate "canned" attempts at exploits.  Of course, going too far
in this direction can be a fingerprint in itself!  :>

> Even if they cannot get into the box
> if they can prevent meaningful work being done by the system (the
> ~100 processors) then I smell ransom ware.  Do you trust that
> cleaning person not to tell one of his friends what he as seen
> on some display.

I can't protect against [D]DoS attacks consuming any/all available
*external* bandwidth.  But, the *functionality* of the system will
not be impaired in such an attack.

>> Instead, I'm interested in *cheap* algorithms to detect intrusion
>> *attempts*.  E.g., port scans, etc.

>> Users, here, will range from "John Q Public" (i.e., tech-illiterate) to
>> small corporations with "traditional" IT departments (i.e., they may be
>> aware of the sorts of attacks possible but probably not skilled in
>> handling them first-hand).
>>
>> As such, it is imperative not to worry the user with false positives as
>> that increases their level of anxiety -- needlessly so.
>>
>> [Recall, the system is hardened so there is no risk of these attacks
>> getting through.]
>
> Famous last words.  Hopefully you have gone through a complete security
> audit on the source code with "outside" eyes.

The stack already has lots of "miles" on it.  And, the changes I've
now introduced just make it *more* robust since it no longer has to
be "general purpose".

>> I can easily and reliably identify obvious unauthorized attempts at
>> accessing services or protocol spoofing.  Since these can only be caused
>> by *bugs* (i.e., a legitimate host accidentally mangling it's
>> MAC/source-IP/dest-ip/etc. in a transaction) or *attacks* (i.e.,
>> deliberately masquerading as someone you are not), any such event is
>> grounds for an alarm (bugs don't exist; attacks SHOULDN'T exist!)
>>
>> The system is physically secured -- though there are ports by which an
>> "invited intruder" could attempt to gain physical access to the network
>> (e.g., sit down at an unused desk in a back office and plug in a rogue
>> laptop, etc.).  I've taken steps to prevent such access...
>
> This may or many not be true depending on the value of the target. A
> recent study put a usb drive in the company parking lot.  60% of
> the people that picked it up plugged it into their work computer.  If
> I put a company logo on it that number will jump up to 90%.  If you
> have a valuable target someone will look for a way to get at it
> from the inside.

Yup.  I've done my best to ensure there are no "easy" mechanisms
for *anything* to be introduced to the system -- even intentionally!
(e.g., software updates have to be signed; anything that "examines"
files is hardened; nothing comes in without being parsed; etc.)

>> The biggest problem is handling "sporadic" bogus contacts that may or
>> may not be hostile (keeping in mind the "false positive" constraint
>> above).
>>
>> E.g., an incoming *legitimate* ping(1) shouldn't ring alarm bells. Nor
>> should a blind attempt to contact an HTTPd(8) which may or may not be
>> running at a particular IP.  (e.g., google routinely tries to do this
>> when you make public posts)
>>
>> OTOH, a ping flood or sequential port scan should probably raise some
>> eyebrows.  And, attempts at exploiting "common vulnerabilities" (even if
>> I am immune to them) would be indicative of hostile intent!
>
> If the target seems valuable the bad guy wont do a sequential scan.
> They can take several days probing it.  Unless you are capturing
> the events and correlating you probably wont see this.

Exactly.  "Not cheap".  So, I have concentrated on identifying
obvious attempts at exploiting "CAN'T HAPPEN" scenarios -- cases
where you know the attacker is trying to exploit carelessness
in your implementation.  Since these things *shouldn't* be happening
in a well behaved client, any evidence of them would be highly
suggestive of an intrusion attempt.  Since the system can "know
itself", I can add lots of invariants that make it less tolerant
of "generic" clients trying to masquerade as legitimate components
or clients.

>> I concede that there is nothing I can do to prevent "bandwidth theft"
>> and the DoS that accompanies that.  But, services and functionality that
>> do not rely on that "external" bandwidth should remain unaffected even
>> in the presence of a full-scale attack.
>
> If this is an embedded low performance device you need a way to shed
> the processing time spend dealing with the attack packets.  This would
> mean 2 different network interfaces if the "private" side must stay
> responsive while the public side is under attack.  With cloud
> computing I can get a lot of horsepower and a lot of bandwidth
> for not much money.

Already addressed, here.  Nodes can reliably intercommunicate (as long
as physical security hasn't been breached) regardless of any external
activities.  Consuming 100% of the external bandwidth has *no* impact
on performance or reliability of the system itself (with the obvious
exception of any services exported to the outside world).   I.e., that
external "mal-activity" never penetrates the outer defenses to actually
*consume* resources used by the system.

As I said, the bigger problem is communicating these things to the user
in a meaningful way as well as providing effective remedies.  A big,
red, flashing "Attack In Progress" light isn't going to do much for the
user besides drive up his level of anxiety!  :-/

--don

On Friday, June 28, 2013 1:14:35 AM UTC+3, Don Y wrote:
> ....
> 
> Instead, I'm interested in *cheap* algorithms to detect intrusion
> 
> *attempts*.  E.g., port scans, etc.
> 

Hi Don,
nice to hear from you. Hopefully that heat in your area
gets more tolerable soon.

The only algorithm which is cheap I can think of is just count
connection attempts/time. Which is probably something experienced
intruders have known for decades now.

Sequential port scanning would be easy to detect but is likely
also retired from use. What would work is not cheap... but here it
is: build a 3D spectrum, on X is the port number, on Y received SYN
segments count and on Z - the IP address.... now Z makes things
somewhat bulky, I know :-). But you can encode things somehow
so you can fit that in 16 bits and below, too (unlikely someone
will use > 64k different IP addresses to attack you in a day or so).
Obviously the time the spectrum is acquired will also be available,
start moment etc. Then you can clear it from time to time.

Just looking at the resulting 3D spectrum should be very telling.

Dimiter

------------------------------------------------------
Dimiter Popoff               Transgalactic Instruments

http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/

Hi Dimiter,

On 6/29/2013 7:12 AM, dp wrote:
> On Friday, June 28, 2013 1:14:35 AM UTC+3, Don Y wrote:
>>
>> Instead, I'm interested in *cheap* algorithms to detect intrusion
>> *attempts*.  E.g., port scans, etc.

> The only algorithm which is cheap I can think of is just count
> connection attempts/time. Which is probably something experienced
> intruders have known for decades now.
>
> Sequential port scanning would be easy to detect but is likely
> also retired from use. What would work is not cheap... but here it
> is: build a 3D spectrum, on X is the port number, on Y received SYN
> segments count and on Z - the IP address.... now Z makes things
> somewhat bulky, I know :-). But you can encode things somehow
> so you can fit that in 16 bits and below, too (unlikely someone
> will use > 64k different IP addresses to attack you in a day or so).
> Obviously the time the spectrum is acquired will also be available,
> start moment etc. Then you can clear it from time to time.
>
> Just looking at the resulting 3D spectrum should be very telling.

Something like this would work well on a primary outfacing
interface.  E.g., "The Internet Connection".  There, you could
deploy a bastion host with the primary purpose of protecting the
system/network.  And, "throw resources at it".

But, doing this "closer to home" gets expensive.  Any network drop
that is (or could easily become) exposed to "invited intruders"
(employees, guests, visitors, etc.) would need similar protections.
E.g., a disgruntled employee launching a attack from his cubicle,
an empty office or a "service drop" on the factory floor.

You would have to treat this portion of the attack surface in
much the same way as the "public" viewport -- run all of them
through firewalls before they get to the physical network on
which the distributed system relies.  (This can render them
unusable *in* the distributed system as these interfaces would
have different behavioral characteristics from the "other"
interfaces.)

If you can keep things "light"/cheap, then you can deploy them
*universally* -- "exposed" interfaces as well as "dedicated"
ones.

I am hoping that expoiting the "can't penetrate" capability
would let me infer that any (failed!) attempts at penetration
could be used to alert to an attack.  ("Young man, why were
you attempting to open the office safe?  What possible reason
could you have to *think* you should have access, there??")

Unfortunately, it is relatively commonplace for semi-legitimate
users/agencies to "poke at" IP addresses without malintent.
E.g., google will poke at *your* tgi-sci.com website when it
notices your *public* post, here.  "Just looking..."

(the log of the firewall I have protecting *this* host as I write,
contains hundreds of such blocked accesses, daily -- many of them
harmless ones from my ISP!)

If you (I) throw up an alert each time *any* such access is
attempted (and blocked!), the user will first become anxious...
fearing his system is under attack.  Then, eventually, he will
become *complacent*... potentially ignoring alerts that he
shouldn't!

--don