Sunday, August 17, 2008

Spliced feed for Security Bloggers Network

Spliced feed for Security Bloggers Network

Freeware and free securityware updates [belsec] [Belgian Security Blognetwork]

Posted: 17 Aug 2008 05:30 AM CDT

Your illegal internet activities are being logged [miekiemoes] [Belgian Security Blognetwork]

Posted: 17 Aug 2008 04:04 AM CDT

I've found at lot of posts at several different forums recently where people are discussing a mail they received from MediaDefender which says that their illegal internet activities are being logged.

Content of the mail:

Dear User!
Your recent internet activity was logged on the following sites:
• Btjunkie
• SumoTorrent
• isoHuntBtscene
• Mininova
• Fenopy
• Monova
• Yotoshi
• GetInvites
• Btmon
We have attached a report about the copyrighted movies, music, softwares you downloaded or searched on these webpages. We strongly advise you to stop any future activities regarding the downloading of illegal content or you can expect prosecution by 17 U.S.C. §§ 512, 1201?1205, 1301?1332; 28 U.S.C. § 4001 laws.

MediaDefender Inc.

This mail also contains an attachement, a so called report of their illegal activities. Ofcourse the attachement is malicious and appears to be a W32.Mytob@mm variant.

Sneaky tactic since many people actually do use P2P in order to get their (illegal) software from, so they will certainly open the attachement (if not removed by their Antivirus already) to see what exact activities are being logged.
If you recieve similar mail, delete it asap and certainly do not open the attachement.

More detailed info and screenshots of the mails can be found at TrendLabs

BelsecTV how criminals steal money online [belsec] [Belgian Security Blognetwork]

Posted: 17 Aug 2008 03:45 AM CDT

security comix crooks are everywhere [belsec] [Belgian Security Blognetwork]

Posted: 17 Aug 2008 02:00 AM CDT


More frustrations with web infrastructure [StillSecure, After All These Years]

Posted: 17 Aug 2008 12:46 AM CDT

As I mentioned in my previous post, the hardest part of recovering from the recent hacking attack has been the lack of process and procedures in place by many of the "utilities" we depend on to use the web.  In this post, I am going to highlight two specific examples:

1. Go Daddy - the hackers best friend. The reason it took so long for my domain to point to my blog after the attack was due to Go Daddy.  The fact is my domain should never had been pointed away from my blog. My domain registered through Go Daddy was supposed to be locked.  In fact it was. I received an email Sunday morning that my domain had been moved from locked to unlocked. This was when I first became aware of what was going on. I immediately called Go Daddy (to be fair, Go Daddy was one of the only companies that had 24 hour phone support) and told them I did not give my permission to unlock my domain, I was under attack and to lock it down and let no changes happen.

I was assured by the Go Daddy tech support person that this is exactly what they were doing and not to worry, Go Daddy was on the job.  In fact he sent me a customer satisfaction survey to fill out about how Go Daddy had thwarted this attempt to hijack the domain. Wong!  When I called Go Daddy back later that afternoon (always double check that these companies are doing what they claim they are going to do, don't trust them to follow through without you staying on top of it), it was as if I had never called that morning.  By that time the contacts on my domain were changed.  When I finally got a supervisor (ask for the supervisor quickly, don't insult the first level guy, just explain that you are going to need a supervisor for this emergency) on the phone it was too late for that. No I had to go to the "undo" department.  By now it was Sunday night and the undo department is only in during normal business hours, so I had to wait till the next day.

The undo department has no phone and you can only communicate via email.  In spite of everything I showed them about my situation, they were writing to the hackers to "hear there side of it", as if this was some run of the mill dispute over who owned a domain.  All through Monday I literally had to call every hour to keep this moving.  Getting a supervisor and making them walk down to the undo department.  Finally,  by Tuesday morning, after lots of help from some contacts in the blog world (thanks Jennifer L), Go Daddy finally put through to give me control back of the domain.  Than it takes 24 hours to 48 hours for DNS to update.  If not for the help and the constant calling it take up to a week for this process!

That is just bull.  Go Daddy and others in this position should do better. Having an undo department is commendable. If they would have taken more than a minute to actually look into the facts here and heaven forbid get on a phone, this could have been taken care of in 5 minutes.

2.Typepad - Again Typepad offers no phone support whatsoever. So I followed the process and sent an email to open a ticket.  I explained that my blog had been compromised as well as the contact email address. I needed to either have the blog shut down or I needed new passwords and usernames set up.  So what did the Typepad tech support do?  They sent new passwords to the compromised email box!  Finally, my friends at Feedburner reached out to their friends at Typepad and someone from Typepad called my from home and helped me out.  So I owned my blog by Sunday night, though the domain was now pointed somewhere else. 

I don't know what I would have done without the Feedburner help.  Typepad needs to offer some sort of process for this type of situation if they are going to be a quality host going forward.

Another failure of Typepad is their back ups.  Many of you have commented that many posts from February to August are missing from my blog.  The thoughtful hackers tried to save some of my storage space by deleting these posts :-(  I have been assured by Typepad that everything on these blogs are kept in a database which is backed up regularly.  Great, right? Wrong.  Though it is supposedly backed up, there is no real way to restore the posts.  The Typepad team is supposedly working on how to accomplish this.  They are busy and will try to get to it and I am patient.  But waiting a week for a back up restore seems a bit much.  What do you think? I would pay extra for a real back up service for my blog with a restore feature.  This seems easy enough, what is the story?

Tomorrow I will talk about the biggest problem I have had with web providers in this adventure: Yahoo!

PHP strip_tags not a complete protection against XSS [ISIS Blogs]

Posted: 16 Aug 2008 09:17 PM CDT

On August 13th .mario, a high-profile member of the forum, alerted me to a XSS issue on the CSAW registration form. I had previously looked through the code and concluded it was safe based on this block of filtering code included at the top of the page:

//don't use this script! foreach ($_COOKIE as &$cookie) {   $cookie = trim(strip_tags(@mysqli_real_escape_string($mySQL, $cookie))); } foreach ($_POST as &$post) {   if (is_array($post)) {     foreach ($post as &$_post) {       $_post = trim(strip_tags(@mysqli_real_escape_string($mySQL, $_post)));     }   }   else {     $post = trim(strip_tags(@mysqli_real_escape_string($mySQL, $post)));   } }

Additionally, the registration script limits sources of user controllable input by only ever using the POST and COOKIE superglobals.

This script eliminates potential SQL injections by calling mysqli_real_escape_string on all user input. The *_real_escape_string functions in PHP are the only safe way to prevent SQL injection attacks as there are ways to sneak attacks by the [deprecated] *_escape_string and addslashes functions, for example, with different character encodings.

After being processed by mysql_real_escape_string, all user input is then filtered through strip_tags, a function which one might think would prevent cross-site-scripting attacks by completely removing any HTML and PHP tags it finds. strip_tags works through many different encodings and very effectively strips tags, however, seasoned web hackers will be saying at this point “there are other ways to inject javascript without tags!” and they would be right.

The easiest way to avoid strip_tags is to inject a quote to close the current attribute, create a giant block with a new CSS style attribute, and make it evaluate javascript onmouseover. Using an example from the forum and .mario himself:

" onwhatever=alert(1) a="

This accomplishes the goal of injecting javascript into the target application without creating any additional HTML or PHP tags, and strip_tags won’t pick it up!

I asked .mario if he would be kind enough to provide us with a proof of concept that would work specifically in the context of the CSAW registration page. He did not disappoint and PM’d me the following attack string:***”style=”a:b;margin-top:-1000px;margin-left:-100px;width:4000px;height:4000px;display:block;”onmouseover=alert(/XSS/[-1]);eval(name) a=”***content,post

*** This has been URL decoded.
*** For the original, please see

His attack utilizes a service he wrote (the name.xss part of the URL) to launch XSS attacks on websites that limit user input to parameters other than GET, as ours does. The attack works as I described above, by adding new attributes into the existing tag that create a large CSS block that triggers javascript onmouseover. Interestingly, this complicated attack string is processed as javascript by all the major browsers including IE8, Firefox 3, Opera 9.51, and Safari 3! I was really impressed.

After verifying the PoC, I made a small addition to our filtering script to prevent this attack by adding an htmlentities function call to each iteration of the two loops. This isn’t the best solution as it escapes input more than is necessary and I didn’t have a chance to bug test it much at all. A better solution can be found in the same forum post I found the attack string in:

…the best practice is IMHO:

Input -> Validate -> Filter (CRLF, Ctrl-Chars) -> Escape -> Store -> Encode (Just the characters you need to encode) -> Output

Validation can be done via type check or regex, for filtering the ord() method does a great job, escaping is done by mysql_(real)_escape_string() and encoding is done by correctly parametrized htmlentities().

We’ll be looking into ways to rewrite our filtering script according to this advice, and also at PHP-IDS, as a way to prevent these types of issues in the future.

Thanks .mario!

.Be sites infected with new spamhacking technique [belsec] [Belgian Security Blognetwork]

Posted: 16 Aug 2008 06:36 PM CDT

It is hacking websites for spamming. You can't call it any other way because they are adding a page to the website - which is hacking. On that page - that you can find with the googledork 


you will find a list of words without links followed by a list of words that have links. Those links will go to another part of the website that has been injected with links.

Attention you need a firewall to surf these sites because sometimes attacks on your pc are started with fake online security scanners. The only effect they have is to scare the hell out of you with fake reports. Both sites may have more pages infected. (this one givea an online fake security scan attack - ( another one in the same series is the (

for the general internet those infections are much more widespread.

Turkish hackers beginning real no war campaign against Russia [belsec] [Belgian Security Blognetwork]

Posted: 16 Aug 2008 06:21 PM CDT

A real defacement campaign doesn't mean that the infrastructure goes down or that some high profile sites are being targeted. No it means that several groups are attacking websites with the same message. We can no say on the basis of the valuable monitoring by that several Turkish groups have joined the no war defacement campaign against the .ru domain and other domains. Even if nobody finds that very logic, the campaigns are being used against all the websites they seem to have found to be vulnerable, no matter what the domainextension. So if you have a webserver that needs to be patched (from windows to Joomla or drupal, you better do it now) as this campaign is increasing. Not maybe in the numbers but in the number of attackers and hackers that are participating in this campaign. We warned you before the weekend that this could happen. Slowly this is developing. If the conflict escalates further (and if you read the ground reports coming from other sources than the over optimistic European ones - than you know that this will escalate slowly but surely because Russia has no intention of leaving Georgia (alone).


what history learns about this European treaty with Georgia [belsec] [Belgian Security Blognetwork]

Posted: 16 Aug 2008 06:10 PM CDT


chamberlain before WOII holding a paper signed by Hitler that would bring peace

XKCD: The issues with electronic voting machines. [Security4all] [Belgian Security Blognetwork]

Posted: 16 Aug 2008 05:49 PM CDT

xkcd makes a bitter but funny point. Electronic voting is very tricky and has lead to a lot of controversy. I'm not going into a long debate but I will provide some resources to read and show why electronic voting is dangerous without the proper security mechanisms.

Audio (mp3) from The Last HOPE:
  • Building a Better Ballot Box (We all know by now the folly of current election technologies from Premier and Sequoia Voting DRE (Direct Record Electronic) systems as well as some of the new, more promising systems on the horizon such as the open source OVC (Open Voting Consortium ) and Scantegrity. The question of whether we can do better will be raised. What needs to be done to make this process better than it is today?)
  • Hacking Democracy: An In Depth Analysis of the ES&S Voting Systems (Last Fall, Ohio Secretary of State Jennifer Brunner commissioned Project EVEREST, a comprehensive security review of the electronic voting technology used in her state. The project contracted several academic teams and others to examine the election procedures, equipment, and source code used in that state, with the aim of identifying any problems that might render elections vulnerable to tampering under operational conditions.)
The Dutch movement against voting machines has some interesting articles:

A presentation from 22C3: e-Voting: The silent decline of public control (Why German voting machines do not meet the requirements of democratic elections.)

(Cartoon under Creative Commons by XKCD)

Linus: Security Circus - Admission Is Free [Infosecurity.US]

Posted: 16 Aug 2008 04:27 PM CDT

In today’s MustRead, a rather curmudgeonly, and controversial view of information security professionals espoused by Linus Torvalds, the creator of the Linux kernel. He really doesn’t find much use for BSD (a statement, of which, we most certainly take exception too)…

Server Upgrade []

Posted: 16 Aug 2008 03:06 PM CDT

So our server was upgraded by our hosting provider.  Unfortunately, in the upgrade, a comment from Christian was lost amidst the shuffle. Sorry Christian!

Please take a second and verify your RSS feeds and all that.  Thanks!


Hey, PR guys, I am listening, even if I don’t respond [Network Security Blog]

Posted: 16 Aug 2008 11:56 AM CDT

And now for an off-topic Saturday morning rant:

In the last week there was an interesting meme led by Robert Scoble (what a surprise) about tech bloggers and PR people. It’s been an interesting back and forth even though it’s only tangentially important to me, it’s still something worth commenting on.

I read almost every press release and PR note that I receive. It may not be a top to bottom read, but I do read enough to make a judgment call as to whether or not the email is of enough interest to me to read further. Often I try to acknowledge that I received the email by responding to the PR folks who sent it, though this has been happening less lately as my time becomes more constrained by travel. If someone’s sending me press releases that have nothing to do with security, I’m more likely to respond and explain to them what I do and write about so that they won’t waste their time and mine telling me about stuff that is of no interest to me. And it’s surprising how many PR folks appreciate this feedback.

A prime example of folks who get how to interact with security bloggers is Connect Public Relations. They send me a two to three sentence email pointing me to a client’s blog post, usually Symantec. Here’s a recent example:

Hi Martin,

I wanted to let you know that Kevin Haley has entered a blog post about some survey results from Symantec on how security professionals are using social networking. Are they like everybody else? You can see what Kevin had to say about it here:

That’s it. They’ve sent me something relevant to me, haven’t wasted a bunch of my time with a long email, asked me for my opinions and let me get back to my life. I usually go and scan through the blog posts even if I rarely use them as fodder for my own posts. It’s not that I don’t find the posts interesting, it’s just that as often as not, I either don’t have the time or just don’t find anything that resonates with me enough to elicit a blog post of my own.

What I hate receiving is the long, drawn out, mass produced, hype-filled, product announcements. There’s maybe one in a hundred of these that I can read past the first paragraph. Quite frankly, most of these announcements are just hype with almost no real substance to them. If you don’t care enough to figure out why your product might apply to me personally or professionally, then why should I?. There are some exceptions when people are inviting me to participate in beta programs or are honestly looking for feedback, but those are rare. I wish I had the time and energy to take advantage of more of the betas, but work, blogging, podcasting and family are more important to me than checking out a new tool I would rarely or never use otherwise.

I don’t want to be broadcasted to or towards by PR agencies, and I doubt many other security professionals do either. I know I don’t have the sort of draw someone like Robert Scoble does, not many people do. I don’t expect to be included in the A-list that’s told before everyone else; but I do want to feel that the person at the other end of the email has take a few moments to at least understand who I am, what I write about and why their message might be of interest to me.

So if you’re a PR person sending me email and press releases, please know that I’m probably reading what you sent me, even if I only skim the email. But also be aware that I’m probably going to spend the same amount of energy or less on reading the email than you spent sending it. So if I’m just part of the 10,000 person mailing list you send to on a weekly basis, don’t excpect much in return. But if you’re actually treating me as a real person instead of another field in your Excel spreadsheet, the chances are I’m listening, even if I don’t respond.

P.S. You could always try giving me a call. My cell # is on the About page for a reason.

[Slashdot] [Digg] [Reddit] [] [Facebook] [Technorati] [Google] [StumbleUpon]

Microsoft Security Response Center August Release [Infosecurity.US]

Posted: 16 Aug 2008 11:34 AM CDT

The MSRC has released their August 2008 Monthly Bulletin detailing a multitude of fixes, patches, and alerts:


· MS08-041 – Vulnerability in the ActiveX Control for the Snapshot Viewer for Microsoft Access Could Allow Remote Code Execution (955617) – Critical

· MS08-042 – Vulnerability in Microsoft Word Could Allow Remote Code Execution (955048) Important

· MS08-043 – Vulnerabilities in Microsoft Excel Could Allow Remote Code Execution (954066) Critical

· MS08-044 – Vulnerabilities in Microsoft Office Filters Could Allow Remote Code Execution (924090) Critical

· MS08-045 – Cumulative Security Update for Internet Explorer (953838) – Critical

· MS08-046 – Vulnerability in Microsoft Windows Image Color Management System Could Allow Remote Code Execution (952954) Critical

· MS08-047 – Vulnerability in IPsec Policy Processing Could Allow Information Disclosure (953733) Important

· MS08-048 – Security Update for Outlook Express and Windows Mail (951066) – Important

· MS08-049 – Vulnerabilities in Event System Could Allow Remote Code Execution (950974) Important

· MS08-050 – Vulnerability in Windows Messenger Could Allow Information Disclosure (955702) Important

· MS08-051 – Vulnerabilities in Microsoft PowerPoint Could Allow Remote Code Execution (949785) Critical

We also revised the following bulletins to update detection changes

· MS08-022 – major revision, added XP SP3 detection

· MS08-033 – major revision, added XP SP3 detection

· MS07-047 – major revision, update detection

· MS08-040 – minor revision, update detection


· Release Advisory 955179

· Revised Advisory 954960

Excuse the dust [Network Security Blog]

Posted: 16 Aug 2008 10:57 AM CDT

I’m playing with the site some, adding some ads, removing others, getting rid of some of the features I added a while ago and never use. I don’t exactly hope to make a fortune from the ads on the site, but beer and pretzel money is always good to have. Or at least money to replace the next piece of equipment I spill coffee on.

Please let me know if/when I cross the line from mildly annoying to driving you from the site. The ad money is worth less to me than my readers are.

On a related note, Rich and I are starting to discuss the potential of sponsors for the podcast, at least for special events like RSA or Black Hat/Defcon. We’d love to be able to follow Paul and Larry’s lead and create some t-shirts and bumper stickers to hand out at the events. Feedback and ideas on this are also appreciated.

One more thing (any Jacki Chan Adventures fans out there?) Rich and I still have some books to give to readers/listeners, but we haven’t been happy with how we’ve done it before. Any suggestions on how to give away books in a way that will draw in more listeners and readers? I received an extra copy of Raffael Marty’s book Applied Security Visualization and I’d love an excuse to get it out of my library and into someone else’s.

[Slashdot] [Digg] [Reddit] [] [Facebook] [Technorati] [Google] [StumbleUpon]

Our web infrastructure needs to be at public utility levels [StillSecure, After All These Years]

Posted: 16 Aug 2008 09:42 AM CDT

This is going to be the first in a series of posts I am going to write about my experience in recovering from the recent attack against me. Many people have asked me "what do you do if this happens to you".  Well first of all it helps to have a team like the StillSecure Security Alert Team (SAT).  In particular, Brad Doctor, the director of our SAT was my rock during this whole ordeal.  Brad quickly saw the full scope of the attack against me and guided me in my steps.

First thing was to take care of financial exposure.  Getting your credit and debit cards canceled is relatively easy. You get on the phone and just about every financial institution had an option for lost or stolen cards or other fraud.  Within a half hour or so that is taken care of.  The card companies will send you out new cards and credit you for any charges on your accounts. Protecting your credit and identity financially speaking was fairly easy as well. There are any number of firms that offer great services in locking down anyone trying to open accounts in your name or social security number. I will give more details on who I choose on this in a later article.

Next Brad had me cleaning out my own computer and my online identity.  I wish that this was easy as protecting my financial exposure. The fact is our web infrastructure that so many of us depend on is just not up to the challenge.  The shame is that at this point in time web companies are almost utility like.  If something happens to your electric or water or cable, you can pick up a phone and eventually get someone on a phone to help you. Not the case with our web utilities.  They are set up for volume and scale, but not customer support in emergency time frames.  The standard response in contacting any of the web companies was an auto-generated reply that someone would email me back in 24 to 72 hours or more! When you are dealing with an emergency, you are locked out of your accounts and your identity is being stolen and abused, that is just not good enough.

As I have written earlier, I was lucky in that I was able to call on people to help me out.  For instance my friends at FeedBurner/Google, Matt Shobe and Dick Costollo, quickly took control of my FeedBurner accounts, including the SBN feed.  They were also to get someone live at Typepad to allow me to take back the blog.  This took more time than it should have though.  Until the Feedburner reached out to someone, the Typepad support team just kept sending a new password to mailboxes that the attackers controlled, even though I was mailing them from my stillsecure mail box! You could not get any of these people on a phone.  Very frustrating!

In any event, if you don't know anyone with some "juice", you have to go through the process.  You can keep sending emails. I think it is important that you write full emails that really explain the gravity of the situation.  Eventually when a live person looks at it, it does help.

But all in all, generally once I was able to get a live person on the phone, I was able to undo some of the damage done.  Our web providers like Yahoo, Microsoft, Google, etc. need to have emergency phone numbers that people can use for these type of situations!  However, even having a phone number does not guarantee success.  In the case of Go Daddy, it was just the opposite. In fact Go Daddy can be the hackers best friend!  More about that in my next post.

Blackhat / DEFCON are over… Next is SecTor [.:Computer Defense:.]

Posted: 16 Aug 2008 01:33 AM CDT

Hey Everyone,

So Blackhat/Defcon is behind us... Instead of blogging about the talks, I've taken a different approach and I've been doing some non-technical blogging. In the end it will be a 5-part series, but the first three are already up.

They are:

  1. Being a Research Engineer at a Blackhat Booth
  2. Competitors Can Be Civil
  3. Why DEFCON Sucks

The last two will most likely appear early next week.

Also, now that Blackhat/ DEFCON are over... What's next? As far as I know the next Con I'll be attending is SecTor. Last year was the first SecTor and I had the opportunity to attend. SecTor will actually make it's way into my upcoming blog series (from above) on the VERT Blog. That being said, I wanted to remind people that it's coming up, after all... it's held in Toronto and I live in Toronto, so the more people that attend, the more people I get to meet.

For anyone who didn't get a chance to visit SecTor last year and is curious about the quality / style of the talks, I tried to write-up everything that I saw.

Of course, these are biased because they're all my opinion, but I do recommend the Con for anyone that can make it up this way. Let me know if you'll be coming up and we'll make arrangements to get together for a beer.

Pics: Liquidmatrix Defcon Side Trip [Liquidmatrix Security Digest]

Posted: 15 Aug 2008 09:54 PM CDT

When at Defcon last week Myrcurial and I had a chance to go and fire off some machine guns. No really. So, we figured what the hell.

Dave with AK Ammo

Here I am with rounds for an AK-47. I had asked for the MP5 but, being a good Canadian lad I had as much clue as the staffer who handed me the ammo that it was wrong.

First on the firing line was Myrcurial. For never having fired a fully auto machine gun he handled it remarkably well.

myrcurial with the MP5

Next up with the MP5 was yours truly. I’ve fired long guns in the past but, never had I fired a machine gun either. It was an experience.

gattaca with the MP5

The look of a contented old fart after burning through three clips. Sure the gun jammed a few times but, I didn’t really care. If you look closely you can see the whole smack in the middle of the target’s head. Myrcurial and I were pretty good shots. Thankfully not a habit forming activity.

gattaca smiling

This last pic left no doubt as to the political leanings of the shop staff. In case your not sure it’s bolted to the wall above a garbage can. Ah, politics.

Google vs. SideJacking, round 7 [Errata Security]

Posted: 15 Aug 2008 04:40 PM CDT

Google recently made a change that allowed you to configure your Gmail account to force SSL. Last I looked at Gmail, it would still leak cookies even if you thought you were using SSL. This latest change from Google appears to have fixed this.

If you enable this in your account, Gmail will not allow you to use non-SSL. If you attempt to log into "", it will switch to "".

Moreover, the session cookie "GX" is set with the "secure" flag. That means you won't disclose if even if you try normal HTTP. I have not yet been able to grab the cookie and hijack an SSL Gmail session yet. I can still hijack a lot of other Google stuff. For example, I'm posting by hijacking my own Blogger account.

Background Info

The following is the cookie specification for cookies sent by the server:

Set-Cookie: =[; =]
[; expires=][; domain=]
[; path=][; secure][; HttpOnly]

When "HttpOnly" is set, this means that cookies can only be set when within HTTP headers but not with JavaScript, thus preventing some cross-site-scripting attacks. When "secure" is set, this means the cookie is set with HTTPS but not HTTP.

When looking at the Gmail cookies, only the "GX" cookie is set with "secure" flag, all the rest can be sent via HTTP. However, the "GX" cookie is the important one that allows me to hijack the session, so that's probably appropriate.

AV versus E-voting [Network Security Blog]

Posted: 15 Aug 2008 03:22 PM CDT

I know this is popping up all over the place, but I had to add it too!

[Slashdot] [Digg] [Reddit] [] [Facebook] [Technorati] [Google] [StumbleUpon]

A clash of cultures [Phillip Hallam-Baker's Web Security Blog]

Posted: 15 Aug 2008 11:05 AM CDT

Readers of this blog are probably aware that lawyers for the MBTA recently obtained an injunction to prevent two MIT students presenting a security analysis of the MBTA 'Charlie Card' payment scheme at Black Hat.

The security community will do what it usually does in this situation: we rally to protect our own when they are under attack. That's why we are called a community: One for all and all for one.

But the case really illustrates a clash of civilizations and different views on how to achieve security. The term 'hacker' comes from MIT and the MIT hacker culture has more than a century of tradition behind it. When I first arrived at MIT as a research scientist there was a police car parked on the top of the MIT dome by student hackers. I don't think the MBTA does that sort of thing on a regular basis.

But there is a big difference between the MIT hacker culture and the subsequent hacker-vandal culture that misappropriated the name. According to the reports, the students did the responsible thing and explained the flaws they had discovered to the MBTA. If the thanks responsible researchers receive is a lawsuit, advance warning is going to quickly become extinct.

But lawyers are not trained to consider such issues, their training is to only consider the narrow interests of the client and frequently only in the case at hand.

Whatever the legal merits, bringing a lawsuit against MIT to suppress research performed by MIT students is poor public relations.

Update: Submitting the information you are attempting to suppress to the court without asking it to be sealed is likely to be counterproductive.

Is Your Firewall a “High Risk Entity” []

Posted: 15 Aug 2008 10:15 AM CDT

Not trying to be overly snarky here, but I was reviewing some GRC product literature recently.  And there was a screenshot of an application window showing how the software helps identify “high risk entities”.  And in the screenshot, there were 5 of these entities listed, each with corresponding risk ratings (High/Medium/Low) and scores (really just non-measurement ordinal numbers).  The screenshot showed that the riskiest entity of the five shown was a Checkpoint Firewall-an assertion backed up by the non-measurement “Risk Score”.  The lowest risk scores were shared by a nameless Web Application and an entity called “Oracle App”.

My friend, I’m going to give you a hint.  If your firewall is “high risk” and your actual business applications are “low risk” - you might be doing it wrong.

Estimating Availability of Simple Systems - Introduction [Last In - First Out]

Posted: 14 Aug 2008 11:03 PM CDT


I've spent a bit of time thinking about how to estimate the expected availability of simple systems. I'm interested not in detailed calculations of complex systems, but rather in rule of thumb type of estimates for simple systems. I suspect that this problem can be as hard or as easy as one would like to make it. I'm going to try to make it easy. Wish me luck.

This post introduces the estimations. Part One covers non-redundant systems. Part Two covers simple redundant systems


I'll start out with the following assumptions:

(1) That you have a basic understanding of  MTBF and MTTR.

(2) That your SLA allows for maintenance windows. Maintenance windows, if you are fortunate enough to still have them, allow 'free' outages, provided that the outage can be moved to the maintenance window.

(3) That you have the principles laid out in my availability post implemented, including:

  • Tier 1 hardware with platform management software installed, configured and tested. (Increases MTBF, Decreases MTTR)
  • Stable power, UPS and generator. (Increases MTBF)
  • Staff are available during SLA hours, either officially working or available via some form of on-call. (Decreases MTTR)
  • Structured System Management implemented. (Increases MTBF, Decreases MTTR)
  • Service contracts that match application SLA's. (Decreases MTTR)
  • You have well designed, reliable power and cooling. (Increases MTBF)
  • You have remote management, lights out boards and remote consoles on all devices

(4) That you understand system dependencies, including all cases where you have parallel and/or serial dependencies, and that you have diagrammed, or can easily diagram the dependencies.

(5) That you understand where you have coupled dependencies. Examples:

  • In a typical redundant router -> switch -> firewall stack, a failure of the switch will cause interfaces on the router and firewall to fail. Those interface failures will trigger an HA failover on both the HA router pair and the HA firewall pair.
  • In a typical dual powered rack, devices with single power supplies configured in HA pairs and are connected such that power failure of one side of the racks leaves the correct half of the HA pair still running. (See Red Power, Blue Power).

(6) That you have Tier 1 hardware from a major manufacturer with platform management software installed, configured and tested. You need the predictive failure messages that modern servers are capable of generating showing up on your SMS device. Predictive failure doesn't change the rate at which components fail, but it allows you you proactively schedule replacement of failed components during maintenance windows. This gives you 'freebie' failed component replacement. If you do not have Tier 1 hardware and do not have platform management software configured, your failure rate and your recovery time will will both increase.

(7) That staff are available during SLA hours, either officially working or available via some kind of on-call.

(8) That service contracts exist, they match the SLA's, and are routinely met by the vendors. In other words if you have a two hour part replacement contract, the vendor has to have depot spares less than one hour away and deliver them in less than two hours. Not all vendors do that. And if your SLA is 24 x 7 x 365, your support and maintenance contracts are also 24 x 7 x 365.

The estimates will:

  • Be conservative estimates, with plenty of room for error.
  • Include operating system issues.
  • Include hardware issues

When in doubt, I'll over-estimate the frequency and duration of outages. Better safe than unemployed.

The major components I'll cover are:

  • Wan Links
  • Routers, switches, firewalls and similar devices
  • SAN fabrics
  • SAN LUNs
  • Servers
  • Power and Cooling

Excluded from the estimates are:

  • Databases, database issues, database performance problems.
  • Web Applications, including performance problems, configuration problems.
  • Once-in-a lifetime events (like an Interstate bridge falling down 100 meters from your data center).
  • The human factor.

A handy chart

A chart of failure and MTTR assumptions might be handy. I'll express failure and recovery in hours per year. The math is pretty simple. Number of failure per year * hours to recover = hours failed per year. It can be converted to 'nines' later.

Component Failures per Year Hours to Recover Hours Failed per year
WAN Link .5 8 4
Routers, Devices .2 4 .8
SAN Fabric .2 4 .8
SAN LUN .1 12 .12
Server .5 8 4
Power/Cooling 1 2 2

WAN links - T1's, DS3's, etc. For all non-redundant WAN links I figure an average of 4 hours per year WAN related network outage. Over hundreds of circuit-years of experience, figuring in backhoes, storms, tornadoes, circuit errors, etc, I figure that one outage every other year with an 8 hour MTTR is a reasonable estimate. Actual data from a couple hundred circuit-years of data is better that the estimate (one outage every 4 years with a 5 hour time to repair).

Routers, switches, firewalls and similar devices, booted from flash, no moving parts other than power supply fans I figure at one failure every 5 years, with the MTTR dependent on service contracts and sparing. If you have spare on hand, or you have good vendor service, your MTTR can be figured at 2 or 4 hours. If not, figure at lest 8 hours. If our network-like devices have disk drives or are based on PC components, I figure their failure rate to be the same as servers (see below).

SAN fabric, properly implemented, I figure at one failure every 5 years as with network devices, but because most SAN fabrics are built redundant, the MTTR is essentially zero, so SAN fabric errors rarely affect availability. The emphasis though, is on 'properly implemented'.

SAN LUN's, properly implemented, I figure at one failure every 10 years. But SAN LUN or controller failure, even though it is rare, likely has a very long MTTR. Recovering from failed LUN's is non-trivial, so at least 8 hours MTTR must be figured, with the potential for more, up to 24 hours.

Servers, with structured system management applied and with redundant disks, I figure at about one 8 hour outage every two years, or about 4 hours per server per year. Servers that are ad hoc managed, or servers without lights out management or redundant power supplies, I figure at about one 8 hour outage per year.

Power and cooling is highly variable. We have one data center where power/cooling issues historically have caused about 8 hours outage per year. We can't remember the last time we have a power issue in our other data center. I figure power/cooling issues on a per site basis, with estimates of 2 hours per year on the low end, and 8 hours per year on the high end.

Your assumptions for failure frequency and recovery time might be different than mine.

Stacking the estimates.

Once you have a rough estimate of the availability of the various components in an application stack, you can apply them to your application.

First steps:

  • Sketch out your application stack
  • Identify each component
  • Identify parallel verses serial dependencies
  • Identify coupled dependencies

application-nonredundantSerial Dependencies increase recovery time. In a chain of components in  series, any failed component in the chain causes failure of the entire chain. In this case, the number of failures of the system is the sum of the number of failures of each series component. The recovery time for each component is also added in series.

In the example to the left, all components from WAN at the top to RAID array at the bottom are serially dependent. Each must function for the application to function, and failure of a component is a failure of the application.




application-redundantParallel Dependencies improve recovery time. Components in parallel, properly configured for some form of automatic load balancing or failover, have an MTTR equal to the failover time for the pair of parallel components. Examples would be Microsoft Clustering Services, high availability firewall pairs, load balanced application servers, etc. In each of those cases, the high availability technology has a certain time window for detecting and disabling the failed component and starting the service on the non-failed parallel component.  Parallel component failover and recovery times typically range from a few seconds to a few minutes.

In the example to the left, if either the green or the blue path is available, the application will function.



availability-redundant-coupledCoupled dependencies, as in the firewall-> switch -> web server stack shown, force the failure of dependent components when a single component fails. In this case, if the switch were to fail, the web sever would also fail, and the firewall, if active, would also see a failed interface and force a failover to the other firewall.

Coupled dependencies tend to be complex. Identification and testing of coupled dependencies ensures that the various failure modes of coupled components are well understood and predictable. Technologies like teamed nic's and etherchannel can be used to de-couple dependencies.


On to Part One - Non Redundant Systems

No comments: