ColdFusion Muse

Tabular Data Stream Error (Who's Using a Cursor Select Method?)

It's not often I see an error that I have simply never seen before. Here's one that some of you may recognize. It happened on a query using SELECT * against a fairly limited table on an MSSQL 2005 database. There wasn't anything unusual about the query (except for the bad form of using the asterisk). Here's the error that was thrown:

The Fix

Sick Server Troubleshooting Part 3 - In the Soup

Ok... you've made config changes and tested your connections and performance and you know with the certainty of Rush Limbaugh making a right turn that the server is fully tuned and as lean and mean as possible, but you still have a problem. Now what? Well, it's time to look at the code. Start by enabling slow page logging in the CF admin during a high traffic low performance period. Take the list of templates that are running slow and start digging into them to determine why they are dragging their little cyber feet.

Here are a couple of code related items that you might be looking for.

Sick Server Troubleshooting Part 2 - Things to Try

In step 1 we discussed gathering information. Without getting good information you are shooting in the dark. Make sure you take the time to know the system well enough to make educated guesses about what to try. That's our next step - trying stuff. I started out making a priority list... as in first try A, then B, but it soon became obvious that it wouldn't do to dictate the order in which you would attempt any of these changes.

In fact, some of these changes fall more under the auspices of "best practice" for configuration and you should probably do them regardless of whether they fix your problem or not. Still, if you are tracking a particular issue then you might need to try something, test, and wait for a result. So here are my “things to try” in no particular order. Your experience will have to help you figure out where to go first.

Things To Try

Sick Server Troubleshooting Part 1 - Gathering Information

I get a lot of calls from folks with dedicated Coldfusion servers who have suddenly developed problems. Usually the server has suddenly stopped responding in a seemingly random fashion. Often the caller indicates that Jrun is "pegged" at 99 percent. If you search through this blog (and many others) you will find a great number of tips and hints on how to attack this problem. The next three posts summarize my own process and give you a quick resource to the posts regarding troubleshooting a sick server.

Keep in mind that there is no substitute for experience. If you are novice at this you will need to get comfortable with the idea that you will spend several hours (sometimes days) working through possibilities. If the server is mission critical and there is money at stake you should consider calling in the cavalry. A troubleshooter is also a unique animal - a "technologist" of sorts. The best troubleshooters have gathered a good deal of knowledge and experience in several areas (app server specifics, database, hardware, networking etc). The best one s are also a special breed that think in a certain way. They make lists, figure out test patterns, know when to make a leap and when to keep digging etc. So with that in mind, let's talk about how to start.

Step 1 is to gather information.

ISAPIRewrite or Mod_Rewrite Rules

For those of you interested in stopping the SQLi attack before it even hits your ColdFusion server, you might try these rewrite rules are from the CF-Linux email list (run by House of Fusion). They were provided by list member Mike Chytracek and forwarded to me by Linux CFG Ryan Stille. These rules are for for use with Helicon's ISAPI Rewrite filter, but with very little tweaking these rules aught to work for Apache Mod_rewrite as well.

# Helicon ISAPI_Rewrite configuration file

# Version 3.1.0.54


RewriteEngine On

RewriteCompatibility2 On

RepeatLimit 20

RewriteBase 

# unsupported directive: [ISAPI_Rewrite]


# CacheClockRate 300


RewriteRule ^.*DECLARE%20.*$ http://www.cybercrime.gov/ [NC]

RewriteRule ^.*NVARCHAR.*$ http://www.cybercrime.gov/ [NC]

RewriteRule ^.*sp_password.*$ http://www.cybercrime.gov/ 
[NC]
RewriteRule ^.*%20xp_.*$ http://www.cybercrime.gov/ [NC]

RewriteRule ^.*EXEC\(@.*$ http://www.cybercrime.gov/ [NC]

RewriteRule ^.*%20@.*$ http://www.cybercrime.gov/ [NC]


RewriteRule ^METHOD$ OPTIONS

Please note that these rules will actually redirect the request to the governments cybercrime website. That's going to freak a few folks out if you end up with any fals positives :)

SQLi Attack on the Rise (Film at 11:00)

Unless you have had your head in the sand (those of you on your honeymoon are excused) you know that the ColdFusion world has been awash in SQL Injection attacks over the last month. Anecdotally I am seeing a significant increase in attacks this week - about 15 times what they were a few days ago. Michael Dinowitz reports that house of fusion was receiving 4000 attacks in 5 minutes (that's nearly 50 thousand an hour). Brad Wood reports no less 90 request per second. The suspicion is that the attack is driven by searching Google for sites with ".cfm" pages. That means the more successful that you are at search engine optimization the more likely you are to be targeted. Conversely if you don't have a good number of pages ranked then you are probably then you will see fewer attacks.

It seems these attacks are orchestrated using infected computers throughout the internet. Some effort is underway to collect IP addresses to see if a pattern emerges. I suspect that approach will not yield fruit, but I still applaud the effort. We (CF Webtools) are continuing to assist customers in any way we can - everything from wholesale changes to sites, to blacklist techniques to friendly advice over the phone. As these attacks accelerate they become more like Denial of Service attacks than anything else. Even if you are binding all your variables and you have great controls you will still have to deal with a bombardment of thousands of requests against your CF pages. I recommend that you use one of the many blacklist techniques out there - at least temporarily. Some folks have started out sending emails alerts when these attacks are underway but quickly discovered that the volume of email can be pretty hefty. I recommend just killing the request - abort it at the top of your application prior to the application being instantiated. Then at least you have kept it from filling up your error log. Meanwhile this round of attacks has had the positive affect of causing folks to suddenly pay attention to a great deal of vulnerable code. Here's another silver lining you may not have considered...