Ok... you've made config changes and tested your connections and performance and you know with the certainty of Rush Limbaugh making a right turn that the server is fully tuned and as lean and mean as possible, but you still have a problem. Now what? Well, it's time to look at the code. Start by enabling slow page logging in the CF admin during a high traffic low performance period. Take the list of templates that are running slow and start digging into them to determine why they are dragging their little cyber feet.
Here are a couple of code related items that you might be looking for.
Coldfusion is so easy and so flexible that Coldfusion developers sometimes develop tunnel vision about it. They start thinking that CF is the best choice for everything - imports, file parsing, FTP batch jobs, batch PDF file generation (ick), automated mass mail, washing dishes, shopping for Mother's day, babysitting, waxing the floors and picking up dog poop in the back yard. The truth is that ColdFusion, like any web application server, is best suited to handle thousands of short requests (i.e. web traffic). It is not terribly great at handling loooong requests that take minutes (and sometimes hours - yes hours) to finish. Now, I have seen tasks that are amazingly complex accomplished in ColdFusion and it can handle the job from a code perspective. But if you really need this sort of functionality then set up a batch server. Don't overload your public server with threads that take up space and have no hope of completing in a reasonable time.
In addition, such requests can often be moved to the database server with amazing results. For example, we had a Coldfusion process that consumed a file of 300,000 records every 5 minutes. By the time the first thread finished it was time for the second one. It was a simply unworkable solution. The same 300k record file using a file driver, DTS and MSSQL task scheduler runs in less than 12 seconds. The lesson is that just because you can do it in ColdFusion doesn't mean you should do it in ColdFusion. Can I just say to all the ColdFusion developers out there - learn to use a cursor for goodness sake!
It's silly I know, but every once and a while a conditional loop is not terribly well thought out and causes problems. Conditional loops (or "while" loops if using cfscript) handle some condition or variable this changed as a part of the inner loop code, as in:
In my years I have seen some query code that would make you run and scream. There are 2 types of this sort of thing. First, there is the programmer who doesn't know how to use the database effectively and he does everything in Coldfusion - counting, summing, grouping, laundry etc. A good example is this popular post aptly titled Funny Coding Errors.
While novice programmers can wreak havoc, their errors are usually pretty easy to tease out and fix. The second class of bad query code is much worse. It comes from folks who know a lot about writing queries. I have seen queries that run on for a few hundred lines and includes numerous in line views, subqueries, function calls, and Cartesian joins. Such queries are often attempts to get work done in a single query instead of 3 or 4. That's a laudable goal, but there is a break point where you are not saving or gaining anything doing it this way. Moreover, extremely complicated SQL should probably be wrapped in a stored procedure. If you run into one of these queries and determine that it is the source of your issue, my advice would be to stock up on caffeine laden drinks and sugary snacks and get comfortable. I would also suggest that you do not try to debug a query like this in Coldfusion. Instead, open your favorite query tool for the DB in question (like query analyzer for example). You will get better error information and can examine execution plans and indexing while you are at it.
When using ColdFusion 8 and the 1.6 version of Java you should be aware that dynamic class loading can cause instability in the JVM. In most cases you can move back to 1.5 without incurring any issues. I'm also aware that recent builds by SUN may have resolved this issue. How do you know if you are using dynamic class loading? Well... chances are if that phrase is a mystery to you, you are not using it. If you are using Model-glue or Coldspring these frameworks use injection techniques that might cause this issue to rise up in an abstract way.
If you make heavy use of CFCs and you happen to stuff them into the application or session scope you might run into this issue with referencing one persistent scoped variable from another. It's complicated, but you can check it out here.
For all my muse readers who spend time slogging through performance issues, would you like to add your tip to the list? Either post a comment or send me an email at firstname.lastname@example.org. I will add another post with "user provided tips". If you are going to write a novel, please send via email and not the comments. I will post a link back to your blog if I like your tip.