ColdFusion Muse

Side-by-side Configuration Error installing CF 11 on Win2008r2

One of my colleagues, Chris Tierney, was installing a pristine copy of ColdFusion 11 on a Windows 2008 server. He followed our standard protocol which is to install the server using the "built in" Web server, then create instances (we typically use multi-server mode) then use wsconfig.exe to connect the instances to IIS. It all went as planned until he tried to run wsconfig.exe (FYI - you must always run this as administrator). He got an error as follows:

java.io.IOException: Cannot run program "C:\Users\ADMINI~1.CFW\AppData\Local\Tem
p\2\\ExecuteAppCmd\ExecuteAppCmd.exe": CreateProcess error=14001, The applicatio
n has failed to start because its side-by-side configuration is incorrect.


Followed by an odd stack trace. After experimenting with permissions and googling he stumbled on Bug 3761543 in the ColdFusion bugbase. The issue is not very well documented. Apparently the MS C++ package installed on 08 is 32 bit. I'm not clear if we installed it or it shipped with 08, but remember, you need the MS C++ 64bit SP1 Redistributable. Here's the download link from Microsoft so you don't have to hunt it down.

The Fix

Microsoft Visual C++ 2008 SP1 Redistributable Package (x64)

One more time: This download fixes the "side-by-side configuration" when installing ColdFusion 11 on Windows 2008r2.

email connection crossover workarounds

As a follow up to yesterday's post (regarding sending mail and having it end up in someone else's "sent" folder) I thought I might put some flesh on the workaround suggested both in the bug report and on CF-Talk. The suggestion is to:

Create a CNAME to point to the SMTP server address so that both websites were looking at different domain names.
This idea is workable up to a point so I thought I would explore it for my readers.

[More]

CFHTTP, IIS 8 and Server Name Indication (SNI)

Guest Post by Wil Genovese

(Muse Introduction)
Most readers know that the Muse is deeply indebted to a large and talented group of developers working here at CF Webtools. These folks solve problems and undertake Herculean programming tasks on a daily basis. They are constantly making me look good and I would not be able to play golf or spend the day wise-cracking in IM and tormenting my assistant Melissa without them on my side. Among these folks is one of my favorite characters, CF guru Wil Genovese. Wil has worked with us for a few years now and he writes an excellent blog at Trunkful.com. If you have not already done so, you should add it to your list of must read blogs.

Meanwhile, a few days ago Wil was trying to troubleshoot a head scratching issue with CFHTTTP and SSL. Now such issues almost always come down to getting the certificates properly installed in the keystore, using the correct URL (correct in all respects for the certificate), name resolution and SSL protocol levels (as in "do you need to lower Java's draconian SSL defaults to allow for less secure protocol"). After beating his head against the wall repeatedly Wil finally decided the issue was on the other end - the certificate on the server was somehow wrong, misconfigured or behaving unexpectedly. I thought this was dubious at best, but as is so often is the case the Muse was wrong and Wil found out (with apologies to Monty Python) something completely different. It turns out a new feature in IIS 8 (Windows Server 2012) was the culprit. Since this setting affects all Java versions prior to 1.7 and even affects CF 10 on Java 1.7, you should probably pay attention. My guess is that you will run into this issue eventually - given the ubiquity of IIS and the coming upgrades to Windows server 2012.

Anyway, I invited Wil to write the following entry detailing his findings. If you want to know more read on:

[More]

Always Check on the Last Thing You Changed

If you can sing this with a sort of smarmy accent like Eric Idle it makes it really pop to the tune of "Always Look on the Bright Side of Life".

Your server's feeling bad,
It can really Make you mad,
JRUN maxed can make you swear and Curse,
When your chewing CFGristle,
Don't Grumble, Give a Whistle
And this will help things turn out for the best

Always check on the last thing you changed
(whistle cheerfully here)
Always check on the last thing you changed.

If CF's being Rotten,
There's something you've forgotten
And that's to check the freaking SVN,
For anything that's newish
Roll it back, don't be bluish
Just pucker up and whistle, that's the thing


Always check on the last thing you changed
(whistle cheerfully here)
Always check on the last thing you changed.


...I'm not sure what was in that mimosa...

Debugging and a Return to Dodge City

One of the things the Muse likes best about ColdFusion is the excellent debug information provided during development. Of course you should never ever leave debugging enabled on a production server. Not only are you generating a great deal of additional data with each request (adding overhead), you are potentially exposing a mother lode of technical information that a nefarious hacker would salivate to see. But during development, the debug information is where you ought to live. Indeed, if you are not constantly checking the debug information start doing it now - make a habit of it! You will learn things about performance, iterations, database interactions, cookies, paths, and all sorts of goodies that will make you a better programmer.

I've had my head buried in the debug information since I started with ColdFusion. Back then (in the Wild West days of CF 4.01) we never heard of newfangled ideas like "cfqueryparam". We just stuffed our variables into queries willy nilly and trusted the good Lord to protect us. It feels like I have spent the last 7 or 8 years cleaning up after code written like that. But writing queries in the raw (unprotected I mean... I don't generally code naked, although I did experiment in college) had one main advantage. As you probably know a lot of debugging goes back to the database. The debug output pre-cfqueryparam was "well formed" query code that could be copied and pasted directly into a query tool like MSSQL studio or Navicat. This made debugging pretty easy. You could swipe a problem query out of the debug, run it and tweak it unit it gave you what you needed, then past it back into CF. But that changed when we all started using CFQUERYPARAM.

[More]

IIS 7 Max Worker Processes and ColdFusion Updated

In my previous post on this topic I indicated that IIS 7 seemed to be a constraining factor. That post lead to conversations with a couple of CF gurus (Charlie Arehart and Russ Michaels) who clued me in to a number of additional settings. If you are truly interested take the time to read the previous post and (especially) the comments before you read this post. What bothered me was that the issue I discovered (a cap on requests) can be affected by both IIS settings or JRUN settings (or both).

My conclusion is that the behavior I was trying to affect is actually the bug that Charlie pointed out to me on Adobe's site (found here). Charlie rightly indicates that this issue is under-recognized (I certainly had not run into yet). The behavior of this bug can be affected (fixed or mitigated) by adjusting IIS as described in my previous post as well as by using the Adobe-provided instructions. This lead to a bit of Muse head-scratching. How do these various processes really work together? This post hopes to clear that up (or at least add to our compendium of knowledge).

[More]

IIS 7 Constraining Simultaneous Requests Limit?

I have been doing some performance testing for a company with a large server farm over the last couple of weeks. Although the farm had 20 or more servers, we started with just one sever to try and get some numbers we could use to extrapolate the overall tolerance of the larger system. The servers were all Windows 2008r2 64bit, IIS 7, running CF 9 enterprise with plenty of RAM. We were also running Fusion Reactor to help introspect ColdFusion.

As I slowly poured on more load I noticed something strange that I had never seen before. Although my "simultaneous requests" setting was set to 48. I could not get ColdFusion to handle more than 25 active connections. Under ordinary circumstances I could pound away at a server using my test framework and get enough requests active to overrun that simultaneous request setting and see the queue kick in. I was trying to max out the server but it was not behaving as expected. Active requests would "cap out" at 25 - as if my simultaneous request setting was set to 25 - but there were never any requests in the request queue. It was a head scratcher - but I kinda love those! Here's the skinny....

(NOTE: The comments on this post are important as well. And this Follow up post clears up some of the confusion.)

[More]

ColdFusion Builder 2 Hotfix 1 Fails

If you have an issue where you are trying to install "hotfix 1" for CF Builder 2 on a Windows 64bit platform and it fails after decompressing the file, you might try the following. Look for the uncompressed files (usually in the temp directory), drill down and find the *.lax file. This is the file that install anywhere (a Java installer) uses to launch its own install procedure. In the file is a path to the JVM - look for the line that says something like lax.nl.current.vm=\\... Then set the path to a known good 64bit JRE SDK. You should have one if you are running eclipse. Once you make the change launch the install exe in the decompressed files folder (instead of the original file). This is a version of the fix found in my previous post on CF 8 64bit on Windows 2003 64bit Web edition. The previous post has all the details and then some. I'm told by CF aficionado Christian N. Abad that the fix works for CF Builder Hotfix 1 installs as well. I suspect as a general rule that it would work for any "install anywhere" process with this specific problem.

Jmeter Part 3: Script Recording and Crawling

This is my last post on the topic of Jmeter testing. I wanted to finish up by showing you 2 things. First, CF Guru Larry Lyons pointed me to a different test tool called "bad boy". This test tool has a good deal of capability in it's own right but the main reason I bring it to your attention is that you can record a test and export it to Jmeter in a few simple steps. Second, as promised in my previous posts (Jmeter testing and Jmeter Distributed testing) I'm going to share a Jmeter script that is able to crawl your site using the magic of regular expressions. First, let's explore Bad Boy.

[More]

Jmeter Part 2: Distributed Testing

In my last post I detailed using Jmeter to set up a simple Web Application test. Jmeter can bring a lot of users to the table with just a single workstation and you can affect some real pressure on your system. But there are times when the resources of the server simply exceed the amount of traffic you send its way. For example:

  • The site is heavily cached or pure HTML and can handle bucket loads of traffic.
  • The server is a huge horse with tons of RAM, fast disks and teamed NICs.
  • You are testing a cluster or load balancer - multiple servers working together for high capacity.
The idea behind distributed testing is to use multiple workstations with a single test plan. Jmeter makes this extremely easy (surprisingly easy considering how difficult it is with other testing products). Here are the details.

[More]

Site Testing With Jmeter

Recently I did a user group presentation on testing an application using Jmeter. Notice that I did not say "load" or "stress" or "performance" testing. There are many different types of testing you can do to your application. All applications can benefit from testing for performance or load - although there is a cost factor involved that can prevent some folks from taking the time to test smaller apps or apps with an expected predictable or lite load (like an internal application for example). Still, it is something that every developer should consider. Generally speaking there are 3 increasing levels of testing that rise "up the scale" in complexity and cost.

  • Performance Testing - To do a performance test you usually you put a reasonable or expected load on an application and examine the internals. By internals I mean memory usage, processor usage, database performance and specific pages that you suspect need tuning. There might be many other things you want to look at here - networking, third party services etc. The goal is to find bottlenecks and improve performance under the expected load. Almost everyone does some form of performance testing - even if it's just loading up a page and looking at the debug info to see what sort of time it takes to run a query or execute a routine. When finished you have a "tuned baseline" - a set of expected benchmarks you can reasonably expect your application to meet (until the database crashes or some data center guy trips over the power cord anyway).
  • Load Testing - Load testing is sort of generically thrown around to mean both performance and stress testing. In reality a true "load test" (at least by the book) means maximum load for a sustained period. In other words, pushing your application to the top of the curve and leaving it there for anywhere from an hour to a couple days to see if it degrades over time. So really it is more like "marathon" testing or "endurance" testing. Load testing teases out things you might not expect - bugs that occur over time in large data sets, crashes that have a threshold or duration attached to them, memory leaks, and timed problems that occur when events conflict (like backups or scans). Most folks do not do any load testing. Load testing is very expensive and time consuming and simply understanding the data set will require experience and patience. If you have invested in a high level application where perforamance and capacity are important long term factors then you should probably be load testing.
  • Stress Testing - The goal of a stress test is to see where your application will finally break, and how it recovers after crashing. In other words you add an increasing amount of requests to your app until it stops functioning, then you see if it recovers. Does it require a hard kill? Is any permanent damage done? Are there corrupt files? Orphaned DB records? Finding out where your system crashes gives you a baseline for capacity planning and allows you to fix issues related to potential crashes.
The following post details just the "minimum" or "basics" of setting up a test with a product called Jmeter (part of the awesome treasure trove of products found in the Apache Foundation vault). You can extrapolate from these instructions to do performance testing right away. With a little extra effort you can set up a stress test as well.

[More]

Preparing for When You Need Me Most

In the new order of things practically no one has a server "in-house" anymore. Unless you are a large company with a heavy IT presence, it is simply easier to rent space at a data center. This works very well for the most part, but one of the nuances of this system is remote access. Most data centers have a number of different methods for providing you with remote access. In some cases they use a VPN like Cisco AnyConnect (which I think is excellent). Once you are connected to the VPN you access the server or servers via an "internal" NAT address - an unroutable address usually beginning with 192 or 10. Then you use something like RDP (remote desktop protocol) to log into the server desktop and configure the server, deploy code or whatever. In other cases the NOC might simply provide open ports to your own static IP for the services you need (like RDP, MSSQL, MySQL etc). This approach opens the NOC up to the your security, but it's certainly better than open ports.

In addition, some hosts may allow these ports to be open and trust the next layer of security to keep the bad guys out. In fact, virtually all hosts offering shared hosting or web panels etc have to offer these open ports of necessity. The Muse hates this idea. Allowing open ports for RDP (3389), SQL (1433), and especially FTP is a bit like Goliath strutting up and down daring the Israelites to challenge him. Meanwhile, all rock throwing David could think is, "how could I possibly miss that guy?" But of course there's more to the story...

[More]

Accessing Both 64bit and 32bit Assemblies

Many readers will doubtless recall the war waged by the Muse against .NET on 64bit ColdFusion 8. If not, you can read all about it in this series of posts on Muse Vs. .NET Integration. It was clear from the outset that using 64bit CF against a 32bit assembly was not working for us. Naturally we went down the path of recompiling everything into 64bit and we had multiple obstacles to overcome. But the fact that we could not communicate with 32bit assemblies always puzzled me. The communication (like most things on a web server) happens through a socket to a listener provided by the integration service (just like Verity and Sequelink). I could not reasonably explain to my own satisfaction why it should be that a 32 bit assembly was inaccessible. After all, our tests using the assembly directly worked fine. I assumed in a vague sort of way that differences in how variables were stored and passed back and forth must be to blame.

A few weeks ago I got a tip from the always insightful Rick Root on CF-Talk - who figured this out with the help of the folks at Just CF (SupportObjective). His problem was different than mine. He had a mixed environment with both 32bit assemblies and 64bit assemblies. In his experimentation he discovered something that solves both our problems. When you call and assembly (any assembly) using .NET integration ColdFusion creates some jar files under the hood. You will find them in the /cfclasses/dotnetproxy folder. Here's a sample folder where the server is using 2 .NET assemblies:



Of course, one of the Jar files that ColdFusion creates is the actual assembly interface which "mirrors" the methods and properties of the .NET interface. It's the one with the cryptic name that looks like "-23480983_3023903.jar". The other file however is special. It's the one that is always named dotNetCoreProxy.jar. It is compiled only once (unless you delete it) on the first time you instantiate a .NET assembly. It provides (presumably) the interface to the proxy listener.

The Fix

Now here's the kicker. If the first assembly you call is a 32bit assembly - in other words, if you call the 32bit framework first - then this little jar file is compiled to access the 32 bit framework and cannot access the 64bit framework. In fact, there's some good evidence that it can't reliably access 32bit DLLs either (it seems error prone). However, if you access a 64bit assembly first then the jar file is compiled in such a way as to be able to access both the 64bit and 32bit frameworks.

Conclusion

To avoid having to run this gauntlet I would suggest accessing a .NET Assembly on the 64 bit framework first and backing up the subsequent dotNetCoreProxy.jar file to avoid future confusion. I'm sure you can find a nice "Hello World" assembly to compile to 64bit. If you have legacy .NET integration code and you are moving from 32bit to 64bit then it will likely not work out of the box until you have done this (or it may work once or twice and then give you inscrutable errors). You will need to get the correct dotNetCoreProxy.jar file compiled (the 64bit version) before you can access your 32 bit assembly dlls successfully and reliably. The good news is that you don't necessarily need to recompile all your assemblies to use the 64bit CLR - although there may be a performance or compatibility reason to do so.

Explained: Error Loading jvm.dll

Recently on CF-Talk I participated with super-gurus Dave Watts, Russ Michaels, Wil Genovese, Jochem Van Dieten and others trying to help Brook Davies overcome a ticklish obstacle. After updating here ColdFusion 8 server with the updater he was unable to start Coldfusion. Instead the /runtime/bin/cfusion-out.log presented him with this unhelpful error message.

"Error loading: C:/ColdFusion8/runtime/jre\bin\server\jvm.dll".
Brook tried different paths and different JVMs - all to no avail. Finally he fell to frantically uninstalling and reinstalling, updating and rolling back and (presumably) consuming large quantities alcohol. The list was helpful with a variety of suggestions. Suggestions ranged from a bit depth mismatch (a 32 bit OS trying to run a 64bit JVM for example), to paths, bad downloads, file I/O, wrong updaters - the whole gamut. But nothing seemed to work.

The Fix

Finally, user "Mack" (a newcomer to me) figured it out. The install in question was using a custom or incorrect version of the msvcr71.dll file. This is a core library on a windows machine. The jvm.dll cannot be instantiated without it - and in this case the version of that file was important as well. Mack copied the newest version into the /runtime/bin directory and the JVM was able to load. There are a couple things to note here. First, I don't now what the exact version should be. My information has Mack copying the dll from a running install into the bin directory and that did the trick. Secondly, you don't need to register the dll or overwrite it in the system32 directory. Apparently Jrun looks first in the /bin directory and then elsewhere - so as long as it's in the bin directory it's cool.

Finally, user Leigh (a princess I presume) noticed these 2 bugs in the Sun bug tracker for 1.6 which could be related.

Now perhaps an enterprising Muse reader will publish the exact version in the comments for us.

Muse Vs. .NET Integration - Part 3 a New Frontier

As we continue our saga against the evil empire our intrepid Jedi, Master Muse, has a new chapter of knowledge to share. So with this post I'm adding a few things you may need to know to work with .NET integration on ColdFusion - in this case ColdFusion 8.01 enterprise in "multi-server" mode. Before I continue it might behoove you to read my previous post on Muse vs. .NET integration Part I and Muse Vs. .NET Integration Part 2. These 2 previous posts provide a blow-by-blow description of issues faced and resolved to get this running. My father used to say "behoove" so I pulled it out of the attic to use here. It means "It might be in your interest" but as a child I always thought it had to do with goats or fauns or something - but I digress.

Here is the latest information we have uncovered in our never ending quest to get .NET integration working smoothly on ColdFusion 8 64bit. Many thanks to fellow guru and very large brained friend Guy Rish for his tireless efforts to uncover some of these items. I'd also like to thank Dennis N for his part, a brilliant engineer in his own right who has kept at this with us till we solve it or die trying. I'd mention his last name but I don't have his permission :) Meanwhile......

[More]

More Entries




Blog provided and hosted by CF Webtools. Blog Sofware by Ray Camden.