This JVM bug seems to be getting some high-level attention in the IT press so I thought I’d lay out the issue where CF is concerned:

History

The bug is in the JVM (it has been since ~2001) and so ColdFusion running on Sun JVMs are affected.
Someone out there has obviously made the link between the same issue happening in PHP and brought this issue to light again ( http://bugs.php.net/bug.php?id=53632 ). There’s a Java related discussion happening here: http://www.exploringbinary.com/java-hangs-when-converting-2-2250738585072012e-308/

How to reproduce

To have the bug show, you must call the parseDouble() method of the java.lang.Double class. There are several ways this can happen. Many people are discussing this as a vulnerability that can be executed at the HTTP header level like so:

Accept-Language: en-us;q=2.2250738585072012e-308

However, this requires a call to HttpServletRequest’s getLocale() method, something that isn’t done trivially on a JRun4, CF 9.0.1 instance (even when calling the ColdFusion function “getLocale()”). Thus, to show this problem, you must do something like…

#GetPageContext().getRequest().getLocale()#

… within your ColdFusion page.

From our experience, a more likely attack could be performed with code like this:

<cfparam name="URL.pageNum" default="1" />
<cfparam name="URL.itemsPerPage" default="10" />
<cfquery name="qProducts" datasource="mysql_dsn">
    SELECT * FROM products
    LIMIT #((URL.pageNum-1) * URL.itemsPerPage) + 1# , #URL.pageNum * URL.itemsPerPage#
</cfquery>

The problem here is “URL.pageNum-1“. This calculation causes a call to parseDouble() behind the scenes which means that if the page were called with “page_name.cfm?pageNum=2.2250738585072012e-308” then the thread would hang in an infinite loop.

What doesn’t show the issue?

Note that in this example, “URL.itemsPerPage” could also cause the issue because it is used in the multiplication calculation. If the variable were not used in any calculations but only output, it would not show the issue. This example does NOT show the problem:

<cfset x = 2.2250738585072012e-308 />
<cfoutput>#x#</cfoutput>

What can you do?

Short term

If you have FusionReactor installed and configured with CrashProtection enabled and configured, the threads can be automatically killed by FusionReactor, saving your server from almost certain failure. To do this, enable Crash Protection and configure a “Request Timeout” value and set it to use the “Abort and Notify” strategy. This will cause requests taking longer than this time to quit – even if they are stuck in the infinite loop bug as in this scenario.

For those of you who are wondering, this is NOT the same as the ColdFusion timeout mechanism and so the ColdFusion page timeout alone will not help you in this scenario.

It’s good practice to have FusionReactor installed and Crash Protection enabled because it can save you from a lot of these issues without you needing to do anything.

Long term

I’m sure Oracle/Sun will offer a new update in due course. However, you can also download the “Java SE Floating Point Updater Tool”:
Download: http://www.oracle.com/technetwork/java/javase/downloads/index.html#fpupdater
Read Me: http://www.oracle.com/technetwork/java/javase/fpupdater-tool-readme-305936.html

Further Help

If you’re in need of help updating your JVM and/or patching it then we can offer assistance in this area from as little as $800. The FusionReactor product is available from as little as $249 and contains a wealth of other features – the majority of which are not covered by the ColdFusion Server Monitor – http://www.fusion-reactor.com/fr/ for more information.

Notes

This article refers to JRun4, CF9 installations. The issue is apparent on a wide variety of Java platforms (we offer consulting for most Java environments) and is more prevalent on Tomcat installations (which includes JBoss).

References

Official security alert (CVE-2010-4476): http://www.oracle.com/technetwork/topics/security/alert-cve-2010-4476-305811.html

Christmas is still a busy time on the web. With new computers for Christmas and days off work there’s plenty of time for Internet users to be out there surfing your site. But what if your site is down? Perhaps then it’s not such a Happy Christmas!

Through a combination of factors – including our geographically distributed team – we can still offer consulting services over the holiday season.

Having said that, those with a pro-active attitude can save themselves some money, time and stress with some simple tips:

• Plan
  • It sounds obvious but releasing code on at 5pm Christmas Eve is probably asking for trouble!
• Test
  • Make sure you’re testing your code – from unit testing to load testing, the tools to help are available for a reason.
• Monitor
  • Keeping an eye on your server is one thing, but if you don’t react then you could be in big trouble.
  • Using a tool like FusionReactor ( http://www.fusion-reactor.com ) can help identify issues before they become a problem.
  • FusionReactor has several features that can help you work pro-actively even when un-attended. Setting up good crash-protection settings can keep you alerted via email or even react to prevent your server failing in the first place.
• When all hope is lost…
  • Sometimes, bad things happen. If your application server fails, you don’t want your users to have to wait 3 days for someone to restart a service. FusionReactor Enterprise edition has a feature called “Enterprise Scripting”. This lets you configure scripts to run when a service fails to respond within given parameters. For example, if your server is polled every 5seconds and fails to respond to 5 polls in a row, you may want to restart the service. What’s more, FusionReactor includes the scripts to do just that – even from a remote machine requiring authentication. Just look inside the <FusionReactor Installation Direction>\etc\cp\ folder where you’ll find a bunch of scripts for *nix and Windows environments along with a helpful Readme.txt – or you can refer to the online help.
  • With prices starting from less than $0.50/day there’s no reason any server should be lonely without FusionReactor this holiday season!

Have a great holiday and we look forward to seeing you in 2011 with our new “jewel in the crown” to the Fusion product suite, FusionAnalytics!

We consult with a lot of different types of company. Sometimes there’s a lot of security process to deal with. This is great when you’re trying to stop un-authorized access but can sometimes hamper the speed of response an outside agency can give.

In one such incident we were trying to identify the IP address of a slow uploading client – this we could then link to a client account and identify where the issue was coming from. At the first stage we weren’t able to access any of the remote clients network. Using good old email the client was able to send me a copy of all the FusionReactor Crash Protection alerts. These fire under certain conditions alerting the recipient of a potential issue. You can read more about crash protection on the FusionReactor website.

Now the emails are a great feature but they’re not very easy to analyse over 100’s of emails. So we created a quick tool to analyse the crash protection emails for just this sort of event. And now we’re making it available to you… FREE!

FusionReactor CrashProtection EMail Analyzer

1. Download & unzip the contents of the download into a folder under your ColdFusion webroot (eg c:\inetpub\wwwroot\fr-mail-analyzer)
2. Create a new folder called “mails” under this folder (eg c:\inetpub\wwwroot\fr-mail-analyzer\mails)
3. Put all your *.eml files inside the mails folder – I recommend naming them 01FusionReactor Crash Protection Alert [xxxxx-y], 01FusionReactor Crash Protection Alert [xxxxx-y], 02FusionReactor Crash Protection Alert [xxxxx-y], etc
4. Open your web-browser and point it at the “read.cfm” file

Now you have a list of all the slow pages (over 60seconds) and which IPs they’ve come from and which page(s) they’ve hit – all without direct access to FusionReactor. Also great if you’ve only got access to the emails (eg your logs have rotated).

Phew – Sound like too much work? Save the hassle and get FusionAnalytics or contact us now!

FusionReactor – the leading ColdFusion server monitoring software – has a nifty Crash Protection feature allowing it to abort requests that take too long. This works in a similar way to the ColdFusion server page timeouts but at a lower level allowing FusionReactor to stop requests under many more circumstances. FusionReactor also gives you the options not to abort the request, but just to email you a stack trace of the slow running page. There are several forms of crash protection FusionReactor provides but I won’t get in to those just now – take a look at the FusionReactor site for more information ( http://www.fusion-reactor.com/fr/featurefocus/crashprotection.cfm ).

One of the ways in which FusionReactor timeout protection is better is the ability to configure include (or exclude) lists of page URLs. This can be done with regular expressions. Let’s look at a couple of examples…

First of all we need to imagine our directory layout:

• wwwroot
  • public
    • my_first_page.cfm
    • my_second_page.cfm
    • my_third_page.cfm
    • my_fourth_page.cfm
  • scheduled_tasks
    • task1.cfm
    • task2.cfm
    • task3.cfm

In our examples, we’ll assume we’re looking at timeout protection and the crash protection settings are all configured for all URLs.

Example 1

Let’s look at exlcluding everything inside the “scheduled_tasks” folder. The first step is to ensure the restrictions are “enabled” and the behaviour mode is to “ignore matching requests”:

Next, we add a new regular expression RegEx for the exclusion:

You can see the RegEx matches on path only (unless you choose to include hostname). Additionally it optionally matches URL parameters and can exclude URL from statistics (eg average request time).

The RegEx’s are standard Java patterns. The online help describes some examples which are available from the FR interface from your server or on the FusionReactor website – http://www.fusion-reactor.com/fr/help/help.htm#creating_a_regular_expression_exclusion.htm

The Java (1.4.2) Pattern docs are available here: http://java.sun.com/j2se/1.4.2/docs/api/java/util/regex/Pattern.html

Example 2

Exclude the public files “my_…._page.cfm” where “….” does not include the character “t”…

/public/my_[^t]*_page.cfm

[^t] = any character except “t”

* = any number of the previous matching group (ie [^t])

Hopefully this gets you on the way to configuring not only your crash protection but excluding your scheduled tasks from server level statistics so you get a better idea of the stats for public facing traffic. We can offer a lot of help and advice and have a wide range of consulting & development services available to assist you no matter the project size.