USA: (978) 496-9990
Germany: +49 7031 221461
UK: +44 207 193 1212

Blog

Tom Meets Fusion Debug – Week 2

First Debugging Session

We left off last week just after I had installed the FusionDebug plug-in to Eclipse. I was surprised at how quick it was to install FusionDebug.  Within just a few minutes I was ready to debug my CFML application on a ColdFusion 9 developer server.

At the start of this week, I was issued a new project to create a ColdFusion application that interacts with the Harvest time tracking software. Harvest provides an API that is well documented for PHP, Ruby, Pearl etc. but not for ColdFusion!  All that is provided is a simple URL based API that returns XML data once a request has been made.  I know I can use CFHTTP to send the request, but the structure of the XML returned is a mystery.

I began my investigation by using the built-in debugging feature <CFDUMP>. By passing the parsed XML to CFDUMP it presents you with the XML structure and the data within it, this is what I got.



I am a web developer; I like to see my XML looking like XML. I don’t want to see a formatted table, I want to see raw XML so I can easily use the data.

I thought about how I could get the raw XML. I didn’t want to write anymore CFML code.  As readers from last week will know, I am new to CFML and still learning, so I don’t feel confident creating code to print out this XML to the screen. It also seems like a waste of time as the application does not require the raw XML to be printed to the screen. So, I fire up FusionDebug.

I discovered this by accident, but I am glad I did! By pausing the application just after you make the CFHTTP query you can actually see all the elements that the request returns! You can see in the image below where I positioned my breakpoint.

As soon as the breakpoint is hit in the code, FusionDebug displays all the variables currently set. When you make a CFHTTP request this creates a variable called CFHHTP.  CFHHTP contains information about the request that has been made, including “Filecontent”. This is where the raw XML I was looking for is stored. Below you can see that by selecting “Filecontent” Eclipse displays the XML that was returned.

Now I know the exact structure of the XML without any ColdFusion formatting making things hard to read. FusionDebug has proven itself to me as a learning tool. I can use FusionDebug to work with API data I don’t understand. It helped me to learn how the XML data was structured and how I can use that to perform my required tasks. Over the last week I have used FusionDebug on several occasions.

Next week I will discuss some even more inventive ways of using FusionDebug to make life as a CFML developer easier.

Twitter: http://twitter.com/FusionDebug
Facebook: http://on.fb.me/cvYlCp
Youtube: http://www.youtube.com/fusiondebug

Tom Meets FusionDebug

The First Encounter

As the new web guy here at Intergral it is my job to maintain our public sites as well as our internal system known as FusionOrders.  Our system is written in ColdFusion, so the Team recommended that I install and use FusionDebug.  I have never used a code debugger, why would I need one?! We are all super coders, we don’t need such things!

Over the next few weeks I will be installing, using and learning why I need a CFML step debugger. At the same time I will be evaluating the FusionDebug user experience.  If I can’t find the information online, you can’t either!  Difference is I have access to do what I like to the website. If the process is not fluid, I’m going to change it!

Today I took some time to install and configure FusionDebug with Eclipse, this is the primary IDE used here at Intergral to develop our products.
As you may or may not know, there are a two ways you can install FusionDebug.  I tried both – and immediately became a fan of installing FusionDebug by using the Eclipse plug-in manager – which I will explain below.  For our next version release (coming soon!), I have recommended we provide this download option as the primary way to download and install FusionDebug. I will remove the complete IDE installer as I found it confusing.

Step by Step as I Install FusionDebug

I boot up Eclipse after installing it and begin to read, to find that the installation instructions on the FusionDebug website are not updated for the latest version of Eclipse. These instructions will soon be updated on the FusionDebug website!
After spending a few moments trying to figure out what is going on, I finally work out that the new version of Eclipse has a new name for the plug-in manger.  I added the FusionDebug update site to my new package installer interface and hit next, everything went great. FusionDebug is now installed and ready to go!
The installation of FusionDebug using the Eclipse Update Site is really simple; it takes less than 5minutes

Next week I’ll give you some insight into what I think after my first week of CFML step debugging.

Let me know what you think!

Do you find the FusionDebug install easy or challenging? Let me know on Twitter, Facebook or just reply to this post.

Twitter: http://twitter.com/FusionDebug
Facebook: http://on.fb.me/cvYlCp
Youtube: http://www.youtube.com/fusiondebug

Why you should track issues… forever!

An interesting issue cropped up today which involved taking a trip back in time to look at our old issue tracking software. Back in 2004 we were using a Windows client/server based system which had a MS SQL back-end. From day 1 employees are trained to track track and track again. Emails, calls, suggestions – it all gets tracked. Added to that, most new employees – even senior engineers – get started on support duties. This gets them familiar with their environments, customers and our software – plus familiarity with tracking everything. This means we have a LOT of knowledge built up in our ticket-base.

One of our products at this time (used by some of the worlds largest corporations – HP, Philips, etc) was called “Tornado” – this evolved into a product called ShareDox ( http://www.sharedox.com ). The product is a knowledge management solution built on ColdFusion technology.

At the time there were no monitoring tools and thus this led to the now leading monitoring solution – FusionReactor ( http://www.fusion-reactor.com ). So, during the upgrade process to CFMX 6.1 with one of our customers, we started seeing huge CPU usage, hanging threads and various other nasties. At the time this was all quite serious and a major thorn in our sides. Eventually (I believe with some help from our CTO but don’t quote me on that – this was several years ago!) this got resolved and all was calm again. Until today!

Fast-forward 6 years, a move to JIRA and some changes in our business focus – ie our old tracking system is just a distant memory. I started working with a client and getting very distinct feelings of de-ja-vu… CFMX6.1, MS-SQL DB, high CPU, multiple failures per day, hanging threads – essentially a whole heap of stability issues.

Now, I knew I’d seen this before. I knew it was something to do with problematic DB drivers. What I couldn’t remember is how to solve the issue. As you can imagine from a company that’s been doing ColdFusion consulting for over ten years that brought back a lot of issues. A little bit of date filtering and some extra keywords and… result #1 of #5 “ColdFusion MX 6.1: Updated DataDirect drivers for 100% CPU utilization and other issues”.

Of course there are many questions here… why is the customer still on CFMX6.1 amongst others. However, my real point is that tracking is your saviour. In a consulting company like ours we’re truly able to assist more rapidly to a huge variety of issues because generally – we’ve seen it all before. It’s very common for us to have identified, resolved and documented the problem you’re having. This allows us to give you the best value for money on your consulting investment.

If knowledge is power and the key to results – you want us on your team! For all your consulting needs – whether issues from 2004 or today, we can help, contact us now.

Cumulative Hotfix 1 for ColdFusion 9.0.1

Remember that now you’ve updated to ColdFusion 9.0.1 (aka Updater 1) there’s a Cumulative Hotfix (CHF) to apply too:

http://kb2.adobe.com/cps/862/cpsid_86263.html

If you’re looking for assistance in keeping your servers up-to-date then our engineers can help you – just get in touch.

Hello from CFUnited 2010

A big hello from CFUnited!

Myself (David Stockton) and my colleagues (Darren Pywell & David Tattersall) are all at CFUnited this week. We have a FREE copy of FusionAnalytics to give away so don’t be shy – come and speak to us for a chance to win!!

Plus, ask nicely and we’ve got FREE goodies for everyone we speak to!

We’re in the vendor area between Adobe & Railo – look for the ShareDox, FusionReactor, FusionAnalytics & Intergral banners… plus this good looking guy:

See you there!

ColdFusion Update 1 – 9.0.1

ColdFusion 9.0.1 has been released for about a week now. I’m sure everyone has done due diligence in their test environments, run a full test-suite and deployed to production right?

http://www.adobe.com/support/coldfusion/downloads_updates.html#cf9

Well it’s not always that simple is it? So if you want some professional help from the experts then call the experts – we’re ready & waiting to help.

Identifying slow upload connections with FusionReactor Crash Protection emails

We consult with a lot of different types of company. Sometimes there’s a lot of security process to deal with. This is great when you’re trying to stop un-authorized access but can sometimes hamper the speed of response an outside agency can give.

In one such incident we were trying to identify the IP address of a slow uploading client – this we could then link to a client account and identify where the issue was coming from. At the first stage we weren’t able to access any of the remote clients network. Using good old email the client was able to send me a copy of all the FusionReactor Crash Protection alerts. These fire under certain conditions alerting the recipient of a potential issue. You can read more about crash protection on the FusionReactor website.

Now the emails are a great feature but they’re not very easy to analyse over 100’s of emails. So we created a quick tool to analyse the crash protection emails for just this sort of event. And now we’re making it available to you… FREE!

FusionReactor CrashProtection EMail Analyzer

  1. Download & unzip the contents of the download into a folder under your ColdFusion webroot (eg c:\inetpub\wwwroot\fr-mail-analyzer)
  2. Create a new folder called “mails” under this folder (eg c:\inetpub\wwwroot\fr-mail-analyzer\mails)
  3. Put all your *.eml files inside the mails folder – I recommend naming them 01FusionReactor Crash Protection Alert [xxxxx-y], 01FusionReactor Crash Protection Alert [xxxxx-y], 02FusionReactor Crash Protection Alert [xxxxx-y], etc
  4. Open your web-browser and point it at the “read.cfm” file

Now you have a list of all the slow pages (over 60seconds) and which IPs they’ve come from and which page(s) they’ve hit – all without direct access to FusionReactor. Also great if you’ve only got access to the emails (eg your logs have rotated).

Phew – Sound like too much work? Save the hassle and get FusionAnalytics or contact us now!

HowTo – Get the most out of your FusionReactor purchase

Our colleagues at FusionReactor have published some great – quick-start HowTo guides. There’s information on attacking problems from two stand-points (pro-actively and re-actively) aswell as a user-contributed user-guide. You can check them out here:

http://www.fusion-reactor.com/fr/howto/

… or read the official documentation here:

http://www.fusion-reactor.com/fr/support.cfm#doc

You may also be interested in our training webinars (which get some great feedback!) – for more details, please check the FusionReactor site:

http://www.fusion-reactor.com/support/training/

Killing Rogue Requests – Going native, don’t stop me now!

FusionReactor is a great monitoring tool and one of my favorite features is the ability to kill rogue requests. FusionReactor is sometimes limited by Java itself. Java has a known limitation that threads running “Native Code” can’t be killed (until the thread returns from the native code block).

What is Native Code?

Underlying all your ColdFusion goodness is Java, underlying the Java is the runtime environment typically implemented in C/C++ code. When you hit a code-block that must “go native” this is inside the C/C++ code typically waiting for an event to occur. When a thread is executing this native method the thread cannot be killed by the JVM.

What to look for?

Some of the most common examples where native code is used are:

  • CFHTTP calls
  • WebService calls
  • JDBC Queries

What you’re looking for is “Native Method” in the stack trace of the thread. Let’s look at some concrete examples…

CFHTTP Calls

Example CF Code:

<cfhttp url="http://localhost/blogs/dont_stop_me_now/slow.cfm" />

Example Java Stack Trace (available from FusionReactor):

java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)[Native Method]
java.net.SocketInputStream.read(SocketInputStream.java:129)
HTTPClient.BufferedInputStream.fillBuff(BufferedInputStream.java:172)
HTTPClient.BufferedInputStream.read(BufferedInputStream.java:110)
HTTPClient.StreamDemultiplexor.read(StreamDemultiplexor.java:273)
HTTPClient.RespInputStream.read(RespInputStream.java:155)
HTTPClient.RespInputStream.read(RespInputStream.java:115)
HTTPClient.Response.readResponseHeaders(Response.java:1000)
HTTPClient.Response.getHeaders(Response.java:720)
HTTPClient.Response.getStatusCode(Response.java:259)
HTTPClient.RetryModule.responsePhase1Handler(RetryModule.java:83)
HTTPClient.HTTPResponse.handleResponse(HTTPResponse.java:761)
HTTPClient.HTTPResponse.getStatusCode(HTTPResponse.java:191)
coldfusion.tagext.net.HttpTag.connHelper(HttpTag.java:850)
coldfusion.tagext.net.HttpTag.doEndTag(HttpTag.java:1140)
cfslow_cfhttp2ecfm1758959420.runPage(C:\inetpub\wwwroot\blogs\dont_stop_me_now\slow_cfhttp.cfm:1)
coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:231)

WebService Calls

Example CF Code:

<cfset ws = createObject("webservice", "http://localhost/blogs/dont_stop_me_now/slow.cfc?wsdl") />
<cfset ws.goSlow() />

Example Java Stack Trace (available from FusionReactor):

java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)[Native Method]
java.net.SocketInputStream.read(SocketInputStream.java:129)
java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
java.io.BufferedInputStream.read(BufferedInputStream.java:237)
org.apache.axis.transport.http.HTTPSender.readHeadersFromSocket(HTTPSender.java:581)
org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:142)
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
org.apache.axis.client.Call.invokeEngine(Call.java:2765)
org.apache.axis.client.Call.invoke(Call.java:2748)
org.apache.axis.client.Call.invoke(Call.java:2424)
org.apache.axis.client.Call.invoke(Call.java:2347)
org.apache.axis.client.Call.invoke(Call.java:1804)
blogs.dont_stop_me_now.SlowCfcSoapBindingStub.goSlow(SlowCfcSoapBindingStub.java:157)
sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:???)[Native Method]
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
coldfusion.xml.rpc.ServiceProxy.invokeImpl(ServiceProxy.java:224)
coldfusion.xml.rpc.ServiceProxy.invoke(ServiceProxy.java:154)
coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:2360)
cfslow_ws2ecfm1005604111.runPage(C:\inetpub\wwwroot\blogs\dont_stop_me_now\slow_ws.cfm:2)
coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:231)

JDBC Queries

Example CF Code:

<cfquery name="wait" datasource="test">
   SELECT 1 waitfor delay '000:00:10:000'
</cfquery>

Example Java Stack Trace (available from FusionReactor):

java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)[Native Method]
java.net.SocketInputStream.read(SocketInputStream.java:129)
macromedia.jdbc.sqlserver.SQLServerByteOrderedDataReader.makeMoreDataAvailable(null:???)
macromedia.jdbc.sqlserver.SQLServerByteOrderedDataReader.receive(null:???)
macromedia.jdbc.sqlserver.tds.TDSExecuteRequest.submitRequest(null:???)
macromedia.jdbc.sqlserver.tds.TDSRequest.execute(null:???)
macromedia.jdbc.sqlserver.SQLServerImplStatement.execute(null:???)
macromedia.jdbc.sqlserverbase.BaseStatement.commonExecute(null:???)
macromedia.jdbc.sqlserverbase.BaseStatement.executeInternal(null:???)
macromedia.jdbc.sqlserverbase.BaseStatement.execute(null:???)
coldfusion.server.j2ee.sql.JRunStatement.execute(JRunStatement.java:348)
coldfusion.sql.Executive.executeQuery(Executive.java:1229)
coldfusion.sql.Executive.executeQuery(Executive.java:1008)
coldfusion.sql.Executive.executeQuery(Executive.java:939)
coldfusion.sql.SqlImpl.execute(SqlImpl.java:341)
coldfusion.tagext.sql.QueryTag.executeQuery(QueryTag.java:843)
coldfusion.tagext.sql.QueryTag.doEndTag(QueryTag.java:533)
cfslow_db2ecfm445915345.runPage(C:\inetpub\wwwroot\blogs\dont_stop_me_now\slow_db.cfm:1)
coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:231)

Why!?

All these examples are in native methods for socket reading. Socket functions (both reading and writing) are the most commonly found native methods in stack traces.

What can I do?

Unfortunately, the only current work-around is to restart your server. But this is a Java limitation that even without FusionReactor you would still have the problem – FusionReactor is just giving you visibility. The real solution is to investigate the root cause of the problem and solve that – that’s where we come in! We’re experts in this field and working on issues like this on a daily basis – give us a call!

FusionReactor Crash Protection – Regular Expressions “HowTo”

FusionReactor – the leading ColdFusion server monitoring software – has a nifty Crash Protection feature allowing it to abort requests that take too long. This works in a similar way to the ColdFusion server page timeouts but at a lower level allowing FusionReactor to stop requests under many more circumstances. FusionReactor also gives you the options not to abort the request, but just to email you a stack trace of the slow running page. There are several forms of crash protection FusionReactor provides but I won’t get in to those just now – take a look at the FusionReactor site for more information ( http://www.fusion-reactor.com/fr/featurefocus/crashprotection.cfm ).

One of the ways in which FusionReactor timeout protection is better is the ability to configure include (or exclude) lists of page URLs. This can be done with regular expressions. Let’s look at a couple of examples…

First of all we need to imagine our directory layout:

  • wwwroot
    • public
      • my_first_page.cfm
      • my_second_page.cfm
      • my_third_page.cfm
      • my_fourth_page.cfm
    • scheduled_tasks
      • task1.cfm
      • task2.cfm
      • task3.cfm

In our examples, we’ll assume we’re looking at timeout protection and the crash protection settings are all configured for all URLs.

Example 1

Let’s look at exlcluding everything inside the “scheduled_tasks” folder. The first step is to ensure the restrictions are “enabled” and the behaviour mode is to “ignore matching requests”:

Next, we add a new regular expression RegEx for the exclusion:

You can see the RegEx matches on path only (unless you choose to include hostname). Additionally it optionally matches URL parameters and can exclude URL from statistics (eg average request time).

The RegEx’s are standard Java patterns. The online help describes some examples which are available from the FR interface from your server or on the FusionReactor website – http://www.fusion-reactor.com/fr/help/help.htm#creating_a_regular_expression_exclusion.htm

The Java (1.4.2) Pattern docs are available here: http://java.sun.com/j2se/1.4.2/docs/api/java/util/regex/Pattern.html

Example 2

Exclude the public files “my_…._page.cfm” where “….” does not include the character “t”…

/public/my_[^t]*_page.cfm

[^t] = any character except “t”

* = any number of the previous matching group (ie [^t])

Hopefully this gets you on the way to configuring not only your crash protection but excluding your scheduled tasks from server level statistics so you get a better idea of the stats for public facing traffic. We can offer a lot of help and advice and have a wide range of consulting & development services available to assist you no matter the project size.