Coldfusion and XML – an introduction

I was doing a bit of investigation into XML in ColdFusion. This is pretty simple stuff, but I didn’t really see any good information about this on the web, so thought it would be worth blogging, as much for my future reference as anyone else’s.

<CFSaveContent> versus <CFXML>

There are various ways of creating XML in ColdFusion. The two simplest ways are cfsavecontent and cfxml.
Method 1 – CFSaveContent:

<cfsavecontent variable="myXML_1">
<country>
	<name>United Kingdom</name>
	<population>61,113,205</population>
	<capital>London</capital>
</country>
</cfsavecontent>

Method 2 – CFXML:

<cfxml variable="myXML_2">
<country>
	<name>United States</name>
	<population>307,212,123</population>
	<capital>Washington, DC</capital>
</country>
</cfxml>

Look almost identical, right? There’s a difference though. CFSaveContent returns a string. CFXML returns a ColdFusion XML object. What’s the difference? Firstly, try outputting both variables.

Variable 1:

<cfoutput>
#HTMLEditFormat(myXML_1)#
</cfoutput>

<country> <name>United Kingdom</name> <population>61,113,205</population> <capital>London</capital> </country>

Variable 2:

<cfoutput>
#HTMLEditFormat(myXML_2)#
</cfoutput>

<?xml version=”1.0″ encoding=”UTF-8″?> <country> <name>United States</name> <population>307,212,123</population> <capital>Washington, DC</capital> </country>

As you can see, the CFXML example has got an XML declaration prepended onto it ( <?xml version=”1.0″ encoding=”UTF-8″?> ). If you were using CFSaveContent you’d usually have to add this in yourself. But that’s not the only difference. This time try dumping out the variables:

<cfdump var="#myXML_1#">

<country> <name>United Kingdom</name> <population>61,113,205</population> <capital>London</capital> </country>

Dumping the first variable is identical to outputting it, because all you have is a string.

<cfdump var="#myXML_2#">

Dumping out the variable created with CFXML displays a bit like a structure. [If you click the part that says "xml document" it changes to the 'long version'. Clicking it again hides the cfdump content.]

So we’ve established the difference, but what advantage does one method have over the other? Because the first variable is just a string, you can put anything you like in there, including invalid XML. However if you try doing that with CFXML, it’ll throw an error (which is a good thing, as you don’t want to accidentally be using something invalid, right?)

XMLParse()

Any time you’re working with XML in ColdFusion, you’ll probably come across the XMLParse function. This takes a block of XML and turns it into an XML object. If you used CFSaveContent, then passed it into XMLParse, you’d get back an object identical to what you’d get if you’d just used CFXML.

You can also pass in an XML object to XMLParse, and it won’t change it. In other words, regardless of if your variable came from CFSaveContent or CFXML, calling XMLParse on it will return the same object.

If you’re getting some XML from anywhere other than CFXML, you probably always want to call XMLParse() on it to turn it into an XML object. If the XML is invalid, it’ll throw an error. You can also pass in a path to a DTD or XML Schema to validate it further.

Functions

When you declare a function in ColdFusion, you can specify an optional returntype attribute. This determines what type of variable is returned from your function (or ‘void‘ if nothing is returned).

When your function is returning XML, you can set returntype=”string” or returntype=”XML” (or returntype=”any”, but that’s a whole other discussion).
If your function is using XMLParse or CFXML to create an XML object, it doesn’t matter if you specify string or XML, it will always return as an XML object.
If you’re using CFSaveContent to create your XML as a string, you can still specify either string or XML, and it will return your XML as a string.
So in other words, regardless of which returntype you use, the variable that’s returned is of a type that matches how it’s created (a string if using CFSaveContent; an XML object if using CFXML).

However if you specify returntype=”XML”, it has to be valid XML, or it will throw an error.

If your function is returning XML to anything other than ColdFusion, e.g. as a webservice, then you want to return it as a string, not an XML object. I’d recommend still using returntype=”XML”, and CFXML to create it (therefore forcing you to create valid XML), but then call ToString() on it when returning.

You probably also want to specify output=”false” (which is good practice most of the time anyway) to avoid getting any extra whitespace appearing at the start of your XML object.

Arguments

Similarly, if you are passing in some XML as a function argument, you can specify type=”string” or type=”XML”. Again, regardless of whether your XML came from CFSaveContent (or elsewhere) as a string, or CFXML / XMLParse() as an XML object, either type=”string” or type=”XML” will be fine. However if you specify type=”XML”, the argument must be valid XML.

If you specify type=”XML”, but it’s an optional argument, I’d recommend doing it like so:

<cfargument name="myXML" type="XML" required="false">

Typically you often see in code default=”" on non-required cfarguments. However it’s not best practice, and if using type=”XML” will thrown an error. If you really need to specify the default attribute, put in a valid bit of XML, e.g.

<cfargument name="myXML" type="XML" required="false" default="<foo>bar</foo>">

Summary

• Use CFXML instead of CFSaveContent; you won’t have to specify an XML declaration, and it will only work with a valid XML packet
• CFSaveContent is just a string, so it doesn’t have to be valid XML
• Use type=”xml” on the argument, even if the value you pass in is a string. It will only accept a valid XML packet
• If the value could come from anywhere other than CFXML, use XMLParse() to turn what will be a string into a ColdFusion XML object
• You can use XMLParse() even on a valid XML object (it won’t change it)
• Even with returntype=”string” if you use CFXML it will return a ColdFusion XML object
• If you might be returning data to something other than ColdFusion, you would need to turn it into a string first using ToString()

2009 goal review – summer

I decided this year to publish some personal goals and then review them quarterly. I didn’t manage to do my summer review when I’d originally intended, due to the complication involved in moving from Scotland to London. So here it is now, better late than never.

Experts Exchange : get one more Master ranking, and improve both Coldfusion Markup Language and ColdFusion Application Server to Guru level.
Half-done; I managed to get those two CF zones up to Guru, but don’t have another Master ranking yet. Haven’t been active on Experts Exchange since I decided I was moving to London and realised I’d be too busy over the coming weeks. Might get back into it now though.

Join the gym.
When I moved down here I thought I’d do this, and had a one day trial at a local gym. However I’ve decided not to bother for now. I have been doing a bit of running though, and my goal is to try and average two runs a week.

Start using some kind of source control.
When I set myself this goal, I’d originally thought I’d have to do this for my personal projects, due to little chance of seeing it being implemented at work. However at my new company I use Subversion every day.

Complete 40 of the Project Euler problems.
I completed my original goal to do 25 of these, and revised it to complete 40. At the time I’d done 33, and I haven’t made any progress since then.

Blog at least once a week on average. I’d also like to do more Coldfusion-related blog posts.
When I moved to London I was offline for about six weeks, so I’ve fallen behind on this, but hope to get back on schedule.

Get involved in something open source.
No progress.

Install and start using Railo.
No progress.

Start using MySQL.
No progress.

Police considering Chinese-style spying measures for 2012 Olympics

Worrying article in the Sunday Times: “Spy bugs may be deployed for 2012 Olympics“.

The police have looked at how the Chinese conducted their surveillance for the last Olympics, and are considering re-using some of their ideas. Such as:

• bugging taxis to listen in to any incriminating conversations
• using microchips on tickets and passes to track the movements of athletes, journalists and spectators
• facial recognition CCTV

What I find most frightening is the final line:
“Alan Campbell, the Home Office minister, has revealed that the Home Office is investigating technology that would allow police to halt a vehicle remotely“

Does anyone seriously consider that might be a good idea? Especially in the hands of the Metropolitan Police, or any other police force in this country.

Thanks to OurWorld OurSay for tweeting this originally.

---

Photo from surfstyle’s Flickr stream. Creative Commons license.

How not to do input validation

Here’s a screenshot from a form I was filling in on the TV Licensing website. I’d just entered the date as 9/6/2009 instead of 09/06/2009. Obviously it’s too tricky for them to work out how to pad 1-digit numbers with a leading zero.

This falls into the same category as sites that insist you enter your username in either lower or upper case. Or that you enter your credit card without spaces or hyphens. Or that you format your phone number in a particular way.

It’s all putting extra work on the user for something that could be done automatically server-side with just a few lines of code at most. Not good practice!

Fife Council publishes councillors’ expenses

Fife Council recently published “Councillors expenses for 2008/09″ [sic]. Although this is never going to be as exciting as recent revelations about MPs’ expenses, or even MSPs’ expenses, it is maybe still worth analysing a bit. If you download the Excel file of expenses, you’ll see it’s been formatted for print, with the page header repeated about half-way down, presumably where the page break occurs. For some reason, in Open Office, the council logo is reversed.

Anyway, rather than try and read through a spreadsheet of numbers, I’ve tried graphing some of the data.  Please click on the graphs to see each fullsize.

Salary
Since the last local authority elections in 2007, councillors have received a salary and not just expenses. Heads of committees, the provost, etc. receive a higher salary.

For some reason Margaret Kennedy received a marginally smaller salary than her colleagues; £15,445.24 instead of £15,785.73.

Travel Expenses
This first graph shows all councillors. You can see the long tail effect, with several councillors claiming zero travel expenses. These travel expenses do not include rail, air or taxi fares. The mileage rate is 40p per mile, reduced from 49.3p.

Unfortunately Google Docs saves the image without all names visible. So this second graph shows just the top 30 names.

Gerald McMullan and Tim Brett clearly have higher claims than anyone else, at least £1000 more than the third highest.

For Councillor McMullan to claim £6993.64 in travel expenses, at 40p per mile, he must have driven 17,484.1 miles in a year. I’ve no idea if that is reasonable. He lives in Crossford, approximately 24 miles from Glenrothes. If we divide 17,484.1 by 48, we get 364.252083. In other words, if he was only doing the return drive from his council ward to the council headquarters, he’d have to make that journey every day for a year to justify his travel expense claim. However he must do lots of other mileage besides, e.g. driving round to various locations in his ward. I’d still be interested to find out more about exactly what mileage he claimed for. For example if we could see his daily mileage claims instead of just an annual total.

ICT Expenses
Only 8 out of 78 councillors claimed any ICT expenses. The spreadsheet says “Communication and IT Costs, Mobile Phone Costs and Provision of Council Cars borne by Fife Council is not generally a cost incurred directly by individual Councillors.” I’m therefore unsure what William Walker managed to claim £670 pounds for. If full receipts were published this would clear things up.

Rail / Air / Taxis / Hotels
Less than a third of councillors claimed for this.

You would expect the council Leader Peter Grant to have the highest claim, on the assumption he’s representing the council at national and international events. Second highest is Mike Rumney (listed as Robert Rumney on the spreadsheet), with a claim of £1,241.31. Again, without more information, it’s impossible to tell how this money was spent.

Total expenses
Again, Google Docs produces an image without all names visible:

And here’s the top 30 names. Gerry McMullan clearly with the highest expense claims due almost entirely to his high travel expenses.

Kirkcaldy swimming pool opening hours

Update: I just discovered that Kirkcaldy Swimming Pool actually publish another leaflet with contradictory times to their Opening Times leaflet: Activity Programme 2009 (PDF). I’ve emailed them for clarification. Until I get it, assume the information below is possibly incorrect.

I was trying to work out the times I could go for a swim at Kirkcaldy Swimming Pool. They publish the opening times (PDF) but it’s not immediately clear when is normal swimming and when does that clash with lessons / aquaerobics / family fun / lane swimming.

So I’ve attempted to re-publish them in a different format, so you can hopefully see what’s on at any time. This information will probably get out of date within a few months, especially when they start building the new pool.

Key:

	Normal swimming
	Adult lane swimming
	Swimming lessons
	Aquaerobics
	Family fun session (last Sunday of the month)

	Monday	Tuesday	Wednesday	Thursday	Friday	Saturday	Sunday
8.00
8.15
8.30
8.45
9.00
9.15
9.30
9.45
10.00
10.15
10.30
10.45
11.00
11.15
11.30
11.45
12.00
12.15
12.30
12.45
13.00
13.15
13.30
13.45
14.00
14.15
14.30
14.45
15.00
15.15
15.30
15.45
16.00
16.15
16.30
16.45
17.00
17.15
17.30
17.45
18.00
18.15
18.30
18.45
19.00
19.15
19.30
19.45

The worst thing in the world

I don’t know what your opinion is of torture, but personally I’m against it. The US Government has published some secret memos detailing the legal justification for their torture enhanced interrogation techniques used by the CIA in the ‘war on terror‘. Mostly these are just self-serving legalistic jargon explaining why the end justifies the means.

The techniques range from relatively mild (slaps to the face or abdomen, shoving captives against a wall) to the extreme (sleep deprivation, prolonged stress positions, water boarding).

The first of these memos (PDF) has the following section:

In addition to using the confinement boxes alone, you also would like to introduce an insect into one of the boxes with Zubaydah. As we understand it, you plan to inform Zubaydah that you are going to place a stinging insect into the box, but you will actually place a harmless insect in the box, such as a caterpillar. If you do so, to ensure that you are outside the predicate act requirement, you must inform him that the insects will not have a sting that would produce death or severe pain. If, however, you were to place the insect in the box without informing him that you are doing so, then, in order to not commit a predicate act, you should not affirmatively lead him to believe that any insect is present which has a sting that could produce severe pain or suffering or even cause his death. [censored] so long as you take either of the approaches we have described, the insect’s placement in the box would not constitute a threat of severe physical pain or suffering to a reasonable person in his position. An individual placed in a box, even an individual with a fear of insects, would not reasonably feel threatened with severe physical pain or suffering if a caterpillar was placed in the box.

Does that remind you of anything? The following excerpts are from George Orwell’s Nineteen Eighty-Four:

‘The worst thing in the world,’ said O’Brien, ‘varies from individual to individual. It may be burial alive, or death by fire, or by drowning, or by impalement, or fifty other deaths. There are cases where it is some quite trivial thing, not even fatal.’
…
‘In your case,’ said O’Brien, ‘the worst thing in the world happens to be rats.’
…
‘By itself,’ he said, ‘pain is not always enough. There are occasions when a human being will stand out against pain, even to the point of death. But for everyone there is something unendurable–something that cannot be contemplated. Courage and cowardice are not involved. If you are falling from a height it is not cowardly to clutch at a rope. If you have come up from deep water it is not cowardly to fill your lungs with air. It is merely an instinct which cannot be destroyed. It is the same with the rats. For you, they are unendurable. They are a form of pressure that you cannot withstand, even if you wished to. You will do what is required of you.’

CFSchedule bug

In a follow-up to yesterday’s post about a bug with scheduled tasks in ColdFusion Administrator, I decided to see if the same bug existed if you used the <cfschedule> tag.

And I can confirm it does, at least on Adobe ColdFusion 8 developer edition.

This code replicates it. Setting a one-time only scheduled task with a start date/time in the past. The task runs immediately.

<cfschedule
	action="UPDATE"
	task="TestTask"
	operation="HTTPRequest"
	startDate="#CreateDate(2009, 04, 01)#"
	startTime="21:00"
	url="http://127.0.0.1:8500/TestTask.cfm"
	interval="once">

And similar to the problem with scheduled tasks in CF Administrator; if you create a one-time scheduled task to run in the future, then update it to be in the past, it will run immediately.

I’ve reported this as a separate bug.

ColdFusion scheduled task bug

I just discovered this bug with scheduled tasks in ColdFusion Administrator today. I think I’ve encountered it before but didn’t realise at the time what had happened.

I was updating a scheduled task to run in the future. I then decided to just run it manually instead. To make sure the scheduled task didn’t run as I’d originally planned, I just put the date into the past. I didn’t want to delete the scheduled task as it’s one that is run occasionally as and when required.

What I found was that when I edited the scheduled task to be in the past, it actually ran immediately! If you edit it again to another date in the past, it’ll run again. Luckily the scheduled task I was working on wasn’t that important, but some scheduled tasks (especially one-off ones) might have more serious consequences if they run when you’re not expecting it.

I found two ways to replicate this bug.
Method 1:

1. Create a new scheduled task in CF Administrator.
2. Set it to run one time with a date in the past, say yesterday at 9pm.
3. Save the scheduled task.

Method 2:

1. Create a new scheduled task in CF Administrator.
2. Set it to run one time with a date in the future, say tomorrow at 9pm.
3. Save the scheduled task.
4. Edit the scheduled task, and change the date or time to the past, say 9pm yesterday.

So basically any time you create or update a scheduled task, running ‘One-Time’ but with a date/time in the past, it’ll run immediately!

I noticed this bug in Adobe ColdFusion 7 and 8 Enterprise, and ColdFusion 8 Developer edition. I’d be surprised if it happens in Railo or OpenBlueDragon. Bug reported at Adobe’s Bug Report Form.

Custom 404 page with ColdFusion and IIS

This is one of those techniques that I’ve been using for years, and only just discovered a simple mistake in it. However, I think this mistake might be quite common, so decided to blog about it.

If you’re using ColdFusion and IIS as the webserver, you can either use IIS’s default page for handling 404 (page not found) errors (and any other error type). Or you can specify a page of your own. By specifying your own page it makes it easy for you to have a 404 page that matches the design and functionality of your website.

To do this is simple. In IIS, go to the Properties screen for your website. Go to the Custom Errors tab. There you will see a list of all different types of errors you might generate. Scroll down the list until you find the 404 error.

Select it and click Edit. Change the type to URL, then put in the relative path to your own 404 page.

Simple, right? You’ll see the above method in many ColdFusion articles, and I’ve done it like this for years. However, there’s one vital step being missed out. I only discovered this when trying to register my site for Google Webmaster Tools. You have to verify your site by uploading an HTML file or adding a custom meta tag to an existing page. However it seems the Google bot also checks your 404 page at the same time. I got the following error message:
We’ve detected that your 404 (file not found) error page returns a status of 200 (Success) in the header.

?! To check this, I went to mysite.com/blah, then using the Web Developer Toolbar in Firefox, checked View Response Headers. Which gave me the following information:

Connection: close
Date: Wed, 25 Mar 2009 20:41:55 GMT
Server: Microsoft-IIS/6.0
Content-Type: text/html
Page-Completion-Status: Normal, Normal

200 OK

Oops, that should say 404 not 200! It doesn’t take much to fix it. At the very top of the custom 404 page (before any <doctype> or <html> tag) just add in:

<cfheader statusCode="404" statusText="Not Found">

Checking the Response Header Information again following that change then correctly returns:

Connection: close
Date: Wed, 25 Mar 2009 20:41:30 GMT
Server: Microsoft-IIS/6.0
Content-Type: text/html
Page-Completion-Status: Normal, Normal

404 Not Found

and Google then successfully verifies the website.