Coldfusion, JSON, and the Amazon CloudSearch API

[RedHaven]

Hi all,

We are using Amazon CloudSearch service for our site search.  We created a front end which uses jquery to communicate with the server to display search results.

The problem we are running into is with creating and updating data on the backend.  We have been creating new domains and uploading whole data sets manually from spreadsheets which is obviously not going to work for us long term.

The cloud search uses has an API that we can send JSON to for updating the items.  I have created the nested json like this (the query name and data has been changed to protect the innocent):

<cfscript>

  variables.dataFields = {};

          variables.dataFields['id'] = " #getsdata.id#";

          variables.dataFields['lang'] = "en";

          variables.dataFields['type'] = "add";

          variables.dataFields['version'] = " #DateFormat(Now(),"YYMMDD")##TimeFormat(Now(),"HHMM")#";

  variables.dataFields['fields'] = {};   

  variables.dataFields.fields['redfield'] = "#getsdata.redfield#";

          variables.dataFields.fields['blackfield'] = "#getdata.blackfield#";

  variables.dataFields.fields['purplefield'] = "#getdata.purplefield#";

variables.dataFields = serializejson(variables.dataFields);

</cfscript>

When I dump that json out, it looks pretty good (there are many more fields).  For now, I added a space before numbers to work around a bug I read about in the serializedjson function in CF8 that sees all numbers as numbers even if they are expressed as strings.  I will figure that out later.

The code to post the json is:

<cfhttp url="http://doc-ourcompanydomain-abunchofnumbers.us-west-2.cloudsearch.amazonaws.com/2011-02-01/documents..." method="post"  result="httpResp">

  <cfhttpparam type="header" name="Content-Type" value="application/json"  />

  <cfhttpparam type="header" name="Accept" value="application/json" /> 

  <cfhttpparam type="header" name="Content-Length" value="#Len(variables.dataFields)#" />

  <cfhttpparam type="body" value="#variables.dataFields#" />

  </cfhttp>

The response I get is "400 Bad Request"

I know that there aren't that many people using cloud search and Coldfusion but I was hoping somebody might see a noob mistake in my code or methods.

Thanks!

Red

[pete_freitag]

Hi Red,  I haven't worked with CloudSearch, but have worked with other AWS services, I think you probably need to authenticate your request with some credentials. Checkout this project for some sample code: http://awsconsole.riaforge.org/

[RedHaven]

Hi Peter,

Thank you for your response.  For some reason, I was not able to reply to my own post.  Maybe I just wasn't signed in.

Anyway, the problem was with the json that I was uploading and not with my method for uploading it.  I was actually able to remove the "Accept" and "Content-Length" params.

I had added a space in front of the ID so that the serializedjson function would treat it as a string and not a number but the API required that field start with an alphanumeric character.  Since I am using the sku number for that product, I just changed it to sku12345 etc so that it was seen by the serializedjson function as a string and the API would accept it.

Also, I had to add brackets around the enitre body because the Cloudsearch API wanted the entry to be an array so I just added:

<cfset theuploaddata = "[" & variables.dataFields & "]">

Once I made those changes, the server responded with a 200 code and updated the item.  Now I just need to figure out how to get arrays past the CF json tag.  Some of our json is set up for the cloudsearch in arrays like colors.  The facets need to appear in the upload as "Colors" : ["Red","Blue"] but the serialized json tag automatically escapes the quotes in the facets.  I think I saw some third party custom tags that I can use as a work around. 

Thanks again for your help.  The Cloudsearch service has been awesome so far.  We are really happy with it.  We are a small to midsized company and the cost of a dedicated search company was 10-15 as much as the cloud search and they basically wanted total control of the site.  The cloudsearch required a considerable amount of work to set up but it works beautifully.  Since we use it for navigation too, I have been thinking about caching the json for the common requests.

Red

[pete_freitag]

Good to know about CloudSearch, thanks Red. -pete