We are also interested in leveraging Akamai instead of using the DAM to render published assets. We are wondering how other large organizations have built integrations into Akamai that make the process to non-technical Authors as seamless as possible and also accounting for invalidating cache.
Has anyone solved this with 5.5.?
I guess there is custom integration with CQ and Akamai. You might want to get in touch with Adobe Services for this.
I think you meant, You want to use akamai instead of web server (Dispacher) to render ?
There is no "officially supported" hook for Akamai, but it can definitely be used to cache DAM assets (or any content). The thing to watch for is the fact that Akamai caches more aggresively than the dispatcher. For DAM assets that probably don't change often (or at all), this isn't much of an issue. For page content, especially if it is dynamic or specifically targetted, it is more of an issue.
Let me see if I can clarify my question. As we began going through the various use cases of the system, we realized the CQ DAM essentially competes with our CDN network Akamai. While this is a non-issue 1-3 months after launch, it could potentially become a very large issue since sending images directly from our datacenters instead of our CDN decreases the value of the CDN and increases our costs. Of course we could build a non-technical interface to Akamai for our Marketing users; but we lose a good deal of functionality - the Image, Teaser, Multivariate Testing, etc. components all require the DAM.
I’m curious how other large organizations have integrated CQ with their CDN (whether Akamai or not). Do large organizations:
- Simply run the DAM and CDN separately and simply ignore the fact that they aren’t getting the most out of the CDN?
- Completely turn off the DAM and push all assets to the CDN (build custom interface to CDN). How do they make up of the fact that some CQ components rely on the DAM?
- Somehow integrate the two together so that the images are in the DAM during the Workflow – but are actually sent to the CDN during Publish. If so, is the process that during Publish, some Listener rewrites the Page/Template links to point to the CDN instead of the DAM?
- Something else???
Really looking for best practices here as we don’t want to stand something up that becomes as headache months down the line.
Thanks in advance for any assistance anyone can provide.
I think the answer is that most organizations using CDN and CQ together use a pull rather than a push integration. So rather than prepopulating the CDN cache during the publish they have the CDN pull images on request.
I have seen organizations handle the links to the DAM images in several ways:
- Some organizations do use a custom Link Rewriter to add a different hostname in front of items to be cached at HTML generation time. This has one significant hole in it - a page must be either secure or not secure - if pages are sometimes secure and sometime not secure you can't cache the rewritten link effectively. In this cache you have to consider a solution where your rewriter inserts a Apache variable, and then have the web server layer do a variable substitution to add in the correct protocol (http or https).
- Organizations that use the CDN to also cache HTML have the option to just configure caching based on extension or path (because all request to the root hostname go through the CND even if the request isn't cached). This significantly simplfies you CQ implementation, but increases the complexity of your CDN and networking adminstrive tasks.
- In theory (although I have never seen this implemented) you could create a replication agent that would push a DAM asset to your CDN on publish. There would of course be some limitations to this sort of approach. First you would limit your ability to do any dynamic resizing of images. You would also have to have custom link rewriter that would rewrite the image links to point the CDN in publish (if this were a different host name you would have the same secure/non secure issues mentioned in line 1 above).
In all cases you need to consider the impact of cache flushing as well. Akamai specifically has some interesting challenges in terms of writing an effective on demand cache flushing agent, so in general you need to rely on time to life for cache management on Akamai (not a problem for DAM assets - only really a problem if you are caching HTML).
Very interesting responses Orotas, we may have our work cut out for us. We only use Akamai to cache binaries (images, video) and CSS/JS files - all HTML is served by our own webservers. In addition, the domains (hostnames) are in fact different. And finally, it is a small use case, but it is possible that some page(s) can be both secure & non-secure (login to see this promo kinda stuff).
Seems we've hit the difficulty trifecta. Surprised Day/Adobe hasn't published any whitepapers/best practices on this - we can't be the only org that has thought of this.
We are facing similar situation. Did you find a solution to the DAM asset and Akamai usecase? If yes, please share who you addressed it.
We use CQ together with Akamai. We basically point our public DNS records xyz.com and www.xyz.com to the Akamai Edge Server.
We have a origin-www.xyz.com which is the website pointing directly to our datacenter.
We do our product imports over night 2-5 AM and we then do purge the Akamai Cache for our html pages, to get the cache filled again before the user rush starts our Google Search Appliance
indexes the pages right after the Akamai Cache purge.
We use Akamai to deliver all content from CQ (html, pdf a.s.o., DAM) and we are meanwhile on a 80% Akamai / 20% Our Datacenter ratio. Beside the www.xyz.com delivered via Akamai we also have setup a static.xyz.com host setup with different caching rules (expiry date of one week), we use this for the more or less static elements such as js and css and images used within the css. We were able to speed up the page rendering process on the user side as well as we use 2 hosts now to build the website (static.xyz.com and www.xyz.com) this means the browser does double amount of parallel http connections.
It usually takes 30 minutes up to 45 minutes until changes are available on the Akamai network (out authors have been trained to check origin-www instead of www and this works out fine) with our current setup and sure from time (2 years / 7 cases) to time there's strange behaviours which causes us to purge the Akamai cache (which usually takes 7 Minutes) but I hardly could think of our environment without Akamai in place to be honest.