Viewing Individual Entry / Main
October 28, 2003
Caching can improve performance by leaps and bounds. How caching is implemented greatly impacts your overall performance. CFCACHE is a good solution and load test show great performance with it when there are only a few files in the cache. As the cache directory fills up performance dropped off quickly. Instead of a file based approach CF_Accelerate provides an in-memory solution.
CF_Accelerate provides the ultimate solution for managing large contents of data in-memory and provides an (easy to use managment interface)* for managing cached entries from the page down to a single entry. CF_SuperCache, a custom tag from the Macromedia Developers Exchange provides a good solution but runs into the same problems as CFCache as the cache fill up. It stores all cached entries in the same server scoped structure and this proves to be inefficient. CF_Accelerate breaks through this barrier by segmenting the cached structure out into a tree of structures this reduces the amount of records to search through when you have cache from many pages and sections of a site. Instead of storing them all in one structure they are broken so each page, section, etc has its own structure to hold the cache.
It can be as simple as just placing the custom tag around some content or an entire page. This allows the content to be cached based uniquely on the page name and url parameters and it is super fast.
In testing on my laptop I created a page with around 14K of data and wrapped it with <cf_accelerate> to cache the content. Next, I created a script to populate the cache in a single node of the cache tree.. In total I added 9000 entries to the tree and ran a load test that invoked the page and retrieved it from cache.. I was able to get 310+ pages/sec (on my laptop!!!! 2GHz P4 512MB Ram) with the in-memory cache. The total memory taken up by CF was 225MB, 9000 cache entries at 14K a piece totals up to around 126MB.. I compared this to CFCache with the same amount of data and I could not even get 7 pg/sec. With memory prices as low as they are it is worth putting your cache in memory. In a production level server it is not uncommon to have 2-4GB of memory, with that amound of memory you are unlimited with your caching abilities.
I have also tested this with customers and yielded the same results. You can dowload CF_Accelerate here.
* I am in the middle of building an interface to manage the cached data
Not that I doubt you (grin), but you make the assertion that retrieving info from a flat struct is slower than a nested struct. Do you have proof of that?
We've got a caching layer in FarCry CMS (http://farcry.daemon.com.au/) -- with an administration interface for cleaning and flushing the in-memory cache. If cf_accelerate is a faster solution, would you mind us incorporating the code into the opensource code base?
Ray I tested this on a 4 CPU Solaris box with 14GB of RAM under load with several hundred virtual users both with a flat struct and a nested struct. The difference really suprised me. Geoff, feel free to integrate it whereever you want.
One addition to Ray's question. I don't think the speed difference occurs during the retrieval process instead it occurs when you do a isdefined() or StructKeyExists() call to see if it exists before you populate it. In a caching tag you have to check if it exists before you pull it out. Otherwise you populate it.
Yes, it would be very helpful. I hate having to switch back into Flash for the debug. Especially annoying since it doesn't even stay on screen when the app doesn't have focus.
Would it be better - or possible - to use named locks (based on the cache keys), rather than application scope locks?
I tend to used named locks because, although you have to be more careful when coding, you can avoid a lot of queuing if you have big updates/reads e.g. If I wanted to generate content offline to store in cache - I'd rather lock on the cache key and update it, as I may do this several hundred times in a minute, than trying to obtain and application scope lock.
Or maybe there's a reason that application scope locks are better.
Anyway, I'll be trying out the tag - great work! I'm also having a look at some of my other large flat structures to see if they would benefit from a tree-like structure.
All the best.
You could use named locks as well. Probably use the scriptname attribute that keys off of the page name. That would narrow the lock down further.
Can you share a bit of info on how you did your tests? I created a large, flat struct, and a very small, but deep struct. I then tested how long isDefined and structKeyExists() took to check the structs, and in my tests, the results were all the same. I'm not seeing one being consistently better than the other.
Ray, Check out the entry http://www.bpurcell.org/blog/index.cfm?mode=entry&entry=964
This tag is VERY cool! Thanks very much for making it available and i'm looking forward to seeing the interface for it.
Brandon, Don't know if you'll answer here but what would be the maximum cachedWithin value you'd recommend for CF_Accelerate - specifically for low volume sites. Would a day be OK? Some hours? Is there a risk that the cache becomes corrupted somehow, so better to keep it at minutes just in case?
Nando, I don't think corruption is a concern at all. I set bpurcell.org to cache most content for 24 hours. It is refreshed if I post a new entry or if someone posts a comment. With respect to your other question (from your email) there really is no easy way to see how much memory is in the cache. One approach would be to add up the total number of characters in each cached entry. It would probably take 20-30 seconds to loop through a large cache and count them up. I provide info on the number of characters in each entry in the admin (http://www.bpurcell.org/cfaccelerate_admin_demo/) but not the total. It is kind of a hack but I used len() function to get the # of characters. Since structs() are java HashMaps there may be some way in the java api to accomplish this.
How did you did the testing? I did some testing myself, on a similair system:
1GB DDR PC2700 of ram AMD Athlon XP2100+ 2x IBM GXP120 120GB in Raid Striping mode ColdFusion Server 5 Ent. Edition
I load tested with Microsoft Web Application Stress Tool, with 200 concurrent users on 2 threads and with 10 minutes of testing with 1 min warmup and 1 minute cooldown.
Caching turned on: Number of hits: 2871 Requests per Second: 4.79
And without cache: Number of hits: 953 Requests per Second: 1.59
Your numbers seem a bit high to me 310/sec although I do believe you. That is a page serve per 3 milliseconds.
My profit was max 580% speed. The minimum execution time I get is 16ms and CF is not dropping beneath that.
Did you make any special tweaks to the CF server?
Micha, If you response time is 3 milliseconds then it sounds like your not throwing enough load at it. What is CPU utilization like on the CF server? You may try bumping up the threads in WAS. I did all of my testing with CFMX but you should see much better numbers than 1.59 without caching. You should at least see in the 30-40 range. I used this tag with a customer, load testing in a production environment and was able to get over 550 page/sec. With CF 5 I think you should expect 75-100 on your current machine. What OS are you using?
I'm trying this out in our dev environment, and it doesn't seem to work with SES urls. That is, if I have 2 distinct detail pages, like
they both seem to cache to the same page. Am I missing something? I'd really love to be able to use this tag -- it seems just perfect for our needs.
Replying to myself here:
Brandon and I had an offline conversation, and indeed the code needed to be (only slightly) modified for SES usage. I've sent the modified code to him, if others would like it as well.
And from my email to him:
> Also, just a few random notes in case others ask you > about these things: > > 1) we're using the fusebox architecture, so the > biggest bang for our buck seems to be to use the tag > in both the act_ and dsp_ files; > > 2) so to get uniqueness for #1, I'm using this > construction: > > <cfset strIDVal = Replace(attributes.object_id, "-", > "", "ALL")> > <cf_accelerate blnSES="yes" > primarykey="act_parks_detail_#variables.strIDVal#" > cachedWithin="#request.dtmAppTimeout#"> > > NOTE: if you're using database-created UUIDs as part > of the primary key, you have to strip out the dashes > -- if you don't, the caching won't happen > (complaints in the accelerate tag), but you might > not know it because the page doesn't actually error > out. > > Hope this helps, and thanks for the tag!
I tested on Windows XP Pro, a development station. I also tried changing threads, but nothing really changed except that the response time per request increases.
Windows XP Pro is not a good test bed for performance. The OS is limited to (I think) 5 simultaneous connections and the packaged version of IIS is limited as well.
If I'm already using Trusted cache and I have a page that is display ed from 6 different templates, am I going to experience any improvements using cf_accelerate over the already cached templates? Thanks, Dave.
Dave, Trusted cache reduces the file IO because it skips the step for checking the .cfm date/time stamp. CF_Accelerate caches content for an entire page or a section of a page. After the first request processes the CFML and renders the content all subsequent requests serve the content directly without processing the CFML (queries, logic,etc...) between the start and end of a tag again. This occurs until the cache expires. A page that takes 900ms to process the first time can be rendered in 10ms for subsequent requests using the tag.
Just wanted to say thanks for making cf_accelerate available. I did some quick load testing with it on my CF5 powered dev box and the results are quite impressive.
help... I got the following error on our CF5 machine:
--------------------------------- Cannot lock application scope CFLock cannot be used to lock the application or session shared scopes without these scopes being established through the use of the CFApplication tag. You must use CFApplication and specify an application name to use the application scope. If you want to use the session scope you need to enable session management as well. Application and/or Session variables must also be enabled in the ColdFusion Administrator.
The error occurred while processing an element with a general identifier of (CFLOCK), occupying document position (168:5) to (168:60) in the template file C:\CFusion\CustomTags\accelerate.cfm.
Charlie, you need to enable sessionmanagement, by using <cfapplication> in Application.cfm.
If you do not now howto use this, refer to the ColdFusion manuals, or references available on the net. These are CFML basics.
Micha, thank you for your assistance. per your suggestion I got CF_accelerate tag working at PARTS411.com now.
I noticed the response time definitely improved greatly the 'SECOND' time a user visited the same pages... However, how can I make it so EVERY user visiting our site can have datas servered from cache the 'FIRST' time?
Our databse is about 600mb in size. Our web server has 1.5gb of RAM.
Thanks in advance to anyone who can help me out!
Charlie, The only way for that to happen is to somehow invoke each of the pages before a user visits them. You could write a CF page that you invoke after CF starts that cycles through and invokes every URL using CFHTTP. I have this functionality built into the CF_Accelerate admin. I have trying to to finish it for months but haven't gotten around to it.
Reading the comments of this topic I read kelly had a solution for SES urls. Kelly said he had a modification to the cf_Accelerate code and he sent you a months ago.
I downloaded the actual code but it seems like that functionality it isn't there.
Do you have it? Do you plan to put in the cf_accelerate? can we have the actual code of this "upgrade"?
thanks in advance,
And thank you so much for the info. you put in your page: it's so valuable for me.
I cannot find the updated version anywhere in my email. It was pretty simple to make the changes though. The main keys of the struct where the cache is stored use CGI.SCRIPT_NAME and CGI.QUERY_STRING. You primarily just need to replace CGI.QUERY_STRING with CGI.PATH_INFO when you are using SES URL's. Let me know if that works.
Sorry if this is a basic question. You mentioned in one of your posts a few weeks ago that You could write a CF page that you invoke after CF starts. How is this possible? Do you simply use application.cfm?
Brandon- This is awesome. Thanks for the tag. Is there a kill switch so to speak that will kill the cache before it is set to expire? In other words, can I tell accelerate to drop existing cache associated with the page and force a recache?
Thanks - Tom
You can do that very easily. There are both NoCache and flushAccelerator parameters that you'll find documented in the source code. I added a customized clearAll to delete the whole cache, as below between the I added this comments. The code below the bottom comment will give you the location in the tag where i placed this:
<!--- I added this here ---> <!--- flush entire cache contents during editing operations ---> <cfif isdefined("url.clearALL")> <cfif isdefined("application.accelerator")>
<cflock scope="application" timeout="5" type="exclusive">
</cflock> </cfif> </cfif> <!--- end I added this here --->
<cfif not attributes.noCACHE><!--- to cache the content or not? --->
<!--- flush based on script name --->
<cfset acceleratorPassword = "true">
Forgive the newbieness of the question, but: when the cf_accelerator application variable expires, is it actually deleted? As in removed from memory? Or is it set to nothing? I'm finding that after a while, cf_accelerate is chewing through my memory and crashes or hangs the cf application.
If you've got UUID's in your urls, or CFID's, then the tag will see those as unique pages, and might cause that behavior.
cfdump "application.accelerator" and you'll get a sense of what's going on in your case. you probably should also look at your cachedWithin setting. You can keep that at an hour or less and still get 98% of the benefit on high traffic pages, and keep ones that aren't viewed often out of the cache.
I wanted to address a few questions that have been asked recently
1. Killing existing cache - right now this is an all or nothing approach in the tag. (admin solves this) 2. Cache expiration - When the cache expires it is not deleted in the tag (admin solves this)
If you have a lot of different URL parameters then it is possible to have the cache continue to grow. I resolved these problems by running a scheduled task that goes through the cache tree and prunes any expired content. This feature is in the admin along with a feature to prune specific items in the tree. I have been saying I would put the admin up for download after I finished it but at this point it is close enough and has enough features that anyone using cf_accelerate could benefit from it. I will post it in the next few days.
A version of CF_Accelerate which supports cache persistence in the file system, rather than in shared memory, is available at http://www.throwingbeans.org/cf_accelerate_1.4.zip. This may be useful when memory contraints override the performance advantage of cache storage in the Application scope, and will still yield significant performance benefits in most cases.
I've created something very similar (before I knew about this tag) which does 2 things differently. Should I merge my code and offer a variant of this tag. www.onlysimchas.com and it has many different sections of pages, many of which are shared across different URL's.
1) My keys are all manually set as this is necessary when 2 pages are to share the same output (my stats in the header, for example), and are often constructed with #fuseaction# and #someID#, etc
2) My cached objects are tag output or queries/structs and can be stored to disk or RAM.
------------------------- All CFLock keys are the same as the cache key.
Keys are created manually in the code, as many objects are "ObjectID's" in database tables, so keys look like "get_object_#objectID#" etc.
Hybrid structure is maintained by having parallel structs.
Application.cachetagoutput ( key-> datetime of last refresh )
Application.memorytagoutput ( key-> tag output ) Application.memorystructcache ( key-> struct ) Application.diskstructcache (key-> wddx file of struct)
View the source of any http://www.onlysimchas.com page and you will see the cache comments. ----
Current stats (after 72h of uptime)
CacheTagOutput: 44030 MemoryOutput: 4467 QueryOutput: 11025 Total Memory: 132 MB Free Memory: 260 MB Threads: 20
How, or can this tag be used within includes ? I have several locations where i would like to cache the output from some query heavy includes, but keep the main page free from caching.
Is it possible to use cf_accelerate inside an included file, and include two or more files that use accelerate into a page ?
When i've tried this application.accelerate struct seem to contain a key for the base page, instead of the included page?
Brandon, congratulations for cf_accelerate. Great tool! I'm using in an application caching header and footer content related to users rights/roles. This is my simple logic, using cfimport to refer to cf_accelerate:
<SCM:accelerate primaryKey="SCM" secondaryKey="header_part15#IDUser(CGI.REMOTE_USER)#" cachedWithin="#createTimeSpan(0,8,0,0)#" StripWhitespace="true"> <SCM:header UserGroup=#session.UserDescr# DescrRightSide="Backup Units" DescrLink="BACKUP UNITS" LinkDescr="frm_all_bkp_unit.cfm?reload=1"> </SCM:accelerate>
Please help me with:
All contents are cached using "header_part15#IDUser(CGI.REMOTE_USER)#" for assign individual content. If user is in Admin Group all Admin content is presented. If that same user change your account(maybe Manager Group) I'll like to kill that previous(personal - CGI.REMOTE_USER) cached content and load your new content to cache. It's possible? Thanks for your help.
Brandon...Small bug in code. The Struct requires syntactically correct variables ... but the script_names / Directory Names can start with Numerals...So you need to modify to prefix with an alpha to be sure.
<cfparam name="attributes.scriptName" default="A#trim(rereplacenocase(CGI.SCRIPT_NAME,"[/&.-]" , "", "All"))#">
Thank you for the tag though...it works great! My $.02 Ryan
Can you post the SES fix here or email it to me ? :)
Can this caching component be used in combination with a site that uses session management? For instance an e-commerce site still needs session management but certain elements of the site could be cached such as products on a category page.
Oh nice I really hope you enjoyed this video! Hopefully, this video will be helpful to you. Please feel free to comment and please pass this on to your friends and to your contacts on Twitter and Facebook, as well as anyone else that comes to mind! Take care! For more information, go to http://www.youtube.com/annettapowell <a href="http://www.youtube.com/annettapowell">Annetta Powell</a>