Memory Leaks: Part II - variables scope leak
In part II of my memory leak posts, I will attempt to demonstrate a simple example of how the CFML runtime engine can cause memory leaks. If you would like to know more about the tools I used to identify the problem, or would like to follow along with the example you should take a look at my previous post, "ColdFusion Memory Leaks: Part I - profiler introduction"
For those of you that have ever experienced memory leaks, have there been cases where you have been convinced that the problem is not your code? Well, you may actually be right. This example will show a very simple case, then I will try to build on this to show a more problematic example in my next post.
The first part of the example is to create a simple .cfc. This cfc will have 2 methods: init() which returns itself, and doSomething() which will return an array of 100 instances of itself.
<cfcomponent name="test">
<cffunction name="init">
<cfreturn this >
</cffunction>
<cffunction name="doSomething">
<cfset var idx = "" />
<cfset var arrayToReturn = arrayNew(1) />
<cfloop from="1" to="100" index="idx">
<cfset arrayAppend(arrayToReturn,createObject("component","test").init()) />
</cfloop>
<cfreturn arrayToReturn >
</cffunction>
</cfcomponent>
<cfapplication name="memoryTest">
<cfset application.test = createObject("component","test").init() />
<cfset foo = application.test.doSomething() />
<cfoutput>foo is an array of #arrayLen(foo)# elements</cfoutput>
Now, to show the results of the test, I created a flash movie that will show the behavior, watch it to see what actually happens.
Results:
What happened? As you can see, 101 instances were persisted in the heap, and not released. These instances will live for the life of the application and NOT for the life of the request as intended. In this case we should only see 1 instance of the cfc in the heap. Here are some of the behaviors I observed.
- references to objects created in application scope will be kept alive if they are set to a reference in the variables scope.
- Setting the variable in variables scope to empty string " " will destroy the reference.
- Setting the reference in request scope will clean up the reference correctly.
- Calling structClear(variables) in onRequestEnd.cfm will destroy unwanted references tied to variables scope
- The reference retained in the variables scope only seems to apply to the last request to the.cfm file (will not grow)
As you can see, this isn 't a horrible memory leak because it won't grow continually, but it could cause unwanted objects to be persisted in memory. This IS a simple example, and I'm working to create a similar, but more complex example to build on this that will show that cfc references can grow in memory on every request. Stay Tuned.
Thanks, you've got me thinking about some apps I've built recently.
And oh wow is this is a mess or what? I think you may have identified where my problems lie with reactor.
Thanks for both these entries. I'm looking forward to #3.
Two questions:
1) In which version(s) of CF have you seen this behavior?
2) Do the CFC instances persist if you change the line:
<cfset foo = application.test.doSomething() /> to
<cfset variables.foo = application.test.doSomething() />
Doug-
-When I say variables scope, I mean variables scope within the cfm page. Variables scope within the cfc would be expected to live for the life of the cfc, and technically wouldn't be a leak. One interesting thing to note is that there is also a reference to the compiled cfm page that is kept alive after each request.
Aaron-
1. The version I am running is 7,0,2,142559, although I've seen this problem for a while, so I'm guessing it's been there since 6
2. Same behavior in both cases, if you can get the flash movie to play, you can see that I try that scenario. However, changing it to request.foo makes the references disappear.
John-
-This behavior only seems to appear when you are creating object references in the function. Therefore duplicate() will not work, because it is unable to duplicate cfc instances. I actually ran a number of test cases because at one point I was convinced that duplicate() was not actually destroying references, but after testing it looks like it actually does.
You're observing the template cache in this case, as per the Caching page in the CFAdmin. The reference is the cached template.
I notice dthat your Flash movie only works when you view the entry by the main page, if you actually click into the entry, the flash movies vanishes.
Do you have any tips on how one could identify and trouble shoot such memory leak problems on a production server with hundreds of sites, as installing JVM 5 and using the profiler wouldn't really be an option as it would break sites and result in downtime.
Thanks for pointing out the problem with the flash movie, this was my first time working with flash video, and I didn't specify paths correctly. Unfortunately, I don't have any suggestions for diagnosing a live production server. To do this effectively, you would really need to have a separate CF server instance with your production code. The developer edition works just fine for this... is it an option to setup another cf instance locally and copy code?
So, if you passed it to the same scope there would be no issue?
And, aren't all unset variables in CF automatically put into the variables scope? The second example is redundant, but most people likely don't know that is the case.
In this case I don't know if you could really say that the array or structure are really existing in a persistent scope. The CFC exists in a persistent scope, but the array is created as a var within the function, which means it will only exist for the life of the function and will not exist in the persistent scope. Does that answer your question?
Also, yes the example is redundant, but like you said it's a good way to show developers the difference between request and variables. (yet another reason to explicitly scope all variables)
That said, I wonder if your results would change if you weren't using 'this'. Since the init method does nothing more than 'return this', just remove it from the doSomething function:
<cfset arrayAppend(arrayToReturn,createObject("component","test")) />
and/or the test.cfm code:
<cfset application.test = createObject("component","test")/>
( By the way, both of your code samples are commented with the same file name )
Good observation, I didn't think about that, I just tried removing the pseudo-constructor and the results are exactly the same! Returning "this" doesn't make any difference. Guess that eliminates another potential culprit. Also, thanks for pointing out the file names, I updated the post.
Also, in my application I'm seeing jumps in memory usage because of this problem (I suspect). I have an abstract cache that expires items. Every time items get expired and recreated, the memory utilized by those expired objects persists rather than being deleted.
My only other thought is to make sure that you should be sure to file out a bug report w/ Adobe.
http://www.adobe.com/cfusion/mmform/index.cfm?name...
All you've proved is that variables scope hangs around until you use the same .cfm page again right?
You are correct, it's not a leak in this scenario. However, I would still argue that from a developers perspective, this would be unexpected usage of the heap. The main reason for this example was to show something that I could build on. I'm finishing up a video right now that extends on this example and should show a leak. Hoping to have it up by the end of the day.
Try calling <cfset CreateObject("Java", "java.lang.System").gc()> to perform an explicit GC after running your test.
Explicitly calling the GC does not make a difference. For this example, the GC is running around every 5 seconds, and I've also explicity called it through code as well as through a jvm console, and it doesn't seem to make a difference.
http://www.schierberl.com/cfblog/index.cfm/2006/10...
-Mike
I'm running CFMX7.0.1, Win2K3, SUN JVM 1.4.2_08
Do you have a link to the issue in the Sun Bug Tracker? I went down that path around 6 months ago, and identified 2 potential known issues that had been identified by Sun. Both were fixed in the 1.5 release. I have now tested the behavior on the following JVMs.
Sun 1.4.2_08
Sun 1.4.2_13
Sun 1.5.0_09
BEA 1.5.0_06
All produce virtually identical results. In my opinion, this would eliminate the jvm version as a culprit, unless this is still an outstanding issue with Sun.
While it certainly good to remove a buggy JVM from the stack, I'm fairly sure CF proper is the culprit.
Thanks for keeping on this Mike!
I think there may be something else going on here. I have reproduced your Part II test, and initially I saw the same results in the JRockit Memory Leak Detector that you've shown. But then, something weird happened. Let me explain:
Part II in CFMX 6,1,0,63958:
In an attempt to 'solve' the issue, I went through a variety of modifications to your code. For example, in the doSomething() function I created a locally scoped 'holder' variable and used that to create the object before appending to the return array:
<cfset var tmpHolder = "" />
<cfloop from="1" to="100" index="idx">
<cfset tmpHolder=createObject("component","testChild").init() />
<cfset arrayAppend(arrayToReturn,tmpHolder) />
</cfloop>
Also, as you can see in the code above I created a secondary CFC (testChild). This made it clearer for me in the Leak Detector screen to see the separation of the test.cfc and its loop-of-100 spawn. The testChild CFC contains only one function, an init() that simply returns THIS.
Neither of these did anything to your results. I still received the exact same 100 lingering child elements you've demonstrated. To be honest, I was hoping that the local 'tmpHolder' would have helped... somehow.
Here's where it gets weird.
I restarted the CF service and the Leak Detector and tested a slightly different situation. I created a new CFM page, "testOther.cfm" that contained only one line of code:
<cfoutput>#now()#</cfoutput>
Then, I loaded the new testOther.cfm in a browser then immediately loaded the original test.cfm. The results were very different. This time, 100 testChild elements cleared up completely!
So I tried again... this time I loaded the original test.cfm FIRST; then the testOther.cfm. The elements stayed in memory again, not clearing until the Application expired.
I did these two tests about a dozen times (letting the Application exipre between) and each time if I loaded "testOther.cfm" first, the testChild elements cleared; and when I loaded your "test.cfm" first, the testChild elements stayed as in your example. I tested both sequences with single page loads, with multiple page loads, and mixed loads of both templates. The results were the same.... depending on which template was loaded first caused different outcomes of the scenario.
???
- Are we 100% confident that the Leak Detector is showing the correct data?
- Is there a possibility that the CF caching Steven Erat and Sean Corfield mentioned is showing us inconclusive results in the Leak Detector?
- If we assume that this code is 'leaky' all of the time, what could be hiding the leak during these other tests?
-If we assume that the code is never 'leaky', what are we viewing that makes it seem broken?