Re: Question about growing memory problem with our plug-in
Re: Question about growing memory problem with our plug-in
- Subject: Re: Question about growing memory problem with our plug-in
- From: Darrin Cardani <email@hidden>
- Date: Sat, 16 Sep 2017 10:31:22 -0700
Glad to hear it!
Darrin
> On Sep 15, 2017, at 12:54 PM, Peter Litwinowicz <email@hidden> wrote:
>
> Argh, I finally figured it out.
>
> Turns out that in my worker thread I didn't turn off GPU processing for our
> code. In fact, I didn't set it up at all. In that case, our internal code
> will use the last GPU device setup used… which in this case is an OpenCL
> context attached to an OpenGL context. And that OpenGL context is one that
> FCP has provided for retrieving its images.
>
> So, basically, I was indirectly using an OpenGL context in two threads
> concurrently. Now, why OpenGL didn't spit out some error messages in the
> system error log or the console, I don't know. But the problem wasn't memory
> related per se.
>
> I just happened upon this conflict by chance. Whew! Anyhow, it's fixed now.
>
> Pete
>
>
> From: Pro-apps-dev <pro-apps-dev-bounces+pete=email@hidden
> <mailto:pro-apps-dev-bounces+pete=email@hidden>> on behalf
> of Peter Litwinowicz <email@hidden <mailto:email@hidden>>
> Date: Thursday, September 14, 2017 at 4:36 PM
> To: Darrin Cardani <email@hidden <mailto:email@hidden>>,
> <email@hidden <mailto:email@hidden>>
> Subject: Question about growing memory problem with our plug-in
>
> I'm having one heck of a time with my analysis pass plug-in that we're
> creating.
>
> If I get an FxImage, copy the image to my own memory, dealloc the FxImage,
> then process, then get another FxImage, copy, deallocate, then process, in
> the same thread… then all goes fine.
>
> Well our analysis works about as fast as a CPU memory fetch of an FxImage, so
> I figured I would get images on the main thread, and process on another
> thread. So now I get an FxImage then copy, then "throw" over to another
> thread for processing, then concurrently (on the main thread) get another
> FxImage. Repeat. Goes about 2x as fast, as you might expect.
>
> EXCEPT
>
> All goes fine for about 500 frames. Then all of a sudden kernel_task starts
> to grow memory by leaps and bounds. Before 500 frames, kernel_task is at 3Gb
> of memory usage, FCP is at about 1.2Gb. Both remain approximately constant
> Then memory continues to grow for kernel_task, while FCP stays constant (at
> about 1.2Gb). Over the course of 80 frames or so, kernel_task grows from 3Gb
> of usage to over 35Gb. Then I get a system error message (not a crash in my
> plug-in or FCP), that "System has run out of Application Memory". Oy. I
> read this is related to virtual memory become scarce. Crap.
>
> I KNOW I don't have a memory leak in my code (it's code we've used for YEARS
> and have analyzed it's memory usage quite rigorously). And the code works if
> the frame retrieval and processing is in the same thread. Of course we
> allocate and release memory in our plug-in using malloc, but the memory
> allocations and deallocations are all balanced.
>
> I thought maybe there's some issue with concurrent memory allocations by FCP
> and our plug-in, but malloc and other memory routines are supposed to be
> thread safe. So perhaps some memory fragmentation? But wouldn't that
> probably happen even if our processing is happening in line (and in the same
> thread) as the frame retrieval?
>
> Anyone have any ideas? Bueller?
>
> Pete
>
> _______________________________________________ Do not post admin requests to
> the list. They will be ignored. Pro-apps-dev mailing list
> (email@hidden <mailto:email@hidden>)
> Help/Unsubscribe/Update your Subscription:
> email sent to email@hidden
> <mailto:email@hidden>_______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Pro-apps-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Pro-apps-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden