Re: Question about growing memory problem with our plug-in
Re: Question about growing memory problem with our plug-in
- Subject: Re: Question about growing memory problem with our plug-in
- From: Peter Litwinowicz <email@hidden>
- Date: Fri, 15 Sep 2017 12:54:18 -0700
- Thread-topic: Question about growing memory problem with our plug-in
Argh, I finally figured it out.
Turns out that in my worker thread I didn't turn off GPU processing for our
code. In fact, I didn't set it up at all. In that case, our internal code
will use the last GPU device setup used… which in this case is an OpenCL
context attached to an OpenGL context. And that OpenGL context is one that
FCP has provided for retrieving its images.
So, basically, I was indirectly using an OpenGL context in two threads
concurrently. Now, why OpenGL didn't spit out some error messages in the
system error log or the console, I don't know. But the problem wasn't
memory related per se.
I just happened upon this conflict by chance. Whew! Anyhow, it's fixed
now.
Pete
From: Pro-apps-dev
<pro-apps-dev-bounces+pete=email@hidden> on behalf of
Peter Litwinowicz <email@hidden>
Date: Thursday, September 14, 2017 at 4:36 PM
To: Darrin Cardani <email@hidden>, <email@hidden>
Subject: Question about growing memory problem with our plug-in
I'm having one heck of a time with my analysis pass plug-in that we're
creating.
If I get an FxImage, copy the image to my own memory, dealloc the FxImage,
then process, then get another FxImage, copy, deallocate, then process, in
the same thread… then all goes fine.
Well our analysis works about as fast as a CPU memory fetch of an FxImage,
so I figured I would get images on the main thread, and process on another
thread. So now I get an FxImage then copy, then "throw" over to another
thread for processing, then concurrently (on the main thread) get another
FxImage. Repeat. Goes about 2x as fast, as you might expect.
EXCEPT
All goes fine for about 500 frames. Then all of a sudden kernel_task starts
to grow memory by leaps and bounds. Before 500 frames, kernel_task is at
3Gb of memory usage, FCP is at about 1.2Gb. Both remain approximately
constant Then memory continues to grow for kernel_task, while FCP stays
constant (at about 1.2Gb). Over the course of 80 frames or so, kernel_task
grows from 3Gb of usage to over 35Gb. Then I get a system error message
(not a crash in my plug-in or FCP), that "System has run out of Application
Memory". Oy. I read this is related to virtual memory become scarce.
Crap.
I KNOW I don't have a memory leak in my code (it's code we've used for YEARS
and have analyzed it's memory usage quite rigorously). And the code works if
the frame retrieval and processing is in the same thread. Of course we
allocate and release memory in our plug-in using malloc, but the memory
allocations and deallocations are all balanced.
I thought maybe there's some issue with concurrent memory allocations by FCP
and our plug-in, but malloc and other memory routines are supposed to be
thread safe. So perhaps some memory fragmentation? But wouldn't that
probably happen even if our processing is happening in line (and in the same
thread) as the frame retrieval?
Anyone have any ideas? Bueller?
Pete
_______________________________________________ Do not post admin requests
to the list. They will be ignored. Pro-apps-dev mailing list
(email@hidden) Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Pro-apps-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden