Jim,
If you're never releasing the images the host application gives to you then you are leaking memory and possibly OpenGL textures and you will eventually crash. You do need to call [myImage release] explicitly on any images the host app gives to you. I recommend using them, and then releasing them in your render method, although you could also get them in your -frameSetup::: method and release them in -frameCleanup.
This is not a problem with Motion (or FCP). I'm not as familiar with Core Image as I'd like to be, but I think what you are doing is the generally accepted way to do things. We just need to work out how to get you textures that don't use the same texture ID.
I think the easiest thing to do at this point is to use a utility function which gets a bitmap from the host app and makes it into a texture yourself. Here's one that I've written, below. It should be no slower than asking the host application for a texture because the host application reads bitmaps off disk (or from cache) and then just uploads them to a texture for you when you ask for them. This does the same thing explicitly.
- (BOOL)createGLTexture:(GLuint*)textureID withInfo:(FxRenderInfo)renderInfo atTime:(double)time { BOOL result = NO;
// Get the temporal image api id<FxTemporalImageAPI> temporalImageAPI = [_apiManager apiForProtocol:@protocol(FxTemporalImageAPI)]; if (nil != temporalImageAPI) { // Get the frame we need FxBitmap* frame = nil; result = [temporalImageAPI getInputBitmap:&frame withInfo:renderInfo atTime:time];
if (result) { // Upload the frame to the video card void* dataPtr = [frame dataPtr];
glGenTextures (1, textureID); glBindTexture (GL_TEXTURE_RECTANGLE_EXT, *textureID); glTexImage2D (GL_TEXTURE_RECTANGLE_EXT, 0, GL_RGBA, [frame width], [frame height], 0, GL_RGBA, GL_UNSIGNED_INT_8_8_8_8, dataPtr);
// Clean up [frame release];
// See if there were any OpenGL errors if (glGetError () != GL_NO_ERROR) { result = NO; } } }
return result; }
Note that it assumes you want the texture in GL_TEXTURE_RECTANGLE_EXT, and that you must delete the textures when you're done with them. (I think that has to happen after the call to drawImage). Let me know if you have any questions.
Thanks, Darrin On Oct 1, 2008, at 10:24 PM, Jim George wrote: Darrin, I am actually never calling release on anything frame or texture related... Things seem to be running ok. The filter is slow for large frame volumes (width of 60 - 150), but the memory foot print stays relatively constant... What is the recommended pattern for allocating/releasing temporalImage frames? Does release need to called explicitly? Could this problem be considered a bug in Motion? or is there a more conventional way for me to go about stringing these filters together successively. Perhaps the CIFilterGenerator class may work in some way? The API for that is a bit confusing, but it seems like it may be useful. Thanks, On Thu, Oct 2, 2008 at 8:34 AM, Darrin Cardani < email@hidden> wrote: Jim,
I'll have to think about why you're getting the same texture ID for
successive calls, and what you can do about it. There's no real way to tell
Motion to give you a different texture ID than the one it decides to give
you. But I'm a little surprised that it's giving you the same one when
you're not releasing the previous one. (And you better release them all
eventually or you'll run out of resources!)
If you want to cache the frames, you could get bitmaps instead of textures
and keep them around. However, this will make it more difficult for the host
app to cache other things. You'll also have to upload the images to a
texture yourself every time you want to use them. But I guess that wouldn't
be any slower than having the host app do it. In Motion, we may already be
caching the input frames as bitmaps in memory. (If your filter is the only
one applied to the footage, then that's already happening.) You probably
don't want to do it yourself, though, because if there are other filters
applied before yours, and the user changes the parameters of those filters,
you have no way to know which frames in your cache have changed and which
haven't. So I don't recommend doing that.
One option would be to draw the intermediate steps manually, and feed each
output to the next input after it's been rendered. This will reduce
performance, of course. Not the best solution, but probably one that can
work.
Darrin
On Oct 1, 2008, at 5:23 PM, Jim George wrote:
good thinking, I did a print of the texture ID. In Motion it is
always the same, but in final cut it is always different.
I am not releasing the texture within the loop. doing so actually
causes a crash.
does anyone have any ideas what to do to ensure that new textures get
allocated each iteration?
On a related note, is there anyway that i can cache these frames into
some sort of LRU expulsion cache. Most of the time, especially during
render, each frame of out put uses almost the same frames of input but
I am not storing them locally at all. If i could store them and
reduce the number of calls to temporalImageAPI perhaps that would help
performance?
Thanks!
On Thu, Oct 2, 2008 at 1:28 AM, Darrin Cardani <email@hidden> wrote:
On Oct 1, 2008, at 12:22 AM, Jim George wrote:
I've run into a discrepancy between the behavior of using Core Image
filters between Final Cut and Motion in an FxPlug and wonder if anyone
would have any insight into the problem.
I have a filter that takes a number of contiguous frames and runs some
processing over them. I am using the temporalImageAPI to get the
frames as FxTextures.
In Final Cut the filter comes out correctly, but in Motion only the
*last* frame gets drawn. I'm thinking somehow that Motion uses a
shared context or texture where final cut allocates new ones, but my
understanding of these under the hood is a bit of guesswork.
Jim,
Yes there are cases where Motion reuses textures. You could check
whether we're giving you the same texture by printing out the texture ID
in
the loop. I would think that since you aren't releasing the textures (at
least in the code below) that we wouldn't be reusing the texture ID.
If this is a cut-down version of the code and you actually are
releasing the textures after each loop iteration, that may be the issue.
Try
not releasing them until after the draw command. Remember that CI filters
don't actually execute until you call the drawImage: command because they
can concatenate for improved speed, so if you're releasing them in the
loop,
they might not be there when the drawing happens.
Below is the basics of what I'm doing in the Render method for FxPlug.
glPushAttrib(GL_CURRENT_BIT);
CIContext *ciContext = [CIContext
contextWithCGLContext:CGLGetCurrentContext()
pixelFormat:CGLGetPixelFormat(CGLGetCurrentContext())
options:NULL];
CGColorSpaceRef cspace = CGColorSpaceCreateDeviceRGB();
FxTexture* inputTex;
[temporalImageAPI getInputTexture:&inputTex withInfo:
renderInfo
atTime: renderInfo.frame - width/2];
CIImage* imageBase = [CIImage imageWithTexture:[inputTex
textureId]
size:CGSizeMake([inputTex width],[inputTex height]) flipped:NO
colorSpace:cspace];
UInt32 frameStep = 0;
double i;
for(i = renderInfo.frame - width/2; i <= renderInfo.frame +
width/2; i++){
[_filter setValue:imageBase forKey: @"inputBase"];
[temporalImageAPI getInputTexture:&inputTex
withInfo: renderInfo atTime:i];
CIImage* inputImage = [CIImage
imageWithTexture:[inputTex
textureId] size:CGSizeMake([inputTex width],[inputTex height])
flipped:NO colorSpace:cspace];
[_filter setValue:inputImage forKey:@"inputImage"];
imageBase = [_filter valueForKey:@"outputImage"];
}
[ciContext drawImage:imageBase
inRect:CGRectMake(left,bottom,right-left,top-bottom)
fromRect:CGRectMake(tLeft,tBottom,tRight-tLeft,tTop-tBottom)];
glPopAttrib();
|