February 2008 - Pro-apps-dev

Re: Antialiassing in an FxPlug
by Paul Schneider 04 Feb '08

04 Feb '08

site_archiver(a)lists.apple.com Delivered-To: pro-apps-dev(a)lists.apple.com Hi, Gary, it depends on how you're looking at the result. - Paul On Feb 4, 2008, at 1:28 PM, Pete Warden wrote: I'm not sure about your FCP question, maybe Paul could answer that? Pete On Feb 4, 2008, at 5:23 AM, Gary Fielke wrote: Cheers, Gary Message: 4 Date: Thu, 31 Jan 2008 11:02:07 -0800 From: Pete Warden <pwarden(a)apple.com> Subject: Re: Antialiassing in an FxPlug To: pro-apps-dev(a)lists.apple.com Message-ID: <4D141ABD-BB0E-4B69-9C9E-7FC4120D1016(a)apple.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Pete On Jan 31, 2008, at 5:56 AM, Gary Fielke wrote: Thanks Pete, just what I was after (the tiling tip will undoubtedly be useful also) Out of curiosity I just tested out CILanczosScaleTransform to scale down from a 2x2 FBO. It seemed to work well but a bit slow on my G4 PowerBook. This might be suitable for the rendering stage but I'm not sure yet how well it would play if tiling of image buffers was required. Thanks again Gary Is that any help? Pete _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/pwarden%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/pschneider%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/site_archiver%40lists.a… I'm not sure about your FCP question, maybe Paul could answer that? If your canvas is not at 100%, Final Cut will show you one field, stretched to full height. This scaling happens during display, after your filter is applied, and so it can introduce artifacts that might make your antialiasing look off. If your canvas is at 100%, Final Cut will show you both fields. Again, this can make your antialiasing look off, for the following reaons: If there is any movement in your scene, then the geometry in the two fields won't line up; there's nothing you can do about that (except maybe add some motion blur). When the media is viewed on an interlaced device (like an NTSC monitor), it should look correct. If there's no movement in your scene, then the problem could be that you're not accounting for the fact that the lower field is shifted down one pixel from the upper field. Don't forget that the two fields occur at different points in time, and at different points in space. If you remember that, working with interlaced media becomes easier. I'm not sure why you're not seeing similar behavior in Motion. One reason could be that Motion doesn't do field rendering by default - you have to switch it on in the View menu. Super-sampling is the simplest way of reducing aliasing but you're right there's more advanced techniques you can use. I can't discuss Motion's implementation and I'm not sure what the typical content of your scene is but here are some general pointers on anti-aliasing: - For the best results, work in linear space, not gamma-corrected. The resampling will look better with fewer samples. This means using 16-bit float buffers at least, and converting to and from 8 bit gamma corrected inputs and outputs. We're currently trying to prepare some documentation on the recommended way to do this in FxPlugs. - If you're drawing large images scaled down, or with perspective transforms, make sure you're using a good down-sampling algorithm, like rip-maps and anisotropic filtering. - With polygons, an old game trick is to get a cheap quality boost by drawing anti-aliased lines along the edges of your geometry as an initial pass before drawing the filled shapes. One thing to watch out for with this is that anti-aliased lines require blending support and that's not available on many cards at float bit depths. Thanks Pete, I draw my scene into a double-sized FBO, then draw the attached texture as a quad to my single-sized FBO. With a half pixel offset in texture coordinates the aliasing is certainly reduced. It's just not as good as Motion seems to do it. How does Motion do its antialiasing? Also, does the FxPlug need to de-interlace frames prior to performing this antialiasing in FCP? With interlaced frames, the output seems a little untidy around the edges. On 01/02/2008, at 6:40 AM, pro-apps-dev-request(a)lists.apple.com wrote: One trick you can use is to just use the default bilinear filtering to resample four pixels to one without a shader. You need to set up your texture coordinates so that the sample point is at the corner of four pixels when drawing to the final output, but it's an efficient way of using the hardware. Doing an extra pass is always costly though. Back when I was coding plugins for another app that shall remain nameless, I used the same general over-sampling technique to reduce aliasing, and offered it as an extra user control, as a filter parameter. If you're worried about the performance hit, that might be an option. Re the "scale down and resample result into the output" stage is it preferable to use GLSL to average neighbour pixels in a frag shader? Or is there a better option? Hi Gary, you're right, there are a lot of mines to step on when trying to get multisampling working in an FxPlug. Since as you discovered PBuffers don't support them, and I'm not sure of the state of FBO support, we've tended to do the same operations manually. I don't have much information on the state of hidden windows, since it's been a long time since we used them, but it's a pretty unusual path to be using these days and I wouldn't recommend it. Basically, allocate a pbuffer or FBO that's several times larger than the final output, and draw into it scaled up. Then as a final pass, scale down and resample the result into the output. One thing to watch out for with this is that you may hit card limits to the maximum texture size, or even how many textures will fit in VRAM at 16 and 32 bit float bit depths. FCP and Motion have checks to limit the output size they request to stay within those limits. Rather than allocating one large buffer, you may instead want to allocate several smaller buffers at the output size, draw separately into them as tiles, and then join them together by rendering them into the final output. ********************************* This email sent to pwarden(a)apple.com This email sent to pschneider(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: Antialiassing in an FxPlug
by Darrin Cardani 04 Feb '08

04 Feb '08

site_archiver(a)lists.apple.com Delivered-To: pro-apps-dev(a)lists.apple.com On Feb 4, 2008, at 5:23 AM, Gary Fielke wrote: _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/site_archiver%40lists.a… Also, does the FxPlug need to de-interlace frames prior to performing this antialiasing in FCP? With interlaced frames, the output seems a little untidy around the edges. Gary, Would it be possible to post a screenshot? (Or if you prefer send it to me directly?) It sounds like FCP is just showing you 2 fields, which would correctly create jagged edges if there's any movement in your output, but it's hard to say without seeing it. You should not have to deinterlace your output. -- Darrin Cardani dcardani(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: Antialiassing in an FxPlug
by Pete Warden 04 Feb '08

04 Feb '08

site_archiver(a)lists.apple.com Delivered-To: pro-apps-dev(a)lists.apple.com I'm not sure about your FCP question, maybe Paul could answer that? Pete On Feb 4, 2008, at 5:23 AM, Gary Fielke wrote: Cheers, Gary Message: 4 Date: Thu, 31 Jan 2008 11:02:07 -0800 From: Pete Warden <pwarden(a)apple.com> Subject: Re: Antialiassing in an FxPlug To: pro-apps-dev(a)lists.apple.com Message-ID: <4D141ABD-BB0E-4B69-9C9E-7FC4120D1016(a)apple.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Back when I was coding plugins for another app that shall remain nameless, I used the same general over-sampling technique to reduce aliasing, and offered it as an extra user control, as a filter parameter. If you're worried about the performance hit, that might be an option. Pete On Jan 31, 2008, at 5:56 AM, Gary Fielke wrote: Thanks Pete, just what I was after (the tiling tip will undoubtedly be useful also) Re the "scale down and resample result into the output" stage is it preferable to use GLSL to average neighbour pixels in a frag shader? Or is there a better option? Out of curiosity I just tested out CILanczosScaleTransform to scale down from a 2x2 FBO. It seemed to work well but a bit slow on my G4 PowerBook. This might be suitable for the rendering stage but I'm not sure yet how well it would play if tiling of image buffers was required. Thanks again Gary Is that any help? Pete _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/site_archiver%40lists.a… Super-sampling is the simplest way of reducing aliasing but you're right there's more advanced techniques you can use. I can't discuss Motion's implementation and I'm not sure what the typical content of your scene is but here are some general pointers on anti-aliasing: - For the best results, work in linear space, not gamma-corrected. The resampling will look better with fewer samples. This means using 16- bit float buffers at least, and converting to and from 8 bit gamma corrected inputs and outputs. We're currently trying to prepare some documentation on the recommended way to do this in FxPlugs. - If you're drawing large images scaled down, or with perspective transforms, make sure you're using a good down-sampling algorithm, like rip-maps and anisotropic filtering. - With polygons, an old game trick is to get a cheap quality boost by drawing anti-aliased lines along the edges of your geometry as an initial pass before drawing the filled shapes. One thing to watch out for with this is that anti-aliased lines require blending support and that's not available on many cards at float bit depths. Thanks Pete, I draw my scene into a double-sized FBO, then draw the attached texture as a quad to my single-sized FBO. With a half pixel offset in texture coordinates the aliasing is certainly reduced. It's just not as good as Motion seems to do it. How does Motion do its antialiasing? Also, does the FxPlug need to de-interlace frames prior to performing this antialiasing in FCP? With interlaced frames, the output seems a little untidy around the edges. On 01/02/2008, at 6:40 AM, pro-apps-dev-request(a)lists.apple.com wrote: One trick you can use is to just use the default bilinear filtering to resample four pixels to one without a shader. You need to set up your texture coordinates so that the sample point is at the corner of four pixels when drawing to the final output, but it's an efficient way of using the hardware. Doing an extra pass is always costly though. Hi Gary, you're right, there are a lot of mines to step on when trying to get multisampling working in an FxPlug. Since as you discovered PBuffers don't support them, and I'm not sure of the state of FBO support, we've tended to do the same operations manually. I don't have much information on the state of hidden windows, since it's been a long time since we used them, but it's a pretty unusual path to be using these days and I wouldn't recommend it. Basically, allocate a pbuffer or FBO that's several times larger than the final output, and draw into it scaled up. Then as a final pass, scale down and resample the result into the output. One thing to watch out for with this is that you may hit card limits to the maximum texture size, or even how many textures will fit in VRAM at 16 and 32 bit float bit depths. FCP and Motion have checks to limit the output size they request to stay within those limits. Rather than allocating one large buffer, you may instead want to allocate several smaller buffers at the output size, draw separately into them as tiles, and then join them together by rendering them into the final output. ********************************* _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/pwarden %40apple.com This email sent to pwarden(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: questions on pixel formats
by Micah Sharp 04 Feb '08

04 Feb '08

site_archiver(a)lists.apple.com Delivered-To: pro-apps-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/site_archiver%40lists.a… This email sent to site_archiver(a)lists.apple.com

1 0

Re: questions on pixel formats
by Darrin Cardani 04 Feb '08

04 Feb '08

site_archiver(a)lists.apple.com Delivered-To: pro-apps-dev(a)lists.apple.com On Feb 3, 2008, at 12:02 PM, Steve Christensen wrote: In FCP, I think it's this: 1) select your sequence in the browser, or click on the timeline window 2) From the "Sequence" menu, select "Settings..." 3) Click on the "Video Processing" tab 4) Click on the "Render all YUV material in high-precision YUV" In Motion: 1) From the "Edit" menu, select "Project Properties" 2) Change the "Bit Depth" popup from "8 Bit" to "32 Bit". Darrin -- Darrin Cardani dcardani(a)apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/site_archiver%40lists.a… I don't remember seeing these specifically addressed so I figured I'd ask so I can make sure I'm doing everything correctly. I'm currently interested in the software rendering case, but if there some differences in the hardware case, that'd be nice to know as well. 1. So far all the images or video I load into a project for testing use 32-bit ARGB/RGBA for the pixel format. If I've read the docs correctly, I should also expect to see 128-bit floating point ARGB/ RGBA pixels in some circumstances. If that's so, what can I do to get that to happen so I can test that I'm handling that case correctly? 2. Can I assume that the input and output images will always have the same sized pixels, i.e., I shouldn't expect to see a 32-bit pixel in the input image(s) and a 128-bit pixel in the output image? Correct. Input and output should always be the same bit depth and channel order. Note that there is a bug in Motion in which the output bitmap will claim to be RGBA, when it's really ARGB, just like the input (when working in software). It's safe to ignore it and output ARGB. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Antialiassing in an FxPlug
by Gary Fielke 04 Feb '08

04 Feb '08

site_archiver(a)lists.apple.com Delivered-To: pro-apps-dev(a)lists.apple.com Cheers, Gary Message: 4 Date: Thu, 31 Jan 2008 11:02:07 -0800 From: Pete Warden <pwarden(a)apple.com> Subject: Re: Antialiassing in an FxPlug To: pro-apps-dev(a)lists.apple.com Message-ID: <4D141ABD-BB0E-4B69-9C9E-7FC4120D1016(a)apple.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes One trick you can use is to just use the default bilinear filtering to resample four pixels to one without a shader. You need to set up your texture coordinates so that the sample point is at the corner of four pixels when drawing to the final output, but it's an efficient way of using the hardware. Doing an extra pass is always costly though. Back when I was coding plugins for another app that shall remain nameless, I used the same general over-sampling technique to reduce aliasing, and offered it as an extra user control, as a filter parameter. If you're worried about the performance hit, that might be an option. Pete On Jan 31, 2008, at 5:56 AM, Gary Fielke wrote: Thanks Pete, just what I was after (the tiling tip will undoubtedly be useful also) Re the "scale down and resample result into the output" stage is it preferable to use GLSL to average neighbour pixels in a frag shader? Or is there a better option? Out of curiosity I just tested out CILanczosScaleTransform to scale down from a 2x2 FBO. It seemed to work well but a bit slow on my G4 PowerBook. This might be suitable for the rendering stage but I'm not sure yet how well it would play if tiling of image buffers was required. Thanks again Gary Is that any help? Pete _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/site_archiver%40lists.a… Thanks Pete, I draw my scene into a double-sized FBO, then draw the attached texture as a quad to my single-sized FBO. With a half pixel offset in texture coordinates the aliasing is certainly reduced. It's just not as good as Motion seems to do it. How does Motion do its antialiasing? Also, does the FxPlug need to de-interlace frames prior to performing this antialiasing in FCP? With interlaced frames, the output seems a little untidy around the edges. On 01/02/2008, at 6:40 AM, pro-apps-dev-request(a)lists.apple.com wrote: Hi Gary, you're right, there are a lot of mines to step on when trying to get multisampling working in an FxPlug. Since as you discovered PBuffers don't support them, and I'm not sure of the state of FBO support, we've tended to do the same operations manually. I don't have much information on the state of hidden windows, since it's been a long time since we used them, but it's a pretty unusual path to be using these days and I wouldn't recommend it. Basically, allocate a pbuffer or FBO that's several times larger than the final output, and draw into it scaled up. Then as a final pass, scale down and resample the result into the output. One thing to watch out for with this is that you may hit card limits to the maximum texture size, or even how many textures will fit in VRAM at 16 and 32 bit float bit depths. FCP and Motion have checks to limit the output size they request to stay within those limits. Rather than allocating one large buffer, you may instead want to allocate several smaller buffers at the output size, draw separately into them as tiles, and then join them together by rendering them into the final output. ********************************* This email sent to site_archiver(a)lists.apple.com

1 0

Re: Matrix and FOV values output from Motion
by Gary Fielke 04 Feb '08

04 Feb '08

site_archiver(a)lists.apple.com Delivered-To: pro-apps-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/site_archiver%40lists.a… This email sent to site_archiver(a)lists.apple.com

1 0

Re: questions on pixel formats
by Brian 03 Feb '08

03 Feb '08

site_archiver(a)lists.apple.com Delivered-To: pro-apps-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327; d=earthlink.net; b=ZEgosZKq8kZdPbCsusVbaupaI4HZu3hZweTuuZCtktPzt7fdOzJGfWGzZh/BHIbw; h=Message-ID:Date:From:Reply-To:To:Subject:Mime-Version:Content-Type:Content-Transfer-Encoding:X-Mailer:X-ELNK-Trace:X-Originating-IP; 1. Quick an dirty way is to set your project to 32-bit float: Edit-> Project Properties -> Bit Depth = 32 bit float - Brian _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/site_archiver%40lists.a… This email sent to site_archiver(a)lists.apple.com

1 0

questions on pixel formats
by Steve Christensen 03 Feb '08

03 Feb '08

site_archiver(a)lists.apple.com Delivered-To: pro-apps-dev(a)lists.apple.com steve _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/site_archiver%40lists.a… I don't remember seeing these specifically addressed so I figured I'd ask so I can make sure I'm doing everything correctly. I'm currently interested in the software rendering case, but if there some differences in the hardware case, that'd be nice to know as well. 1. So far all the images or video I load into a project for testing use 32-bit ARGB/RGBA for the pixel format. If I've read the docs correctly, I should also expect to see 128-bit floating point ARGB/ RGBA pixels in some circumstances. If that's so, what can I do to get that to happen so I can test that I'm handling that case correctly? 2. Can I assume that the input and output images will always have the same sized pixels, i.e., I shouldn't expect to see a 32-bit pixel in the input image(s) and a 128-bit pixel in the output image? This email sent to site_archiver(a)lists.apple.com

1 0

Matrix and FOV values output from Motion
by Jean Marc Noel 03 Feb '08

03 Feb '08

site_archiver(a)lists.apple.com Delivered-To: pro-apps-dev(a)lists.apple.com Regarding the Fx3DAPI protocol : I get strange values for the Field of view of Motion camera. ex : in motion with a framing camera : a FOV of 1° output 2062.59570232 a FOV of 90° output 18 a FOV of 180° output 0,25134374 what is the logic with those numbers ? Best regards. Yanomano _______________________________________________ Do not post admin requests to the list. They will be ignored. Pro-apps-dev mailing list (Pro-apps-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/pro-apps-dev/site_archiver%40lists.a… This email sent to site_archiver(a)lists.apple.com

1 0