Keying out something with similar detail inside object

So there's this recording of this 3d rubik's cube I have( It's onstantly rotating and changing shape). The BG is pure white, and the cube has clear black outline. But if I try to key out the cube using color or luminance, the white parts of the cube itself also gets keyed out.

 

Is there a way I could somehow limit the keying to only outside the outlines? Maybe any way I could choose how many pixels in an area to remove at once? (the bg is a huge area while the cube stickers are small areas)

I remember seeing something like this in After Effects if I remember correcly.

 

Attached is a screenshot of the vid (https://imgur.com/wOIsoEJ). Thanks in advance!

Comments

  • edited May 2018

    I have this video capture of a 3d rubik's cube moving around. But as you can see in the screenshot, if I try to key it out using color or luminance, the white parts of the cube also gets keyed out. Manually freehand masking is not an option as the cube moves around a lot.

    Is there a way to limit the keying only outside the black outline? Maybe somehow adjusting how many pixels to key out in a single area at once?

     

    Thanks in advance!

    screenshot

  • @mangonerd ; I took your still image and went the entire list of keys and had the same luck you did.  About the only thing  I see that you can do (and this will be majorly time consuming) is to export the video clip as a PNG sequence and take each frame into a program like GIMP and use the magic wand tool to select only the background on each frame.  As I typed this, I thought about the set matte tool in Hitfilm but I'm not proficient enough to make a reasonable guess if that might work... 

  • @mangonerd "Is there a way to limit the keying only outside the black outline? Maybe somehow adjusting how many pixels to key out in a single area at once?"

    Not in HitFilm. (At least, no way that I can think of...)*

    *(Assuming that the cube is rotating in 3d. If it's always the same bit of the cube that's visible then you could track it with Mocha)

     

    Nice idea though, to automatically "key" out regions based on the area of the region... it might be possible to make some sort of custom tool to do this.

    The tricky bit would be getting the computer to identify each region as being distinct... I guess one way would be to identify areas of contrast, to find the edges...

    Just speculating. No idea if it would actually work.

    But if someone were to try and make such a tool, OpenCV might be a good place to start

    https://opencv.org/

  • @tddavis I'm quite new to hitfilm so I don't know the use of set matte either.

     Also I think just freehand masking would be easier than the png sequence idea you said haha. Though if I  could make some kind of batch action in photoshop that might work. Thanks anyway man!

  • @JMcAllister No idea what openCV is... Also that region thing I think I saw in After Effects once.

    Also yea, somehow identifying edges would also help, can hitfilm do that?

  • Triem23Triem23 Moderator

    The lesson here is that keys work via contrast. And, yes, keying an object with white in it on a white background is going to put holes in the key. 

    The method for selecting only  part of a frame to key is usually to, well, manually mask or matte first.

    Is it possible to tell the key to only happen outside the cube? Not automatically--and this applies to any program, not just Hitfilm.

    Ok--the computer has no idea what your image or video actually is. The computer is merely processing lots of dots with color values assigned. Automatic selection tools can often work, but, again, those require enough contrast between color values to define the different areas. 

    For the computer to "know" how to limit things to certain areas is to define that area for the computer, which, once again, is some kind of manual selection, whether masking or matting.

    @tddavis has a possible solution in rendering out a PNG sequence and using a magic wand tool to select the bg--but that's manual selection. 

    Otherwise, you're still back to manually masking or matting in Hitfilm. 

    So, lets define masking. Masking is defining transparent area of a layer by drawing areas directly on the layer. Depending on the movement of elements in the layer this may, or may not be fairly easy, or fairly hard. Is the cube moving around the frame or is it staying in the same part of the frame and rotating? If rotating, is it the entire cube, individual cube layers or both? Hopefully the cube is staying centered in the frame and just rotating... We'll come back to masking in a minute. 

    Now, let's define matting. A matte is taking channel information from layer A and using that information to define transparency on layer B. Any channel can be used, but, for this discussion we'll assume we're going to mask layer A then use its alpha channel to matte layer B. 

    Ok, back to masking: this is assuming the cube is staying centered in the frame with all motion being rotations. 

    So, another essential  point here is, often, one just has to use multiple layers to isolate an object. You'll need two copies of the cube layer. Mute the top copy of the layer then key the bottom layer. You'll lose white squares. This is inevitable.

    Unmute the top copy and set its transparency to 50% for now. 

    And... Start masking. Remember, you can use multiple masks on a layer. You DON'T have to mask the entire cube. You ONLY have to mask around white squares. Unless the white square is right against the edge of the cube where it meets the background. Otherwise, you can use a sloppy mask that overlaps adjacent squares. We'll call the three cube faces we can see in the screen shot Top, Left and Right. The Top face has three white segments. I'd probably use two masks on this. The corner white segment abuts the background and requires one precise edge. The rest can be slop overlapping the cube. The other two white segments are nowhere near the background, and can be a sloppy mask overlapping surrounding faces. Yes, you're animating multiple masks.

    Animating multiple masks is actually the standard for roto. It's actually faster to break a roto'd layer into multiple masks than deal with the object as a single shape. You'll have, probably, six masks going--two for each visible face--occasionally moving a mask completely off-screen if/when you don't need it. 

    After making, take the top layer back to 100% opacity, select both cube layers, right-click and convert to a Composite Shot to bake both layers together. 

    Ok, so, what if the cube is moving around the screen, not just rotating? Now we probably want to matte. The setup is similar. You still want two layers of cube, with a white plane on top. Key the bottom cube.

    Motion track either cube layer. Probably just a single point tracker, although, depending on the movement of the cube you might have difficulty finding a static point... Either way, you apply the track to the white plane layer, set that to 50% opacity and mask the plane. Hopefully the track drags the white plane around to keep things mostly lined up with the cube.

    Once the masking is done, set opacity back to 100% and convert the plane to a Composite Shot. Now mute this layer.

    On the top cube add the Set Matte effect. Select the white plane comp as Source Layer, set the Matte Type to Alpha, and the blend to Subtract (check the Invert box.) 

    The bottom line is masking, matting and roto are essential skills. Keying isn't always going to work (like a white object on a white background), while masking and matting, while tedious manual work, ALWAYS works. 

    Or, do what tddavis suggested with a photo editor and magic wand. Even there you'll probably end up doing manual cleanup as the occasional white segment overlaps the background mid-rotation, or the black plastic catches a strong reflection. 

    Roto sucks. Still has to be done. Next time you watch a large blockbuster, look in the credits for roto artists. You'll probably see a couple dozen names--those are working professionals whose entire career is doing the manual work you're hoping to avoid. In this case you do not have any other options. You're just going to have to do the roto work (and frame by frame magic wanding is roto). 

  • edited May 2018

    @mangonerd ;

    The easy way to save time & headaches is to track down the artist who posted the animation and request that they export the sequence over a transparent background.  Done Deed.  I'm sure many people have successfully created similar animations by now.

    Else, track down (and prolly pay for) a properly exported animation that fills all your requirements at the outset.  

    Here is the animation sequence you really want:

    https://www.youtube.com/watch?v=1nkJU-w9L6o

    From 2:22 to 2:26

    (  !  )

     

     

  • Triem23Triem23 Moderator

    Other options include creating your own animation. If you're Blender literate there are several free Rubik's cube models on Turbosquid--some in native Blender format. 

    If you're in Hitfilm Pro, there are several free Rubik's cubes on Turbosquid in Hitfilm compatible formats. A bit of a tricky animation rig to set up the points, but it's possible. 

  • Thanks @Triem23 ! Made it work with your masking the blocks idea. All this time I was thinking of masking the whole cube haha

  • Stargazer54Stargazer54 Moderator

    Well... if you are using white to do the key then everything else that is white will key out, as well.   The black outlines won't help.  The keyer doesn't care.

    You need to create a matte for the cube.  The best way is to render the cube at 32 bit so you have  an alpha channel (the matte).  Otherwise you have to create a butt load of masks and animate those to compensate for the missing matte -  which can be done but is a pain in the arse to say the least.

    Best to render with alpha to begin with (assuming it's your animation file).  If it is just a motion file with no alpha, you'll have to animate the masks.

  • Triem23Triem23 Moderator
    edited May 2018

    Merged the redundant threads... 

  • What would be really slick would be to have essentially a keyframable "magic wand key" effect in HitFilm.  You specify X and Y coordinates for the wand "source" (with a possible option to drive this position via a point or other layer), and then keyframe the coordinates over time as needed to keep it in the zone you want to key.  Other properties would let you control what portion of the source layer to look to for comparison (i.e. luminance, color similarity,  etc), adjust the tolerance, control whether or not the end result is contiguous based on the source, and an option to invert the alpha.  It maybe be a minor resource hog as it recalculates the alpha for every frame, but would be really useful in situations like this.  Just set the source coordinates anywhere outside the cube, set the search to be contiguous so that the white panels of the cube are ignored, and possibly clean up the resulting alpha with the Matte Cleaner effect.

    Methinks I'll add this to the wishlist thread... :)

  • Triem23Triem23 Moderator

     @jsbarrett so, basically, you want a Hitfilm version of this.

    https://www.premiumbeat.com/blog/free-download-magic-wand-tool-for-after-effects/

    Other than this 3'rd party add-on I can't think of ANY NLE or compositor that has a "Magic Wand" key.

    One thing you're correct about--it would be a resource hog compared to most other keyers, since resampling a source color from a potentially moving area with a check for contiguous pixels on each and every frame is a lot more complex than what most key do.

  • @Triem23 Yeah, pretty much. Didn't know that other tool existed, but that about sums it up.

Sign In or Register to comment.