Hitfilm 3D floor plane moving crazily after mocha Hitfilm camera solve?

edited December 2018 in HitFilm Pro Support

So, I grabbed my phone and just held it recording out my window into the street. My hands did not move much. I tracked two good surfaces in mocha (the street and the house opposite me) and everything was good. I camera-solved it as small-parallax and got a solve quality of 98%. But, when I put it into Hitfilm as a comp, Hitfilm's 3D floor plane was moving around the shot crazily. I tried to figure out why, and I made a point in 3D space and parented all mocha points and camera to it, but it still was moving around the shot like crazy with no pattern or anything.

I looked in the top view of the 3D space, and I saw the camera moves around a little at first, then suddenly jumps really far back, which  I never did with my phone. Is there a way to fix this? Did I track something wrong? I followed all the tutorials I could find to the letter but still am having this problem. Please help! I can give more information if needed. Running latest version of Hitfilm Pro. 

Comments

  • Ah, Mocha. 

    Small rant. Mocha is a bit of an art, and I've yet to see a comprehensive tutorial (because it would be three hours long) or a ground-up series. Which means one day I'll have to do one (after redoing "Essential Hitfilm").

    Here's a couple of facts I picked up buried in the middle of other mocha tutorials. 

    1) phone footage is almost always problematic - it's low bitrate and variable frame rate, which causes jittery movement and occasional jumps. 

    2) Mocha works better with PNG or TIF image sequences. Video is highly compressed and has to be decompressed before manipulation. Oddly enough a codec might not always parse a decompressed frame the same way (especially VFR). 

    3) When doing camera tracking track as LARGE a spline as possible, but define your surface as small as you can get away with. Large track spline, small tracked surface. 

    4) track as close as possible to where you want to insert objects. Mocha surfaces/tracks define two (or more) rectangles then measure how they move past each other. Mocha has no idea how "big" those rectangles are and how "far" they are from each other or the camera. Two rectangles moving can generate a literal infinite amount of mathematical solutions, and mocha picks one. I've had wonky tracks I've fixed by moving one spline a couple of pixels and re-solving. The farther away from a tracked surface, the more room for errors. 

    More later, maybe. 

     

  • @Triem23 The reason I was playing around with Mocha today is that sometime early 2019 I'm making a short film and there are a lot of 3D model alien spaceships flying around. For the scenes when they're just in space, I can handle that, because it's a completely 3D environment (with actors green-screened into the ships when needed). The problem is, I have a few scenes where there's supposed to be a spaceship (about the size of, say, a TIE Fighter) landed in the background of the shot. How do I integrate it into the background? The only camera I have to film with is my iPhone 8 (which shoots in 4K and hasn't failed me so far).

  • @Triem23 Many thanks for those 4 facts - they more or less explain the various issues I have experienced with the Mocha plugin. Looking forward to that guide

  • If you're doing very simple pans or dolly moves, use Mocha. If you're doing more complicated camera moves, even handheld shots with parallax, you're gonna need to use a dedicated 3d camera tracking application and not Mocha which only approximates a 3D camera move. 

    I tried matching footage from an iphone camera once and more recently an android phone. I never use video, I always convert to an image sequence. You may be seeing duplicate frames. Convert your footage to an image sequence and delete the duplicate frames. If you don't delete the duplicate frames, you'll get the exact thing that is already happening to you which is your camera will jump back and forth. 

    For a new user, Mochas 3D camera solver can't touch a dedicated 3D camera matching application. Mocha is actually really easy to use and I've gotten really great but never 100% accurate approximations of a 3D camera even when using many splines or as little as two tracking splines which is all you need. It works great with slow deliberate pan and dolly shots but tougher with handheld style shooting so you'll need to plan your camera shots beforehand if you go handheld. 

    If you're okay with the Mocha 3D camera solver and the approximations that it spits out, then go ahead and use it but if your doing more complicated camera moves, I suggest using a camera matching app. I have a free addon that exports an animated and or match moved camera from Blender into HitFilm Pro that always gives me above a 99.99% camera solve every single time, that is less than a 0.1% solve error but I always seem to get it just under 0.01% solve error, even with handheld shots. It is awesome and even exports your camera sensor width and focal length.

    I think experience will tell you this after a while or you can watch every Mocha tut out there and find out for yourself but Mocha is best reserved for roto work and very slow, simple and deliberate camera moves although it can help assist a 3D camera tracking app with blurry footage but that is a whole other process which I've never had the time to show anyone or create a tut for... yet. :)

    I've been semi obsessed, on and off with getting anything built or created in Blender to import into HitFilm Pro since 2011 - 2012-ish so I've experimented and tested and started over then repeated and repeated so many times that I just know by heart not to use Mocha as a stand in for a dedicated camera tracker because you will have such a tough time trying to line up the Mocha data with the HitFilm 3D floor grid. Where as when I use the built in Blender camera tracker and export with my free addon, everything imports lined up to the HitFilm 3D grid automatically so no fiddling around with trying to line things up.

    Here is a link to a few camera matching demos on my site. Scroll all the way to the bottom and check out the first three short videos. I export camera data from Blender into HitFilm Pro. Free camera exporter in my shop.

    https://www.owmycgi.net/blender-to-hitfilm  

    And here is my thread on the Hitfilm Forums... don't get lost in there. :)

    https://fxhome.com/forum/discussion/348/blender-hitfilm-presets-addons-owmycgi-net-design-and-vfx/p1

  • @spydurhank So what you're saying is if I'm using a simple panning shot like on a tripod Mocha is probably fine? That's all I think I'll be doing for my actual movie, I was just testing it with a handheld shot.

    Also, what is the best free 3D camera tracking software, then? What works well with Hitfilm and all that.

    Lastly, what did you mean by convert my video into an image sequence?

  • Hey @HeySiri ,

    Yes, nice and smooth camera moves like a pan or a dolly work really great in Mocha. You can get pretty close camera solves with handheld footage in Mocha but like I said, it will only ever be close and that is not good enough for adding 3D objects and stuff to your live footage. 

    Honestly, I don't know what the best free camera tracker is. I use Blenders built in camera tracker because Blender is free and in the past couple of years the Blender devs have added planer tracking which works marvels for me. It is fast too, it tracks the texture or surface of whatever you place the tracker point on. You can get away with less work. For example if you are tracking a floor and a wall, all you need is four tracking points for the floor and four tracking points for the wall and some parallax to get a good camera solve. There are at least three Blender addons that can export a camera from Blender to HitFilm Pro. One exports a camera in .ma which HF can import but it has problems and doesn't always work. Another is an old addon written by Mark Edwards from here on the forums which exports a camera and empties over to HitFilm Pro as a Hitfilm composite shot in .hfcs it however was eventually broken because the Blender Python API is always changing. I took Mark Edwards addon back in 2013 and started re-writing it for my personal use and have kept it updated ever since then. After correcting and adding a few things to the code, the addon currently in version 0.20 works really good.

    Video from a camera phone duplicates every other frame so you get a stutter type effect when you attempt a camera matchmove. You could either edit the video in HitFilm or other video editor by adding your video to the timeline and manually deleting the duplicated frames and then export the result as a video or a series of images/image sequence of jpgs or pngs. You can then use the video or image sequence in Mocha or whatever camera matchmoving app you decide to use. Image sequences load faster into memory making them easier to matchmove. :)    

  • @Triem23 You have no idea how much I’d love to see a 3 hour tutorial specific to all the functionality available in the Hitfilm version of mocha! I would definitely watch the whole thing ;)

     

    Ive got one particular clip I’ve been using as my benchmark - like once I can successfully make that clip work I feel I could do everything I need with mocha for the foreseeable future. Made a huuuge step yesterday finally getting my surface to track accurately through the whole clip (finally understanding that shapes aren’t tracks and can be moved around on the same plane to GET a good track, was huge). But now I’m stuck because although my surface looks good in mocha, when I come back to Hitfilm and try to place an object there it stays for a bbit while my camera motion is small, then goes all wonky when the camera moves significantly. I’m sure I’ll figure it out, and I’m definitely learning along the way since I’m pretty new, but I’d sure watch a crazy-long in-depth tutorial if you ever feel particularly bored enough to make that lol. In the mean time thanks for those 4 tips!

  • edited May 5

    So I’ve been working with this shot, attempting to place a CGI box in the background of this shot. It’s a handheld shot but I thought I’d try to track it anyway.  The ground had a lot of texture as did some spots on the side of the building we see next to. I tracked it in Mocha and it seemed to work fine, and I came out with a 99% solve. When I put it into Hitfilm it seemed to look fine at first, until I put in the 3D box. At first the box looked fine, it moved parallel to the camera’s shaking. But as soon as I started walking backwards with the camera in the footage, the box moved backwards with the camera. So while it stayed on the same place in x and y space, it moved with the camera in z space. What could cause something like this?

    Edit: here’s the original footage if anyone wants to try to track it. What I’m trying to do is place a digital box next to the fence near the wall, so if anyone can accomplish that please let me know.

    https://drive.google.com/file/d/1xhGVjb6l2osClwYn9oAtqSk_3XGJ9RVR/view?usp=drivesdk

  •  @HeySiri the grid option in mocha hitfilm there is an option to see what the track is actually doing but I will be happy to give it a shot

     

  • @Andersen01498 thanks! Please tell me how it goes.

  • edited May 5

    I had to stabilize your footage first before using Mocha. I got a 98% solve using 2 planes, up high on the wall and down by the corner of the fence, ground and wall.  The center point of layer 1 (the ground) does not jump around...but it's sliding. The camera is twisting left and away at the beginning  and you can see every wobble foot step (in and out and side to side) the camera operator is making as they walk backwards. Tough for me to get the points to stick solid.

    IMHO - Need to work on stabilizing the camera more and get the twisting out of the shot. Also need to up the shutter speed. A bit of motion blur going on makes it hard to get a good track  as well

  • @GrayMotion two things: one: I found a stabilizer that I can use and all my new test footage is smooth and nice, and two what shutter speed is good? I thought 1/48 if I'm filming at 24 fps, but when I tried to set it to that outside (I'm using an iPhone 8 with Filmic Pro) the image was just completely white and I had to increase shutter rate to actually see anything.

  • edited May 6

     @HeySiri I went as far as converting the video to a PNG sequence and then manually removing the doubled frames along with renaming the files in order. Then took it into Blender and then tried a track. I am getting a similar result to what @GrayMotion is talking about using Mocha. The solve is solid for the most part but not as low as I would really want to use, and the floor plane was sliding which with the type of ground that is in the footage makes it really visible if you are planning on having something in contact with it.

  • @FlyingBanana78 I completely forgot to convert the video from VFR to CFR, which is probably where a majority of my problems laid. Thankfully, that was just test footage and I really don't have any tracking like that needed in my movie, this was just for testing. When integrating 3D shots with live action footage for my movie I'll try to keep the camera as still as possible, for sure.

Sign In or Register to comment.