hi,

I'm searching for the best method to extract the movement of the camera and the movement of the scene out of a video/several videos. the content of the videos changes between indoor/outdoor. The system has to run in realtime so a simple approach would be preferred - if there is one at all. It should work accurate but speed and simplicity is more important.

I imagine something like this: http://studierstube.icg.tu-graz.ac.at/handheld_ar/panomt.php .... but used in a more artistic context.


... now my description sounds like I wanna buy something but that's not the case - just wanna some tips into the right direction. I already thought of using optical flow for that task, but I'm not sure if that's the right way.

edit: watched the video once more - seems like their using sift / surf for that task - could that be?

thanks in advance
andreas

Perhaps you should take a look at
viewtopic.php?f=9&t=1807&hilit=optical+flow+motiontracker

pong0 made a MotionTracker class based on Rui´s ofxCvOpticalFlowPyrLK (Anyone correct me if I´m wrong).

It´s very easy to use but a little bit porcessor demanding.

Hi,

You may use BM or HS algorithm which is i think faster for your needs (depending on the pyramid size and number of tracked features).
Anyway you will find what you need at http://opencv.willowgarage.com/document ... cking.html

Bests,
F.

this is called 'ego-motion tracking' in the computer vision literature, if you feel like reading journal articles.. ;-)

thanks for the tips. anybody knows a ready to go library for this? Will try it with the opencv functions first.

nice sunday

ok, just remembered the PTAM system (Parallel Tracking and Mapping for Small AR Workspaces) from the Oxford robots group: http://www.robots.ox.ac.uk/~gk/PTAM/



something like this? :-)

looks good - but you must initialize the camera first .... I wanna use (ready made) film material so this system seems not working for my purposes. what I don't need is the z axis movement, just pan,tilt and x,y movement of the camera.


what I need is something like this - http://studierstube.icg.tu-graz.ac.at/handheld_ar/panomt.php ... but they are not willing to share their code

andreas

if somebody is interested, I did some further research in this topic and found some interesting links:

Libraries:
http://www.doc.ic.ac.uk/~ajd/software.html
http://www.robots.ox.ac.uk/~gk/PTAM/
http://www.robots.ox.ac.uk/~bob/software/index.html (fork of ptam)

Some interesting works collected here:
http://www.doc.ic.ac.uk/~ajd/index.html

Hi,
to determine the movement in an image i've used optical flow. simply average all the motion vectors. it may not be the most accurate but its quite simple and for me it worked great.

hi pelintra,

that's not a option for me because I need the "egomotion" of the camera - if there are moving objects in the image but the camera stands still I get wrong results.


... the whole topic seems more complicated than I thought in the beginning.

recently i've discovered some method for bg/fg segmentation in opencv that seems pretty advanced, google for opencv codebook. don't know if it's something new in opencv 2.0 though.

perhaps that, mixed with some kind of optical flow like lucas kanade, can be the solution, but you'll need to measure the distance of the background in some way to know the relative movement of the camera.

also, if using optical flow take a look at cvGoodFeaturesToTrack to find the points in image that are good candidates to calculate the optical flow.

thanks for the tips.

using the codebook method is a good idea - and no, it's not new ... there's a chapter about in the learning opencv book

found some good solutions for eliminating the ouliers (parts of the image that move in a different direction than the whole scene) - tried it with ransac (http://en.wikipedia.org/wiki/RANSAC) but now I'm using a m-estimation algorithm (http://en.wikipedia.org/wiki/M-estimator). I'm a bit under stress at the moment because the project has to be finished next week but when I have a little time I will post my code here.

... by the way - never got the opencv function cvFindHomography to work.


m9d

Hi~ for cvFindHomography( const CvMat* src_points, const CvMat* dst_points, CvMat* homography, int method CV_DEFAULT(0), double ransacReprojThreshold CV_DEFAULT(0), CvMat* mask CV_DEFAULT(0)) you could use CV_RANSAC in the parameter "int method". it works apparently better when there are lots of outliers

besides, homography matrix is only useful for planar scenes. that is to say, if your camera view the scene from two different viewpoints and gets two images; you extract the feature points from these two images and make a matching and hence get a set of corresponding points(such as the routine of cvsurf). you put these corresponding points into cvFindHomography() to calc the homography matrix. but these corresponding points will obey a certain homography matrix only when the scene is planar. if the scene is not planar, you need to use the fundamental matrix or essential matrix