Most motion-based tracking algorithms assume that objects undergo rigid motion, which is most likely disobeyed in real world. In this paper, we present a novel motion-based tracking framework which makes no such assumptions. Object is represented by a set of local invariant features, whose motions are observed by a feature correspon-
dence process. A generative model is proposed to depict
the relationship between local feature motions and object
global motion, whose parameters are learned ef
ciently by
an on-line EM algorithm. And the object global motion is estimated in term of maximum likelihood of observations.Then an updating mechanism is employed to adapt object representation. Experiments show that our framework is
exible and robust in dealing with appearance changes,background clutter, illumination changes and occlusion