andthe3DOFglobalpositionand2DOFfortheorienta-tionofthebody.Posevariabl - PDF document

andthe3DOFglobalpositionand2DOFfortheorienta-tionofthebody.PosevariablesthatarenotconstrainedbytheKneedWalkeraremodelledusing(damped)2nd-orderMarkovprocesseswithzero-meanGaussianacceleration.Tosummarize,themodelstateattimetisgivenbyst=(t;t;xt;kt)wheretarethespringparameters,tistheprocessnoise,xt=(qt;_qt)isthedynamicsstate,andktdenotesthekinematicDOFs.Themodelalsode-nesastatetransitiondensityp(stjst1)fromwhichonecandrawsamples.Thatis,aftersamplingthedynamicsparameters,(t;t),wedeterministicallysimulatethedy-namicstondxt.Then,wesamplektgivenxt.4.TrackingTrackingisformulatedasalteringproblem.WiththeMarkovpropertiesofthegenerativemodelabove,andcon-ditionalindependenceofthemeasurements,onecanwritetheposteriorrecursively,i.e.,p(s1:tjz1:t)/p(ztjst)p(stjst1)p(s1:t1jz1:t1)(12)wheres1:t[s1;:::;st]denotesastatesequence,z1:t[z1;:::;zt]denotestheobservationhistory,p(ztjst)istheobservationlikelihood,andp(stjst1)isthetemporalmodeldescribedabove.Likelihood:The3Darticulatedbodymodelcomprisestaperedellipsoidalcylindersforthetorsoandlimbs,thesizesofwhicharesetmanually.Thelikelihoodisbasedonanappearancemodelandopticalowmeasurements.Thebackgroundmodel,learnedfromasmallsubsetofimages,includesthemeancolor(RGB)andintensitygradi-entateachpixel,witha55covariancematrixtocapturetypicalcolorandgradientvariations.ForegroundpixelsareassumedtobeIIDineachbodypart(i.e.,foot,legs,torso,head).TheobservationdensityforeachpartisaGaussianmixture,learnedfromtheinitialposeintherstframe.Opticalow[9]isestimatedatlocationsponacoarsegridineachframe(e.g.,seeFig.3,row2),usingarobustM-estimatorwithnon-overlappingsupport.Theeigenval-ues/vectorsofthelocal22gradienttensorintheneigh-bourhoodofeachgridpointgiveanapproximateestimatorcovariance.Theobservationdensityforaowmeasure-ment,v(p),giventhe2Dmotionpredictedbythestate,u(kt;p),isaheavy-tailedStudent'stdistribution;i.e.,logp(v(p)ju(kt;p))=logjj 2n+2 2log(1+e2)+c(13)wheree2=1 2(vu)T1(vu),n=2isthedegreesoffreedom,andcisaconstant.Thecameraisstationaryfortheexperimentsbelow,sotheowlog-likelihoodformeasurementsonthebackgroundismerely(13)withu=0.Tocopewithlargecorrelationsbetweennearbymea-surementerrors,wedenetheappearanceandowlog-likelihoodforeachbodyparttobetheaveragelog-likelihoodovervisiblemeasurementsforeachpart.Toavoidcomputingthelog-likelihoodovertheentireimage,weonlycomputelog-likelihoodratiosoverregionsoftheimagetowhichthe3Dbodygeometryprojects.Then,thetotallog-likelihood-ratioisthesumoftheappearanceandowlog-likelihood-ratiosoftheparts.Thisyieldsthelog-likelihood,logp(ztjst),uptoanadditiveconstant.Inference:WeapproximatetheposteriorbyaweightedsamplesetSt=fs(j)1:t;w(j)tgNj=1,wherew(j)tdenotestheweightassociatedwiththestatesequences(j)1:t.Giventherecursiveformof(12),theposteriorSt,givenSt1,canbecomputedintwosteps:1)drawsampless(j)tp(stjs(j)t1);and2)updateweightsw(j)t=cw(j)t1p(ztjs(j)t)wherecisusedtoensuretheweightssumto1.Thisapproachoftenworkswelluntilparticledepletionbecomesaproblem,i.e.,whereonlyasmallnumberofweightsaresignicantlynon-zero.Toavoidsevereparti-cledepletion,following[7,10],whentheeffectivenumberofsamples,Ne;t(Pj(w(j)t)2)1becomestoosmallwere-sampletheparticlesetusingimportancesampling.Insimpleparticleltersonere-samplesstatesattimetinproportiontotheweights(treatingweightsastheproba-bilitiesofamultinomialdistribution);thenewweightsarethensetto1=N.Here,following[4],ratherthanre-sampleatthecurrenttime,weresamplefromaprevioustimets.Thisaimstore-samplebeforetheonsetofparticledeple-tion.Italsoallowstheproposaldistributiontodependonfutureobservations(i.e.,thosebetweentsandt),sincethequalityofasampleisnotalwaysimmediatelyevident.Asaproposaldistributionweuseamixtureoftwomulti-nomials,onebasedontheweightsatt,andonebasedonweightsatts,withmixingprobabilities\rand1\r.Importancere-weightingisthenneededtomaintainaprop-erlyweightedsampleset.Sothenewweightsaregivenbyw(j)ts=(\rw(j)t+(1\r)w(j)ts)(uptoaconstantsotheysumtounity).Thus,mostofthesampleswillcorrespondtoprobablestatesbasedonallinformationuptotimet.Theremainingsamplesareprobablestatesaccordingtothepos-teriorattimets.Withthisformofimportancesamplingwere-samplelessfrequently,andthetrackerismoreef-cient.Inpracticeweuses=3and\r=0:95.5.ExperimentalResultsWenowdescribeexperimentalresultswiththeKneedWalkeronseveralimagesequencesofpeoplewalkingonlevelground,withocclusionandchangesinspeed,andonhills.Inallexperiments,wehaveroughlycalibratedthecameraparametersandthelocationofthegroundplane.Weuse5000particleswitharesamplingthresholdof500.Theinitialstateisspeciedcoarselyintherstframe,butwithalargecovariance.Onecouldalsoinitializethetrackerwithdiscriminativemethods(e.g.,[1,26]). Figure3.(Toprow)Compositeofimagesequenceshowingawalkingsubjectandanoccludingcyclist.ThegreenstickgureintherightcompositedepictsontheMAPestimateoftheposeonselectedframes.(Secondrow)Examplesofthebackgroundlikelihoodandopticalowmeasurements(yellow,blue,andredowmeasurementscorrespondtoslow,moderateandfastspeeds).(Bottomtworows)Croppedframesaroundocclusion.Thegreenskeletonandblue3DrenderingaretherecoveredMAPtrajectoryfor10consecutiveframes.Experiment1.Figure3(top-left)showscompositeim-agesofawalkingsubjectonnearlylevelground.Thescenehasharshshadows,backgroundclutter,andacyclisticthatoccludesthesubject.Fig.3(2ndrow)showscroppedex-amplesofimagemeasurements,includingopticalowes-timatesandthenegativeloglikelihoodofthebackground,earlyandthenlaterinthesequenceduringtheocclusion.Theyareparticularlynoisyduringtheocclusion.Despitetheocclusionandnoisymeasurements,theesti-matedmotionwiththeKneedWalkermodelagreeswiththesubject'sgait.ThegreenstickgureinFig.3(top-right)de-pictstheprojectionofthe3DkinematicmodelfortheMAPstatesequenceobtainedbytheparticlelter.MoredetailcanbeseeninthecroppedimagesinthebottomtworowsofFig.3.ThesecroppedimagesshowtherecoveredMAPestimatesfor10consecutiveframesthroughtheocclusion.Thelastrowshowsa3Drenderingofthemodelfromadifferentcameraviewpointtoillustratethe3Dposeineachframe.Thevideointhesupplementalmaterialdemonstratesthattherecoveredmotionnotonlymatchestheimagedata,butisalsonaturalinitsappearance.Experiment2.WiththericherdynamicsoftheKneedWalker,wendthatthekneesandtorsoareestimatedmoreaccuratelythanwiththeAnthropomorphicWalker.Forex-ample,Fig.4showsresultsonasequenceusedin[4]inwhichthesubjectslowsdownfromroughly7km/hrto3km/hr.ThecroppedimagesinthemiddleandbottomrowsofFig.4showMAPestimateseverytwoframesfortheKneedWalkerandtheAnthropomorphicWalker.Thesamelikelihoodandnumberofparticleswereusedinbothcases.TheKneedWalkerestimatesthekneeposemoreaccu-rately.Interestinglythisistheresultofasimplerpriormodel.Thatis,whereBrubakeretal.[4]useasecond-orderkinematicsmoothnessmodelwithanadhocdependanceontheanglebetweenthelegs,ourmodelusesapassivekneewithasmallamountofnoise.Thekneebendatthebegin-ningofastrideandthestraighteningtowardstheendisafundamentalpropertyofthephysicsoftheKneedWalker. 6.ConclusionsandFutureWorkThispaperintroducedtheKneedWalker,acomplexphysics-basedmodelofbipedallocomotion.Aspartofthismodel,weintroducedamethodforhandlingjointlimitsinanefcientbutphysicallyrealisticmanner.Wedemon-stratedthatawiderangeofrealisticwalkingmotionsonslopedsurfacesandlevelgroundcouldbefoundthroughtheconstrainedoptimizationofenergy.Whenusedinatrackerwithasimplecontrolstrategy,theKneedWalkerwasabletorecoversubtleaspectsofmotionsuchaskneebendandtorsolean,evenwhenthesewerenotstronglyindicatedbytheimageevidence.Further,asseeninthe3Danimationsinthesupplementalvideo,therecoveredmotionsexhibitahighdegreeofrealism.Inthefuturewehopetocombinethewiderangeofmo-tionsfoundthroughoptimizationwithmotioncapturedatatolearnbetterstochasticcontrolstrategieswhileretainingthediversityofmotionsfoundinoptimizations.Finally,weexpectthatadirectgeneralizationofthemodelandtech-niquespresentedherewillallowustocapturerunningandother,moreenergetic,humanmotions.Acknowledgements:ThisworkwassupportedbygrantsfromBellUniversityLabs,NSERCCanada,andCIFAR.References[1]A.AgarwalandB.Triggs.Recoving3Dhumanposefrommonocularimages.PAMI,28(1):44–58,2006.[2]A.Bissacco.Modelingandlearningcontactdynamicsinhumanmotion.CVPR,v.1,pp.421–428,2005.[3]B.Brogliato,A.tenDam,L.Paoli,F.Genot,andM.Abadie.Numericalsimulationofnitedimensionalmultibodynon-smoothmechanicalsystems.Appl.Mech.Eng.Reviews,55(2):107–150,2002.[4]M.A.Brubaker,D.J.Fleet,andA.Hertzmann.Physics-basedpersontrackingusingsimpliedlower-bodydynam-ics.CVPR,pp.1–8,2007.[5]S.Collins,A.Ruina,R.Tedrake,andM.Wisse.EfcientBipedalRobotsBasedonPassive-DynamicWalkers.Sci-ence,307(5712):1082–1085,2005.[6]J.Deutscher,A.Blake,andI.Reid.Articulatedbodymotioncapturebyannealledparticleltering.CVPR,V.II,pp.126–133,2000.[7]A.Doucet,S.Godsill,andC.Andrieu.OnsequentialMonteCarlosamplingmethodsforBayesianltering.Stats.andComp.,10(3):197–208,2000.[8]A.ElgammalandC.-S.Lee.Inferring3Dbodyposefromsilhouettesusingactivitymanifoldlearning.CVPR,V.2,pp.681–688,2004.[9]D.FleetandY.Weiss.Opticalowestimation.InMathemat-icalModelsofComputerVision:TheHandbook,pp.239–258,Springer,2005.[10]A.Kong,J.S.Liu,andW.H.Wong.Sequentialimputationsandbayesianmissingdataproblems.JASA,89(425):278–288,1994.[11]A.D.Kuo.ASimpleModelofBipedalWalkingPredictsthePreferredSpeed–StepLengthRelationship.J.Biomech.Eng.,123(3):264–269,2001.[12]A.D.Kuo.EnergeticsofActivelyPoweredLocomotionUs-ingtheSimplestWalkingModel.J.Biomech.Eng.,124:113–120,2002.[13]R.Li,T.-P.Tian,andS.Sclaroff.Simultaneouslearn-ingofnon-linearmanifoldanddynamicalmodelsforhigh-dimensionaltimeseries.ICCV,2007.[14]C.K.Liu,A.Hertzmann,andZ.Popovi´c.Learningphysics-basedmotionstylewithnonlinearinverseoptimiza-tion.ACMTrans.Graphics,24(3):1071–1081,2005.[15]T.McGeer.PassiveWalkingwithKnees.ICRA,V.3,pp.1640–1645,1990.[16]T.McGeer.DynamicsandControlofBipedalLocomotion.J.TheoreticalBiol.,163:277–314,1993.[17]D.MetaxasandD.Terzopoulos.Shapeandnonrigidmo-tionestimationthroughphysics-basedsynthesis.PAMI,15(6):580–591,1993.[18]J.NocedalandS.J.Wright.NumericalOptimization.SpringerSeriesinOperationsRes.,2nded.,2006.[19]M.Raibert.LeggedRobotsThatBalance.MITPress,Cam-bridge,1986.[20]M.H.RaibertandJ.K.Hodgins.Animationofdynamicleggedlocomotion.SIGGRAPH,pp.349–358,1991.[21]G.E.Robertson,G.Caldwell,J.Hamill,G.Kamen,andS.Whittlesey.ResearchMethodsinBiomechanics.HumanKinetics,2004.[22]L.Shampine.ConservationlawsandthenumericalsolutionsofODEs,II.Comp.Math.Applic.,38:61–72,1999.[23]L.Shampine,S.Thompson,J.A.Kierzenka,andG.D.Byrne.Non-negativesolutionsofODEs.Appl.Math.Comp.,170:56–569,2005.[24]H.Sidenbladh,M.Black,andD.J.Fleet.StochasticTrack-ingof3DHumanFiguresUsing2DImageMotion.ECCV,V.2,pp.702–718,2000.[25]C.SminchisescuandA.Jepson.Generativemodelingforcontinuousnon-linearlyembeddedvisualinference.ICML,pp.96–103,2004.[26]C.Sminchisescu,A.Kanaujia,andD.Metaxas.BM3E:Dis-criminativedensitypropagationforvisualtracking.PAMI,29(11):2030–2044,2007.[27]C.SminchisescuandB.Triggs.Kinematicjumpprocessesformonocular3Dhumantracking.CVPR,pp69–76,2003[28]A.Thayananthan,R.Navaratnam,B.Stenger,P.Torr,andR.Cipolla.Multivariaterelevancevectormachinesfortrack-ing.ECCV2006,V.3,pp.124–138.Springer,2006.[29]R.Urtasun,D.J.Fleet,andP.Fua.3DPeopleTrackingwithGaussianProcessDynamicalModels.CVPR,V1,pp.238–245,2006.[30]R.Q.vanderLindeandA.L.Schwab.LectureNotesMulti-bodyDynamicsB,wb1413.Lab.forEng.Mech.,DelftUniv.ofTechnology,2002.[31]A.WitkinandM.Kass.SpacetimeConstraints.SIGGRAPH,V.22,pp.159–168,1988.[32]C.R.WrenandA.Pentland.Dynamicmodelsofhumanmotion.IEEEFaceandGesture,pp.22-27,1998.