From 485518cb59ea00b452c2a6f35b900f6f02b9d8b7 Mon Sep 17 00:00:00 2001 From: krzychb Date: Fri, 4 Aug 2017 22:17:31 +0200 Subject: [PATCH] Target change, clarifications and formatting updates --- docs/_static/app_trace-overview.jpg | Bin 0 -> 26583 bytes docs/_static/app_trace/overview.png | Bin 8554 -> 0 bytes docs/api-guides/app_trace.rst | 395 ++++++++++++++-------------- 3 files changed, 195 insertions(+), 200 deletions(-) create mode 100644 docs/_static/app_trace-overview.jpg delete mode 100644 docs/_static/app_trace/overview.png diff --git a/docs/_static/app_trace-overview.jpg b/docs/_static/app_trace-overview.jpg new file mode 100644 index 0000000000000000000000000000000000000000..559051f8da390a4c25db6d2346bcd9e586140396 GIT binary patch literal 26583 zcmcG$1wd5Y)-XKG(B0jwAl=>FA`Q|ZAl)L}4IMB!aoE0JU;hc z@B4oL_y5B=XZBuu?bU0Yea@M~?Wfxn0J@x%tP}tW3JPEd{sY|31H=LFu&{8jFz|42 za0m$Sh{)I|$Vf=Y1ejQ8*d&Bxq$GsI#N^aWbmWwbRK&#e+z%L8SUEU2$mnEzu-~_+eC?mkmCO?JVs)}@fz*Ae@clRPC?F*^;tFb1~ zu;Kd$5*QoOu@M-X}( z=e2}-@7Jh1z5XKTpsWKn6!-&wLKXj~>C$neqFEVmBAWIM=r5IV4E6&_UB!D)Lj){E z?xs-nN6Y+SE)=PR2drIzuf-fx*5;cE2Jn|9>`VQhMv{&rj{U<#pg`ye0HFEFH0qgv z?@yfHR6OpEkEW&ygI^QliTz_kIIj{i6Rf@OkSquC<+x$Ri=>;c>mp-OAmPboC?bJp zDaY9sy?$AQurDkLjQtE*L&b71Q{x8 z+~O3B1uaB!iEFs|rLShnEW>EYlow0}_9q4K@Gs{;dthu3{DAcl0mIllhmTEC=O%IR zd|u+30$<3eDYH^)(KItZu#4S7-+Sv$dO0smRR;TbWZSq<{7x41OScv6fTZ`NV^3bY zgm}bmfyw+L2X6jy<6mKx-(3E1W+B{;D?VOE<1W(f1@}{aX#aNGhWt8{=N$e8{}JOl zBmnLxp<*n?r07UFs_6wmaEFOoFV8*8s^9Vthmh}Vj}wnZzX+p`AHD&HBbX!j^UJ4T z75mx0Bj5Nw(dlJh;MAvL``BAeQ|kc|%o$?}ME|N|0OKXa9)&;3VZt*Jxh}=#hRK8L z0C5&FgcwGnYHvb;_(5ux_EX2tIImsBOX^nIrh%mk%NAR1M9;cf9@HQwwURl}Pel%w zcAK2T+$GYz3^A0va`7xT7KUD{;F8sgoBdf07G3-sd+QpaltFiZ>LLq~JL_7@Cy4E*zvpCb% zW4w7Q>e};tRAn%p?H0AXKfrQA-Gvjc*YWe!{F4}@CASW48z1% zkkGccN!kZT^15i|)p!fE5N1um+xw`$1-!X}M0xG1roHl=CBT*;_&X;odI}Mn{oQ|Z z16;7PF5KnDHx30bz18gg&k$bV5ca?rfPCL4E~It$`8&l?4X(We0RWtHoK4@KgJ20A zMIh}MO5qeU)~`&?k&5BF+I9um5Fq69%QXPNT_gbj3|H$YD|bV83jUn_rIVp~@QF*I zA1EXFkM%&pjm_|WCPbN24P3^_+yfb=6E0)$ank?mA-C!M@UKMp$>#SjNEz6x9;43A!pOVT%6=Hg^98|Po3T0bI6;Ut@#{8a(}ozQ(?l-2lpr6eETMf!d0auAT% z7d8-NUaVD<+yc5-zew&6_qRzB+dixjbf$*+J1+l?O$pBkjfK z`)`x~{{#qwS3S(=04NX;1P2XX7Xd&JZXf^<8VUvzn*$vK3x|cBlbni8go0YdP?FU* z7Q7sS2jfA(0B-?H&l=zANik93VMK)XbQ`a+I^Z`%{t+-(PW7-MvSf{|H1xjtH>SlU z-t*!kPQz#IQ+}l&4cIaiEyBwGZ8xv37VZ*p<{U?t^2rbdjVX0mU}9{1HKvA`T)(U~ z0s{n8sw!joRN@xk>*M>@$4KJj2Lo5gPSz3H5l+QEedbfs55yGf4H9E)5!`YtQaRqZ zwrCTAZlG%z7eu_$mi#Do z7FlR^#msUz`OY~-H~55fJZVm9;7HDQy7Cnt-|`%Zlf$r<#Nd(Bdelhb=LA_qSl`@> z#l(zAURl7=eY3WI_dDvHfcIPicfJPkd(YZ6Ke{k$+t1W3+HKpfUwnO4z3i%cIJih~ zZ0wCSV=vsUsk)bR{C)e5S>6=$`q`L%`z-)UU%F(~6`%l86A`NV+(0IK*4bN|RvD+7 zuWp4yC@J-OLrbPI&WN&8GB5%G7n70tPu)-jltv zAh;l26&c0e+Q-vq3exCLbo|Q!CG*NpesjvhNi{;;0n}0|sVG_ry?^)DoeQH-W&Gfy z8L`3ib9pv~%YXCd-!A6hrG%#BsdUAwUJg%{MuaySplADs>wluE`wfH`ROO6cV0D?S z9$N^7>g()bD>Hwnv2>N6m~XL@Gl*VXSG@%QPIbJz2|W4TqCOc6y?nU7ZfyCT`}y(* zh(=gLr)6^_@u#j|h-yz#5pc3}%Fc9*o}8i1Lhz-pK@(O7r)E0`OWa0YikGD^QDJ#I zi_JbVKUvku0dlr;i|dDL+nh-Cblsxnj@IL}UX{FUM)$@%wWpObdoPHCYx68b5XU07 zG-@d>jqnygJXL!mIgylC@4gsxz`a&3#6=sE=X&si!o|8sRp7ncNRITo-bC^>d_&!q zXR*>O!SO5UYQ*cLvN|?jsvi%Ha%~O1evmj)Lypu%2jyB#iNXrO5-bi;Y-QbzGx!0O75;vtF?0ZL)XzK)b zLQ({ZarGM;9o9|~AP$6B(T^C(!umhT+SVr@c)AjH_`+2kW1U5w37P08S)vaR^(J5T z<$j$1Bp1a4bcvQp(^P-;N+wyG%CwcFxpyBNUL>c>4#!m-c2uy_7(vhb;sUJUJL{@} z7uD*HjrxBXx#p%^fC4*Gru!w``w-fteWO|P@%^OS5Z!WloK@o%M$Mq`qbdVjhRz;$ ztqLRv=mBc8+1n*fvZ|fh^K%Wq_`z`(b!#h4_$QXVtlXxp#K<$uKYFf0Q`)ec+@C|G z%fFR&GJ8iGt*A2)k02Mf>4OsOwbW-AdkgR$=HRY-gJCNNv1H+0xa<>jt7EaOITiJ) z)w+-b?oeiHSC8b(LX6e=;4WtNJs%0yF-8-69_OO;V4?K6L5-T^M+Q3_T%FuSGs>7%_9X`;nyy#bwoN1rLoom5_8zFwls z-1)yfvqN8G{{0Ta$m`K>;iiv%p9%Dmneh^1mkQDC<*ByIw3s|3E zmL_zzeD}#sREhVBG~z#pPr>0^NWt6{sh3QF#>MbAkYh_g6{OT{vFd*&liyUOX3dk{ zu7}Z|1Ml&LpeAbdwR5;!{`8ZHF|9doI)R@&&0lavaQAL)Jg{};jk zL5=&(-&Xri@%-3LqxMHoH=}$O8|9XSUVlUih7%=|spN2IV;mj?q!HJm;wo$V6qm8n zpZO0sQt4_yBw`-=A2AXpgd`sK6aPsIqY&HIzYi6)^IZ4%aq^ld@>(SEU(Dq5N)7&E z8E>Lc@d1b1#aj8(Ro}ih<$2oiS%l_UkK&@v$8(|MD>9k)aop>laVtKDr#00T?t8DF z3&r_R*!4|6dOCx{2ZqRfkU;kIS9x!fA~(&Frp<^2a&$I|)|?SG*~L4-z4lfkJ(jN5 zk9fFx)-&~1*syF3-r-HCGSp?BpKrbex^1WdwR=3(kARzKM5_x(4Z{gxj+8IDfxx` zvr&a394`8U@}s-%u_jqhlVC2=x}&w{#`M|1+s_HQGqm;PigtRbOO2eZU~rmU1lDfE z)n>s_{a>ZCSd#psk`#8r5$8z}w>-Nq>!9ngqLsR=cCe5M{?>>|=`c4t-*nx_Pu<+d zM72+snK%AUmS@&-n;ImigDJSY;U}-<(?n9M@jI*Iq+~(^*!OVzOI|p}MHBn0M6riK zKgZ$%GrkUbj{_-N-bD4b@(y#n(nCGOUna|D#;Yv^b8M1Y5;K~_>4f}VY=`G?%3LH5 zQ*AXSnhiv=Y_!xtceVM)e&t2_n}`>C75De@#nu=cQdtN0qS(?NmlkDI)fR;S`eJ~( zxon~W4!*Q_-V}ui1C@%Mja1dXFD*)(B^!m$i}$fV4Pfy@&t2>Re%#;<6|4QZezO;Y z;wgNwY}9K{U$!yY#8@4Y_-M6mh8VnD<}XR$b1J0vaJS|eZU)9Ex{%IIN9CErOes<_e*GOl16H|3* z5h|F1)=C6cy!y@0v`EOEcAK!1+G28sHwa)KNK=oJDA-I=3iwB_^VBg!nR~@NN#?@C zpEY_31Mz{`-)2NF!$Ih@m%xuf$WvW)mP7H6WJQ$Gk&}kidlCu9_^6bHtJI%~$|NZ4 zt@WMzew&I9iXE`^z!xQqq_4z{Te2LzGT z>qD5EJ|>XVt}1HiTr*NP(NIRA* zPP3x2Yta;zN0MH6fVcjaG)2|DE>zw08@6~CL*TQol1UgednAh!X1Vc`VY$XIP1c87 zGx4ImJN1P_Y{1r3A<0lvEm!hpu4 zK*wTb;}jKBb&9K^WMLOEa&U}?A*XT<$i?PRF*Nq8?wB}I!x8t-Sv`LI*M$hIF!(~` z3uit|^x|id=4pPnGS@F!2TN7+ttR!Jeb3MSSKu-)w^GtS&A$cMYoT2>4O}p)REhN< zVgH|fdHZTFUcmT!(h0i^>T!L@LWnJq;jU%OBeK_);##SWwCym%pZ6bRt%rGK@*Ii2 zcL7&5s^Qmgsf3FK9EMQ-^|W=;0xrT%`i~hwD)8x8nfsWeuUKeSyZouW7G18>h2CjQ z*AH$5Eu|3JEhoCG0 z7CaM%H_aQpJItEE>W#@<3x6nvstsXb@f|0m31}1x)kS@t{#l}gQ6d;=aYLfFj*mEn z%hv;~4C}K*1R`lb3Tox`$4va0&JgnL5PL+{;)xb^RDz*zO5UG;49$CJDQu*nQuEk^ zTEyTgCV84_tTH$EwZXfLI;0o>x-%?@Ar;7${?bE$?)+@^K)0{#rz_l*4?cIz)yHUgq?gSw zH!vP$&Cj&(Rxz+1(02E~g@!{2J9C4`wfEdLJEV3h#X0NYJG#NZwHGCynrcIg`Rx4F3ta@DSuL)w5d-w*!1)c{L*0R!;99R z0|}e@%)jtoKKbyX=;~=ApGU@MO{b5YLFATCgPj>HHvp4Ff6K8(uPN=C2DRZ6a`cXE zt41X#t4=K-<0VTdit`&bC-N`(b%&#!16=FZdY=X8G|m1?{9~O4T0g@5a|-ZR=`BzM z2BET^1{&$ty!unV{7X6^QY)XqFT76hrDyg?OK(@~gV^oNqf?EfYURVjUn zq&8~Pp+vog7q)=U-*>7s66~`%Y00Rpfk!quh@t*V{{E83_ZCpl9j3#erL#I#cWlIY z6!^fgZiEk-{FBE#MFy-lvAC;yP_ZhJ z&WsK<@73$cd!On11m-x#)jWh9Y2hv5ZecHO8+ioDgY(8+vXJl3TMvUfSAFB2_vh}h|lMcXv6I`fr?E|4V-DClKeg}m9mk}b|ysCuozS0 z1|vNi;Y$!k;LqY;jH9iH~EdMl&}h^DG3zs94%ur`tKdwaC7kTE!`5?$j=CvxuEzjvQN5DLhUmn)~r^I zPU~c#5@`+GKs_3yA~tGho8jOR(;s}ZMWQ|W#O60yqo4m}F-Tw?F0DP;+;Vj$HDZZn zqc_!k_gcQ>ymFrzjb)4-Bd0t(7Y5O0hAa%?d z61N3kI(|$pW709Pr?KM}0kE>ztgSE>`o$VDjv_3~TKj{0!X8%Iw%aGNm|uMePK@}=92s0y5v*-L4Xt&=#`O4gVmUm4}dhO6j(*t&DBhyHUncg76! ztvIhDI2zZlT6L6Lgbs3vHI!QzCQ!JHT`5Jdc$8{&w%AIefh>s{M1&p1xtb>Xiya~* zesu zTB1>6w1x<#jM%NJF=HU*z||xgfb)w%ool%eVZy9GTtC;QEctRHnPnLxXY^fhMuK|% zgspXsT#a*jc9$Vc7mlya*s5p%%7{1RU`~Nq;d$d-^3T30&9DgEI=m0$w!A`>DK{fA zYIZJS7(9Le7CnY2frL zy}j@@bOQ6z{3G^gMX(%J`Zn*VC3VB-#=m7B(-Y^T01FG&{~&t~2xRe2GHx`hffq%U2k8{twA}#TRbDc+zcSz|1d;qEAm7F`y!H5@0IzN{)<&zgByeF>_CZ$ z(;vK7o%iZ733+htR@gXVUnH)_ukilWjzZ8tIRUS$vQD?R+Fs7LFfl5vt=g5h8IW@1 zyDeUF(lKZowmH_u5&?yewTZ0sbb%ok>Fy6P78Xjzhm*|w5HY3(@99F&pww^?5!caH z(bCaq!4*fHe1_5Mzl`XEvGDx9kaULN5=+$$Lp!Dk2&cMOCWLRpX}&JQrAz8LkEVJ> zlm>TFEYOxxCz)Jwr7M>I%D8_RB_fcI_^Z4Q%PgVFfK^=$m;9wBI6e>I2KD2g1noVE zXkyYhL&e@!txo+b$oGHz04KTdFF?i7tACzHXWx|QeMYWTdj9wN()B(}cpH9I=EJe& z|9UR|o5rj9AGjPEJ;jtAMT}W;CuRSOy;#D?iiuWgUn7zx5C#z=TTRx)%tS-^jmQ6H zSC=vNlS@mLdg$ASz{3J^__4eoROU)Z>^2>_4V*9@IIVd2KkZ@2Di0nz$@Hk;uhPK+ z?v__1QCJQ}P~yRs*u{0(#{a5BE(|b%H6*^;n_=+xJ)1>ky}Hc2AqnZO$}_nH>(Ft# zJ@s+BKzr=tTAVYiCcNHNCN%O$ZClGM*U8Q0;=+ad;+c4IzE59>mUpU7<1~R~n%Uya z^5W$3DsMputIRDR2DK6#Ku%?B6THo@?4VREcKWc?9XA#)ALmNtHYGsUJ}I?T9#H9f zNDa6cDYv55_LpOxn8!pR8Q=7;V9-hp?6=Q=)%ZL8&y?+G_M;t|_e# z!lMF{ib#6|E56vnvdyR-x~U6teySXj*g8};-PDrhBC@jIn4sAAIZ3}%Ppzx08z~c} zL8Uou9C6BAXEYXbLZGZn3@6G{V@@hlRnq{kzbZ#Qx>eULUrdu_jy&G8BMaP0#j_6z zWTY@cf>*CkhngZ-47t=2>%>K`3<|klkdT5D3J|e^7v|(X*^cA~uGSB~4PIv3ncoth ztsf4lu}g8b{mK;8W;Rhj+{E=QN$PG)pXkLc^&qu!(c# zu*I;qKCC&_mt0XA!}?T{)VQr&pvPO!Bm8^ASIF?gC`{;w(eG=OEFd~OwQG5am1?P% zZ!_puV~6@I>I-rsym$%Hvc_{Pr66LUI70xk5lR0Qd}5D*{2SlQGdi&_4MV)a4i4gp zpbt!xMyqf!h6F_;&s4)41afiER1>X(EOMmT=YcfCL( z)}9d;Yk+`^s)7vbu=Kk(=94(mDLn6@w}z25Sdfr!0VBM=p@TyYK6*={b)*obs~wNAwoqq((-gGbk$LAWekFk=*jjZeY=nw7p}!R9)zqy+uS zjfTy$yrVGD7!vlkQT6v&Tagy%0NWZjpGDKd@Ft1dG&Z?9&mU9PS=;POQN>W&w__1( zekvfpdJv~m_s&(%C*t7ZQpxJqI-6-|SH)tTv=w|I@S1P_1T~ToxwRF>vLJdJt z3|87vAxCe7mn*J<{5RFljcY`iC@K=^)Y$WwHQ9cC;B|S zt;gF;t6?);6ZK7n9Yq6OFEpG|Oh&ec#F_x&)wI-qF%4GcI&G~&_P8Fgy@rv^#R95T zGJU)$u(BDt`;+bAj6+AkbPCZcJ9mqs3%|jaY=$M0qUy4JKwwTRW}-=eT7_n!&O73h zcvx&MBkGaE=cMB~o#}gFPRhr3vGPkx`W1xuE<<&9n-Fw$d#Cg*+(G@_iWd2~)SLcM zja>`GDENs=j?r375B=pPYh!Vz3^gUA%2zw~Lb2z#Jh(j}5$49l%&kP5$3ejwJ1fdw zpY(xO5sW9qoJUX2HLpkx!P4b(dZp?*WH^5_d?Xs;3`N z&r(~92Ai9kkC!~bv6n@tb+mV7*gQxC-)Yk*R1ZL@s~l)hu_h#(@J0KNyfw&^J5zw9 zDYbDd09Eo9O;{>YBz%$HItfF+eFYH*1j~&*e03fslfXQFsV5SVuVjvOj`YM$r>8q+ zOPo~9U`uuJ*DBu)7R(YODuxUcTMx5!efzwXwRUjjeS`V?5^AWx&N4P7r5$@HElq}& zXr<&x(bz`GDPwf4|XuVT~YEtBiG=eszgROuKNkHL!cx|V0+rPFqf zZcW3nDh1LFj9FapZAhBDO^ntMYt&8U`EJRu=5$e-u${z0F1x8^to;VsZr~Q zb;stD3g11^^0rt?yiwcI>k^LHFhK1&5e{jocMCc%`x2HGOENR6+ZvlvgvO)eoI3HL zM{)ZKWrE2#1DWY>_Jn)RufVJExK9(@xainC789qL@*?fk-gJ%wrx`WO;6ARo3J;y)|`0$`l%iG7R?pZ4Rig%qLH>Lv1E2FuK&0WlBne)w4ED` zkkdyncMy%Gty@!8|aT9KiB~QVIV)) zx&I|I6!_jdtEj5c5xPTM&M}K$#{{`Zm5Sl&Uw6r&gekx))#u-Wk^+rx0TI0{w}75i ze4n-BNZ}LBNH8eAg$nIDaUI`hqXNF`7JvtVMRY>Y4pYI$?wf*0;bXoT4lp>BBvs1N zKn%{Dsz&Stoq?4k6)*R$b1G8S?P_TdN|GEsf;0iEAZ>2gif-ENYL4JZKHF8pVi3w@ zzdwwZivk{pfI|1jThaM{pTGZsUF=l3MrH)5T?EqJd|!OtC}Z~i{Shlq%h;;t!Hk~) z-mE*B*{XL^(76HJtou{5mAfX4JI<=6pTSVBg|tN9hm?Fia#%}e5snZED3N!eP|1&S zp|Z8he6{`O$Z_xCBYY2DTRJQ3?6f?z*p*=E-_ky$_VQ`}45gbn^tJvyJmDVjgiL9O zu^O@gNpicyVH8bc9$S98(zlW54_|7RN2G zaG&;cJh6c^=9Z(mz4fpt)DL@4yL=~@_kCGB#6^LWHNJOG>?343( zALR2v1E2TJ7rRTWfH-rT^0U>t$qa5|Q(FDSk6JqTS+wCPsutr72COzYxUZ{v4m*w5 zj-dP_KgeXtR!dkA#}ZfSDLwJI1-L(`l)xp21wRo*IFa-9{eXXwX9imUdWQq1&+|9^ zVu;QURDC;Nqo}PaWcQy5;&90DyjA_0M9JnsIqg#~P8Hrst_C29lgMb`s;sXe-$aVU z9zHt9lK^@%AR$|x!@o9UaSnrPGLcN1mOv6$@iUW0$Z&k}Pd4S>?C17QKi|YxbtK*C z=69-{#?Yq6!0M}nu6w*5&BIrT- zby0~>d$XR+A!G;=;=Uu$J6srCJe6UQASnT9M69iEj(QL>Uz_iSIqNq%X$ zP??%d8{Lr>zSmdufm0<4$uqKQTh$ZnbU_t^33cO5L+K0tWqy7tWx?_JF7_QxflBx z{P0Mr7~LL5Mv6wDf@`3;`BV-TNe)&=^Z|Ub!nX#fa4*mIR~T7t%3d!`!X*^&0w%tb z?{t9xFI)Xlt-R(>Pl!~ItdVMd^i{Er;cKa24Nq1lNMKLei8lD}OGIBk-2P-0lXB*U z^J9^UYZas~VjzAwYxWw09bo{YaBS1;wYH(GozR;Ytm{5S5$RapPPo(kyDVX>L}#@s z>iF0NFs(;ICa%id%8Jkzh4Lj$e60Y+vBY8A7l(3bmdu^;JD$o_BEaw=7MEr;8uh zpA+dDWV~ds9C+SvdLH@ub2Rjp-w?r})#d)2r>>N|OCK4a7s7;F?@=C_y!p)OUW8J+ z2^ss|A;VRZlx>%F>Bov5TY^V3WPbW*8n9Lu5{OiS=#DF&UP!$&};UE_G?#L!AE&CP3ti(y`0gd)vHabw)md9T<;6jbzDK;zTVOHng^GFLQ7sh zWdtc9hllapJ3-nh^?EPf&XD=aug!Jsf-3W=F|^`rF;}&mE)NJO5Q-96T4+#7qmrdt zm=O#xvu1K;V$5Q4&{hTG8nA`&-qf1b>p2`YK-AIuOP!8L9apsNvC=i(3+Ib^l=2Ft z_t0zyn5JaJQ@&=vH%;pS?%3Z$gn+_VdWYQS3ie*(Bn2RBa&{=S)YlpCSe=iM)DTEh z0UW$&-YV{7_`stRU8r6aMql4IWvE0D`vFawm9!TO9RNsL9pB2mutOGY{$SaTs0W`& z!PjfIfH;DCVamyIS70a*ad>A3WIrV;&BXzTTnRBe3Mxa_;3r~yzq*Uy`(ki8_;oJc z0uZk?CgK+z+TSKie?kh>mT7kI?*sr#VBrx^MGT9+(iaeJNUKlK5U^NSs|zMc7HkFt zI~-B5!?V#^TeOmpQ4&wFtU@egPs?As8PFL0K6<3^4ZkyS6(Ha00Tlank=?5`*53_#6EggE!@t)>m2?RmoUGPO^-iS@ zy4v~@5x-FseI^c7K)rINhQ}h&57g-nG=KgUo!P2^xc^NotOZ*%=VhtRbs&3*NUrtZ zNtW5B(9ocF(yJ@x2j>Ql_ge~QJ(R2O!D>i?sX7{;qX{1I`~A4H8173$s1?83WW9yB zK)e#eOjD&oXpIfMx7hut3$BrIU)e$2)K|*KLLaZC`QGh6BPw}BQ;nq*wgSYQ-7GWZ z9-L!Q7CaoDl zJEo%|M)af0Fdk4u;xKm*#N$yP-{&!DCM1ly5;EGqyMPmw2cB@pD^}=799KuTKa412 z-z+|rE#-RLCkupvlm%a^EDi)SWemn=|vqi{OJHQtJ z$iC)^d~KcQ$wGrfI?}T0=3VspuR=;7@*Oa*rJsc|gJjwDVYcoEH+K5yGnehX$n+t~ z=S53A;H8i;sxFV8Ff_&^g`llLLQxjIje0%0(SD6e0&%GiPi;_GVyvIHtFX}Ke8)zh zLGKEbA9?Iy1OUo+&?5%8X0y&p3>*+UlECYIVudPu^(7YuQ!3|e$vKyZ2QyYkm9CWR zSSOuBbT}ZrTNd9RWQ-Q_WNYxNnMZZYtT=%$)W;8y4CM(3%d~F?3!3x~LG9XJmdDEV zrx?i3;%(;+uYHYx;yZ=S=S4o zyRRFeX*Jq20uZhN$?pi3f%QiJ=-Y_q2X)cvqH{g!YW_r4__SB^vJv=Cia#0M(Lo92 zh;1+(GKTGG2=L2TF|_q>7D9eU&ZJEeTI*YB_M2dmjk%_7zcWX=)&u+%J-Yyd*{1ov zFJGN1g_YhHpwhbW6yrDTSKYu7|FRd9m-%PM_w`=}xzkyMKYjjgIFK<)eH>Bbc#p)@ z%J=om*_Y=YDIf^!{&Uuu(~bcC4mDO6R0E=GR2D1V)PRo_!cRVCW8~K9(%l+RA=pQI?E=jMCJ16LNg)pQ6ML2 zpTlqRJ?(mFLanE?A0`5u;4FR)6-CFc;t5WLR%4=hZr#Kmi6eM*#I*Td(?LO&@GKGi zYzWm4bPW1U+M~Is;(41KZ~}M*`e=|rP6A)EJ_Q7obnzp& z2OCNEaf!jfkP*+SLrM3V_p1YAB1b9%Q4xjMxjNB#=r4!oF$2+zetfl&Ll+FW7^!*~ z9OA<$ms5RR#b7kXDEG-}2x}Hg@|xCas00U)w~l3#1M-iFRIJWs$y-mOB&WaJ>JLTa z7pyV;#r94H37FyYk+<-nNToJ&;Bi6Qd(=S?q7ZMjZzouW*diD*f;1xpcuJ;mCl-mm zryqSu+NY@SM7lnrw1*j1U8=ObPDCJ{-ulSDOu}W?*e}|@ixhm6W`IhpzX33|gzxn3BMJ?VxLTCzent_95?0MZG03#_ z&KXGs9V!;SNPp41^P!Yn0BbKSRR3OpJ`3aOLih@O`?qI93o7IZoj~+>!U`K&UlDoV z6?}19Rc98d{hTlrJt%nD042x%4Jto#&^a|(JAO~0eXsS+<=?LRwJ`IYe$!9h9!B8W zY4K>759$^$MM11a7KP+SXU>z`4=0v7#Z$$9C5n4hKrtT<^t;(sj=jJJTuz`wRJMDw zzapqKRSJ2=)Y$`z8*jWFM;l}3LgdH-Rq1UhP1o(OpT3EOtjE~1p_7^WPSjw80{ku# zzyvp=i{SDW@B*K`1&0bxWplORd4sML>FbdSpo~uCTH)tK^!)ZvP{b??kr*_bFHE;e zppbHhzaKgCa5W48Ri!<2v{YEkChit=r4`gkG?lyLn-xrWXyEc?MNT|S&a+X^JR?}r zY^dUR5>z9Tp4|ynwpcU!TOBcGVGnP z8=zgJU$VHLDIYeFbvuE_GsMy>myjkBn9mcJo6erDFg@nUXGtZm$fk7Nuvy3Gg8c{C4Doq93A~E=_yG8NUrg1hfz*xTD8<@puhQF*(N05% zlgiMMNl8u~m-$MPChA?0upg6sft1hjo*9d1NU8UCuf!+<0*@We^FT*rBIy{;Y#qF` z9qc6W09(Qk3%CNdxgftA4Dd4P$OsA5vkrsbijWtH0K-LW0(L0lBb7K(v3wl~R$&)I zj7IP)Rrp{DG`Vu`=#XRz5@knAfHZ?4HsuxRy`ET zKL@~WBQ~87k8U}eZA>F$ll+j~7gIlI9H>VDSYOC>B)5aN1QBX1aM`-!s^FneyFsaU zSLQdNzU|M(#E%q^rz%g+T|p%{kbh7MXCi`t{P95s(s+OWJUkC3oNhSwvq-8UC?LQd z36Uywa0CQ5xm!>Wiry}Ml?w$eFZ3AQeUcFr7h*rA{@|4W6sMPL%OEBO?6}j%qtdKv zi#6e8IFMj4%{oxioaa|`14KS(3GSyJkT*Q6IDrQu_nV?clM89y5D=o$L-8Gp{IdYiWUBRH#C z3(!LAEcZFi0^f0aDPmFCJ^wpXsNjKsMO3NMYJ@k28hb3Y+2nKSg3rUZ0-W-+s)vW+ zcMN5k3S>TkGBofd0D(-@XL8!|dck^MXt;?TMl|w*&#FN4XzN`H9v0xT@ ze?r*Miy|flhessgbYC)%71&?p6?7!y4YlNwN0e>jb0<5!`grP~n?<(__F4vn`HGMq z*9T3Dj@R96+62BS=u*s*fves~si7>h(==Lw;QQ3}jfJkfhwc(v)$&eld= zvfXZ)QYk!F`;DHXLRU;mH8E{NNoDI_s>?Gh?EE)A^DCPTO*>r7I=ZBLpKI`Z24j{* zJa;$`LVe`HletGk{+g!rY*AIIrhdTjEF-UJ<1_E~!$!W*m(8WgF6lqqG+ElVh4m+j zgAV9eX9<%HJ~)xY{i9RKYylaW_2T7?9?S!GX0^bF`OW*I49+vUt6RXDF!*QEcmGx5 z?uASAuVVuLA8jn_BOF~Z93RpC3qj)l6|ZQX{}F9%v}XRxe_;d}I_X1Z{~xs*5;*xE zaYi}*MV^oUK4lcl@lI@Ifv@0gL;y4bhTKgYmHTZv{nxSgCoy-D4+;S98)J9fzmN3m zfiLDG1QNiI|Ace$8uWFHzG{fr5j=ggY8i}!#sOjaHwA&OeY>}SF?g0PWL7ZJ$g2E} zKxYi`*X@>nAa(t?8lgYw7CwQ<`k$GB;U{D8aS=j!Ap*yL!htAH{dH6TqL}heI_vk! zgQxsq1N|$UXKeVN;*AM_ak~D2#>_hdCU|ewpOitPEFfg;KXJK--y1s0qfk}w`cIg^ z7>NIDAY7&CG*BVZgYB53ztctM0GmnB;t{;9>k6TFj| z<77ynrR$G$u>N73Ka%=);{W#$@Y5g$$jcZ&5G>qZ|I_B~Y0#0V(S(YsLr&EyI=P6w zUt9;v-*0xHUrc|i#;iOlA3;x>7tjoqo(OB11+Jc3<f!c+9oVXj5UWDLz=K~@0l%OE z`GBo@boY>LLdC%^E~lzvmE2zB82zVNA#c2(Urj5?rrJ%G?)p^7ZmWyq&c=iPIf(GI zaUW?9l82%X$~N3dsPQ~E^}~@z<7MK^k5fGbn$^W08!rVcGrVCF%Mh@9si7g4BQZny zbVD`Qs`;y^@ujf&N$P1XheK9uCqQw@!+JrC|RDpoutd9xA{edn$Ft0H4@voz4{e46GK9$J4Ibg=Rt|}J8bsZPPFOh z#ROwZBjo5kn45!~$e9&4e23n}Q_$d-{CK-A_V^mEJ%wNI1=Mz_6WV<=-aH-vMe})y z&26q;>IBr8acU|duCR?+ww<0=q30!af6fY&9I~+KoP#Y)JM@prFL?F zQACB~HKdOS5Ds~^RhCwW&4}jY(1Wz3eSp44GqCidR+)FdDzI5HVv~XxUP-QwIC_Q+%Yr*v-C zQ*l0IpKbxUXQod#mVe-t=6<0CKd5MV*8e_c-)F)zm2xNQOG0zqXV`0L9h9qPgWyoB z(4}PVRly2z2~J9;WbOrJ>j@xf9G(KvAzs~1(K88sdb@_lIee9ZLF0z49ZF&@;|hd^ z9bW$F!A%c93b=R;5BjaMlQ}w!z@V=3@cxf5L5GM=s7*)#kU`dA>vEO{bPCqWDIt|NMM=+6kfT+y-f&t3h{VN6vNND z&+%j-pn^b{eWnDM!%)Xmuvn9n7w|4rlAsSsV42bPq`xCH83FgpMq*Pajfrtms?D2Y zZ%W_gXY9`R9qZR6ITgmNH9s3@uD|>GVIUlSp!3e=VEwXus+9I{3EVB?ClT<$nSXi- z8Ot`N4=wK|D*-ovlng4O-^=HK1!@DJ(%>srKThhOvm(^;f|>|-KwUD z1hxbO!norHgMt2xKp5rXLx-MHU>*|4k=o`KAR3*9f`vP6#2+ESV~NmCJVaM~b-or( zp9|i?g2oB4>GtbKUs$sWwr8E<7gtWmTeGBxVWD>l4hdK=SKDtq#UDnq$NK?`sKE86 zE?G$ep(@ZDSLQr)&QugH)S=i6KUXavw9U@g&^)&=1px}qP&4Y`hKd?)m`)(=anY{F z=?8h>bEa!=Qud@-Nev;+6*wC!{6b}Xx*&B+dBbVi2l&o9N?>Ua&q!QFO6qHF$6>V^+j&R!;*)v!s7}Bnr~Tc z1|&8_(vqd$1b>%{bR{W=4MZ79!;Yh-2WcxVe|H1vuMD#JhNbxiQPR8;BS#5!)GgNj zVb@Sf)62aYzqV+C46n;%Y<$@fKK>o!-S_f%PP7MB4a3Ousku_nNQrSNTJUw1iLB!B zZ1}Pd0&PZHz{XjkSd3uBYrqE^88gM8wx@f4|p2ZC7@8D)f za~uPYz1RPt9eSM;1NNxd&gg!a#D-zoJ3jvaG5Yc8|HhZ#bLMZmgGAp^6$~Say+X!7jL`gJ=({=J@vBNgL*$l#=9? zmkx6wD&K)>?P=%|wGK#Jbg3#;J)T5Q%&0H%!{8-~qQ=drM}*Ut@A;EZl^3VGhSRx# zVf?#0m2>nNQ_y@8yE{ja77L%m&Y4lIDu0RU1Xd0!MWt$Ui4nL}skMhbcG^Ox6`Sdkv&}o@J|;YV zlwm+6a0`&{bnb4aH8ElLLHO_$IwdM{t6 z2d&*r1zK~7KcIDkDzt@&dyA-|L)(l2&&$vY-$lsYU8Og*vg%%2-ew?~8PB$_?^dNluVT0Ftz{_@8NEd#%7ma;5iFt zTYvkoC(sD{w75FlVEN+_X;7$89CEmTn;^bVqcAc_bWdW-ZPLaz#l zQl&&tdKIE52-0jw5fSwSec$ta*E#=o_t|~+o|$`QcCWp53m<}CXNGId|Df)EdO>>j zG70E*6%^Uf3y_pfi1|eK!>g70FM! z&`DECP4#f_;Qt47l|LAFxxS<8%DPM3s9?@-z}HXTEJH@FB%0O{Qu?^>XQ;Sq*c!W| zR+U1vI+ECxU2Ju)mo=Kz(EFRAeXmVG!(sD$K%(xA36MD28pR-oJn8;Ic40UNAu~kr z&83?`WXu!K5|IMJ>P?XhY2C8k_}%kHa#?Z_?_(=Ig!*siT~MeF(WvS92EAZDP?V91 zfA_%iq-vy?xOYtk9+1T$q0RiBD4J z>1zAQ)&oms*9ECaD9^imh4+!&5G^FLOUhV}t{$-pm!H)pKzKQ{=z8_OH_^~T@;AVX z6WFTRXr8yGKgzL@2^&A|ESd12@$y=g~=~J@9*$$YEF5Z=Nv!;f(qT9BQQVWwI z!Cw6a09X9ENPOePWFBQ18Og`_dz0!>QYzGwWhR*d{uDq_qyy9b?jlWPK9>6`Lx@iz z>syCEeE%3Bo4R?!JZE-e{~h_3as6}jrxQk0ZvkI4bcT}+3+J7^D{#;Y& zhqg)DtOt=dN}_-m`DscKYUYeO6DVla&_DQH&2eE1sWp~9wXI|bp^`5*QG*@@6=Bi^ zE+6s!beq|gV{Om#X1dIuos!(LwE(~MwNCfBF>)mj=|nlbK}l#r%3tWe;QzORLdtja zqtv3R69$}T&_Po>+&f!`KeO=j2NpGlS)pO&*X}+#Av@b>aJf+L*|*#8%6SW%O-OVg z%u}Rxglv(f{Bu26t`3$@d=G&x$~nVCDxelIn?=lBZ}-@Utm6 z?G}#Ha7y;+y~lns#DB3jhW{kt@mCGJlX+KnC;}h9X(cCEe$hC2b(*^pVpBYnCVOq7 zK^q9@`OIx=f^9Y+684{j4RNI47`!UEx&Y7 zRZ1L(mxcCgWd@A8CrZ@JO~LQ)Ua2y+*!hx-Uky-<1xgwCm7+HwGOk#z>=degLb)u# z90)foP6WZUC2Y|`X1xmP#|`LTR+^f5_ANw0@M*3%FvozQ=bjo2wnACCH?&p|1!F-H zZ7d}WcktWwf}?X?Z>udTZ(&2j9@;*gOUy=Qb!+lEcgC{?=e?vC`Yu^l^I1ViJ7PI# zT65nLRKqh~5f5@tg^y*V3AEp3mJ+rXwVJ^`hyYr=WIt}t|6+~Cz_2`8;L~A|@4O#q zygo{cPCLpey6W;-?)NQuQlNOe$()8z)4J#F48TxzjkWbXQ+ZZ<`G8OE*-R*oT`Y$i z?5yLZbWMZvXe_HzNiaRp!vL$M33XW0QMQ_;In9Q%hHI=C)#f13c1bIU?I@OyT-|-E zXk5W3K`AvhcA`aBzsQojghiCOx@JLGKGE3(KEv z@5HR{(lc03#jfuj#TbJmJ*i7x=TRfo7|SAsb^Byv~c3a|`r3YF1!%C?_ zuVc#$#E_8wKK)k>HR-@O>%o2b2RlV2CO(P1N9x#d9N!+(r*E_fjc6frG8ef}6S)cY zUbQmJn4R$q0Khw3m6f$g!Pjq68M~_vL)!XTc z`p@yuC=-J6Y~CXI7P>x)HT%;$<2nfDsj>@EibGDoXX2G8A{TUSS^FM;4i~t`0JNSI z9AgHI&T>?mG~jv-$DOc*{JO?^M;#_2$&!Jbm32_q)q0Xr!rYt>6If^m?V?S@^W5iE z#L==$jI(@8ymmZ!9Uf3}d6mkU=u(+%)0pz4dHuIQkyG|B@^2jbMng24a{KzS;A=q9 zB?mMufUh3EOfeSsjHjtQffXYV5%;$KS$W>d$1k+dk-NWe=f-W!)|C$#DW7e#TeN|E ztx-QRLPqR$jw|q@uyFKL1oXOA|lPyS4+b_#E}B%vDAEKyRI?-%*t@1qRyOBACU- zI&H5XF{MzVb?v`w@{r5j#m9{0L2qXQ=nq!X5zqgG?%_kPp1D6*XJ}Y+he&_~TaECe zz0*-r`Efgr2LgCmHqh(F5bU%v>tP7uDrn#Gakw)6P zjiWCFcrO(56Ok&6NwyVYW@x{XdfFEa4`(LmC(H%-t&dKGb-(6^R4VP2V2-+{3hUHD zqJaz2*knZ*RPCXKlcZF4bKiaIXyemRcbbQ2>A%8Wa})5fqjv!=rfNb9uGnhT6; zb42UWxx?o7|3JUwST?ob!pDerj{3Qk8gJ#<`e6Ds& zFZ;-2ao^k?y^c4(NpjLnm+oxjtW|p=*{n9|y``B(>;4lUi zSwkL8zH+L%SjdPx3Hh)+)D(}qJrpGv&|$N@35|b{{BV5+M8>5r!q>+B%oubqHRBm& zcE6oH_Z55EgZ!=6YVUptG~3S7=-rl|JMHO5^!AW-c}-spqI{oRGf3+OwY?orH$Be* zntVkJ6p-r&6Z}~rU}-@WBkpr0qTV@mP@&4~*w}8-d}Hb1D4R4BRn|1|#X z!Nqd<&YqXT>_n~Gvijs}|F7d_RK5{jORe{X_aAuKl*-61Nu%<}>9#|-z;nXe#*`1X zGY*8l7liivTP-gxf6<_|&cA8=Ta+=CxD=!?2YCY$6T_ESuDql=Yu;3sk>rLpcj0|f z*|tC{_>yZgs_%TV*|zAO_hZ&kfp;3EHu{c9a&wlBk1+NW~C7WC1 z^Xtec*4QV6=4_Tg4J$kF-|L=iMBFuJ%9#lnuu0N8q(Rg*
  • +G6M>A0uX`^*%Xz4pp)n^YRhp6Y=(%4S>N+;=-E0n5nYo1#q@B8=9OLS4!t>n6FkDCW z{b%{B7TcvnHcGekD{n2yI>E)lB|a{lI}F3w=rW5y8P-C+-f^c_jxb{wR%s#n1&YT( z_~qQ)1FAlPOA#kO_VJ5XRTIRSV_Bmb zC5f1<#<6Uxk!gb@${K%U_kp9Bqd8n77QU)ZuM}f!p&%D?5HxJB1mR6JCZ&wFRoi@r zT>A~U_e%={XIJatVx8ZJFdjd~-ch>w;@=(#<<8YD{-Y(`$o!OrP1r z8J(Blai$z2mCWdD?*WSb-yj|VWWrP$5#M4S2oz(t&T zuAJ_3%gx-$TBm(E1cV*ogTQ+|y|9tpsZs|i@6R z`bu$fI!7(>4ph|RG=H<~d%j{RAC-*x)QR!|lgwgS_u|J6IV=Evhxf-qYD$L2jIQF+ zf;>4PLT6$T)gYEUs5_)U8wci#a#LYr`QYd%#)qYbGYQ2|bU86YlwV&;iLzvhEJvK3 zyMH-YLMnqU-HEs41!zDVLI)GZYdZd9s-e0+PjY07DCTk+X8?h*kUXqUg`(RVi^su# zr5U106Zx++Gz)YlF&R&t4oQ4tPg@)-fMpF+gmFC5;b*5O&-Hz{RLAUv<(}A&r{e(W zxyXf5gfpj2_yAyqTJ+BjI4c_#fT!$o%<2ZPEyg`(6C?OX_b8yJ97pBI&Vx!y$dsr` zX{Cv9ff6QGCB^(xXf3iq4ELCRB#N8tZ8rb~i?gJm=;6-2^_t>DQA&1yj`%t*BMms7 zsAM|YbgZc-tWkg!q*HF84y83FtN-1O?t7;#|NY+Vee``Y%9`Nzz+IZ9)b%{iu+tQ7o>gJcof174)o?FhfCOQo+;}L|rX2PujG_HjDH>WAzkz{D{Arndm znL<{kW{dQ%PASX(j5q}7)eD@|w|Oeb%M$y9AM_^ODNz(-J51#;{uQZamqe`-g;M~LpBejp zpoK)rK7RcTSnT?~U)6ZO87u9`s|^%p)6nvt`vdCc_eDn9rCxz{gJ0du3QT4rX_04@ z68f^1fe!MFbGFywSYs`a)Uv=mCZ6L;{P*NKqbB4Wt$=7(>2S7bnIbr*pFzn@AbV!% z>COa_#aQ#o#X_y3hJojxtl$%zymcBvocuhm1)F+P25ex~pDTxo3>X;<(mdSRd@qNc z2#GLYx*>m!JkVB3d*wr;E?Ywl-TLh%Lr-uAeL#*hIzhm!GZvH3%^`Xo!(AyR$VSKM zK)+qQzI=5W5x22c>^XMhs~Jy2EXl-}7YQGffQ^Zm7gspXD?13@U5kcaqbyux{0&(B zR?Uv9WO(U9rAaOk6}R|s^`FIAIDKWH4+8cn?PF)sg}U4{Sy7`r(-(_EN{uSA)oh4^rC_K*KYb1z`@O)@1Ae87Y!^{4 zd`?l;rQT8!vIzlXT~6lZ^nn8rO&q+?kVM7>$-52K!zwTrzEm+cDWRIl9gU38 zO+UfAx3}N3!XX1KGeOdcxc>tXv-S~b%Sw4Ep~MhQi+-x zA@^Wo;_^>%DZGvuNQl<1lh+=wh+`L#R)I-PlH?glnU8ImOnG7XJ6Ken2-@#U2k6xb zd^1$id&^)%R>Ku8pZ)whgZF1}JIn+94p!0!LPx8hIss;w0J(UX9j*m5$EI0a#09lCykY~<1&x>Elw$SMWZJzk&H zM`#6o+}kYN)K?I0Eg1Ff`r+Z+XT?eVxH3}ad$w!+dtl{KyY-j%&9d{`PA~FVKS}%s zWIoDN;pgNwUZr)4vEzDnn^r%WH?TVLT4u5?^R15B{mD~j!c`8={>-yfC7BQ2Xk6^y zaOzlkmMLJ3oBa(C^gActfcHv%MtzYerHm+eL#uZIjZZ6V7QWN}Dw6EES$Hh3w;hfM5o*$P{_>G;Z5x;;QanykZw}BQ6eIh3d#}jKE|t^z2M5U` z&W5C*;lo<)7yscy&<#7gxo(_h4QV!EjJcqwC@VePVwEQNdv|yDmoHzi*nez!`1wPc zrwj}YFMFk=r1W%mPYV|p7yt3DeE**15w>}1^qYH#;L76S;>?VsFLAy@;o$B90e@s= zW##945vK6QQ5Fvm4`!NRrQ`>s^_kIERKedD7ad8Vj~_o~CQ~AAzB*ES?%k(P?UnMX z=WkuR_HN8$a2m*^qo*IM@Dju6 z3q>0n8ygxJwA{9LapA*CNJ|%8z`iaommy86s;WLl*Hl-V^%0hrm!Cd;y0Eao&Z6H} z(|wXuw_0;B*4dd~T3R|RQh(nhPOE?UEkc}UBAuZ&X#Z#)Ig@p{&YbaC^hqOz&vEoG* zO`cOuPD^GUId77Z&OKGUX<{-w&O{Mdjp~J15B^M>EoBQwC3h!odl`2y1Hxc>?9>61tYA2 z^zPmrX?e%*IeM&?@#*|g-wue?ImY?&@M@1gzw5TQy_J5X-%62k8M*Rg|MAhILDfQN zpY`d?%*?H=ElH<-MkOU><*CWZ?Cfl{muI>>C(;rFd{@V+D!f)_*S!7yuAB~4A+Sl% zUFUi0KGz;`_G)1sP3HP{S}|)oyVYL>7kGJ9-_0C5@{~&zsjN)Mq9C6Tbw7T_gGKc2 znUg2qym@o$)~$jE6W`Hzc`PFn)4SilEtIG>YzmT+l7@zi4Oq#9;j-2K91{qanVH$` z+`Cu3|0K;asj8`!S5)}<`m(dJIggf?{5?j^ee&cw3Y5A8w^X(sH8Mx$L2!Etd zl;7o}5TE};+J)t97HTLf2kn}Ymaws|o|j&}e90y6`z7;5O3FP$L&LI0Npr=GXYnru zPWbxzq91SH^uLEdTv7f!Q>}#l@p}|339-jGfLB!s3*imusOOyda!^5P<=sIp>V0GZ(d+G3= zjEu~KACyTt=g0`e2}k@>^y+{J!SkuuzMa z^xGo99bR$DZUGP;75@myrJ?Zyq4iG^lboq^?NwIMyHpGe$RBJ82?=_7dX#QW6nX&E zgkJ@=)%_a>_m;G$Wr7vK&W<<26fDOG{b9=&!$)GtWD2HFNn9W@60y`v{$ScI_X&xy^*-Rs^m@a7c@ai9w4d#-*n6 z0h&`JxD4c)=H})el^U(h%E<7UijXF!xZS@`*j}63Nm=jQ`dt?^TH!U7chx-@hns<| z^B6s5zqm45@d7PAt(}M-9UnJ`>!pN+g$0D6ySlnG-(OsaY*ikHdp%kmi*WU(fFuk# zjMD$&V9)M97O>+CZ7nb72TDT!FH-9GS2l=keo@iRS~C*b@l64b2M@%>#VuSqKFY>0 zx*hEMqZ1EWh4X$Ws9t1o9V%|=>FF6A^~c9v607{9WMB~8-Fo`;X@Y6;O(SFD_^7C; z*RN||PESt*7iew8%(_6IhhF?7hd=PsyaDkI2IoH-xz$D#HqCL+!F~B6At7PEN8BII z&lLI_pa#GMXgH=n&%1fabZ)usT;1G;ljGsb&D|Z^b=Lh&un9NkkfSEhs!^4NvR8&_U5sX|j9gOare{&xm zy#8OWfdQ{Q^5&^fETa{$Bw#qeA0{U+LwcCy#{1gX*f1c`1nuBt54_S|byV||gI-i- zNx+`3l7_v#y^)cTlany_bWYh=JY?Zmm7gGvmR0nFxaG5&nghaM5eElHTpBYov)oz( z)BgTG1qB6k&ZzFspFekYcCJk|Qyx18<>L8q1`;bpErMMkU>87}?!<}My4=Ro@wa4T zvr0-l$0~i`WZ)rwvvGE@bvrF!7AsSp5FoJ7ua%($YFwT9xi|^s)g<{US;)ZVC`ep8;o5Ip1Y1O2@iX zTG+kDb-Ly;sZfh==@!aA+&@RCac*OHXsB3#1Buk3f*a}-FVm7oNgH$2+&XBIb(!YS z(9rvs7*v@i1_Oj+8VRe%|NQB_~-Io_e$(uD!jTmzOsuC+9Kx z0Lmq*JCuokc6z#@sYw^}WP5^=m6dh#7*ft#k*c7$I0`A}wK7ud&=boL-htl99?mra zx`!`i&Fb9T?c2B21*<%l-e+Yw`T12m-U9+FD_a3X8MBaU zTJcNdSIJi;6o%Ep#?G!NG{nar#bjFXC_B8=mY=AqkP$PG|MgvmMoz&r2qVP;pm0 zS~mXrV4@+Esb+hU*478d*?Iv>82Q#SK0e;lvytFheW%DaBJc3E+Lv`Z?pXP8o6L); zS-#W-%bNncdMZf&PX4Yrpfl{3tgu>qnmRg>VPRIs;kJ^y`NbtAoao12hY!x2IRoHv z6O*Q$@IE>kl#YZg{!E5G2nbcd(JH^cpH;(qS@mC_#{+h9G`+)KKsye~h-|i0tvOyc%=_Gtq8|Ax1mv5v^HA~Y*RKNu1LIJ3x)_~La-^7= zUDf>s^8gYK2=A{0Z$H1H#PfvN@X%08m+_h>iKEXi)$qdWjvY zl(Wtc_Q;-)Z3dh|7E`J~%fEdi4Hnr6;#RA7R^IYzriUeEOQdE+qUE()m?61{*NHx) znV2${QH&rE_ma}m+>DImC|g_GS8S-~canhCO#gTT{ZOoCw~*XQe4_3zfCB-vyEA?e z0J|S8d|MZjrjuxP>((I;I7&&M?iw9|PP_3jUc%v25&y&nNli@+m}0-<)vH$pM{|IK z+%Y!3Y^VI3jzh|I?69-YrF3NIr7^oH-?_sU+58TJ(|1Yh>g@bIIqCLy#nfQ6BzK2W zGP+MiCz!4pR04=C$vspT@OHb8hAbD2;Cr7L1B+!BQwEwZrVp9QqkfHy$oZ3rZ3)s@ z<9u4gTVA#|=paXD)gC^4=;pTWHSTZc?ELZL$11xHuFr(*1T8)eNk?Y)>N}_J-McsY z94L1D@M1=OedsE0Ky9L1UXJ>6igw6t}k00zyg!MrGxCX!RhiHXrQmH?+29W@KFFd-qY^ufMrDWOOETVA-n3w(UIj*U-=_ z4(XMO)#?k_bl*Hd0o|~j(2v+TINrT|`-)3m#<5pmd6DQr zneg#rPCzrx(Jp;)O1X`&zPq=lsi|r1yTJQKqv2EQ>$E>9)|ZvNy|>qAewsgCQ`MFV zPVH*$?k>#8XaT`sSy@of1$uR8D4ua0HWC#C9NrWt;>^#Opr9a}u7kU~JJ4b{8~%uk zWSK=>uWpYL)HBR88}03FxbGs_IFoRz1d&x*+TY#n1_~Wgs5uhpYHuGa=d&JmS~%CN z3Ratb*fmGKK8%##I>X)Y>(@2_4VWQzxv>kF@MvAfo>Xn<^m?22-~p(stCy)7oeKSe z!v$CQEf19tiNv2VV$xDl@d*h4-{Fvz^=3Vc##YoW#;Y`mGjPKTKZKioSlWVL@I7kQBkM>R(xLX#*e52t3rGwLqStmV?kfQARjU6a;9#(sGmW`3I z_QPfCW@y>Zj?n@|16av_v^UG2oxGQ4@{ftblJazC?Rh*HtLn zM;|AFLV!ndxQB%{wC*EX1`kW)00D5c`M`Gf;I+Hk^op~>d05I;oS&cn+&K&4eAoFm zFXUBW>4*J#?vtOWpy2g=Y|LXx5krjG6Z54a*Kzc0nuDd~S`&wBzB;pH4SukxDYd%# zlP;tH2MbHn2~nf`z|s75jxVL9TR{5lZtz$A9rMFSW<#a+_rJgDtRh(Dz4lqj#sDS6 ze0xaI-+yl^Qlb59WV8kkp3RDsl0@ifYiom|ih7hD=-&^aSs@VcBh?-Yx}Y6!#ksld z{ry|Oqf(=Qc7+gnfKJOhbWUQRC7U(?TT$HcX228=`Scjo5Gz`{7#ol`Is&XSo>6{Mr3y(hYs z92M4RbqLiGxxS`i?(j|P8?~1zVxUUAOt7Al4b-!DAZ1L-T%v1kXl(ra05%J=s2bXr zD6qlEZT)v;G^!i&sji9tHpW&}g-$j`X*- zw7g|#n40R7led|XlVhVCot3pjiG1AI)m10=g+!fawqj8+`cQC`Et9-GzqJu;pr_X} zGNL8u?CR>};ZfNOk(E_cRtD8%eWR^VA`(4n ziwTZteRDw_=<1Q9N8yb7_wUDE!#%u-mJ$AVYe^$#e=n(6*=I3@!Qf;R)M@5<`J(@# zT}hTiK`}9D6sl(>Jm9F#f^?+-4jLG6PcVb53XC=!tqQy+8$vN+ulnW4#igay z4MbOzm(wHByLnIcg#G$Tg^bmaaayyhtE-^l;XTcX#uOhS8>ac9L?}9{8}vLDi`|73 z%q*q;+@ZL%IQ z?Kcq12t)hd*qBdg`hD=HTZViSla-Yh)a&c$7?BSwX==5Xmxe>=qiAQK*geb@teAD?s79 zyC>)8?JX@YA$q9)c}5W%%(xCnOpjMl?f}jUN;Z2*lo-u@!;{YEkOHuXbi)m2z-FeQ zroIGu<#}Q6`|z;IfJh+b9ls7&zMd(BM;0&+gK32{KW^K!UkZo_%x0lHoT9r-y%ZAu zI3ERr<#xxia$-b8+fDht=y&hjxBpmznvRNMHvM}9As-mjX>3w#-^IA^{K(=rhgnjd z01m2`7fTqgIXE1XxKJVxM2OXV7E~xmKqe*dwhqlU(=u=m$rRvKpYr?BjK{xho|>p- zKp^a?4&QSBaoE30O8WZxPE!WJ97y>EIw|H5&8b3Pz2d`e@9Z#eNZkid8A4OOLJK1z zSrufb!sh1Y|lRq)pfLdvrT$j z`~rAnC#n7c7rfm01YAY|E~6>O)zQ`#QML5@cT&AASsB;{s5Tj_N?efx-QC|so2J$vZ|g6v&d&=Rj*ljQ zUR7|OqzQ(Qt}~Igw>?b#5x<#Lkb&G9Jf}`GGmDpPY-|7@gWuHL0DizTVL)M;=r)Uw z1BnNsL`79KROK^}bfq7fVpYVSVl`1QF^93Fq(7^nr6u=p5<9yIzD!YJVQ-l)oCcOX zaELu(j}9=@u_69U@;n$r32m*3L z+MbFm$pw!&c>bvPKiXhofhhnD=mHi@pZg-< zBj+})0Tvq$$IET~9ZL=3cdXPoVRjP~@iN$i7+kjt2TOS4snCX@q4I&Ov55Z~Fl9Bm z$92M*;y!%X1zO@%Xoe7{Vp}Yy2pszyh2c46>ebb3eK~HHR$WaEeEv&iaZJjPx$pS2 zg#-k=$qQ*FCMGGLKb!e)8YpQ51XOn`qiMj82B|#-RtOB6{QUeLJ%%Y5H8nMCF?&A) z9o^>oI(Tq9!{hs@j=8|~*f==GD}7LWe74|1#OK`yOHWJdCHW-q{LD-tQd**{S9@I@ z_qlUP!=clC0|PZcNk9-lnaOdv!lpx@&i-KK@*89PzoRHK05&zX=eZS~xRH^&=H}+J zr8fnvU9kVMZe!!)pJTD0e?V^EHZ+`_o9l=}r42vBQ|6E(P3V?ZN4}wCWPE{HW-h|` zWQR;HEh%YiYU1PP&oirvZ=UireaRVRey2^HiW^9|Zbor(GB>~-nD=d&f>(gog6q+O zo&j+6%=Y970G1Zt=bNti9r0o5|hY7GBPru^nBor_&AW!V< z>=!Ryq~Zo-0-6rw#@PA!LGyGn@O{v24n3*heA{oUpc6YP@d7y5!e9B8nEFA)8L1O7 z0tn(U8bpxJ-MeSG4<*~ED0D>=Gqd(F&qt4dPbNpfZh!>`lf%CLeqBs>SXgCM)#%7b zFYC$hMx;OSO>nR(sO}!M|KC8H?@GbzmFKuRIr&XB(P11sJxieQ6whsgI_ksk5Q;i- zjZ4DV5&@@RN?~)E4d(&c1*S=u+lKWggX@44gCflQ_2EgOW1o z?5gKtFLuvO6BLYFebojpj zF!PFL8v*|$$D|B4p{lYn7o0Sh=W*rusi^$3Gqa@`BPu9Y4l^&X?Y8DIc25=aT}I0x zo`Bjg-US`b${OZ!dSv>ncIr7@5 diff --git a/docs/api-guides/app_trace.rst b/docs/api-guides/app_trace.rst index c3fd79a06d..7e50bb5459 100644 --- a/docs/api-guides/app_trace.rst +++ b/docs/api-guides/app_trace.rst @@ -1,10 +1,12 @@ Application Level Tracing library ================================= + Overview -------- -IDF provides useful feature for program behaviour analysis: application level tracing. It is implemented in the corresponding library and can be enabled via menuconfig. This feature allows to transfer arbitrary data between host and ESP32 via JTAG interface with small overhead on program execution. +IDF provides useful feature for program behaviour analysis: application level tracing. It is implemented in the corresponding library and can be enabled in menuconfig. This feature allows to transfer arbitrary data between host and ESP32 via JTAG interface with small overhead on program execution. + Developers can use this library to send application specific state of execution to the host and receive commands or other type of info in the opposite direction at runtime. The main use cases of this library are: 1. System behaviour analysis. See `System Behaviour Analysis with SEGGER SystemView`_. @@ -13,24 +15,23 @@ Developers can use this library to send application specific state of execution Tracing components when working over JTAG interface are shown in the figure below. -.. figure:: ../_static/app_trace/overview.jpg +.. figure:: ../_static/app_trace-overview.jpg :align: center :alt: Tracing Components when Working Over JTAG :figclass: align-center Tracing Components when Working Over JTAG -.. note:: - - Currently only JTAG interface is supported as transport. -Modes of operation +Modes of Operation ------------------ The library supports two modes of operation: - - **Post-mortem mode.** This is the default mode. The mode does not need interaction from the host side. In this mode tracing module does not check whether host has read all the data from *HW UP BUFFER* buffer and overwrites old data with the new ones. This mode is useful when only the latest trace data are interesting to the user, e.g. for analyzing program's behaviour just before the crash. Host can read the data later on upon user request, e.g. via special OpenOCD command in case of working via JTAG interface. - - **Streaming mode.** Tracing module enters this mode when host connects to ESP32. In this mode before writing new data to *HW UP BUFFER* tracing module checks that there is enough space in it and if necessary waits for the host to read data and free enough memory. Maximum waiting time is controled via timeout values passed by users to corresponding API routines. So when application tries to write data to trace buffer using finite value of the maximum waiting time it is possible situation that this data will be dropped. Especially this is true for tracing from time critical code (ISRs, OS scheduler code etc.) when infinite timeouts can lead to system malfunction. In order to avoid loss of such critical data developers can enable additional data buffering via menuconfig option ``CONFIG_ESP32_APPTRACE_PENDING_DATA_SIZE_MAX``. This macro specifies the size of data which can be buffered in above conditions. The option can also help to overcome situation when data transfer to the host is temporarily slowed down, e.g due to USB bus congestions etc. But it will not help when average bitrate of trace data stream exceeds HW interface capabilities. +**Post-mortem mode**. This is the default mode. The mode does not need interaction from the host side. In this mode tracing module does not check whether host has read all the data from *HW UP BUFFER* buffer and overwrites old data with the new ones. This mode is useful when only the latest trace data are interesting to the user, e.g. for analyzing program's behaviour just before the crash. Host can read the data later on upon user request, e.g. via special OpenOCD command in case of working via JTAG interface. + +**Streaming mode.** Tracing module enters this mode when host connects to ESP32. In this mode before writing new data to *HW UP BUFFER* tracing module checks that there is enough space in it and if necessary waits for the host to read data and free enough memory. Maximum waiting time is controled via timeout values passed by users to corresponding API routines. So when application tries to write data to trace buffer using finite value of the maximum waiting time it is possible situation that this data will be dropped. Especially this is true for tracing from time critical code (ISRs, OS scheduler code etc.) when infinite timeouts can lead to system malfunction. In order to avoid loss of such critical data developers can enable additional data buffering via menuconfig option *Component config > Application Level Tracing > Size of the pending data buffer* (``CONFIG_ESP32_APPTRACE_PENDING_DATA_SIZE_MAX``). This macro specifies the size of data which can be buffered in above conditions. The option can also help to overcome situation when data transfer to the host is temporarily slowed down, e.g due to USB bus congestions etc. But it will not help when average bitrate of trace data stream exceeds HW interface capabilities. + Config Options and Dependencies ------------------------------- @@ -38,16 +39,16 @@ Config Options and Dependencies Using of this feature depends on two components: 1. **Host side:** Application tracing is done over JTAG, so it needs OpenOCD to be set up and running on host machine. For instructions how to set it up, please, see :doc:`JTAG Debugging <../api-guides/jtag-debugging/index>` for details. -2. **Target side:** Application tracing functionality can be enabled by ``CONFIG_ESP32_APPTRACE_ENABLE`` macro via menuconfig. This option enables the module and makes tracing API available for users. Actually there is menu which allows selecting destination for the trace data (HW interface for transport). So choosing one of the destinations automatically enables ``CONFIG_ESP32_APPTRACE_ENABLE`` option. +2. **Target side:** Application tracing functionality can be enabled by ``CONFIG_ESP32_APPTRACE_ENABLE`` macro via menuconfig. This option enables the module and makes tracing API available for users. Actually there is menu *Component config > Application Level Tracing* which allows selecting destination for the trace data (HW interface for transport). So choosing one of the destinations automatically enables ``CONFIG_ESP32_APPTRACE_ENABLE`` option. .. note:: - In order to achieve higher data rates and minimize number of dropped packets it is recommended to modify JTAG adapter working frequency in OpenOCD config file ``$IDF_PATH/docs/esp32.cfg``. Maximum tested stable speed on ESP-WROVER-KIT board is 26MHz, so for this board you need to have the following statement in your configuration file ``adapter_khz 26000`` instead of the default ``adapter_khz 200``. Actually maximum stable JTAG frequency can depend on the host system configuration. In general JTAG clock shuld not exceed APB clock / 4. + In order to achieve higher data rates and minimize number of dropped packets it is recommended to modify JTAG adapter working frequency in respective OpenOCD config file, e.g. ``modules/esp-wroom-32.cfg``, if you are using ESP-WROOM-32 module. For example on ESP-WROVER-KIT board the maximum tested stable speed is 26MHz, so for this board you can update the following statement in your configuration file ``adapter_khz 26000``, instead of the default ``adapter_khz 20000``. Actually maximum stable JTAG frequency can depend on the host system configuration. In general JTAG clock should not exceed APB clock / 4. There are two additional menuconfig options not mentioned above: -1. ``CONFIG_ESP32_APPTRACE_POSTMORTEM_FLUSH_TRAX_THRESH``. This option is necessary due to the nature of working over JTAG. In that mode trace data are exposed to the host in 16KB blocks. In post-mortem mode when one block is filled it is exposed to the host and the previous one becomes unavailable. In other words trace data are overwritten in 16KB granularity. On panic the latest data from the current input block are exposed to host and host can read them for post-analysis. It can happen that system panic occurs when there are very small amount of data which are not exposed to the host yet. In this case the previous 16KB of collected data will be lost and host will see the latest, but very small piece of the trace. It can be insufficient to diagnose the problem. This menuconfig option allows avoiding such situations. It controls the threshold for flushing data in case of panic. For example user can decide that it needs not less then 512 bytes of the recent trace data, so if there is less then 512 bytes of pending data at the moment of panic they will not be flushed and will not overwrite previous 16KB. The option is only meaningful in post-mortem mode and when working over JTAG. -2. ``CONFIG_ESP32_APPTRACE_ONPANIC_HOST_FLUSH_TMO``. The option is only meaningful in streaming mode and controls the maximum time tracing module will wait for the host to read the last data in case of panic. +1. *Component config > Application Level Tracing > Threshold for flushing last trace data to host on panic* (``CONFIG_ESP32_APPTRACE_POSTMORTEM_FLUSH_TRAX_THRESH``). This option is necessary due to the nature of working over JTAG. In that mode trace data are exposed to the host in 16KB blocks. In post-mortem mode when one block is filled it is exposed to the host and the previous one becomes unavailable. In other words trace data are overwritten in 16KB granularity. On panic the latest data from the current input block are exposed to host and host can read them for post-analysis. It can happen that system panic occurs when there are very small amount of data which are not exposed to the host yet. In this case the previous 16KB of collected data will be lost and host will see the latest, but very small piece of the trace. It can be insufficient to diagnose the problem. This menuconfig option allows avoiding such situations. It controls the threshold for flushing data in case of panic. For example user can decide that it needs not less then 512 bytes of the recent trace data, so if there is less then 512 bytes of pending data at the moment of panic they will not be flushed and will not overwrite previous 16KB. The option is only meaningful in post-mortem mode and when working over JTAG. +2. *Component config > Application Level Tracing > Timeout for flushing last trace data to host on panic* (``CONFIG_ESP32_APPTRACE_ONPANIC_HOST_FLUSH_TMO``). The option is only meaningful in streaming mode and controls the maximum time tracing module will wait for the host to read the last data in case of panic. How to use this library @@ -55,186 +56,184 @@ How to use this library This library provides API for transferring arbitrary data between host and ESP32. When enabled in menuconfig target application tracing module is initialized automatically at the system startup, so all what the user needs to do is to call corresponding API to send, receive or flush the data. + Application Specific Tracing ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In general user should decide what type of data should be transferred in every direction and how these data must be interpreted (processed). The following steps must be performed to transfer data between target and host: -1. On target side user should implement algorithms for writing trace data to the host. Piece of code below shows an example how to do this. - -.. code-block:: c - - #include "esp_app_trace.h" - ... - char buf[] = "Hello World!"; - esp_err_t res = esp_apptrace_write(ESP_APPTRACE_DEST_TRAX, buf, strlen(buf), ESP_APPTRACE_TMO_INFINITE); - if (res != ESP_OK) { - ESP_LOGE(TAG, "Failed to write data to host!"); - return res; - } - -``esp_apptrace_write()`` function uses memcpy to copy user data to the internal buffer. In some cases it can be more optimal to use ``esp_apptrace_buffer_get()`` and ``esp_apptrace_buffer_put()`` functions. They allow developers to allocate buffer and fill it themselves. The following piece of code shows how to do this. - -.. code-block:: c - - #include "esp_app_trace.h" - ... - int number = 10; - char *ptr = (char *)esp_apptrace_buffer_get(ESP_APPTRACE_DEST_TRAX, 32, 100/*tmo in us*/); - if (ptr == NULL) { - ESP_LOGE("Failed to get buffer!"); - return ESP_FAIL; - } - sprintf(ptr, "Here is the number %d", number); - esp_err_t res = esp_apptrace_buffer_put(ESP_APPTRACE_DEST_TRAX, ptr, 100/*tmo in us*/); - if (res != ESP_OK) { - /* in case of error host tracing tool (e.g. OpenOCD) will report incomplete user buffer */ - ESP_LOGE("Failed to put buffer!"); - return res; - } - -Also according to his needs user may want to receive data from the host. Piece of code below shows an example how to do this. - -.. code-block:: c - - #include "esp_app_trace.h" - ... - char buf[32]; - char down_buf[32]; - size_t sz = sizeof(buf); - - /* config down buffer */ - esp_apptrace_down_buffer_config(down_buf, sizeof(down_buf)); - /* check for incoming data and read them if any */ - esp_err_t res = esp_apptrace_read(ESP_APPTRACE_DEST_TRAX, buf, &sz, 0/*do not wait*/); - if (res != ESP_OK) { - ESP_LOGE(TAG, "Failed to read data from host!"); - return res; - } - if (sz > 0) { - /* we have data, process them */ - ... - } - -``esp_apptrace_read()`` function uses memcpy to copy host data to user buffer. In some cases it can be more optimal to use ``esp_apptrace_down_buffer_get()`` and ``esp_apptrace_down_buffer_put()`` functions. -They allow developers to ocupy chunk of read buffer and process it in-place. The following piece of code shows how to do this. +1. On target side user should implement algorithms for writing trace data to the host. Piece of code below shows an example how to do this. + + .. code-block:: c + + #include "esp_app_trace.h" + ... + char buf[] = "Hello World!"; + esp_err_t res = esp_apptrace_write(ESP_APPTRACE_DEST_TRAX, buf, strlen(buf), ESP_APPTRACE_TMO_INFINITE); + if (res != ESP_OK) { + ESP_LOGE(TAG, "Failed to write data to host!"); + return res; + } + + ``esp_apptrace_write()`` function uses memcpy to copy user data to the internal buffer. In some cases it can be more optimal to use ``esp_apptrace_buffer_get()`` and ``esp_apptrace_buffer_put()`` functions. They allow developers to allocate buffer and fill it themselves. The following piece of code shows how to do this. + + .. code-block:: c + + #include "esp_app_trace.h" + ... + int number = 10; + char *ptr = (char *)esp_apptrace_buffer_get(ESP_APPTRACE_DEST_TRAX, 32, 100/*tmo in us*/); + if (ptr == NULL) { + ESP_LOGE("Failed to get buffer!"); + return ESP_FAIL; + } + sprintf(ptr, "Here is the number %d", number); + esp_err_t res = esp_apptrace_buffer_put(ESP_APPTRACE_DEST_TRAX, ptr, 100/*tmo in us*/); + if (res != ESP_OK) { + /* in case of error host tracing tool (e.g. OpenOCD) will report incomplete user buffer */ + ESP_LOGE("Failed to put buffer!"); + return res; + } + + Also according to his needs user may want to receive data from the host. Piece of code below shows an example how to do this. + + .. code-block:: c + + #include "esp_app_trace.h" + ... + char buf[32]; + char down_buf[32]; + size_t sz = sizeof(buf); + + /* config down buffer */ + esp_apptrace_down_buffer_config(down_buf, sizeof(down_buf)); + /* check for incoming data and read them if any */ + esp_err_t res = esp_apptrace_read(ESP_APPTRACE_DEST_TRAX, buf, &sz, 0/*do not wait*/); + if (res != ESP_OK) { + ESP_LOGE(TAG, "Failed to read data from host!"); + return res; + } + if (sz > 0) { + /* we have data, process them */ + ... + } + + ``esp_apptrace_read()`` function uses memcpy to copy host data to user buffer. In some cases it can be more optimal to use ``esp_apptrace_down_buffer_get()`` and ``esp_apptrace_down_buffer_put()`` functions. They allow developers to occupy chunk of read buffer and process it in-place. The following piece of code shows how to do this. + + .. code-block:: c + + #include "esp_app_trace.h" + ... + char down_buf[32]; + uint32_t *number; + size_t sz = 32; + + /* config down buffer */ + esp_apptrace_down_buffer_config(down_buf, sizeof(down_buf)); + char *ptr = (char *)esp_apptrace_down_buffer_get(ESP_APPTRACE_DEST_TRAX, &sz, 100/*tmo in us*/); + if (ptr == NULL) { + ESP_LOGE("Failed to get buffer!"); + return ESP_FAIL; + } + if (sz > 4) { + number = (uint32_t *)ptr; + printf("Here is the number %d", *number); + } else { + printf("No data"); + } + esp_err_t res = esp_apptrace_down_buffer_put(ESP_APPTRACE_DEST_TRAX, ptr, 100/*tmo in us*/); + if (res != ESP_OK) { + /* in case of error host tracing tool (e.g. OpenOCD) will report incomplete user buffer */ + ESP_LOGE("Failed to put buffer!"); + return res; + } + +2. The next step is to build the program image and download it to the target as described in :doc:`Build and Flash <../get-started/make-project>`. +3. Run OpenOCD (see :doc:`JTAG Debugging <../api-guides/jtag-debugging/index>`). +4. Connect to OpenOCD telnet server. It can be done using the following command in terminal ``telnet 4444``. If telnet session is opened on the same machine which runs OpenOCD you can use ``localhost`` as ```` in the command above. +5. Start trace data collection using special OpenOCD command. This command will transfer tracing data and redirect them to specified file or socket (currently only files are supported as trace data destination). For description of the corresponding commands see `OpenOCD Application Level Tracing Commands`_. +6. The final step is to process received data. Since format of data is defined by user the processing stage is out of the scope of this document. Good starting points for data processor are python scripts in ``$IDF_PATH/tools/esp_app_trace``: ``apptrace_proc.py`` (used for feature tests) and ``logtrace_proc.py`` (see more details in section `Logging to Host`_). -.. code-block:: c - - #include "esp_app_trace.h" - ... - char down_buf[32]; - uint32_t *number; - size_t sz = 32; - - /* config down buffer */ - esp_apptrace_down_buffer_config(down_buf, sizeof(down_buf)); - char *ptr = (char *)esp_apptrace_down_buffer_get(ESP_APPTRACE_DEST_TRAX, &sz, 100/*tmo in us*/); - if (ptr == NULL) { - ESP_LOGE("Failed to get buffer!"); - return ESP_FAIL; - } - if (sz > 4) { - number = (uint32_t *)ptr; - printf("Here is the number %d", *number); - } else { - printf("No data"); - } - esp_err_t res = esp_apptrace_down_buffer_put(ESP_APPTRACE_DEST_TRAX, ptr, 100/*tmo in us*/); - if (res != ESP_OK) { - /* in case of error host tracing tool (e.g. OpenOCD) will report incomplete user buffer */ - ESP_LOGE("Failed to put buffer!"); - return res; - } - -2. The next step is to build the program image and download it to the target as described in :doc:`Build and Flash <../get-started/make-project>`. -3. Run OpenOCD (see :doc:`JTAG Debugging <../api-guides/jtag-debugging/index>`). -4. Connect to OpenOCD telnet server. On Linux it can be done using the following command in terminal ``telnet 4444``. If telnet session is opened on the same machine which runs OpenOCD you can use ``localhost`` as ```` in the command above. -5. Start trace data collection using special OpenOCD command. This command will transfer tracing data and redirect them to specified file or socket (currently only files are supported as trace data destination). For description of the corresponding commands see `OpenOCD Application Level Tracing Commands`_. -6. The final step is to process received data. Since format of data is defined by user the processing stage is out of the scope of this document. Good starting points for data processor are python scripts in ``$IDF_PATH/tools/esp_app_trace``: ``apptrace_proc.py`` (used for feature tests) and ``logtrace_proc.py`` (see more details in section `Logging to Host`_). OpenOCD Application Level Tracing Commands """""""""""""""""""""""""""""""""""""""""" -*HW UP BUFFER* is shared between user data blocks and filling of the allocated memory is performed on behalf of the API caller (in task or ISR context). In multithreading environment it can happen that task/ISR which fills the buffer is preempted by another high prio task/ISR. So it is possible situation that user data preparation process is not completed at the moment when that chunk is read by the host. To handle such conditions tracing module prepends all user data chunks with header which contains allocated user buffer size (2 bytes) and length of actually written data (2 bytes). So total length of the header is 4 bytes. OpenOCD command which reads trace data reports error when it reads incomplete user data chunk, but in any case it puts contents of the whole user chunk (including unfilled area) to output file. +*HW UP BUFFER* is shared between user data blocks and filling of the allocated memory is performed on behalf of the API caller (in task or ISR context). In multithreading environment it can happen that task/ISR which fills the buffer is preempted by another high priority task/ISR. So it is possible situation that user data preparation process is not completed at the moment when that chunk is read by the host. To handle such conditions tracing module prepends all user data chunks with header which contains allocated user buffer size (2 bytes) and length of actually written data (2 bytes). So total length of the header is 4 bytes. OpenOCD command which reads trace data reports error when it reads incomplete user data chunk, but in any case it puts contents of the whole user chunk (including unfilled area) to output file. + Below is the description of available OpenOCD application tracing commands. .. note:: Currently OpenOCD does not provide commands to send arbitrary user data to the target. + Command usage: -``esp108 apptrace [start ] | [stop] | [status] | [dump ]`` +``esp32 apptrace [start ] | [stop] | [status] | [dump ]`` Sub-commands: - .. list-table:: - :widths: 20 80 - :header-rows: 1 - - * - Sub-command - - Description - * - start - - Start tracing (continuous streaming). - * - stop - - Stop tracing. - * - status - - Get tracing status. - * - dump - - Dump all data from *HW UP BUFFER* (post-mortem dump). +``start`` + Start tracing (continuous streaming). +``stop`` + Stop tracing. +``status`` + Get tracing status. +``dump`` + Dump all data from (post-mortem dump). + Start command syntax: ``start [poll_period [trace_size [stop_tmo [wait4halt [skip_size]]]]`` - .. list-table:: - :widths: 20 80 - :header-rows: 1 - - * - Argument - - Description - * - outfile - - Path to file to save data from both CPUs. This argument should have the following format: ``file://path/to/file``. - * - poll_period - - Data polling period (in ms). If greater then 0 then command runs in non-blocking mode. By default 1 ms. - * - trace_size - - Maximum size of data to collect (in bytes). Tracing is stopped after specified amount of data is received. By default -1 (trace size stop trigger is disabled). - * - stop_tmo - - Idle timeout (in sec). Tracing is stopped if there is no data for specified period of time. By default -1 (disable this stop trigger). - * - wait4halt - - If 0 start tracing immediately, otherwise command waits for the target to be halted (after reset, by breakpoint etc.) and then automatically resumes it and starts tracing. By default 0. - * - skip_size - - Number of bytes to skip at the start. By default 0. +``outfile`` + Path to file to save data from both CPUs. This argument should have the following format: ``file://path/to/file``. +``poll_period`` + Data polling period (in ms). If greater then 0 then command runs in non-blocking mode. By default 1 ms. +``trace_size`` + Maximum size of data to collect (in bytes). Tracing is stopped after specified amount of data is received. By default -1 (trace size stop trigger is disabled). +``stop_tmo`` + Idle timeout (in sec). Tracing is stopped if there is no data for specified period of time. By default -1 (disable this stop trigger). Optionally set it to value longer than longest pause between tracing commands from target. +``wait4halt`` + If 0 start tracing immediately, otherwise command waits for the target to be halted (after reset, by breakpoint etc.) and then automatically resumes it and starts tracing. By default 0. +``skip_size`` + Number of bytes to skip at the start. By default 0. .. note:: - If ``poll_period`` is 0 OpenOCD telnet command line will not be avalable until tracing is stopped. You must stop it manually by resetting the board or pressing CTRL+C in OpenOCD window (not one with the telnet session). + If ``poll_period`` is 0, OpenOCD telnet command line will not be available until tracing is stopped. You must stop it manually by resetting the board or pressing CTRL+C in OpenOCD window (not one with the telnet session). Another option is to set ``trace_size`` and wait until this size of data is collected. At this point tracing stops automatically. + Logging to Host ^^^^^^^^^^^^^^^ IDF implements useful feature: logging to host via application level tracing library. This is a kind of semihosting when all ESP_LOGx calls sends strings to be printed to the host instead of UART. This can be useful because "printing to host" eliminates some steps performed when logging to UART. The most part of work is done on the host. -By default IDF's logging library uses vprintf-like function to write formatted output to dedicated UART. In general it invloves the following steps: + +By default IDF's logging library uses vprintf-like function to write formatted output to dedicated UART. In general it involves the following steps: 1. Format string is parsed to obtain type of each argument. 2. According to its type every argument is converted to string representation. 3. Format string combined with converted arguments is sent to UART. -Though implementation of vprintf-like function can be optimised to a certain level, all steps above have to be peformed in any case and every step takes some time (especially item 3). So it is frequent situation when addition of extra logging to the program to diagnose some problem changes its behaviour and problem dissapears or in the worst cases program can not work normally at all and ends up with an error or even hangs. +Though implementation of vprintf-like function can be optimised to a certain level, all steps above have to be performed in any case and every step takes some time (especially item 3). So it is frequent situation when addition of extra logging to the program to diagnose some problem changes its behaviour and problem disappears or in the worst cases program can not work normally at all and ends up with an error or even hangs. + Possible ways to overcome this problem are to use higher UART bitrates (or another faster interface) and/or move string formatting procedure to the host. + Application level tracing feature can be used to transfer log information to host using ``esp_apptrace_vprintf`` function. This function does not perform full parsing of the format string and arguments, instead it just calculates number of arguments passed and sends them along with the format string address to the host. On the host log data are processed and printed out by a special Python script. + Limitations """"""""""" -Curent implmentation of logging over JTAG has several limitations: +Current implementation of logging over JTAG has some limitations: 1. Tracing from ``ESP_EARLY_LOGx`` macros is not supported. 2. No support for printf arguments which size exceeds 4 bytes (e.g. ``double`` and ``uint64_t``). 3. Only strings from .rodata section are supported as format strings and arguments. 4. Maximum number of printf arguments is 256. + How To Use It """"""""""""" @@ -265,6 +264,7 @@ In order to use logging via trace module user needs to perform the following ste 2. Follow instructions in items 2-5 in `Application Specific Tracing`_. 3. To print out collected log records run the following command in terminal: ``$IDF_PATH/tools/esp_app_trace/logtrace_proc.py /path/to/trace/file /path/to/program/elf/file``. + Log Trace Processor Command Options ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -274,29 +274,18 @@ Command usage: Positional arguments: - .. list-table:: - :widths: 20 80 - :header-rows: 1 - - * - Argument - - Description - * - trace_file - - Path to log trace file - * - elf_file - - Path to program ELF file +``trace_file`` + Path to log trace file +``elf_file`` + Path to program ELF file Optional arguments: - .. list-table:: - :widths: 20 80 - :header-rows: 1 +``-h``, ``--help`` + show this help message and exit +``--no-errors``, ``-n`` + Do not print errors - * - Argument - - Description - * - -h, --help - - show this help message and exit - * - --no-errors, -n - - Do not print errors System Behaviour Analysis with SEGGER SystemView ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -305,71 +294,76 @@ Another useful IDF feature built on top of application tracing library is the sy .. note:: - Currently IDF-based application is able to generate SystemView compatible traces, but tracing process can not be controlled using that tool. + Currently IDF-based application is able to generate SystemView compatible traces in form of files to be opened in SystemView application. The tracing process can not yet be controlled using that tool. + How To Use It """"""""""""" -Support for this feature is enabled by ``CONFIG_SYSVIEW_ENABLE`` menuconfig option. It also enables a bunch of options related to that type of tracing: - -1. ``CONFIG_SYSVIEW_TS_SOURCE`` selects the source of timestamps for SystemView events. In single core mode timestamps are generated using ESP32 internal cycle counter running at maximum 240 Mhz (~4ns granularity). In dual-core mode external timer working at 40Mhz is used, so timestamp granularity is 25 ns. -2. ``CONFIG_SYSVIEW_EVT_XXX`` enables or disables particular SystemView event. +Support for this feature is enabled by *Component config > Application Level Tracing > FreeRTOS SystemView Tracing* (``CONFIG_SYSVIEW_ENABLE``) menuconfig option. There are several other options enabled under the same menu: + +1. *ESP32 timer to use as SystemView timestamp source* (``CONFIG_SYSVIEW_TS_SOURCE``) selects the source of timestamps for SystemView events. In single core mode timestamps are generated using ESP32 internal cycle counter running at maximum 240 Mhz (~4 ns granularity). In dual-core mode external timer working at 40Mhz is used, so timestamp granularity is 25 ns. +2. Individually enabled or disabled collection of SystemView events (``CONFIG_SYSVIEW_EVT_XXX``): + + - Trace Buffer Overflow Event + - ISR Enter Event + - ISR Exit Event + - ISR Exit to Scheduler Event + - Task Start Execution Event + - Task Stop Execution Event + - Task Start Ready State Event + - Task Stop Ready State Event + - Task Create Event + - Task Terminate Event + - System Idle Event + - Timer Enter Event + - Timer Exit Event IDF has all the code required to produce SystemView compatible traces, so user can just configure necessary project options (see above), build, download the image to target and use OpenOCD to collect data as described in the previous sections. + OpenOCD SystemView Tracing Command Options -""""""""""""""""""""""""""""""""""""""""""" +"""""""""""""""""""""""""""""""""""""""""" Command usage: -``esp108 sysview [start ] | [stop] | [status]`` +``esp32 sysview [start ] | [stop] | [status]`` Sub-commands: - .. list-table:: - :widths: 20 80 - :header-rows: 1 - - * - Sub-command - - Description - * - start - - Start tracing (continuous streaming). - * - stop - - Stop tracing. - * - status - - Get tracing status. +``start`` + Start tracing (continuous streaming). +``stop`` + Stop tracing. +``status`` + Get tracing status. Start command syntax: ``start [outfile2] [poll_period [trace_size [stop_tmo]]]`` - .. list-table:: - :widths: 20 80 - :header-rows: 1 - - * - Argument - - Description - * - outfile1 - - Path to file to save data from PRO CPU. This argument should have the following format: ``file://path/to/file``. - * - outfile2 - - Path to file to save data from APP CPU. This argument should have the following format: ``file://path/to/file``. - * - poll_period - - Data polling period (in ms). If greater then 0 then command runs in non-blocking mode. By default 1 ms. - * - trace_size - - Maximum size of data to collect (in bytes). Tracing is stopped after specified amount of data is received. By default -1 (trace size stop trigger is disabled). - * - stop_tmo - - Idle timeout (in sec). Tracing is stopped if there is no data for specified period of time. By default -1 (disable this stop trigger). +``outfile1`` + Path to file to save data from PRO CPU. This argument should have the following format: ``file://path/to/file``. +``outfile2`` + Path to file to save data from APP CPU. This argument should have the following format: ``file://path/to/file``. +``poll_period`` + Data polling period (in ms). If greater then 0 then command runs in non-blocking mode. By default 1 ms. +``trace_size`` + Maximum size of data to collect (in bytes). Tracing is stopped after specified amount of data is received. By default -1 (trace size stop trigger is disabled). +``stop_tmo`` + Idle timeout (in sec). Tracing is stopped if there is no data for specified period of time. By default -1 (disable this stop trigger). .. note:: - If ``poll_period`` is 0 OpenOCD telnet command line will not be avalable until tracing is stopped. You must stop it manually by resetting the board or pressing CTRL+C in OpenOCD window (not one with the telnet session). + If ``poll_period`` is 0 OpenOCD telnet command line will not be available until tracing is stopped. You must stop it manually by resetting the board or pressing CTRL+C in OpenOCD window (not one with the telnet session). Another option is to set ``trace_size`` and wait until this size of data is collected. At this point tracing stops automatically. + Data Visualization """""""""""""""""" -After trace data are collected user can use special tool to visuailize the results and inspect behaviour of the program. Unfortunately SystemView does not support tracing from multiple cores. So when tracing from ESP32 working in dual-core mode two files are generated: one for PRO CPU and another one for APP CPU. -User can load every file into separate instance of the tool. It is uneasy and awkward to analyze data for every core in separate instance of the tool. -Fortunately there is Eclipse plugin called *Impulse* which can load several trace files and makes its possible to inspect events from both cores in one view. Also this plugin has no limitation of 1000000 events as compared to free version of SystemView. +After trace data are collected user can use special tool to visualize the results and inspect behaviour of the program. Unfortunately SystemView does not support tracing from multiple cores. So when tracing from ESP32 working in dual-core mode two files are generated: one for PRO CPU and another one for APP CPU. User can load every file into separate instance of the tool. + +It is uneasy and awkward to analyze data for every core in separate instance of the tool. Fortunately there is Eclipse plugin called *Impulse* which can load several trace files and makes its possible to inspect events from both cores in one view. Also this plugin has no limitation of 1000000 events as compared to free version of SystemView. Good instruction on how to install, configure and visualize data in Impulse from one core can be found `here `_. @@ -378,10 +372,11 @@ Good instruction on how to install, configure and visualize data in Impulse from IDF uses its own mapping for SystemView FreeRTOS events IDs, so user needs to replace original file with mapping ``$SYSVIEW_INSTALL_DIR/Description/SYSVIEW_FreeRTOS.txt`` with ``$IDF_PATH/docs/api-guides/SYSVIEW_FreeRTOS.txt``. Also contents of that IDF specific file should be used when configuring SystemView serializer using above link. + Configure Impulse for Dual Core Traces ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -After installing Impulse and ensuring that it can succussefully load trace files for each core in separate tabs user can add special Multi Adapter port and load both files into one view. To do this user needs to do the following in Eclipse: +After installing Impulse and ensuring that it can successfully load trace files for each core in separate tabs user can add special Multi Adapter port and load both files into one view. To do this user needs to do the following in Eclipse: 1. Open 'Signal Ports' view. Go to Windows->Show View->Other menu. Find 'Signal Ports' view in Impulse folder and double-click on it. 2. In 'Signal Ports' view right-click on 'Ports' and select 'Add ...'->New Multi Adapter Port -- 2.40.0