From 97b85779d0978b0dc685407d81d4a0105119f613 Mon Sep 17 00:00:00 2001 From: Blake Date: Mon, 24 Feb 2025 23:28:55 +0900 Subject: [PATCH] Post(meta-cm-sql_and_ml_xai-20250126) --- .../meta-cm-sql_and_ml_xai-20250126/image.jpg | Bin 0 -> 41742 bytes .../index.ipynb | 1473 +++++++++++++++++ 2 files changed, 1473 insertions(+) create mode 100644 posts/meta-cm-sql_and_ml_xai-20250126/image.jpg create mode 100644 posts/meta-cm-sql_and_ml_xai-20250126/index.ipynb diff --git a/posts/meta-cm-sql_and_ml_xai-20250126/image.jpg b/posts/meta-cm-sql_and_ml_xai-20250126/image.jpg new file mode 100644 index 0000000000000000000000000000000000000000..3ec04c8c4e169aa7cfbd882821eb0d436b62c93a GIT binary patch literal 41742 zcmb@tbyyt1vnaZ_y9Ny*I0Se1;I_C03-0a^90DZ4-E9{M&f*XxEbhVGU4wHszjNL@ z_kG{Hf4x^T(_K?t-Bn%PGt*mL`}*f~9e}GSryvJ_gM$NngdKp_O?V0gX=zh+O*J_M zWm#AO0058aVBzQvj|%`eI(fKh%1gb~(>HjFvJ8L&ya6Bpm;i!i7Va*R>gvjX|0CSa z!MI^C&GxUY|0CP~djYzomAeJ3=m~b2Sh%=(003}CFdW0%!{r}t1H%c;?f-)Z{KMT~ z3c~2${^8dDg%|&a=fCil|L_1crC~fHFkHg=f5XlHH~hcl!dL(Z7F_?b|9`>5$_J)D z0HB};0Q|G*f3^NUw)!6$0x?VlZrE59{}0W%5C9OkfziMH56wIS0BDW?07w`AhxR@n z0Kf?c0Gh@vJl(whO9p@pmJh=XAy^>&K()uX#T6&gAm^dEVEV>ID*QtZNyFRt#NNdZ zDW*0vpwhN0aQfE)$Ls6s5+ED^kAQ%Hh=7QMgoykPP*G7(P*Jha&|wD_0VXyk>?FV= z#KVDIRHQ^iq*Sz&6qK~gbac$TT)eWfMsWXM8+h#n;Gx2S;6o7L-T~n8;1KZOUi$$2 zu)#!v&EtPF_P+rRrX?~w3Mv{p901|}SpXHk^sO1H9}DTTQn?#gW~|e8Q^K~A|fH$O2Wy&$*9W$B-IH4L~x*YF%;{F zgfKk_agV;issey&vLr8D zi3B2&vILxDlo2Cpeh3^P@GT~c2L^DkULC=nrWB%o3K8PM2>*oq2jJAPwS`m=j|Fkb zLTIZcU^3w(;E3{Z35Bqc;E({q4(8DjvZPAUJSa9B>i=m4R*ewPZ~b3wxPOYl>;T7s z9P%FwmIwm%^Y=>8L!!vk0l0u}xLZPiWC@-GP$M7ypH*PeVL$};-==}J@UNS2GO$El z0+##-i2kLxvMPkI?n@xRv?Y^N7m`4PjS#HQu*m>31+0z#u7*(j?@o9JHrkRBl9B}z zD5781U62Rin!NJ=Y+{7{6cCo!R>%D(22t3N7YUINpE-Y01NMdiz;!#o#zjUQ2+;U1 zYyY#9E?i-6znrh9``bS9drlMAwGsh#{@@88xeq!SQ9Nf#miEh*GV@^cHYzY;U8guXmVdV8PIz+k$5(KHT(*YC@uRUelk9DXB{++zx8|Erji}%cU^IO z>C~a}bbC2_BTCI+J?`?7*S}X5QG0kH6qh)8B!g(yZQxnI3%;DC>ur;-qH0pCb zP`d^dAn3go+L3%0P5JhpvM`{GLdq%LOFuKYR{cC?q>xc(lcgk&pVyZ0Rn&FgBJXID zr!$ZJYsXFR?I5)I6`)`v#CboicASzUXnw}Z;$-z#`UAzD`{_n!=N=^jHn`j)Rrr$f zPhSb;>Uu=KmBppQXVj`K2*wHL*S`$I+Jag2& zRm&8)&4b2ThH+}`dj&Ag8{GRf70EYKfbSX-NQk-)z!}eiBu+`oCptd0y(#TJ#;Im9 z!xOc#q22LEmHD#%jR!UTthMpIMgQU4PCrAYxr!7S? zFt0%B~tV*KPztnWK7`2%>di>e*%GW7!F|>b8 zWO-D!s;lL*_HenRu5Q~`ao#+wB3JHsLgky0v+{RVWp%#BqJ!+JeuXQYtIVJkk)u4` zjO>!6DS?vKUnOHU4dkDW^&wB!bz@3Z(+q21Ji8sawxnhxQ(xmQRuny_tM>8c4?Izh z#ve>FV(A52+xdW`Y+xkzpw{nbNb&8J+k^$ zfCCw9Q#-|rXInuS&2P*WfZFO7{>;o#u_=XyRDlPd{@B*8Az5+l6(#ZArgq7av{nJ< z$a5MDWIF;IW}DoAW-~ywZ|veTqS;E5^gcl|?{{{3=GZ>+`z}q2yaKWYt;}``g37+; z*AE)!5PvrQaDDI3977-wu5~@)0jf#eYB_wsG8KE!yC3@9SmE6^#_g)9hrgtg=2Q!zJa(>(%3EeVP`w@YGcmiu!5`Qt|-OF+eGB`YyIZ+>j&Z zy7QAUvPbK$i#s$k)m!8%zppF3!j!G#DjqQMg5g4IBPNskElz8)9@sH@UNQq{)*n~Z zq8p=bUjY=uI)WJ;SQJbLGLyT1Wx7Y6(XP4}fT*B7Q!E43#GK_eS%9*W%W90~&GLPq zK5L~Kw^x-+$ZWr?<YrsuGL!cIJjNMd36{Lt#*vd_mpS8STnv z75e7-?@N`^zK%abB`dOWwmY+zMNGwxu3mJ_lxGrh<+4-qTm@6ARkR{^>=>l4vxN#D zJ{VGtcboGS5`MCTSsP66NWOMjh+vOb944P0H|U?`Iq%n0iLhH=1$KS9 zJd=;D6D@Wc6L7f|oD!|L!@27mOc^j1uirI)b~a1cJRaPgdj-^B1aWC|v{jwe$d9g= zdr`XHZdFl^U%Pu$iWhrYyN7OKj(d0o>}`f!=i%*Jom&n+olQQy0xEsH|LEgjOy7Ge>5Wf5 zJ?p&0A_^o=Jqv&Ao!B=h$(`05q!+xX){xh4s0)?E2k;770yijzN2u zfq8MFT{jksozw?iEJ2=+d)xgr{S86!Z@~RsB9G!D zhH5$&>@D}7X*N3ux)N{RK1~lkvCrFbWdsFF+*Jja_p;VHxf?SWvZ^8%YySjFeCqd%JC8N_nyC^Zkv87y$h<@Rp~l0?Rf==A3Oy@t}=qv zwyT3F>yLIuxSLBB&4$a)j6URn6J7fD=gg{a#{myvJ%5AcJI9^^uWO)P63@|1Pks;5 zNjMe3d?FTCOGhHkpclj9!e^XqR;Sv@XKH?LC6g8LvI-Hd`mioFW2wQ4-NQikzTG|d z<=^~G2BZpw?e)roCk#$UmR8Lby;qSCfBw#x4eYVp9PPdA;knIcnxS^PXAGRQ0A2^f zi`M!1U@Tv;>@{$Z%hgii6m{Jz0HNwhP5Ob|`@(+Zg?s->9NsT6E2!f92Jej9RQ@XA z_qdB{Vd?jR1?>x)9Lp8eVBs`l!x8)X_QH-Uzvr6OPV`CN+=O|O&uifR%(DsZjKhqG z;p;2Ay5-URy*N*Ls`{UUhqdi1&m=A>!-5>iYtE`UmqOqC_Wu%C4zQOHcpq?Kb>!|GLSb z%GIc!s54kJFQ(Hx=X$+S?d^B2Qg|s{E!mpYjm5)mA z#Ik+}Y&5C9Z~gYMJ}%L8#e?6$zCbQ@M6p3j!DB|Ybb6$-<~|9lSj%89{0fu){)oYH z>(XPbKkBf~*H9>dXB>XDwM>-Bf8GRHXH_(qLHUD~U?-lR{Hg!nak87r2d+EFq{jep zob_&?zym``!0f?X6}PxVKr1QMa`_j?Z$sfy!SYlr&L&f{l00L+1Y|ONucdi{6T44> zUEp1E9gaJRBn$1OtRUtT0-qEE)nB}&h0K%@B69`&?n=C(k#un}DkGHcb16v?!-;8ep#8@;rfR$c@ zh`KVqhJC7+VG>!ACCSU^_%%0HINyj{6NoihIf#+|x8ss0V z^R>@~-Rg$Fkgdk{uF-FLhUsyX?z62Z1v`G3eCa)1Hs-Q^fTxp^qo?}%BIKyLt)#C! z|GOZWn-Lpr_Tj};pu*2e)aUmFtilEpjyC}T13F~MP*+|)p_Z8-%0#}c+BTIJLx{VE z1)J40Hy4Pf^)yl{CR^o)j<4fkeS>egrvB)7JKKtveUl9hkeAS%F1pyPX*zxT11R^+ zvGH5p=}M8Y$ZYqrG!?T+G8E3^dYw_^Xr`-2HSvLxmCreLz0K8c7MpAYszM&WifgL< zoB{f7IoX!<f+$9%tL*P+{{~peu?Fo|Dqe5lgiYRNP&B>Pc z?6G1)&u=;tq8!lmkd6nX(W%EB%Zj$DbKCW`$@lNwf?V}QTaFJ%yvF^duK*HF&@a}* znHmu#PyB(p8JfDP?!$#?o|9eg3@^OLRw9o>QFlhG6PE+i*&Xt+zYP`ciltOz?@idE z-hSTpCQK8gVfmQKyDy#0G_#fOlK6YYxpp4r5|V;BDi9Fh5fKqk;9$NUm}3a$>A^$5 zr$OYzC6LrWdS~YPmMfHymP1POdjU7Hd2P=mk%Y9m+XnG_myj<%F3w@zAyk+@2~HgT z6)^ql6+nyio?7jGFm>hXJnqw;s2G=m_@i_ltNhd1u+x3sGuGGOlOUoJz0=|3oguev zqM;k3?<;N@D{cq*i*}24Ot+@q3%Wbo61}_f6_6yUBV0$Av*4z?r1anTjCEjcJuHD5g}L+HY_I6WOKm-xxAZ zPB&~ZV)p=qAcDt~Zt-T5DB#(BV8V}TXv^48qfxwOogaO}t&X6TBArRKX56G-3;5gZ z1_f7mAu)8bL^1b`43$Xhcg_em*pkQvizT({lP;29-WS}`?G^5cse}@~2p%NPNyU}| zRQ&~WebS+d)K+(TvA>YM8 z(fRsWbUMA?!NCmg&PA6MY)}lFEn9YuBvdi2x-=!u9&ePHB=AG%_>ymo%Djw{W(~$Q zOFH1+$k1{@K8U7JvIw=~cs%$S1ha(R%2Bj~Q32V9#O78wSxiqqWIm#oEquJ(AmgcZ zc!2KIcHp&C8%t?S3E}2zWvB|3PVkZ7q^*cfeMXtYx}5l#nWtF{F!@B2JlCkpEMawmsA- z`>*6zD$qLNaz<&JHy}q|acz`wNdR@c?s4MiOPHXWy$>K-bk0Y)kQn%oAL8aw;8yx4 z!c=`q-pHRpOhZpI5pcJF!+CJSV&&HHJI@3W37wc2Pd*&{y-DbzOu40O(p`vt`kNc= zMgy26nI-uNu7ocs)+w*mksj$S%Q!9x+lnshTT#N4-Q(=jgGR#zqf0G2!vzw*(mZX4 z8};t&KoX9E!?5ohMET)*#fss(O?X4yDi%qSvSp#J5)VbF<>>vNBD6dc_O-r3E1;bA z4llv|m@a*^=xWX(q|8;HXRscdnuJ*uIS9*5tVq?Zq@W^f%o9JW@1zwS@gs1<41`?a z`YAW437&74wxF09dIoq_e)AhQ9FDGpy1nLEu_XhK%GBZOs$nzd-;7?PTkCOjNY{qP;E;!Z9)9fv{Qawfi=r|mAok{vh3 zpTZu#R3Cx9v0!j=!c?s%^H6bPVi@FbyOBKcn@g?m;oQL`fkdi9%Oyp#1obl9V!fes zl2IMzrO%VKHR{|!chp?{-Ok!oXiv6UQC*z=phC{vxRCIiqPR|Vuozpt1c!Szlhv;L zL7zD{qa?>Uuh2UD(C$7%W%6rE!8zF&XzD?8Z3E}{hPM2-&8KJw=$uVEzrwv?Zg`cM zGskY;D*!8%AQwkMH2K1jLn{LgUzkysa5$FCuX-V`w@R^1Rgzj3QOuu6^DE!Mni*3F z0;f3e!INyRmBWiE*-lV2;k1jcX^6qI?0i(G(7~2-iA8I}^SYs@Lg9qoJKSD{#aHDO z0OHEVJPuQY+z!eN4oarHf4BKH5tK4sHC&&56s#s|F_$K!s%Jm?1&#_5ItFlO0kwrJ}H46pO`-LAR!Zr znkDKh-~y!)VPmwU{!siCP_7#I?(EtKszQo72Rf4(7*j~Lv#{FBIgb#e5ldh0FhPM< zM2_bM#ij_!-nWVp5o(Ds^ah+268@UrMYidYLRrgB zTIc3w$j04=kWUiUtHyMIy@w!V&WXWg;p4b*PuCQwDxEgyC6V8(y^txe<)OfChAjTw zR!h!~`ypE63XKxf(V92G0f*lu6gqv#gPCve%u#+dXHEc-U6SGpN*-)-e+3+#pWP~5 zX=VU0?^BEP*H5S66p!TY##?8f6GfVXQNvbvoM{)d%DZJp7`kNatVp2cwwnt}4R5s> z>dr(43ul4nM}|YZqLxy=NNR;wq;J$5ShBo_6M<*kKmS~7W_>BE!}OKPlEOfnRg+-= zd4?*e!lUE{!bb!5J4=Ta&6ESybD`ky(+rLDlrg+I_d`LkxIb_n)^k2ul~5VMpK9gF zCzAf#pv=WYfF9|%bjK`NZE3mSv5nX9*L2zLYZcJ$aoA}0B2K5q z54UPKpld0hqxvY^uH{C#Tsu(fP{>Zm75zrB!=GxdV?s{iuvAs?z2CF8P-ykAbd=@c z$@}huN0z?5k!qZvxkm=hUoLV&g|#c%fwY%t9%~^^_bUF81Gqo5!-KS4q8r#b_bdHa z)1kQhq{&7L3+|%uyD3Ve8u_j7PyLv0$S<3*{$_~I?wF|VAUCaChT$B_5ZVqatW6+# z@V-&}i*-vuIxlRsi>{BIk(>R2CH-apTU@q6$=ZIr^YjtvnH0v?``!m-}$i?(oAzuf0EBlKQ)Gp|?zx z%Oc&BbrEtgeZRvOyLlR#@0F`$Py$!}449F;T~Pf&#$d@rU$4k0I0oK*s)Q5RQM5SZ zEpEKM?vbgGn^>*O9*E>w>H4Udr7^moJ*!*uET2ZTgC#0G8JMwR=0-W8NXYDIBdvns z0-9_eY2K%C`=x3^bS>{WT0%R_oVIY-_RyQwl2Tb9msHRiIU~M8Dkog{S8p-`v0U1- ztf0)r)hnNi50Cqixz=@kePF0=(^!b~O;DvsF%*QKDJbKBBQ=J1iXJvQDEXNHu2f)vf|n;t7T zH0#TbCL|<1GFLW}xX_ZH=ZfiVP%r{_@TJMlJPl1Nz*j)dC9_9FDh6S{cbDoa6$Mu8 zDK}K|Mm($;8@cohzA6?uA@B2vv!u(w4u=)>?PJ6>@`@?0s;MO@fpSD_#NoUH)wP8} z{M^Uc8Z090kz=^_ z)iS~+2;-J!v4&QWiywJP2ATBxEb7`OtV|&HC};Q65^SJmn&C1JEcmm_JmGw>zzDq2?ln z)mwz;MoV0QnG~%9*_92PKg&M1i^p_yi?tRD38?30dY(7}notY)Gst zzhY_1`R>{#!l*GRlp?KJYoE#SC+k}veH)2<>HZtwrzm)S$`n7p&&Yo+k3Y*;d5$9! z*JV8ZH1cAoBR?%3+vkS55&v#UdS7D+t@%7ab{>1fY&ietJZW6;6j&lUsMD>LM}d8Il&3L1+#cc#cV zBkO7ceF)m>ArZgUXUCRXLA#!eyh01x%{{zp1F!vi=Z^8VlG+!$31h5eCcPt*UR2or z$s_-VFnx(8!(9ez=Bi7awuc>#=_D6n4~apUY3Zb$Uk?wDZ+9DfGo4J{lve2!Ytj6= z!g*xwTDZ7XvITk!%%_0P$r>t*c7%u^WX$}L)W2u%jf?q1d6?*ri#$S8=xk4Oh zkHewvia&2Pdev=DE18pILgdlJG03JiK7_82ZZ7)Xf0?=q%1Tq7pQ%f}{#tfdohG%! zBeiLmmY1PjQ&Dbpl?1`a+#`*NPqJOn9o4S7Q5AEXykU-qKjgJ@*PGx++?k$0lpjy01d!*{|RqUY$HZ2=> zN*A27IRX!7_aI%hW?5aOAxptr_a}JJB$o4y-p8i`jbe)PiaY8Eo{1dgC?}3a19p_@ zgH^Fv&p`m6#bMWF&$;B+ns@KgQYivmvJ^w7o6kBg#)jWb^i?U$n++ca} z#AkYY*I5Y}<@kE@%;7X{i>T7?dXeYy%e z1{>Jn&q>IKxScPya;m8tT@LC;eav`HP3jAfO|%f40Shz_?Jlk8Twn#d*w|e+p$-p- z_uB>VRnc?y7Wh+s|1)T*zr%TRt$}euMD_=<%-+)zfL*ClgkKCTr~<33uRn&L1TwdN z<}0Ff!5(jYBpM6Zcq}Lm+~%!*k47V|W~!!Aga6?se9^HSjW=`pPo?pTZNrD`tuhb~ zTO!Az>!i(fGo)T&%=Wmk6GbZ3(6koI1Cu!jORiw7G%NTX;l-{i;xZ>Q{zjWo3v2Lf z#f@T6df>1rvqT{Elpj6h1Lt{S{iT?cspxt1je3~kPODp9e)RxKYgXp;O#X}UyF;F@ z4t5kfWc?=o)$UF1aZv}^^hU3Mk;zjtnZ$IQs|{vy>(Pt_4s2+Sv62z$dv^5mGCSE> z8(%f!W&>xWvJ&>BmbXN*$?3vsP~wVQ^3iffcTkkC@hiXvIzJ08w+$d}j1)WH>37Rs zG~11fQ|S51V0Vc1(Oqw#^&K0o_-S56`F#>Y>tU97F&ke&tM9@%B~3|J>v@reD*_jikF5P_PD2JzWX-cnnd+JHUUz9;kxK8S`puGQ3B)5OmeaPpLuHww!sbZ z**zwX+}`?smCviRN*j^2T;6W1h_+SfWKU&{XVr0b6H*WfHO84$2?+O2_@tJp4mKQS z4|KH2UV`U}bdtzc+(AGBhY{+RVGXGcq3mYv@uwZF^HKN3d340ETD5a(LZo_$xDKhB z4YDfn8vwPs?gEbItrkYgat)(gMaG98L~0~p&1j4K(EWHjku2IPbV;RX6t#wm|S#9-&N-^1h7)FCXnsyQAK}13V{z+DASy ziU4%jcmaFEQOQm@g;kC_sg2yqn-bAC0%|2?g=>e)e#kXVGY)zP9}khW;to+nx_o-wKg2A#b&wpVgbp&DJgOQ{pz}qk#|?Cllu^2#}YeaBJAUSfWMQuS{$=P5I5eL?~Ld zyK=;a0d&!>oJ zQM`q#VxmZ=jg*>Trb;g5+78!XV{yZlHiCh-`Uf5F8RD;Kqj~Mp#Xz}%;PClFMr}l+ zL7qokkj_lU@*qVnsJdY6i&EL$11+fHA$Gziy|z+_57;oWcfSulXs5rO+YOn%()|2U ze7|Aw3}Mz_(Du)o$2ju2-LC#edDm}^RBy_rzeA@c{`y^%iY7UdNmm4Ni?_17_5Nhh zpDFkx^MJ&r)rYWPiWRNv5>Yxm1N#Wk*#y_KezSWcURGDVhBn1LibdLltp=LCJ7TAg zCtl(99eUAL(UxzAEHHO=HYz$=^6+>$1U->Lz3|tSpMH9YfpW-~?|pEmw!~t$zO9Y6 z22sd%KZO4|?d$?tz~j9iEVP%gMuDhcwJ0R71V?I>If^o55vW6K1A>#(>lVGN*^-`4 zGoJH7%nhr1FQq~#mr?QSbDj(w1XyE4;&?@dANRycM(y*dQ|zB)UjgQXjB}5bN#0#D z3OX&vI5^DN;N&uK2HfgRWIAEuK}*G!QjF+V0MRJn@~P=*v&dTpzLmfFnp&plyMrM2 zWmV;T^gRr^(_P_Dl=Nbt=&80n-_OCSL~PWN!h7>ei4&^+3_ivNU#b*UOCLP?Afw&X zt%S@56l-Y(Ks!ict7XEG2HA=*hK&|+!$zl8D_FUq2hzDAE+K8zi11XuvmYUkJgD$I z;j2<9b-qS0qh7%pV8Qqbuw@a;N{4M>a2^I#EJ9Zvq}+Q%#jOH4>)cD!dYM{M2oLZv z1>KFxLfx|`*hsIH0p@Z#ahi*UI2>H);H|}GlxcSAvY{7Hi7o|X=!P>K<+O@{hD^D1 zIBs!^(~gLYvWZT+oI)AXBj+yi2DqU{67L|MeB{Mfi;Q>Jxqz ziaV;WVFR>yu@Pa|z{ueJP3Vd^Y`y=f{3x>r;9=lQkUxZ?it3>h%sl(ko@XT1z>$vT z6@WKBTy$@cphb=4d2!zcUN*c6#1pYHiu)!nDA`OouWdP%xPUHsh00l-iie2l|4<0c zKq*swOTUaN>{muS_6h+0uAjMKl-ZHi+5m(}`xTNOS0T8Sm{J1wFmm@v$ZKbPykq#) zOtMDlJEv<6;?+XuC$vKjorZS4_MGNx)>Sclkvw?;6=gK7410;ojN))byR-1LzXE>! z*?5D(DDZ_KcTXVqC>PrQr6zTfgtu!mPo>${RJ`|XVti$}7X_{0*54HB<+8dVl>M4C z`Wf0!povPR##0C0B|@l#F>T@AROz`2u2tT+2k+;>VE+JbZ4ioCRbvHu)T3DiRx>SVaGvyNY3JjkbSBY*F9U;fNc)JcyU4QjdT4k7L?K)1`-?z;N zy`PCx$o6$_U^ucDier*Cn7gHh4*kd?NKrWE&Ei--TkuJX-)|)2!wC9Y0Z*`6y-w&7 z9>ge)2u{io$K@#d`o!N9avF3wWN(h6ksIAo#Fgc+ zy%nA?)c7vWbc{a?5cDm3uVecCj|!VZ$SL-v0bTzJJcu<=QnA2OFs>=ZNn2u|4SLk! z1&j;jB&VDqH(vv_D0<2bz(b76)^qV|=;V8ehS4>4u5-TGe?-hHEZ?|{58WiyQb~Al zwPH?KsLMw18@i*B+?=5zZ+f22J$-sVh#dL*NA|5kq!h1RTQyX1()OCjV>aGV(qbu< zzonJlGDBTru4M7;W2|m7*c0QHDB-l<{^X8SIgw7ujj03cp#} zeB|XUvWROQx;(it+MNzll#T*Egtz>#ZY6>_>G6`Gm_8Z6{b__*!30-+Oa^Y%9&Uza zWDTTxx9vsYqF_tw8q5Iv#2$BA(B;4~4snwosMhV9&^uu6VLZKPih?+cB|EzeddJFL zZnob=E`lfE7-BAFNj8l3Vcb%Rz;+WoxHSPUf*K&jwj2OYqgbYKol7x|BvzD%pZWd_ zt*77*cQ0KByjIzt2 z=cM=+$vq^p#O6sjw}v7Sr(XIXco!5vFg&@_{f10~v8aI*yrF3G`X>6*cZy)Vr1N! zOxyfRK#GudFgP^LI6N>V9X~(xG^76c`)Ajbl{u~y?H{EJr*{Vp)m%{x%~06ZfB4S^y~; zP5++D()mous7{i_JmS;LhtEgJeA=SqUj>(_pkRZAW;p7RIGCZ)o-fI*`>%w#N)#04IdSwHyAJMPHCne*wJ@6(dut zbpxOv&vvV_;kx10a{i-41Y1QwEg=tq!E@eY?;h}??%Fg_z{%fQ9`Ww^+e90<2<@1a zLmjc8v;rUEsLF<TsXXkm~raJLpZdi|zm=1Jc@N$ydp zlFM%tEiURpdqUx9c*47rP;_}a2-l5p4$XawkGJ9X-lb70h1rK=M|6}EU8)gpKNezC zJiZbC4V6juk0dlj*$Oxd^E74exata@k9RMYRMK`48)o~z!#sCT^-^W)FBFiJpkAQ< zqH~DT%O)06U`23lQP;wBt*0p*fDZ)6rZ6*{M-gW;N%VfJ(GA~gd31ejxIj5K7nuEI zMrGSAj>K0x7Na&Tqb~qH-9V`(e(>41M`iWjZq%kG(us=*Ant%<-A|E(VT({ zfI%%+|BpG@tvlLQc$eb)ouhiwogC@&0{3!SnUd=ilG5sXwxc~uzKX-y6$kyqwU2F{ zRW)Po)o%`ZQ?XW3%634}Rxi5B%bU6y|_e(75VjOxOD%^0Wez%lU&J4=>9K*Kf z<4>dS=oDMLCm1u=7I0_jrzHe(i)$utlYti4u7s z&u|=2(5jP7xS;ly$TnDxWO-|kw8f+R%5vrNRP&rwKmdV_LP?YDgw9nj&KvMQ(VkCv zRww(wG4~P@E|!3E)UIn7WIu_1w>gIReA6D2U5(u=3r~Os!w18apA96A%$(eLqBq}U zU2As_5V5DM!95Y5RuGJ$o;JJ!=F46I>LzrtzX^sOstzuYh>i@7g|o@XNQ-%D`Z$)R$J+mI)prXi5Cq z+jK@o>w(wM;l{}OI{ZXdv-u`pXWsd_i<)oN^qGPB;_4N^^$Iv)r@oMS1w5;1v~>ob z9;U8n$HL-bo*05ZnqI30Nd^~dr@aEy^x7W!%1B`~E)`$md0zqKu-J?Hc4#uximQaH zL^0(R;HbHxhWVZ(+H;&q)Rfittusl?#vReQF8O{g;YC@Zh{Bc{ybs=m82VjW;-MKB zCs?Pe=;($23mV2U#SmKeu0~%Gk*wd{J>6sMsfmFhH#gG*l48oW5yWk0IQ7&J;_SBI z(9qB|7rGPm0J#NuK1;TQ4Km>F#Fj)?^qLr$uy1nK!KyJmP(I{(Tv?}Gh){DLx?Xa7Ec<=fM8sshEihJrl=B9f4YH%#uLS#yre^ ze?wkBGn+8K_U&;zHevg1*Oopo zq_JI~HZOmIu!jT@DD1_0vdxSLE;e78SpC?(%5=W_I>l#%I3vuwC3Fgf7MrwK<#$ zO#=$Vl_f!Ub-%_4UR%GAE_oOWa1;Oj{89DoPqw-oJhxW>^Ow)DB<)hC0`}=PKAAKO~O%5C#R$P4^_eWTHg=*nlnmqc9S95~BrHMB8G*euB zsbDf~a^)0B2-dKYe|i8Vqrd!G_qjW1*=gEgT(>kFh8Ri14Z zX!V7|!Ubyd=6etOx3~1V>(RJVlxDacZJPw{QK9TxDa@y6HQgC*3@uAjg*~nLM9vj* zJJ&S1>qN_+dpRJA`ZJsY2XEZbvxDfB$z3UUDR=oU=O5(Y`wxw!Sl)Jd@Yq=0r+xYD zBf)EPQYZdUMnsX><%MXg2zR9*H^)9vJRhYb;i~&q?as)7~=TFLspgl z_8#_xb<%kA;<=x2mh{S2fq-~@#GJ+@wbw=WDIeb0u`o~HyjhRwhW=(&uVV8CU>LQ9 zbfb*G``|V&E^)*X;c;B?3J^*xB{tCL4Vrkm{~m*5qb8NN374a?%cnp7qP<+?N>^7y zW>9eW+~+Xv9s3I4EaMuU0pW}gN1-ft!s!~)A$m1S|0PaWhO($N7%JL5J?|B;ZfyZo zZip6*Lpm@V@?yr6n?QN+fnkKZ#wneTb&isr4?Uz&GtibQmjrvlwU~FUTyQ z8@_5Co2QUm5~##yHBacm`3e{^Zwx0PINh+4jD$Zqt|Q;8%C6@y#<;whsloi(VOao4R_sHnFOIAa8%CUMH(Or z^msS1Z~xM~jPtiMn_kak4%S%cJ;fwzn+Bj9YIdc^e=7{l%sK!&vMb=1z(dC&RyNC9 z!FT~BHkuuZqjt<@v+Ce4tqJ^SL%1|m_tMA7O6$72+9WIsk%^rcVbB|qz!JZc!p7}1 zS^3}%V#Zb5C*|S9=KE~tmhFbBj?%hLfhJ^^T?u`_DUXD5~2B zI(YH3`-y<}$=;MY0!G}+6kS$NZ+R0~(_gyna8YF`Gb8#&NyxM*S&%X zl0F?RuC3&`z_?ZTu9tqu*Yw4$Y#1YggPf>%z}%!u>G{{n6Q1j7>6>f(mUQ%9H+DSC zx3Q3M40EhuGW5SZ5)EVZ%0Z~Z_ON&OeJLuDd;6_Xz7v5|mS#%Rf=_Oh9ZBI1#G}9a zydYC9d*88L`Rr8YC(l$fk9A0$tKlfjZH^#X395p9>33vp_a*=Ain*?jw-_ej6(i6z z=5Rb0WwJMG=eCGaK z;aaec#sn*=D*MO%DPIR%HnN?`g#1#5i;4CjN6`|_Bh9V?+H)*?d6vSPStb?dDDu=p zztJN40RrziRQ{|hUyc8iy3dYWc~DWa{wFYhaMvMi_f3o5S%u$@F?tZKwA2I34R3;( z*t&50`9qeH?=EHiUwGpgby6O4K^xJ&8E!$i6<2bQ?@rpxBSczFz;rvo=R|s~Mg!xR z%9@Ow*iO6!o2Z2)BVl||S3lqO?~L{-7JbUR3xDX}JA`D5VUm${`3;tfzh60yGJMb$ zR0Ab8z78l6sn2Yq$S31(G2bAXE!!+!r1z((7OCV@%;Xcf7q}dCi{|;1^p{t>GZ9$dfco5D-4!*R9$#4%*L_y!{Yk5I#}4U|)Wjp&=Y`EGRj zuM3Z+E;rv)J=UiSk>etkAhapxMT_W^NqHo-hO5;3Sw+=1uAmmuD@;V`B&zR$h2-=kk+- z${7DEdwabhepM7*1`F2lnL27ndkIQgc(ZGP?(_GYfsaE1&53>H*tZ)`Bd06|f6wQ8 zVi%w5$x{2D>7J)0YVr;nsanh_`$17Q_5FS92_dOalu{HL-{c>12p@#?T_RfTE&$ZT zCc-AP6(564`)FL{gr7&H*@E7-s9$z5h>uW5)(zW6a);fg-Tv98^eMlMxhGHKIb=~V zgU6u?0?_gNvY9+dh7_1%tuIrAo!~FX!2MJoyp#BF{MRq_?+3GP8;a(2+H(A6RCHGH z_TM<>BEN>LpG)yf!!&fOhAB>sN{zpyruK!Z^mKn)nlboJ(aV>r-@#sfx45JHBfMgE z;<>PWJh73)Vl#u(RwKjv6**BX34F$ZdA!M0tp6^bto zIGFfNVy_Vq%vuzCmYcZ~ENtGqT(;w4lO{a_A$}I+fKtvK&n!UmERc#@TK5NnMb4XI z@sjv}rfSZjoU4`v^YBKVonE3)CmaU%XCAYsAh63*Z;jrjPH z)%tn%_#2KT{vM_nmiFciDfQF_<@#MX@fwU7Ll(Pw&MrH-X$UEynpXV$2ea)KelD+n z-7z9uUVns2lWSetJEJLCcxB@8t|lepQ6j^^iilntfTGfqPylKN?!ZPcL%$p9RsNA{ zE?>pVm^;B)ZY8lkAoQNS%R?&0FFL5gk zz=r98%PLh%DXvKB=r1n)L;nCejF7I@4z$N{EcV?qPOTTZI*6&nq$;+&#*M*hWO>>) zOe(g3HP=~U2;=ULc)6!prSsQzWLE(JRW-Z?ot+_SmSFDP|Y97}|o4#a-ocnS!20G8aL;-k13 zh&C#tpBRiWgeibL;HDJyD4NVPSHmjY!0saQ%ZOb= zJPA%P&RBRqltrjZEUzlymg1>7Y{z`dUB3`dq`xZJWVBHpTS1op08tdh+|8#(e#LHi zbJn3S{^8%M3%|T=<|^|7$ArF8Fl!$8203BZY`)?D0L(O?QZu0nXou zv?wvc#xbJe>f*SSMNxu%&b931s3R5Cb62=Vd1@6m#_k$$Vt^k10GHt$4)Us}_Y1tz zt6h6kHzCD9n&`(9GgCzY=APowGRiGQ+QQ>V0YYfxT*KNB|QC&H(r+)))(iLx^rs!k-rVm}2Wwxuhi65~N` zTv!X#zl&%(l?e-)mtTl2hvJKCUS^^h+y-kDE8niABG+ZF1(@|K;sC)q+^isBO;jD* zmD(SRV_vV;W>c@#svpb}y4GL>Kr#~@`ib58I*!0OE)GvesN}OsLzc5GW@QF@J1drX z1S}r;Czx1P(P=+|B8JA>Egf!QSL}D{Y5-?}d!9}ULkGS3h++M}16LWb=*FKh<)q

O|mh2mkH`Lcr2Em-*&Bq%QH^%sh#?Il( zR=O$Z&Sic~qLY&CW*UtKjSKEC05huWOgFVG*gJn{n417&{6UkgF!_3l*cUl}uBB1l z&^CVJErStDZm}{lENtTyfpWO(Ow4dnkIa4S%sn527i2gu$UY@&o_&O}l@&)n+8e^Z z2U8h^lBUWk`;Ry!^L{QIa6{d4_WuBJ>s3cIb1Kdaf<(Tc<-d3#N~q^aVxGbJS2K`q z<1a_1VZd8o&h_pSOvXg6ZD0}oAB}Kk7_LJ7scs~*XW3|3QkYRH`GMw8nF@$_0<6E1 zBCQ-8LFDcURcw3|k@pOxl2F$2eqaU-Z%cy8CJn#tdArVdAH zswYWS&u}6NPedi(+{T^bZ3%@nxu_G$bp=jGA7HMY2x2j8ItLcHzr;)C0q@Yxe^6Oo zOJL>66^%4_4F{+&?m(+5a39?V1;Vcf`%LNi%vUBN+Q2bC7>I%N9JmL{l{5aeK;c@$xnk+@1yVR^c zF`r>^9TI?7Q(8Zm)|qb|7g&@UzLw`m<45y|#a?QzdZ>&ULvUA{9ZI3jIGIQcc!4#p zc5JM~jC8!0!wvGl$2Sd3(w{MtN82z9e{>`jtz1xLiGC7K44fy1Ui^>Nu!wIL2D1?O zxs>6#bZ&g>?JTYi2m1`EyWa}ExQBRmq&`*_+B5lyVX)sImFS+J!{?!6*PHCtW3JtA$&vOWoa6L|?y*j#CxC`miGlm*P-HZx1 z`Dz7T4U3NzE}g-$H)9ix8o^Jzj||KqH0U2dTg>jm8V|fw3y`}vy+XWefG4MOrk8vh z{{T?#1a4czYr{^zyw0*KFsXay0?2b^*Wx*7G#NR7;lqD;UvC!IULN2yqXpr-wSrvL zQL2}=G-je?)_(9wam<(EKQ+KYpTD%o$e}X|>pNwekw)}xAK8V*{{WhTaB>BbmiJ2I z?Qjm;@xiG;&I(OlOXCb=yxlc$ezh`qQAv;!4#7X2n)$?LwcdX|lE?QVch4{fbX}&W zrLPT_5SdtRgZU~_Zv2g3c#cO^H2(lK9H0(M&*=oqr}~yVue76NO>{ox`NbZHpO)$^ zWJS!di(#01aAwtRm@gfKuwuc_nOiMpL?-oZ^BcKNCcZ}IVOb>!lym+OO?!>J4>GHi z4+FS!hqA}-4YNYxs-OHkN&v{`yOqa<62RCgD5;Pqn*RWB+i%G1cr;7wo)-;kEQ{tL zbS)>^lG|Eq(cPHJM)ql*Vz!x%KG3T+X=%9XXRQuuCUY#gzqHOCJwaJ=%82M?KJmAi z#u%5kG#$s2E?yst1ze>#EFJ`|CF1;%Y*yH8f{S(05K!AurJZ#=^CBvmtSYquD79-6 z`HoQOv)OOdy}Ch*1DHWaDihVZ^A%8RsBu7T%@QTJlzEFhc=HExjQ(PO32AO$0ZG#bS@r@Ka+bF(+(~~VfZS|j zTjSXt2a1+!;)Eu4QTwx`ezhrztQVbEk8>AYgkrpTL{vGvw*?kH#8&RiC~^5&qUrZl z%i?69Xj`u4Ax&JM`Im>5S%<|kp8 zsHW=MV2uWC0_<|*nTJw?0e1@JI%Ba+$`}saZfYBd;4=Yb%WfUVD!$e7z%Z-pfFU$K z>6tkDW>A6trV66w%GlJi{gDOR;Ffe$78a*uAO5a8$WZ73wDMdQHB zF7EcHx)~+30Xs^(J~fpokNWRn*eKSWi=B=@?mi1 zZdoVFk`lpfFQ-*Dz{ea+5Z7sk@eYn^6i(W4%b#&n%`5%MJ=;5%zaFM{@&?zxsDc8x zDfl5dg+(mst|bC0uSeQ728Gm8)t;5r{{S&*6E704Y+#JGstKG+id++zS+h}?ho+%< zxhwB1A-jI#Xl@2($*u{%ZWh2VgYK5H%iN*I{{SoCwAbO7l`czyZMkqKXrn>%FoyQI zs-Vy2Xtb^3Sot8_UA!vsy^kkzAcKzO&O{lqSTeW4_ly4k5nshYsiRk^Z^=-X5`$hD z%ttvqu2ftRZZiyn63o{^F;vI{#kcv2b1o@Vpfawi9qpnx;Pd*Q0uVN4YowP|aW7rU z`}vo8SZt#GP4E>6uV;3|T#qWodMAE&1D_lcgm~BM-R9tN72LU%#nj>tdX*jgKS8=O zpMelts60LVg`h{_egjNO+wiz*<-#2bnP*%F2Kj-xXrXMO{2)2Ya{7yrg5W2lU3)NX z449a^Y4_wSGoniZq8Y3L*>Vj-lX}|o!Mrggk(%F}6 zsSeM)vpDcC;YSpNcTP0vAMq}2opP%&wAGoa(|>3nx~gO0sMgDgT&-Lw`;D9fK`A`* z4Ft{MTa;m*?qWG@ZVoD1U(mRCM`@fF0|e?>)Fo8?Awsbew9>rFlO`TBFjKlM+;$pu zdHGbd_B~vxs0MRWV_^QK2<&uAj3uLWQ|evlm5q{tc1p_cv`*lAXm=i7I{3C?m^2-C z_LRk<+Eto(+V|Tq;%;7OrBRFbTY((>=PRBGX#(cT&8qxQBdB)x4I?Qjo97^rUqb?=#^#@;z9?w$1XDaPhP{CLzj7}#o@@Iw1sp%?fCS2XYaBd_D*KNXL z$P{x=82A_^N@U7GgI{lx6P0Ezg;7P8FrOCzc{Ko_1ptSMsK}-Wg9rp@J7b0pxr8|` z1*c(0-f!T3vEHf|WJ;p}n5eB1!n=z2dYV)`PZ`!Fpc)jMoJ2sZ#<5JN6E$2$sQH!d z{Y57={h)baW~N}VxQ8pUs`D&`OiZ@(#Ln!@rURxVouD6x0w69|h`_1cRnB0NqN4d026g|jnFt#e&Kdi-r%oI?yh62^&3 zj7o-VB)6CY2Ox;Tuon&efXr$!KJ;4B)@jA6i$4o zf|m6f-B1@DOASC-W2)r}1>vI2bxDb>Gj4Lu+{3rhHAEpzLA7wTz+Vxi)w6vSD>%x? zHZUHa;ge2b2P585xGQSh5HP$ z_Y1`RqXS(RE+VjNxavbi4@4op9Y$2VY9R*1{n0wW9LAAyV!MwA6M%>5+z}kLZrO{w zgDzjvd^PkNnq}MY571in&cL`-CK19)F1P!b7(U47Krldq_IEBkE;y`aD_Ga=4<<1j z1k667GQ_K93yi=y%Pfr>xKR{mz|6vlaYv7*$Yy3`M{z;&Pe@$EVqzKMw*<02cTrq0 z?%J5?!OFq%CE`A!c#1Ycsl2c5o!=LnN`gy+HtH6`E5YH23xe)3vx_>z3b~STDMM}} z7`s+)^(!9+J#C4ZV%cUTF-f=&b&xF?~b_1|wDQDyf6d!v_!1NM-zs65lL1o_-zPQ4XtAoBUzqlTN{E$ ztQ@og*OTK%nQzU@mj)Yx{tAR!Wu90$sZ<;BJW(5^#a=(5RL$`!1W_qIZc#Ha^N*bc ztFYBF%HuKf^pxx)o7_r?Rr4=C7Zt1L;_>hzTL^LTM5X}4Xr|0c8e>a@GY4xEq+@7u z*Tlbqr5w+Hg4ejfv_5Q?kAvtL_!&$;9ow&mI6^UEj|EIz1yy$iL9^?{9~8Cju@&Z8 zu*?^@dH6ANaMGp53z*eJMyfAa13q9*H{~x?EC5>0>VJo#q_Fk3re^9uo&X2?VK}G5S zptM`z@YS{aL=a#z{{RyKL~I8w6ght%7ykf4C1Q{8YVhOv^%<38O*i}%f7kJ9gN1lI zQRAxqFKYnYyPIzme;z&-dQ}@$@8va{$MC=U_PDu@S@~Qfh0%Jy=Kla;R$jlDE%<*6 zz<(;*&Hn(!DxUcGstsZP0EYhn4nKp3WKv~UkAy~(_?hV6_XWHDyTyOR792m>igEoY zsQyEpf7T*D%}0R$0E_wliaY+pd-VRF`s{*b0YMdzL|@`VNAe_zs7<`rQJ z$`}6tOL*7CH8)ky^R)Cph#d57{%e4v9~^v@{bhe!U(romroZVy{`)BZ0HCIq{)KQ9 ze+yOCKcxWEP1E)LSpG#{@<0$9+1K=g{{S5ME0(|EDz?mjYg|9dur_~kN|efFCR%uW z9#Z{JDQRhGX-ud78~uBC@lzxJ000-_rMiOhgL8?}_-bI~2zQWXL@CTxZpZ9yjPZQq zl2yu9KpGOKOQ-SfY-fM|1b!hYP+;VrU&X!|{AIjG4E1V+{cf0UjY0J?FIc{+Mu}(M|GID?$2&W`qwPv;(K^1{mJdOO#8Ld0cz(>_j+clxqHv{;+ZKt69_Sf(%q`{q8caxyDWmv2R$9&RK|j(#?IHQIu7=YmX#W6gY$x=^ z(7zGyBi}d69aj9I%1o*jIe$e&Y5{R*bQGsFiJEcELI9@4*%;tmXdcuZMf;d& zA4GR3_eC&Y0p@{6aG2R&qE*-;DL=$=l1w>fboTTX#MLNC$K;|{!6jgKS|#CnyHEZ? zDOdL>-UIGXm_-6w{Ds^8qWK7>6c8iM=Z5?4@ zM+ALDYLXx6EUs-063GhSP=*;RmVlIAbz^bz!28U20(%&gewm3sbQMCDm3ps zOx6fxv-_3WrF>%eL;DSOtobjBs8+PZULy->>;=Jjkt~s4;ub)DW0DFYfx`!{*%c*0 zAO2t}Z_1gxBRmTO+VI34EcC>yYRum6=KP{>tNwZT5uloPDArAY#ac9PF&<^u2~!pi z+W}4rGlwZwhp<&l*yZ)8y#k&=FbIzsj4@wOE``vHKXD>F+`9Y`FZn9pqLRh4UI1OF zOMNEYble$jwXCm}vVgvru=T3e-vU4mrgberE>;6egwV@#J4GL*FY^pg?$Nt<2B;pa zt`>SKF^CI*ja(ZR>-90l_8dm2@75Rr=u!S-mH2#R)wVbcznIt(E`qKdzwuFc*nQNt zOZkJU-jtz=T$;!WQul`}cxuZ{ZDL(TLK4>A4_7X^_FzsI>QmTzGx7dq7)RQPLqHl| zvt|!#7%_g@ii*|8AJxk=t2nYyO2Rd;%9l6r{{RL48Hi#x7$+{XkH_;K=%_5SHALBr zph5PW!6|5{1&$Ud3BhXMy|7FttErR{IIx`6_JR}f&x1qv!OIn*{ScQnjtYScDYltZ zg70cRDUwVmu?}ALi+;g2Ni5iV15;tDxkSCB(BN@mqE*KFzuC7Q&6T# zpbNMP1ISwhYoN<)M?pk(Z_z8-o$K-JCfrUaNK0>A6fOy@E8PW~k7yZH3h)5{%77>* zoIF913&})Qig2vm+XOzY0gf0@1^UBVi;BcH$~g&wDPgogYKBtVykh(Z)XoqZ>|&eA z5W7E9fX*}y;assUGKL`Im?n3>=+~(GB5({f)eUzSEV7hO5~`e`zsUuBd71+H!l8YO z+dD*B(-wO)$thvM-Nz8i(GEjgt5A@t^0f#;OMM%x9bFATtK z1+rLsnYFY~USo#$xS7iOGR!i*%Gw7+NCBOq*cxqS@wB`OHjqjF=uIHDqbSs97nmf3 z=zY|xn@~8opkLh72ZU5^O1;fUcVup`KBBa;v^g0|p=3ev#jtN-o)Q4;mT)sv0t>R9 zqAa0`z^LQ9fbcL5bsfcjiKMeLxh85Cv1>AqGNHY!i{xtvoWSg-N+0fA-D<|-DE9dU z{{TP2rmi$x&|U46sKi=ZGX$Ir(>MhnO+#?H8UeEqQFOR+a}ZVE6M?V=5vySWuoQhk zT~0z`ghPx=tTXlg<%ZFq(E%Awxquov%q+=vDasv4{mWX8#}M5Xh^meT3f-2f6j!CB zdu{QUVG2axr|`u2mxpokmMvUDCJ>;dxft=HQ%0SYP;jZDEg_ZG{ey)-pbMEf0a~|i zQJJlqMk5lB!r@zT9aIZigQ7Lf`nDkm`x)$vX31l`tenhZ#vm&sjm1+flwR3NFurC? zhJK|58+ukEnezH%`eH=U(XsqX$t@s#>~jSL3+19cq48ZpKqChfB zeQM)qBA9lcQFJ9$P_C{9LcFMCl@dKJ0jMQFxeNT+mI2ccrK$=bKm*WapbA}uT%i*b zUTnIpR1Bnc>R{4n62JIGO0~D*&k}+DiqW0i$_NT4^td{g*)xLq1ugB|%W&Uz$8~kH zh~p91YTzw_)iFVb?gC&_QEj37BZZN*lybgHc;qPzo4VZXdu$52Wwy|F9ISOJ7UIw& zZABf$kp+&OdaGkQU!Z`X<=Za~2XzBmDOQRYvpfNjM*gGtknL+WOZp+L03p5oOcJ`q zCgD45+wVq+pctL<%7p;EOsfSDA@!`b$_B-jDY)6J!GxKMuP$5_rfFD8jL`F`!*a`l zQIsas%Ij4vi3Re0;|r1+1W!K;_!Aq%WfG>%xHb|BhRlFV_EK!(TG3{UXY@wkJTzbW zgoc#*$I>(22Y&V=YlhM?5;tNBz@cD$JtF6BMXbt|JXmN+MGc4l0MLpl)T%1_OEDZK zjz&^$a@~KJn5jO|mMO+m40YDo+S%h!=F^EKIdI+Y>$_gz?c3#MB$gniP8(jA(bQk)IX6_?X@i-V7VAFI8957Y-)>&+- za~n@C7{Oet7yTYj&l02~ZzUgyRKIL&A3+3b&_6^P1xD+i%&q}CjEEK+IsX7qEbfa> z#JDFxG5RvXb~k#yf&+pq!3U+ZR{Xaao$g0!=4+yENrLLwg|L7o$zQ5qmvNF&Mk0>Z zl@mldd9VpjdhZ%&_XC4P3W-f*8Ch9kMUqOQ=0rbOK>p)vzbJ{ESmq#6RIAA}fJ%_G z#^bQYDuoL3?FHi@Je%2(qYCulJapmj3kv zXnU@ir<7rRw9@OCL1T7`TpPYHYPknP6zV7^84O#q>4PmW5In%R<>FtehTU0tXo$i| zgvN0BZT#wx_%fPV`$D54M$EVIhC3$ zmXHt*4QXzoxdg~LJY_M)S8X?%LDypJ06KDvvZP_mDt$SFY|Of=%PDex!8FKourw~E z(5&G^xsFx2lxnhLQjQjH0_A6VNmun6rClX3t=Y2-Zso;yI*(XcpaP$=Lkc`lwYLh& z=>`Qh`(+?77KXRvx#EKW@`+E(K;mZ7MjMX8)O$2xD8}ee{J^a=a~-brETwe#5GDfv z6%iv*D(6Zxz*5}}m4%ucg3Fn2{VG+cs0hqcq9o|93OWFhz%B^qs}QeGI#g1&j= zLUTM_IB|YMb^aDVaU7V0gH!;b3d|9^*0}qb0_$oMf-yk8$s1xkEq*4b^k^rPBk2+& zSlAkD?BXa;ayBnFQkI>T2af`)2NjYltuXN|;0!_*H3yoE$5dfk7q25*Q%1%CcWpT! z^Vtewj~6KntJI=j%+L6O#pGDGg?%a;seb7dt^g|3tE?4MISyr_>o`|X;{8_(p?rBY zYrzAxToS|8NX*t=!$T7lre;V0Iv&7k%3+%Gcaecs{{Sf&a7(;Z!3WiRqXAMk?6@fG zbdawJ7cC2FEVIIJ!dKX15NOC5D{RYV0d8m`uHT^I38nzjTBWzR z>AnY;wPI~n)%9g5JTSR#xifo|ucw3ktgGRRe)ZH4Mnxdy27nG%#Jk`SsZ>;>A=s{X zid(Q|vA$v{sYQyabgMIb%aPe)r-7h?woL1AxtpnEI^ zoJBy2w(bTHr0-=B3kO!3A~dm~e8#&-4S88vUKkWd$ioE|1?pi)$yV4oF5wrTB0-85 zi1-DCHyC6dMX2hs)#>|TwF#9+cUi_nalJrAi<&u)`pC8RO_oFKiPT7_7IU1jZWO1| zIoJ{fQ*YHfl-Xny$xFn?G9)q_tADv!L0$FgDdRgfA_ln4<>#S;;B(E$k@`%8>w##fFS6 z8&|PadB<#ABku&1s<|E^mAIy#t_74CSZYm0$WeM{F%RNR$1MSamlvI7 zv~&&#yjT^;9r8SsUC`A)XZ>a?oC|El<*-{(5TFN|?tzxpunJ_Ov@q_f8A@ou>6TI8 zFjFiiS&6E)gfK!nP*JE#0tV~@gj>;U zuh62UU3c3kj077XEe~xMeyt;bAi8iYVL?IF;>hJH4HuTKr4GdcBhWdHXaNthATY$UTc+EhD&GMRureF_We`&{ zghMO4d-5t8mP7FhOA!_=6!JJ~D_yryYfeU#Of(m1W9$JoIWtQCH-&$DSem;CR)V`c+Iht zq1&g&326kvr4DIgO8_$qnhj@}WfrhbNG!&-aMt0nuZH&F25G2D0tWDmLfYLGLDv|K zR~k&0xH)@WCMja%IHn6lyKQexk!FQ1v1|S%C|i-3mLu!M>l8x08=0UISXHpVMr!WwKW%Z4X<%G1gz3!jGLJGWN=K5rE$A-SIyEY zgs?;k=9gzHxX}$-D0gGS5OB4{T()`=_&+eht&z}Dg?&UILFyD(F348QW1?9_E?B&? zyZc_;sCEXzqW=Kc(`D|4bVU`?*anc6GU7HBo4QXDSxV)I(%iEF?Gy|yiD`<9FxSGB z-;=;Z@X)<2vt7dI18Efa;I{=vj?^V>M6Dw$QGPC!Wwbf@9kHk4Ck~ofl9=obgfwB8mxP>OCjmO5pIHQ*>1>xIJ$!< z4CA<17Wy=gR40PrbBC0^ufejl?xlmu9RC31Tw^tsTDmS?rH7_p3swW0DIC2yQYx$^ z3oto~AkEbYwks8;S5d#k%pc*6FDpfT62=~GRJuoM>Ggv}nh{IR%S)zqMb$>`KPVX* zz+v|)QNRbRj09@i!H60POeZnC3JtfOQXK~x2=ZOGQ3fG!OC@5-ux;>~hkkTs~JmM~(Bh;em$($!kv%YQ{B)4J4 zCIuRB3If;Bx0t>OhD@bm!0d^Q(u?8lP^eONTEL6|-OI}k0+oq^-ttaWmu5SfHLf3u}xGR4m z6lPJe+C`7sP&Zjk24>sOeUJ>=vgm-QP+3T+b(rwrR9>sX?p=X00@a-t1j7}fHaS2z zR6)TpZUG+Ll>!jKBHDF5bS%oq%LNcUL3%^r)X)u%DA-CFqpKpFso275!w#11LJ?xF z=xhl|*9nY^)jEtvOm+#02(tx+W!zaj$7Zw#JPyq9v3 z&X%XRg>M6j0tKSfM1<)bhSMV4!EkT6pWQO*5Kpni zMF8CZX<{{`4jq$_*AcDg{nc8<2HjqdoKjJWe2TF&jI6t$))-5_%Z+yl7V3<0{%Joe zr{XX}=ryw9O*XIb3^l8yC@QDj2=^xB=};O! zGWR@j^ge&2kx;iTzAM~(B|$s-Pij%4f;@Ox!F%aNfbnthbb_U@ots7&C_=GdtKeZK zHttznu$G}5Jj0eS8R+_&%IR7B5{;7F`xYofACq)NJ*=v3MC{K>AxlKo7Pa|!VPfqQ z2X=hjBlxgX4AfRYYRQfk$W~2uAVQ|J6J|?yvSb5{*;MuKz_CqmmPMw<6w-Fhz*t0I zdIZ4WadF@C%2iB#fjA;U3f0v_IZ$pC6acEJiTuL>aSX3yKEiLy&Qg#89W~Ry6k8IHrp1@Aa3_)2lH%)lszgtCgtfmLOu@v_am%#arVsfu~tzo|NG&0Z0o*TquG1fVsDK7Wzqr)b^sP9JR|F z9X89fFEtSb#)S<;DNtuEltBop3-8DWEHSf@w63XL{veZ017T?iMhVoHl|kwE8#B6C zsc63y!MAT8I4L8MVyuS0>4kFf*C0Ydm-5)MMM-VDGV@L-!` z4B`o3l;GLhZEIr;-a0L7ad3kQ#*wgu+7wmmRS%0S*#$2f`iwJ~sI<`rrBGKO9Dt1$ zl)@Q?IC67LdiXr9QYxm*aE=mGBoi#CE}#-Ji9ik3S!QHNf)?@97;qX^DXW-3XVI61 zUX~`UuL-aI$3PBGpfSq@0}$FIR{LlRLCYC4%mvU$O_zu+<;k&3sL8wXP-iss*WI3g zupz{D&f?qbECGHo7B~r=w+HQfjyb|ykkEz6IgKJ$2?!NQ4X+wzunU%MjmJZk++!cD z3;+;`*bH1DOSHQh*!2(g5BlsF0vim^@`a}ez=tyXGa zUbP5efOcR~(OzCsp};wWaGB}`Z6JYhN9Y^jjaxKJn~&L1O4%KZYdX(ZB^gatLMnw@ z>JW22!nD2NC{Nf-N>5b7=-)YIR1@eTxAAHsn`=`rd9k9%owej}U&O&(KqP)_Tn02t zE+4v*+|*wE@DUb59oW)k(M%rJn2Fg0Q#!B=TcKK)jq(7W-p*^Ms|D%-eQO72}Ei$8LW z%skZj(GLKdmo_|%vzyjqMhTECG(EsA8)?hY85tE-wUsbg$S@?=P2IY=Yyrt81hQiQ zgjHFY=m|#*xP3V#gn}yyb9~$;lmZrw7QkRRU+N){9)O0U)#Av^H&)Fy{Y$R%Dm9=f z11zgn6hnk$^TA3as7YH$BEq&{1$_i8JFCE>h-A@ZR%pA=%w4Ly^CAQ}Z%{R~Af>di zn1ls*34!EMU92dfOJ` zF7epOP=-Z7*u5di1$GN(r@h3UIjB9bcwt6E1(wJ#n!9q(g-kf&8=B80vfyFrTu?1g zT&R5$>aOaSY)B;OH5V!>^xOa=r8 zZ3v*=K&Z6bEfgAb^BRJWpuo>+xtSwEP(~ofvKOU7oU*kJ8GViPc$+4^*>Gw$ShavG z+Z&iF?{|HZzI65%;Bvv6TA*EY4{q6tz^n}yc1{tyb-2_viv2=d90)qOBvNFh3c2hU zmnI0%Zd_QZBj|~W@iRFEb5XTinlB}{Hpra|PD1v?!Z2CDoj{cptc74LOM(=UiwajT z@PO?BN)+r_Z$JGbM9$MR7ZPso?lx4eAW*0Xu+E9U!$(sL1r!WxU?iaS6yoJi40U7{ zU4A)%D(DurzNSM3RCRpD7Q(X!WlBte-pszvvD9N+w+EwOB?rg2D++GJg@ILhIV&YN zxr&9$Qll|9m1-CdMwGxnn}IP%c(oRS8xk!svCK*WmVuYE?c8mz8?k@^$n2T}L!6Do zpDp`W8*mlEKv*qa(gN24D~;A=^{R_9-A~M4fKCQtUNQ{_Hv&#>v)pT=vSy{DE?|m? z;3!cWl1W&ybD@cl4hB{Dxqo0HD{pb8nuM)`$1uaYSp>AQ0Q1QyPfJl+E*B0GR9`Je zx)~lTRRJY7=!#{jTaE~mgJNnlt-;7*FWI5AfAN%E8DUkoTwfW%DRG&$SK?DzN+a=R z$rjb$4to2AE`kTa)d^bXx=Ad+26+ppS#+o+XhrtKU19g=@%2bg($S>sV%1FM-ryxEh@py$APpthkeL)!?=gNKC~2LLWX4`@ z456HxIW&>-;YEzei*3liUJpq^p;-yJc*-ei!wS2K8d)O)KQKsV##XrdeTS!c+@LrHbz}#AiXqo7~E=47uDvbaSvE2Q5Vk z_B_B~{{S#{sP^$WIMukmLRczjH(brG^_b@tE?w|F4lV&!^7(EV6)#wd+xf$mIFiQ3GNZO{ z-cv^iRr*$7#sMsc0+rZSAVZY3=-wjM^G@H1Vl9gqx{AMxWu^OoR9zS%3z;Ju5;vZiTDnv(E>(to9{5~7hzYafS?^*=bD!;1l=nwaDB_c33pFu%YlJl0bY<& zfn2b1_0;3QM7%XonyMW+{lj1uR7c?br2ryu{9-xs&dyjN6C@D0;%eN57ib+FK|;CM z!AwhaW?ltUVQRtD(1X8f1**iY3i#G6*C5OIxy(_~)Uje7tD2d?s|KStsk0o@OjL`w zvL#oZ<6ybP%ezi`Wp60I{{Rf4svAQhLuG{7P%Iw7Qo1fF5pH-3(J^TSQ~u!Eu|)Ob z)6B-;oJNc@DtIX1k2e^GU=m+K?&>0gf}Bc{)=;qOTm_aDM6@`Kaew@dX%@VCfg{$F zhv0f(Dh2_}4X|J=Wxj3z1gEgTC$*V?UcqU3GBSnN1}q9=w;>m8K1S$@N_~utf|pca zkr1~r)?(4ST{TVWTH|LYP#QMOpxD!x0b2$M&AQem1-?r7G&6dHO)Gp7*fvnNH>^R@ zmW}L+apl4fN();NLQ?lJ_CeKJ{X{YxXBA8YjB7^zf@0WJ*3b7a;XbmA3>8=g2~-G& zXVP^HN($RKi@eMBBbFj*iAVXh)NYBuJVC0H1j8^=DH?uhLLeH@DW&%M!8ij*F52LX zG}=FLQ9)FBz(QUMDwGVsY^CcOAgu8U=AzY9hk|KTvH&u%?RgfovK#~OL5v$@$F&&8 zYV@mv`fq*1m9wc$#KpJjgO0X|L?m+7QmNA#B9%(F)y9)!%%fah!Zr<@0KFVV1j0CA zmAO;mhb0ZiEU9Yp&ORGQ+%h}k82b_ON<^*g;I&apiZ-_VXEwMcw>gK5#N%LB6bv`p zEBlIDXoF~9(PTeR>~@-`*E2nW;VmElba(7AL27oR0}D)|c7slDRK!WMH$f@(a9WP{ z1}Sb@H8bX~uI3jddwT>Gfq`~J)(ET-G2 z8j9sjL5>I{gKHK3e&Nbig#&+cZ5E(Ko@?E}Kq=FbHtc;%l+uZU7Z5lrhE;`u@E9)Z znvArmR4T(!cT+eHU>2xC!_vneQtm5|#Ky-=0sqoHF)?)Kgm+fbx33#Amo=IB+5N}-lORy=oC6TFk z1V=TdcDNg5msKk#3{FwsAP>;#l}skk;5+8-6p%w$JUgh~y<=78YcTQ2k@M-77Lo&m z4U*TGGnS&REAep}5iW?kRLaIv5Cd6Shrt`JI)E#I*NjwYr$6~W#w;>w{$g!Mq)m!1 z?mY`GH*f%;lB-2f!*9&TS5Mq=F;BAgtOsX@vggu=O>j+m#C?}WutTG`+KWa88EBm% zi$7$q&HLvP&JwH^S1S}5^x~J8uVMwaT+1xhxgMCr3AFLd5FjXeMg@e)C!$EOXTsTX z<;2|JJ&K&)#layEEdKxl1O5z!!Y;F^zjR;p|CVgu#L^ zn_~fGfz>rIWptVRaMjpB=cV=hYU)JobAU@iD4EdD}6jfZMju1<5ak;6_i7+o*`M-L}&ENi} zgJ2F_MIf-5W6q(Q0fVwtbF)yfTT*Q&QVv>Pn zOCK+b)By-F+GDcTL@x5hp%gh`u*gmV@e2x?sO_{=vamat`E(ypDB7&tLiTIO75H4u z=onF^F4UlOKrG>x@DL5KMTk9^iqUfZ6>RewI2{~Aa?6?;31$1En$^?b8@WuFz zSHwiT8)d?6vTb^quep=)$FnGcSmuz31|S{_$pa!%%nVw~C>|~}g(x5(iw@C#vejZ_ zD&Uo6&o)H6Hf!tu0NF!?s5FXPtPub?-@>bz zm8pvN*x251kw|<46gD;rem*VkSwJC++g}?)KwO1pFThl)06StVIEPPB3oXikHAUy& zhe?EQJ!TMLVjL*y@8BpdV7!~cJnX%}fKztfOMcms{F{al?@9UbD-j2lzgc#;@P5u863-Crr5oZi0pnY z2y9`IL=?lr?Z&TRBPRQs1K^w+p#CZ+Jc~!Ex zgm4gMP-;ATf@KoPWTSb`DDEPU5jlj@k(`<079844jMYXtVp0TI3fsxJ@o3qmHNjGz zF}>oTz@5}pm@T+TX_wuLNV}B_YA3}NUzQMCh|~Nm$~0V|j~tSgc_3NKXxVdzXAv8- z-9zqxdXb=uw&m4%I*5{&3n&H`ErLb(7-}s*&n(PN3u}WE-YzW`xGLg;s<87l1`6Fm zAa-UHdrt$_DFmwKy=0eJS~_KRN~;&iiL*=zq1TB5d=}wrB7o(Ubjk$|D?Ce-yim!w znskKZBg831Y@C6WBK#mbAE{D{5ooIUDV)JZ-M%qAkfkg}0KJ4o!@-MtN0^o%duDFX z@}tfeK|<6}q_DQZ`;S$Nmi36U3{DBw5vZd$X@a7`hES(NQm_928$OYjE%+SFOZYu!A)J6;!=^@g6p3W zC(%Pxv1Qdj;;knHRmvI3j4=FE5;kRwm|XxlVRxHkY-&~#dPr>Ghc$47e4|2=)0?%B zIcF|(#lCgKrYa#p0Wp*_f&f8#dPHH*%cJDhW5BzZ<7u_Fb8*yQmX3u?+_BUdkThZJ z2m@_U0~X>nb5JQ;OvUq zEw6Icv}n|+WdX($EPZC<)FA#(vM0ihEhQC-azM@JA)H9#Po@~9`qV*j@n07f6$i`X zE>gx3-^c#|e+&GX{w&fi@a7t;X5=M&QWA+@CW%-J%7B0fQ(f~_8~LFTOJ>i}@c#f5 z{1^qIEt0rf7nJ`1J%m9lf7E~xrSM~z3UgJ@6%}l{Plc2yBARYvARd)7RchkkP?c?Q z6Y<2y>wX#_(kf5$Uw~`>0Qg7crEeq{sTy2qj2mfS%vyMauhMRY;%K>N8Qq+tt_@ta zi{fz Cm+F%pPw`T!~ixC00II50|EpD1O@>G0RR910RjLK0|XKwF+l|o6Hy{z zae)*dGLa)NLQ|o!QU(=3Lts$<+5iXv0s#R(0RI5?z`OG!r}9OA@tl9=gV+B6B!Bc) zJ>his6V*xTk7GSU{{R`iy@cJ8yGwt0@%z9V+{$lrHn+K(Ting9HZ?Kjja|ND#XF&r{e<@9hb5-fVLhOOd)MF+4L=-7z=q7s2&C)koCD@MrGmAn=6m zlzoK%0JdpVTV+4A9vh?`+`*vHkLW+V{2r70U*>)nP5q$;pH2Od4h=u{mK=CT{-utj zABRqd?@>7?ew9beHP1^={jm-WKlb6lrT+kr`G-%Y_Au!%dzf^%eb4fX4|%+M8M*zX zG4B@%J&BmSJ5v2I7@iH6`khuUh3OZD%*=g)!KTnB{<40iE8!Kl3Jv7pv!&dtS$`HPczn@qZmv70La4Rcb=)6+um<8ahs34-7HU3 z9)uaVAIwM-aF%A)iG|yulQH;ibJPX;nu+0f$Lh;}K^IG-$Nd-eFghnd2hc{(f7K3X z&)1U<|yo0HgJXTZvy{tFieGA;`pc5wgbkfcTHsq!tBSAXE$h`=+U!c zPzg273*yWUc3EGv-axpy9FK~=(>|wQCuub?YF+x_U_NjYEZn(psJo0Tb&IEUVt4g0 zSnuXYg^yuABLYl%JnzvWcwir`!1)N8T{3h_KR6n=6bH&!1{dS3Ich#xm9mjZ#K(=) za_qVY>9$|{K zBdH^UQ_{(_0^?LuSFV;`xn=@T9l=z<)v-S(ZJ6I@xL5Wdfhf_v8j zd_KY!fr%%BDNPtzRMcl8wt~d{C`D*YE(F{q#pAupip|dvi`cIsNF(7qz>ul27anzI zEb2icXaa~IGjL)HbwsBSSN5T=OkMZ%O1F+un8FA?_(+C;~u%KZl( z@ichH>Nb8;833Baw!8bn;KPuKgH+jze+wdjcrGgbgRG!yh)>VUw-NFM2EpKUsp38? z*fuKScP(@P_V)m;x07V99^A?I?oO9T~?zz`nWRV zvq5$mA5$-mkZ!WOO##;6T7CiKOsvYJfK7HCW}i^mP&cIMeV9>{ZW{-oF;4@E`At4b zuw&*5FQ2fuauR4jz>+K)>pAv{7>ap4&(i()Fz zTA1{hk$A?S%-AsGQP6UyfC=NL1~Q$su(wVoG`2qh6+jao5#Fb_aWXzMUR0YPP#4$f z6A~f?RRDGZW5gBSf{Q)6$5pMQUCe5Paem*lDVcI6&^A>YJj-8rg%AY>=Id%A#A9{tlm6$!Ei!6|_`U(qf)@SnykdcXu1`r^^)cLv z9nZ*-4qb@O$BBbE%z?HtgV}`{!rA6422e*rAXVbWA`u$V2@7Yin{_U~7S!%SbN)<3 z08Cj+7ycV427zujOPo|8ICI4hdJ|Er#esJ{*U3wNsgqxXG^4OCD4xiS|2}K2%fIE{PQ-dk=(=iRfC4Ij0_-XxC#15yEQE|D*7&|CD zmEsOAE?fb~83yK~*SBeQO>tdKPCCPr8zHgBvk8rHq5!HC4!}@{DF^iME18!zJUH@g zMzVV;Jf1FO)D?6-q2Wzq;?K{XW{+R%<3>2yHxt1k0!tCJ@C3~HSTc(pK^+_FCcDQQ zNu6<<$av&+P<_P5)9WXhOvTd)B~C!|sc>v6#-X||tu7!lSOEqb-gkFXQA;DE%*xVOT>jdR|2 zyZc7Xi8h9C3Ft4Z)F!7`D;`sbD|1v^r}&yr3CE@-hw(BW)bGm}Yn4+*x{lWbTeds6 zzLixqdiBzw$(F02qsjXjtMTm4#S+m^bJR^1q0U?+C&$w!L&sh~9`PTafl;dLlg)^S zA^kuPkTX-L%Zwe081@||9~#SpRM;~p4suyteYlx@M8rU5 zEP7Ch;GQJeLCvqoKp@Z&FtD$Po;~JVIbULjaYuIUIrau+$CGFyDN3SES)tpR;)6L& zn6)4~*&B#nCXuu0a5&XvbUW<{e}vy_nk=~A5|nMrTpmsR%wcm|;0Wg{?+1Cayk6P> z3Xg+TgH!xE7|8*)qZKB)^L6%%tkYw}A1_G;Z^kxUdqOrl_`%@BN4By^Jp~RR$Kod|Ud^4pE>HNGyfeh# zP_DYsjYm}rcl()xL$6Gh`rYeYHUSwkfk1=T?-L>q$hV~{y8WE~XpD&mq*9mn449zdod z7WjFZ(U;LFjdGH}@~uRC?V{;?U2>iHn4YTcc_90d{-#_Sr3`Xu`VKpb_MR2s@0mBg zM#cf*K-%AVrqOAzrp#&p;{D@dS5mAvF))A#yA$vj#;oSrWYBFRv$rKpZ(q-_>8FB5| zLy^HN{-%#W`e9zBuVn&<-`vDnzcE)f%ACV1h!kDU{#A2t&Ji0Lw@?h$ct)2ls}@bp zkGK$2`*SeheePUc8dttpeCyk^$(uHpCFCuW8?^!yS`vC1?em*do33gmopGOlm0ICA zp*PW@57cyk+_+j2Jb(z_3JuITHCJBou`@8YMZ>Q26gr%tMz&t^w#`_E7Qc;OsIv;oxp3_m` zcyr~%?#=u&Y6MUK^5e|G=ex+<>F39lL&Pg!^8f%;BE?K^dzOu_dZ?bZ^SG#5;0wRs z%VO@ZNctE~K*|8HNwQ5;C&*l^zNq~>EAMi})DiJ-DQrMfe0TX7>D$;*(usfp01(CU z<_nf!Cx{Hc%V)~}03kBWK3C=3LVC;cxIC;&08C6-e@S0!>nL$5aZtqu;WGTL1LO~t zk1I6+m-$RST_IsE{QJfui~D&JAw0nkmJ9(151}K zTqn-u%a;ZF&C=m8+#+x$;R%;FONYq$iEvAo1ivCg+#_~?eoOKn$-0}R&c+ir7-LJ9 zFUa{+GW@~fZWEe>hzuqJFY=a`IfRg4JV0fSAWYmLsm!qn5U-KBa>wKV+2_dliEtX5 zjrm{XQ|1hB)dvD2-{dE(?JtiZ%r*?(XIW6(zt8`~05uT+0s;X71O);G1OWvB00001 z0ss*M5+MW-F+l|rB2i&+ffOJ@Qjsz zueh4mSvgo=_7S+uE7u^0#4z?^euClg1?-M~)end_ zY)AQv#qkb(ILGrp8N}UEiTJ)J_mAR!Cy77#f5h|nkNt)=Ou&D1n7MNi{l+Mhc}5&t4X2bQl@JyEpuo0v5MmR{TAp0dG>M?i1`I@X zQY97q#enZM`vPUTntkEWqBj+UbT{7L=bzhKcAR>bOBCU|~$7szt$-n1C<7 z+t7`KYoPxCawnH&qhdg5Z|^X0x~OUb=3W^Pe|1Utunfj9#3HF%rrlF{iJ|iYA zUVuRP+|O0nB+4^NirxlcwLln)H5EYy$OHYx19%o`xungZ-?t`J6}`m^Q)-H!gSgrp zi+!MV19|4*V1ZGAt;sik;vCw#z-aEj&mgw|5kOijEFQqr)(IN{8HXSvEgrWYGAC0T zh-1)K$(aSH{{WLwJ%?x!y`tg-%-k|zRsR5$7@o>*zVgVu`oYp)#J1JG!YVN;K^vZv zgYflzOk4mftR5S{;sM#gB981R&&O~W$)#6q#>=(L`IWMg1rM02*<1m+CNgPqr(z=G z#iaU$*Yg>aUCtlMX91W2IG@2TFGI5%-)a%Eu9LUYE@jeGsrZkP7t>pgyWGd0hGL^4 zu>)erZ9I@aznH=3dTPeodqIfT zc;u7PtF*`IraOt7Gh`s>RjsxaG2+RQsX-<(kYWcSgvrBU7;W8`DCnAc`%NZeC3hrH znYic76?CWr*+R$0zF9i`K-EI6x6FK55vYyf#jxDj6Uw3}?>xFO#skA+q+t8o6R{np ziwAj+Kt7;r;LT>t;j{3mh#Z(T@B4_EP7>mcDfSmo)$I@xVCxh>+D_sUYn5_bVDEov z!}wNw$Xo!cb!(-0fT8BfrS7HY;H`!j~cSEqtN@0 zp`1U5aQTy2BOUEsiI3pf_cka2weD^lY^148kG)^m&Le@yaN>2w^cQ>HD9ELTo$81p zCX{e;DKtl?q{oS~Wuz8K=^a=a(@`r>wAJw+8w1iU#wt$02#Oko)FKfd9-8$7V)mU^ zr`|3cj0p;M>oB$+7R%$-NVVL7IFAu*^bes&MO}md&|)lKL*Aj@6*Qu|d)(o8HsTJ1 zvE1KDgTrtlhGHCn+US|YVL^%Qsb~~ki8b%-89M4Lw^RVyqi7v0pK>P)gANsM`T7U;*$ru`sq?J;J> zj4>*tpG|M~B4f*pxAwzz+UQxH9&rY@!b*S!!&dVzmzN^>YMC~sAt=s26Q))Sv~I}Q zSJ+H#Q-<-ANV*8pC}}-r4E#Ic8uuU)ps)-+HmQ?`%K0?o!sM?|0W}|o@l~=@GN{<~ z0%Kk=52sKwHcZ)q`HBvaS5#&Z!lFHvSFcGCod{Z>M_tbI&hX)lYhJ$ezx|lJOMqNO zTZU&OY)JxT<2WZu94?dZst3Yv7_*X3Z)h@s0(2CH+L~{nNf>*jc4XJ z7F>amL;9?mqJHtPVtjyOWl-CI%msZ+95!&O0aAK^tO?)g0b)?OWjj-RPAM~D7BUxC z-(B@H9}&gyxo;smkApFI#x-((44^k!CSc<@7FBjCpxD!-U5{ftOFk_5ODQ0p)FDyt z`9$A$qD9r2i;vrrXqF%t@MP?=lQPC{0Ay zsGom`$&u8sQA<}stF*_5BRH{OP$rE*kwbt<(k3oHCO5ryenzRp@tE_mAxfWeZe?(5 z-VM(^M|c=ZUv5CnW~;O$P?rPEK+7>QGW0Ws@hs_u)Tng}M*eo0b0}2`YWFMH8IKk& zAuAXOkw)OQ%st1<*1#3B*Q9bv8#u7=CX7Z&&$a1b*KskjxNCV_s#K_XP}`kA45(n6N|gunkpL7X(`lv}+iqb^%zy?ypnj9T zJKk+jDTQGYrE;s}0hKnnQl`_qBdn-7)y)P~a;`ucHOL7@r?))$e8MPI56Ct!%Y%`C zO)!aE_nTCyQn^%M4N$43p%6`=X;%WV?FZ5UEAok4s!`@(9LR(6*CQxGB|s%y0#x2q z(<_h=sGGt0H-nW!&ZSD^1h33_RMR%70EeupMpPncXbI&;0DmP){FVqCS0ni=m2E0l fD&UWM13ISc@D6SN0m#VD`-D35KsTv;iVnw literal 0 HcmV?d00001 diff --git a/posts/meta-cm-sql_and_ml_xai-20250126/index.ipynb b/posts/meta-cm-sql_and_ml_xai-20250126/index.ipynb new file mode 100644 index 00000000..d23203c4 --- /dev/null +++ b/posts/meta-cm-sql_and_ml_xai-20250126/index.ipynb @@ -0,0 +1,1473 @@ +{ + "cells": [ + { + "cell_type": "raw", + "id": "de6ec63f", + "metadata": { + "vscode": { + "languageId": "raw" + } + }, + "source": [ + "---\n", + "title: '[DA스터디/6주차] optuna, Autogluon'\n", + "author: 'Kibok Park'\n", + "date: '2025-01-26'\n", + "categories: [Python, optuna, Autogluon, 202412Study_DataAnalysis]\n", + "execute:\n", + " freeze: auto\n", + "toc: true\n", + "draft: false\n", + "format:\n", + " html:\n", + " code-fold: false\n", + "comments:\n", + " giscus:\n", + " repo: kr9268/giscus_for_blog\n", + "---\n", + "금융권 데이터를 활용한 분석 스터디 - 6주차" + ] + }, + { + "cell_type": "markdown", + "id": "5e861fae", + "metadata": {}, + "source": [ + "# 개요\n", + "\n", + "* 아래의 목적/이유로 참가한 스터디에 대한 기록\n", + " * SQLD취득 후 장기 미사용 & GPT를 통한 SQL사용 등으로 많이 잊은 SQL을 복기\n", + " * 기존에 사용해 본 Optuna가 아닌 Autogluon이 커리큘럼에 있어 익혀보고자 함\n", + " * 기존에 관심있던 XAI(설명가능한 AI)를 익히고자 함\n", + "\n", + "* 6주차 요약\n", + " * 모델별 주요 하이퍼 파라미터\n", + " * optuna\n", + " * Autogluon" + ] + }, + { + "cell_type": "markdown", + "id": "6ef59b99", + "metadata": {}, + "source": [ + "# 5주차 과제 내용정리\n", + "\n", + "* SHAP Force plot 여러개 비교 = Row(표본)별 비교\n", + "\n", + "# (추가)Multi label에 대한 Catboost실습" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4c707081", + "metadata": {}, + "outputs": [], + "source": [ + "# importing Libraries\n", + "import pandas as pd\n", + "from catboost import CatBoostClassifier\n", + "from sklearn.metrics import accuracy_score, classification_report\n", + "from sklearn.model_selection import train_test_split\n", + "import ipywidgets as widgets\n", + "from IPython.display import display\n", + "import joblib\n", + "import numpy as np\n", + "from sklearn.datasets import load_iris\n", + "import matplotlib.pyplot as plt\n", + "import seaborn as sns" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9eb347b9", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAq4AAAGJCAYAAABLvrEVAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAALfpJREFUeJzt3Qt0VNW9x/F/QkjCK+ERIUTCG0GeesPDFEt5SQBLpdBW1ErgIl4soJBWaa4ghuoN1isgGEC7kEgV8VFBpQqVQIKPRHkYwQfUUBQUCIomgSgJDXPXf681czN5kYSQM5v5ftY6JnPOmTP7nJnIb/b5n30CXC6XSwAAAAAfF+h0AwAAAIDqILgCAADACgRXAAAAWIHgCgAAACsQXAEAAGAFgisAAACsQHAFAACAFQiuAAAAsALBFQAAAFYguAIwHnzwQQkICKiX1xo6dKiZ3NLT081rv/zyy/Xy+lOmTJGOHTuKLztz5ozccccdEhkZaY7NnDlznG6SNWx4fwHUDsEVuAylpqaasOOeQkNDJSoqSuLi4mT58uVy+vTpOnmdY8eOmcCbnZ0tvsaX21Yd//M//2Pex7vuukv++te/yu23317pl40LTaW/JPjS/m3atKlGzykoKJCkpCTp16+fNG3aVBo1aiS9e/eWefPmmfcbwOUvyOkGALh0Fi1aJJ06dZJz587JiRMnTM+m9twtWbJEXnvtNenbt69n3fnz58sf//jHGm1fw4IGCe3duuaaa6r9vH/84x9yqVXVtr/85S9y/vx58WXbt2+X6667ThYuXFjpOhMmTJCuXbt69dJq0P3lL39plrm1adNGfDG4/upXv5Lx48dXa/1//etfMnLkSDly5Ij8+te/ljvvvFOCg4Nl3759smbNGtm4caP885//vOTtBuAsgitwGRszZoz079/f8zgxMdEEop///Ofyi1/8Qj777DPTa6WCgoLMdCn98MMP0rhxYxM4nNSwYUPxdSdPnpSePXtWuY5+8Sj95ePbb781wVXn/fa3v73oNhQWFkqTJk3Eaf/+979NEM/NzTVfvq6//nqv5Q8//LA88sgjjrUPQP2hVADwM8OHD5cFCxbIl19+Kc8++2yVNa5vvfWWCQnNmzc3p2a7d+8u//3f/22WaYAYMGCA+X3q1Kme09J6elvp6Wk9jbtnzx4ZMmSICazu55atcXUrKSkx62hdpwYmDddHjx71Wkd7ULWGsazS27xQ2yqqgdSQ9vvf/16io6MlJCTE7Ov//u//isvl8lpPtzNr1ixzmlv3T9ft1auXbNmypdqBdNq0aaYXVEs49LT3M888U67e9/Dhw/L3v//d0/YvvvhCakPf59/97ndmf/RLSqtWrUyPZdntuctLMjIyzPqtW7eWdu3aeZanpKRI586dzTYGDhwob7/9doXvY1FRkekl1p5gPTZ6PO+77z4zv/Qx1OOt++3ev4reU7e//e1v8tFHH8n9999fLrSqsLAwE16rou/lT37yE7P/ug8xMTEV1lRX9Zl3W7FihXnP9TPdokUL8+Vw/fr1Xut8/fXX8p//+Z/mfXZ/Rp5++ulyr1edbQH4f/S4An5I6yX1H2M9ZT99+vQK1/nkk09Mz6z23mnJgf7jm5OTI++++65ZfvXVV5v5DzzwgDlt+9Of/tTM13DgdurUKdPrO2nSJNMDeKFT1ho+NMRozaIGvGXLlpnTw1qn6u4Zro7qtK00Dacaknfs2GFCpZYWbN26Ve69914TQJYuXeq1/jvvvCOvvPKKCXjNmjUzdcMTJ040p7E1GFXmxx9/NEFPj6OGXy3jeOmll0xoy8vLk3vuuce0XWta586da4Kjhml1xRVXSG3s2rVL3nvvPfMe6PY0sK5atcq049NPPzWBqTTdJ30tPXYaLpWur+3V46jt0m3oKX4NWqXDrZZf6HHU46PHXfdl//795vjpaXx3Tavun154pgFY11NdunSpdB+0rEVVVOdbXY8//rhp22233SbFxcWyYcMGE+A3b94sN954Y7U+8+4yk7vvvtuUOej7dfbsWVOu8P7778utt95q1tGeYS3zcH/J0eP55ptvms+W1um6L7SrzrYAlOECcNlZu3atdhO6du3aVek64eHhrmuvvdbzeOHCheY5bkuXLjWPv/nmm0q3odvXdfT1yvrZz35mlq1evbrCZTq57dixw6x75ZVXugoKCjzzX3zxRTP/8ccf98zr0KGDKz4+/oLbrKpt+nzdjtumTZvMug899JDXer/61a9cAQEBrpycHM88XS84ONhr3kcffWTmr1ixwlWVZcuWmfWeffZZz7zi4mJXbGysq2nTpl77ru278cYbXTWh75VuX99Ltx9++KHcepmZmWa9devWlfvMXH/99a5///vfnvlFRUWuVq1auQYMGOA6d+6cZ35qaqpZv/Qx/+tf/+oKDAx0vf32216vp58BXffdd9/1zGvSpEmF72NF9HOqn9fqKvv+VnQc9Lj37t3bNXz48Bp95m+66SZXr169qnz9adOmudq2bev69ttvveZPmjTJ7Ie7LdXZFgBvlAoAfkpPg1Y1uoCeKlWvvvpqrS9k0h4rPVVfXZMnTzY9mG7aE9W2bVt544035FLS7Tdo0MD0fpWmvZ2aVbW3rDTtBS7dQ6g9dHq6Wi8gutDraBnELbfc4lVvq6+rF1bpafq6VrqnWi/S015wPY2v7+/evXvLra898Hos3Hbv3m2eo/NL10Brz6X2uJamvcfay9qjRw9Tb+uetDxFaY92bWgvZenPxcUeh++//17y8/NND3LpY1Cdz7yu89VXX5me7Iro50VLG8aNG2d+L30cdFQPfV33a15oWwDKI7gCfkqDUlVh4Oabb5bBgwebU7p6il9PNb/44os1CrFXXnlljS7E6tatm9djPdWqIau29Z01qQPV4cLKHg8NYe7lpbVv377cNjTEaSC60OvoPgYGBlbrdeqClifoaX937W5ERIQ5da2lCRqiytLyhbJtVqVHL1AaYsvWCX/++efmdLtuv/R01VVXmeVa/lEb+qXgYodw05IAPX2vdcUtW7Y07dISiNLHoDqfeS1j0S99Wuag7+XMmTO9Sgm++eYbc2yfeuqpcsfB/SXOfRwutC0A5VHjCvgh7eXRf7DLhpGyPVQ7d+40vWR6kZBefPTCCy+Y3jOtjS3dK1fVNupaZTdJ0Au7qtOmulDZ65S9kMsXzJ49W9auXWvqKmNjYyU8PNwcQw1lFX0JuZj3TLfXp08fM9xaRTQ814b24H744YfmQr3abEMvJNP6Vr1IcOXKlaYXX3u69biUvhCqOp95/ZJx8OBBE4R1ufau6jb1y4EOv+Y+plrTHR8fX2F73CNBXGhbAMojuAJ+SC+OUXrqsiraMzhixAgzaRjRsTf1ym79h11Pl9f1nba0x65sENSLY0oP+aQ9m9qjVZb2DOpV7241aVuHDh1k27ZtplevdK/rgQMHPMvrgm5HL77RcFO617WuX6c0vXJeA9Rjjz3mmacXAVV0DCtrs9L3YdiwYV5DVGlPeOn3Rssn9Op//bxc6PjX5P3R0+7PP/+8GQVDh3SrKQ2E2tOqF9xpr7ObBteafuaVjnihvbM66YVeOlSXXliobdOeVf0M6Rcp9/pVqWpb2mYA3igVAPyMjuP6pz/9yZwS1jrFynz33Xfl5rkH8ncPbeQe47O6IehC1q1b53VKWEPX8ePHzcgEpcNRVlaW+UfeTXusyg6bVZO2jR071gSNJ554wmu+Xg2vAav0618MfR29EYT24pUOgDokkp4y/tnPfiZ1TXsJy/YE6+vp/laHDs+kIyXoFfDaVrfnnnuuXGnEb37zGzMKg65bUcmCe5QC9/tT3c+N1jprT64GuszMzHLL9TOj4bKqY6DvY+l91tBd9s5d1fnMa71vaVoKo+Pt6jHWGmJ9LR1hQsPyxx9/XG57WkrgdqFtASiPHlfgMqYXFWlvngYOHaJHQ6uOU6m9aDrEUFU9OjockJ421aGCdH2ty9PTmDr8kXssTQ2ReoHJ6tWrTS+ThpFBgwaVq5OsLq091G1rLaC2V4fD0nKG0kN2af2hBtrRo0eboHTo0CHTE1d2OKWatE179LQ3UcOPBhodW1VPDetFOnqKvaqhmmpCh3568sknzfBXOr6t1ojqvmhdo+7rxV6AVBEd3kl72LVEQEORBj/tXa5q2K6yYUrH+NWSAz1lrsdcj5GO+6rHpXTPqQ5XpTWhM2bMMD2UWi+qYVE/gzpfezzdN8TQcVS1HdqrqfXF+r7o+1MRPa2vw49pD6ae7tc26LZ1vtbU6ul+7YmvbCxX/Qzr6+hnRoeZ0s+yjkurny3tAa/JZ37UqFHmAjt9fa2D1Zt46BcefY77/Vu8eLHZf90f/ezqcddQrBdl6T67A3J1tgWgjDKjDAC4DLiHNnJPOnxTZGSk64YbbjBDS5Uedqmy4bDS0tLMcD1RUVHm+frzlltucf3zn//0et6rr77q6tmzpysoKMhr+CkdJqmyoX4qGw7r+eefdyUmJrpat27tatSokRkO6ssvvyz3/Mcee8wMnRUSEuIaPHiwa/fu3eW2WVXbKhou6fTp0665c+ea/WzYsKGrW7durkcffdR1/vx5r/V0OzNnzizXpsqG6SorNzfXNXXqVFdERIQ5rn369KlwyK66Gg7r+++/97yeDrkVFxfnOnDgQLn2XmgIteXLl5vn6DEfOHCgGdoqJibGNXr06HLDTD3yyCPmvdd1W7RoYdZLSkpy5efne9bTNgwZMsS8z/q61Tl2ui8PPPCAOWaNGzd2hYaGmiGt9DNz/Phxz3oVvb9r1qwx76m2qUePHmZ/a/OZf/LJJ027dYgw3VaXLl1c9957r9e+ud9n/ZxER0ebz5P+/Y0YMcL11FNP1XhbAP5fgP6nbJgFAKAqWqer9Zxak1lRaQAAXArUuAIAqqQXc5Xt49B6ZD3lXdGtewHgUqHHFQBQpfT0dHOrV71FqtbGaq3mmjVrzHBOWqtbk7F6AeBicHEWAKBKehGZjp+6fPly08uqF9HpXc70IiRCK4D6RI8rAAAArECNKwAAAKxAcAUAAIAVgvxhyJZjx46ZwZzr+vaUAAAAuHhauap3wdMbkpS+JbbfBVcNrXpRAQAAAHyb3r5b71bnt8HVfds8PRBhYWFONwcAAABlFBQUmI7GC93u+LIPru7yAA2tBFcAAADfdaGyTi7OAgAAgBUIrgAAALACwRUAAABWILgCAADACgRXAAAAWIHgCgAAACsQXAEAAGAFgisAAACsQHAFAACAFQiuAAAAsALBFQAAAFYIcroBAPxHzL3rnG4CUM6eRyc73QQA1USPKwAAAKxAcAUAAIAVCK4AAACwgs8E18WLF0tAQIDMmTPHM+/s2bMyc+ZMadWqlTRt2lQmTpwoubm5jrYTAAAAfhxcd+3aJU8++aT07dvXa/7cuXPl9ddfl5deekkyMjLk2LFjMmHCBMfaCQAAAD8OrmfOnJHbbrtN/vKXv0iLFi088/Pz82XNmjWyZMkSGT58uMTExMjatWvlvffek6ysLEfbDAAAAD8MrloKcOONN8rIkSO95u/Zs0fOnTvnNb9Hjx7Svn17yczMrHR7RUVFUlBQ4DUBAADAfo6O47phwwbZu3evKRUo68SJExIcHCzNmzf3mt+mTRuzrDLJycmSlJQklxrjUcLXMBYlAOBy51iP69GjR+Wee+6R5557TkJDQ+tsu4mJiabMwD3p6wAAAMB+jgVXLQU4efKk/Md//IcEBQWZSS/AWr58uflde1aLi4slLy/P63k6qkBkZGSl2w0JCZGwsDCvCQAAAPZzrFRgxIgRsn//fq95U6dONXWs8+bNk+joaGnYsKGkpaWZYbDUwYMH5ciRIxIbG+tQqwEAAOB3wbVZs2bSu3dvr3lNmjQxY7a650+bNk0SEhKkZcuWpud09uzZJrRed911DrUaAAAAfnlx1oUsXbpUAgMDTY+rjhYQFxcnK1eudLpZAAAA8Pfgmp6e7vVYL9pKSUkxEwAAAPybTwVXAABQHkMwwtfscWgIRsdvQAAAAABUB8EVAAAAViC4AgAAwAoEVwAAAFiB4AoAAAArEFwBAABgBYIrAAAArEBwBQAAgBUIrgAAALACwRUAAABWILgCAADACgRXAAAAWIHgCgAAACsQXAEAAGAFgisAAACsQHAFAACAFQiuAAAAsALBFQAAAFYguAIAAMAKBFcAAABYgeAKAAAAKxBcAQAAYAWCKwAAAKzgaHBdtWqV9O3bV8LCwswUGxsrb775pmf50KFDJSAgwGuaMWOGk00GAACAQ4LEQe3atZPFixdLt27dxOVyyTPPPCM33XSTfPjhh9KrVy+zzvTp02XRokWe5zRu3NjBFgMAAMAvg+u4ceO8Hj/88MOmFzYrK8sTXDWoRkZGOtRCAAAA+AqfqXEtKSmRDRs2SGFhoSkZcHvuueckIiJCevfuLYmJifLDDz9UuZ2ioiIpKCjwmgAAAGA/R3tc1f79+01QPXv2rDRt2lQ2btwoPXv2NMtuvfVW6dChg0RFRcm+fftk3rx5cvDgQXnllVcq3V5ycrIkJSXV4x4AAADAL4Jr9+7dJTs7W/Lz8+Xll1+W+Ph4ycjIMOH1zjvv9KzXp08fadu2rYwYMUIOHTokXbp0qXB72iubkJDgeaw9rtHR0fWyLwAAALiMg2twcLB07drV/B4TEyO7du2Sxx9/XJ588sly6w4aNMj8zMnJqTS4hoSEmAkAAACXF5+pcXU7f/68qVOtiPbMKu15BQAAgH9xtMdVT+uPGTNG2rdvL6dPn5b169dLenq6bN261ZQD6OOxY8dKq1atTI3r3LlzZciQIWbsVwAAAPgXR4PryZMnZfLkyXL8+HEJDw83gVRD6w033CBHjx6Vbdu2ybJly8xIA1qnOnHiRJk/f76TTQYAAIA/Btc1a9ZUukyDql6kBQAAAPhkjSsAAABQEYIrAAAArEBwBQAAgBUIrgAAALACwRUAAABWILgCAADACgRXAAAAWIHgCgAAACsQXAEAAGAFgisAAACsQHAFAACAFQiuAAAAsALBFQAAAFYguAIAAMAKBFcAAABYgeAKAAAAKxBcAQAAYAWCKwAAAKxAcAUAAIAVCK4AAACwAsEVAAAAViC4AgAAwAoEVwAAAFiB4AoAAAArEFwBAABgBUeD66pVq6Rv374SFhZmptjYWHnzzTc9y8+ePSszZ86UVq1aSdOmTWXixImSm5vrZJMBAADgj8G1Xbt2snjxYtmzZ4/s3r1bhg8fLjfddJN88sknZvncuXPl9ddfl5deekkyMjLk2LFjMmHCBCebDAAAAIcEiYPGjRvn9fjhhx82vbBZWVkm1K5Zs0bWr19vAq1au3atXH311Wb5dddd51CrAQAA4Nc1riUlJbJhwwYpLCw0JQPaC3vu3DkZOXKkZ50ePXpI+/btJTMzs9LtFBUVSUFBgdcEAAAA+zkeXPfv32/qV0NCQmTGjBmyceNG6dmzp5w4cUKCg4OlefPmXuu3adPGLKtMcnKyhIeHe6bo6Oh62AsAAABc9sG1e/fukp2dLe+//77cddddEh8fL59++mmtt5eYmCj5+fme6ejRo3XaXgAAAPhhjavSXtWuXbua32NiYmTXrl3y+OOPy8033yzFxcWSl5fn1euqowpERkZWuj3tudUJAAAAlxfHe1zLOn/+vKlT1RDbsGFDSUtL8yw7ePCgHDlyxNTAAgAAwL842uOqp/XHjBljLrg6ffq0GUEgPT1dtm7daupTp02bJgkJCdKyZUszzuvs2bNNaGVEAQAAAP/jaHA9efKkTJ48WY4fP26Cqt6MQEPrDTfcYJYvXbpUAgMDzY0HtBc2Li5OVq5c6WSTAQAA4I/BVcdprUpoaKikpKSYCQAAAP7N52pcAQAAgIoQXAEAAGAFgisAAACsQHAFAACAFQiuAAAAsALBFQAAAFYguAIAAMAKBFcAAABYgeAKAAAAKxBcAQAAYAWCKwAAAKxAcAUAAIAVCK4AAACwAsEVAAAAViC4AgAAwAoEVwAAAFiB4AoAAAArEFwBAABgBYIrAAAArEBwBQAAgBUIrgAAALACwRUAAABWILgCAADACgRXAAAAWIHgCgAAACs4GlyTk5NlwIAB0qxZM2ndurWMHz9eDh486LXO0KFDJSAgwGuaMWOGY20GAACAHwbXjIwMmTlzpmRlZclbb70l586dk1GjRklhYaHXetOnT5fjx497pj//+c+OtRkAAADOCBIHbdmyxetxamqq6Xnds2ePDBkyxDO/cePGEhkZ6UALAQAA4Ct8qsY1Pz/f/GzZsqXX/Oeee04iIiKkd+/ekpiYKD/88EOl2ygqKpKCggKvCQAAAPZztMe1tPPnz8ucOXNk8ODBJqC63XrrrdKhQweJioqSffv2ybx580wd7CuvvFJp3WxSUlI9thwAAAB+FVy11vXjjz+Wd955x2v+nXfe6fm9T58+0rZtWxkxYoQcOnRIunTpUm472iObkJDgeaw9rtHR0Ze49QAAAPCL4Dpr1izZvHmz7Ny5U9q1a1fluoMGDTI/c3JyKgyuISEhZgIAAMDlxdHg6nK5ZPbs2bJx40ZJT0+XTp06XfA52dnZ5qf2vAIAAMB/BDldHrB+/Xp59dVXzViuJ06cMPPDw8OlUaNGphxAl48dO1ZatWplalznzp1rRhzo27evk00HAACAPwXXVatWeW4yUNratWtlypQpEhwcLNu2bZNly5aZsV21VnXixIkyf/58h1oMAAAAvy0VqIoGVb1JAQAAAFCrcVw7d+4sp06dKjc/Ly/PLAMAAAB8Irh+8cUXUlJSUuHg/19//XVdtAsAAACofanAa6+95vl969at5iIqNw2yaWlp0rFjx5psEgAAAKj74Dp+/HjzMyAgQOLj472WNWzY0ITWxx57rCabBAAAAOo+uOptWZWOt7pr1y6JiIioydMBAACA+h1V4PDhw7V/RQAAAKA+h8PSeladTp486emJdXv66adru1kAAACg7oJrUlKSLFq0SPr3729uvao1rwAAAIDPBdfVq1dLamqq3H777XXfIgAAAKCuxnEtLi6Wn/zkJ7V5KgAAAFB/wfWOO+6Q9evX1+4VAQAAgPoqFTh79qw89dRTsm3bNunbt68Zw7W0JUuW1FX7AAAAgNoH13379sk111xjfv/444+9lnGhFgAAAHwmuO7YsaPuWwIAAADUdY0rAAAAYEWP67Bhw6osCdi+ffvFtAkAAACom+Dqrm91O3funGRnZ5t61/j4+NpsEgAAAKj74Lp06dIK5z/44INy5syZ2mwSAAAAqL8a19/+9rfy9NNP1+UmAQAAgLoPrpmZmRIaGlqXmwQAAABqXyowYcIEr8cul0uOHz8uu3fvlgULFtRmkwAAAEDdB9fw8HCvx4GBgdK9e3dZtGiRjBo1qjabBAAAAOo+uK5du7Y2TwMAAADqN7i67dmzRz777DPze69eveTaa6+tq3YBAAAAFx9cT548KZMmTZL09HRp3ry5mZeXl2duTLBhwwa54oorarNZAAAAoG5HFZg9e7acPn1aPvnkE/nuu+/MpDcfKCgokLvvvrva20lOTpYBAwZIs2bNpHXr1jJ+/Hg5ePCg1zpnz56VmTNnSqtWraRp06YyceJEyc3NrU2zAQAA4G/BdcuWLbJy5Uq5+uqrPfN69uwpKSkp8uabb1Z7OxkZGSaUZmVlyVtvvWXuwKUXdxUWFnrWmTt3rrz++uvy0ksvmfWPHTtWblQDAAAAXP5qVSpw/vx5adiwYbn5Ok+X1SQAl5aammp6XrV2dsiQIZKfny9r1qyR9evXy/Dhwz0Xhmlg1rB73XXX1ab5AAAA8JceVw2R99xzj+n9dPv6669N7+iIESNq3RgNqqply5bmpwZY7YUdOXKkZ50ePXpI+/btzc0OKlJUVGRKFkpPAAAA8NPg+sQTT5hA2LFjR+nSpYuZOnXqZOatWLGiVg3Rnto5c+bI4MGDpXfv3mbeiRMnJDg42HMBmFubNm3MssrqZnWcWfcUHR1dq/YAAADgMigV0DC4d+9e2bZtmxw4cMDM09P3pXtGa0prXfUCr3feeUcuRmJioiQkJHgea5gmvAIAAPhZj+v27dvNRVgaBgMCAuSGG24wIwzopKMD6Fiub7/9do0bMWvWLNm8ebPs2LFD2rVr55kfGRkpxcXFZqit0nRUAV1WkZCQEAkLC/OaAAAA4GfBddmyZTJ9+vQKw6Celv+v//ovWbJkSbW353K5TGjduHGjCcVablBaTEyMueArLS3NM0+Hyzpy5IjExsbWpOkAAADwp+D60UcfyejRoytdrkNZ6QVVNSkPePbZZ82oATqWq9at6vTjjz96wvC0adPMqX/tjdVtT5061YRWRhQAAADwLzWqcdVT9BUNg+XZWFCQfPPNN9Xe3qpVq8zPoUOHes3XIa+mTJlifl+6dKkEBgaaGw/oiAFxcXFmDFkAAAD4lxoF1yuvvNJcQNW1a9cKl+/bt0/atm1bo1KBCwkNDTU3NtAJAAAA/qtGpQJjx46VBQsWmNuwlqWn9xcuXCg///nP67J9AAAAQM17XOfPny+vvPKKXHXVVeaiqu7du5v5OiSW9oiWlJTI/fffX5NNAgAAAHUfXHXg//fee0/uuusuM16q+1S/Do2ltacaXnUdAAAAwPEbEHTo0EHeeOMN+f777yUnJ8eE127dukmLFi3qvHEAAADARd05S2lQ1ZsOAAAAAD53cRYAAADgFIIrAAAArEBwBQAAgBUIrgAAALACwRUAAABWILgCAADACgRXAAAAWIHgCgAAACsQXAEAAGAFgisAAACsQHAFAACAFQiuAAAAsALBFQAAAFYguAIAAMAKBFcAAABYgeAKAAAAKxBcAQAAYAWCKwAAAKxAcAUAAIAVCK4AAACwgqPBdefOnTJu3DiJioqSgIAA2bRpk9fyKVOmmPmlp9GjRzvWXgAAAPhpcC0sLJR+/fpJSkpKpetoUD1+/Lhnev755+u1jQAAAPANQU6++JgxY8xUlZCQEImMjKz2NouKiszkVlBQcFFtBAAAgG/w+RrX9PR0ad26tXTv3l3uuusuOXXqVJXrJycnS3h4uGeKjo6ut7YCAADAT4OrlgmsW7dO0tLS5JFHHpGMjAzTQ1tSUlLpcxITEyU/P98zHT16tF7bDAAAgMuwVOBCJk2a5Pm9T58+0rdvX+nSpYvphR0xYkSlpQU6AQAA4PLi0z2uZXXu3FkiIiIkJyfH6aYAAACgnlkVXL/66itT49q2bVunmwIAAAB/KhU4c+aMV+/p4cOHJTs7W1q2bGmmpKQkmThxohlV4NChQ3LfffdJ165dJS4uzslmAwAAwN+C6+7du2XYsGGexwkJCeZnfHy8rFq1Svbt2yfPPPOM5OXlmZsUjBo1Sv70pz9RwwoAAOCHHA2uQ4cOFZfLVenyrVu31mt7AAAA4LusqnEFAACA/yK4AgAAwAoEVwAAAFiB4AoAAAArEFwBAABgBYIrAAAArEBwBQAAgBUIrgAAALACwRUAAABWILgCAADACgRXAAAAWIHgCgAAACsQXAEAAGAFgisAAACsQHAFAACAFQiuAAAAsALBFQAAAFYguAIAAMAKBFcAAABYgeAKAAAAKxBcAQAAYAWCKwAAAKxAcAUAAIAVCK4AAACwgqPBdefOnTJu3DiJioqSgIAA2bRpk9dyl8slDzzwgLRt21YaNWokI0eOlM8//9yx9gIAAMBPg2thYaH069dPUlJSKlz+5z//WZYvXy6rV6+W999/X5o0aSJxcXFy9uzZem8rAAAAnBXk5IuPGTPGTBXR3tZly5bJ/Pnz5aabbjLz1q1bJ23atDE9s5MmTarn1gIAAMBJPlvjevjwYTlx4oQpD3ALDw+XQYMGSWZmZqXPKyoqkoKCAq8JAAAA9vPZ4KqhVWkPa2n62L2sIsnJySbguqfo6OhL3lYAAAD4cXCtrcTERMnPz/dMR48edbpJAAAAuJyDa2RkpPmZm5vrNV8fu5dVJCQkRMLCwrwmAAAA2M9ng2unTp1MQE1LS/PM03pVHV0gNjbW0bYBAADAz0YVOHPmjOTk5HhdkJWdnS0tW7aU9u3by5w5c+Shhx6Sbt26mSC7YMECM+br+PHjnWw2AAAA/C247t69W4YNG+Z5nJCQYH7Gx8dLamqq3HfffWas1zvvvFPy8vLk+uuvly1btkhoaKiDrQYAAIDfBdehQ4ea8Voro3fTWrRokZkAAADg33y2xhUAAAAojeAKAAAAKxBcAQAAYAWCKwAAAKxAcAUAAIAVCK4AAACwAsEVAAAAViC4AgAAwAoEVwAAAFiB4AoAAAArEFwBAABgBYIrAAAArEBwBQAAgBUIrgAAALACwRUAAABWILgCAADACgRXAAAAWIHgCgAAACsQXAEAAGAFgisAAACsQHAFAACAFQiuAAAAsALBFQAAAFYguAIAAMAKBFcAAABYwaeD64MPPigBAQFeU48ePZxuFgAAABwQJD6uV69esm3bNs/joCCfbzIAAAAuAZ9PgRpUIyMjnW4GAAAAHObTpQLq888/l6ioKOncubPcdtttcuTIkSrXLyoqkoKCAq8JAAAA9vPp4Dpo0CBJTU2VLVu2yKpVq+Tw4cPy05/+VE6fPl3pc5KTkyU8PNwzRUdH12ubAQAA4IfBdcyYMfLrX/9a+vbtK3FxcfLGG29IXl6evPjii5U+JzExUfLz8z3T0aNH67XNAAAA8NMa19KaN28uV111leTk5FS6TkhIiJkAAABwefHpHteyzpw5I4cOHZK2bds63RQAAADUM58Orn/4wx8kIyNDvvjiC3nvvffkl7/8pTRo0EBuueUWp5sGAACAeubTpQJfffWVCamnTp2SK664Qq6//nrJysoyvwMAAMC/+HRw3bBhg9NNAAAAgI/w6VIBAAAAwI3gCgAAACsQXAEAAGAFgisAAACsQHAFAACAFQiuAAAAsALBFQAAAFYguAIAAMAKBFcAAABYgeAKAAAAKxBcAQAAYAWCKwAAAKxAcAUAAIAVCK4AAACwAsEVAAAAViC4AgAAwAoEVwAAAFiB4AoAAAArEFwBAABgBYIrAAAArEBwBQAAgBUIrgAAALACwRUAAABWILgCAADACgRXAAAAWMGK4JqSkiIdO3aU0NBQGTRokHzwwQdONwkAAAD1zOeD6wsvvCAJCQmycOFC2bt3r/Tr10/i4uLk5MmTTjcNAAAA9cjng+uSJUtk+vTpMnXqVOnZs6esXr1aGjduLE8//bTTTQMAAEA9ChIfVlxcLHv27JHExETPvMDAQBk5cqRkZmZW+JyioiIzueXn55ufBQUFddq2kqIf63R7wMWq68/4pcDfDXwRfzuA83837u25XC57g+u3334rJSUl0qZNG6/5+vjAgQMVPic5OVmSkpLKzY+Ojr5k7QR8QfiKGU43AbASfzuA7/zdnD59WsLDw+0MrrWhvbNaE+t2/vx5+e6776RVq1YSEBDgaNtQ/tuVfqE4evSohIWFOd0cwBr87QC1w9+O79KeVg2tUVFRVa7n08E1IiJCGjRoILm5uV7z9XFkZGSFzwkJCTFTac2bN7+k7cTF0f958D8QoOb42wFqh78d31RVT6sVF2cFBwdLTEyMpKWlefWg6uPY2FhH2wYAAID65dM9rkpP+8fHx0v//v1l4MCBsmzZMiksLDSjDAAAAMB/+Hxwvfnmm+Wbb76RBx54QE6cOCHXXHONbNmypdwFW7CPlnTo+LxlSzsAVI2/HaB2+NuxX4DrQuMOAAAAAD7Ap2tcAQAAADeCKwAAAKxAcAUAAIAVCK4AAACwAsEVjklJSZGOHTtKaGioDBo0SD744AOnmwT4tJ07d8q4cePMnWX0ToCbNm1yukmAz9NbwQ8YMECaNWsmrVu3lvHjx8vBgwedbhZqieAKR7zwwgtmjF4dlmTv3r3Sr18/iYuLk5MnTzrdNMBn6RjW+reiX/oAVE9GRobMnDlTsrKy5K233pJz587JqFGjzN8T7MNwWHCE9rDqN+AnnnjCc0c0vX/07Nmz5Y9//KPTzQN8nva4bty40fQeAag+HRtee1410A4ZMsTp5qCG6HFFvSsuLpY9e/bIyJEjPfMCAwPN48zMTEfbBgC4vOXn55ufLVu2dLopqAWCK+rdt99+KyUlJeXufqaP9e5oAABcCnp2b86cOTJ48GDp3bu3083B5XjLVwAAgLqgta4ff/yxvPPOO043BbVEcEW9i4iIkAYNGkhubq7XfH0cGRnpWLsAAJevWbNmyebNm83oHO3atXO6OaglSgVQ74KDgyUmJkbS0tK8Tt/o49jYWEfbBgC4vOg16Bpa9WLG7du3S6dOnZxuEi4CPa5whA6FFR8fL/3795eBAwfKsmXLzNAkU6dOdbppgM86c+aM5OTkeB4fPnxYsrOzzUUm7du3d7RtgC+XB6xfv15effVVM5ar+1qK8PBwadSokdPNQw0xHBYco0NhPfroo+Z/Itdcc40sX77cDJMFoGLp6ekybNiwcvP1S2BqaqojbQJsGDquImvXrpUpU6bUe3twcQiuAAAAsAI1rgAAALACwRUAAABWILgCAADACgRXAAAAWIHgCgAAACsQXAEAAGAFgisAAACsQHAFAACAFQiuAOCjd/vZtGmT080AAJ9CcAUAB+itjmfPni2dO3eWkJAQiY6OlnHjxklaWprTTQMAnxXkdAMAwN988cUXMnjwYGnevLk8+uij0qdPHzl37pxs3bpVZs6cKQcOHHC6iQDgk+hxBYB69rvf/c6UAnzwwQcyceJEueqqq6RXr16SkJAgWVlZFT5n3rx5Zr3GjRubXtoFCxaYsOv20UcfybBhw6RZs2YSFhYmMTExsnv3brPsyy+/NL25LVq0kCZNmpjXeuONN+ptfwGgrtDjCgD16LvvvpMtW7bIww8/bEJkWdoLWxENpKmpqRIVFSX79++X6dOnm3n33XefWX7bbbfJtddeK6tWrZIGDRpIdna2NGzY0CzTXtzi4mLZuXOnec1PP/1UmjZteon3FADqHsEVAOpRTk6OuFwu6dGjR42eN3/+fM/vHTt2lD/84Q+yYcMGT3A9cuSI3HvvvZ7tduvWzbO+LtOeXS1JUNpjCwA2olQAAOqRhtbaeOGFF0xdbGRkpOkt1SCrgdRNywzuuOMOGTlypCxevFgOHTrkWXb33XfLQw89ZJ6/cOFC2bdvX53sCwDUN4IrANQj7QnV+taaXICVmZlpSgHGjh0rmzdvlg8//FDuv/9+c/rf7cEHH5RPPvlEbrzxRtm+fbv07NlTNm7caJZpoP3Xv/4lt99+uykz6N+/v6xYseKS7B8AXEoBrtp+/QcA1MqYMWNMgDx48GC5Ote8vDxT56rhVoPn+PHj5bHHHpOVK1d69aJqGH355ZfN+hW55ZZbpLCwUF577bVyyxITE+Xvf/87Pa8ArEOPKwDUs5SUFCkpKZGBAwfK3/72N/n888/ls88+k+XLl0tsbGyFvbRaFqA1rRpedT13b6r68ccfZdasWZKenm5GEHj33Xdl165dcvXVV5vlc+bMMUNtHT58WPbu3Ss7duzwLAMAm3BxFgDUM704SgOkjizw+9//Xo4fPy5XXHGFGcJKRwUo6xe/+IXMnTvXhNOioiJTDqDDYWl5gNJRBE6dOiWTJ0+W3NxciYiIkAkTJkhSUpJZriFZRxb46quvzFBZo0ePlqVLl9b7fgPAxaJUAAAAAFagVAAAAABWILgCAADACgRXAAAAWIHgCgAAACsQXAEAAGAFgisAAACsQHAFAACAFQiuAAAAsALBFQAAAFYguAIAAMAKBFcAAACIDf4PY8iCI9rMXX0AAAAASUVORK5CYII=", + "text/plain": [ + "

" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Load the Iris dataset\n", + "iris_df = load_iris()\n", + "X = iris_df.data\n", + "y = iris_df.target\n", + "\n", + "# Split the data into training and testing sets\n", + "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n", + "\n", + "# Create a figure with specified dimensions (8x4)\n", + "plt.figure(figsize=(8, 4))\n", + "\n", + "# Generate a countplot to display the distribution of target classes (y_train)\n", + "sns.countplot(x=y_train)\n", + "\n", + "# Set the title, x-axis label, and y-axis label for the plot\n", + "plt.title('Distribution of Target Classes')\n", + "plt.xlabel('Class')\n", + "plt.ylabel('Count')\n", + "\n", + "# Display the plot\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "567f18e0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Accuracy: 1.0\n", + "Classification Report:\n", + " precision recall f1-score support\n", + "\n", + " 0 1.00 1.00 1.00 10\n", + " 1 1.00 1.00 1.00 9\n", + " 2 1.00 1.00 1.00 11\n", + "\n", + " accuracy 1.00 30\n", + " macro avg 1.00 1.00 1.00 30\n", + "weighted avg 1.00 1.00 1.00 30\n", + "\n" + ] + } + ], + "source": [ + "# 모델 설정 및 학습\n", + "model = CatBoostClassifier(\n", + " iterations=100, depth=6, learning_rate=0.1,\n", + " loss_function='MultiClass', verbose=False)\n", + "model.fit(X_train, y_train)\n", + "\n", + "# Make predictions on the test set\n", + "y_pred = model.predict(X_test)\n", + "\n", + "# Evaluate the model\n", + "accuracy = accuracy_score(y_test, y_pred)\n", + "report = classification_report(y_test, y_pred)\n", + "# printing metrics\n", + "print(f\"Accuracy: {accuracy}\")\n", + "print(\"Classification Report:\\n\", report)\n" + ] + }, + { + "cell_type": "markdown", + "id": "87231608", + "metadata": {}, + "source": [ + "# (추가)VS CODE 팝업이 떠서 찾아보는 Tensorboard\n", + "\n", + "* Catboost는 지원하지 않아 타 모델로 향후 실험" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "99dbcdf3", + "metadata": {}, + "outputs": [], + "source": [ + "import tensorflow as tf\n", + "\n", + "# TensorBoard 로그를 저장할 디렉토리 설정\n", + "log_dir = \"logs/fit\"\n", + "\n", + "# SummaryWriter 설정\n", + "tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)\n", + "\n", + "# 모델 학습 시 callback 추가\n", + "model.fit(X_train, y_train, callbacks=[tensorboard_callback])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0a202d44", + "metadata": {}, + "outputs": [], + "source": [ + "!tensorboard --logdir=logs/fit" + ] + }, + { + "cell_type": "markdown", + "id": "09976f5f", + "metadata": {}, + "source": [ + "# 6주차 수업정리\n", + "\n", + "## 하이퍼 파라미터\n", + "\n", + "* 하이퍼 파라미터 튜닝 : 모델의 초기 설정값을 최적의 값으로 구하는 것\n", + "\n", + "### 하이퍼 파라미터 - RF(Random Forest, Bagging)\n", + "\n", + "* 주요 설정값\n", + " * `n_estimators` : 트리의 수 (Default=100)\n", + " * 증가할수록 계산비용 & 성능 증가 (일정수준부터는 크게 상승하지 않음)\n", + " * ↔ 감소할수록 계산비용 감소. 그러나 과소적합 발생할 가능성\n", + " * (경험적으로)천 단위에서 마감하는 것이 좋음(만 단위에서 유의미한 성능향상 X)\n", + " * `max_depth` : 트리의 최대 깊이 (Default=None)\n", + " * 증가할수록 모델의 복잡도 & 과적합가능성 증대(복잡한 패턴을 익힐 수 있음)\n", + " * ↔ 감소할수록 과소적합 발생할 수 있음\n", + " * `min_samples_split` : 노드를 분할하기 위한 최소 샘플 수 (Default=2)\n", + " * 증가할수록 과적합 방지 & 성능 감소\n", + " * ↔ 감소할수록 과적합 위험이 커짐(트리가 얕아짐짐)\n", + " * `min_samples_leaf` : 분할이 모두 끝난 노드(리프노드)의 최소 샘플 크기 (Default=1)\n", + " * 증가할수록 과적합 방지 & 성능 감소\n", + " * ↔ 감소할수록 과적합 위험이 커짐\n", + " * `max_features` : 각 트리를 학습할 때 사용할 feature의 비율 (Default=1.0)\n", + " * 증가할수록 성능/계산비용용 향상 & 과적합 확률 증가\n", + "\n", + "### 하이퍼 파라미터 - XGB(XGBoost, Boosting)\n", + "\n", + "* 주요 설정값\n", + " * `n_estimators` : 부스팅 단계의 수 (Default=100)\n", + " * RF와 뜻은 다르지만 양상은 비슷함\n", + " * 클수록 계산비용/성능 향상(↔과소적합). 천단위 마감\n", + " * `max_depth` : 트리의 최대 깊이 (Default=6)\n", + " * 증가할수록 과적합 위험 증대\n", + " * `Learning_rate(eta)` : 부스팅 단계에서 학습률을 조절 (Default=0.3)\n", + " * 배깅이 아닌 부스팅이므로, 학습률 개념이 있음\n", + " * 증가할수록 학습속도가 빨라지고, 초반 데이터에 가중치(+최적해를 놓칠 위험 있음)\n", + " * ↔ 낮을수록 느리지만 최적해에 안정적으로 수렴\n", + " * `subsample` : 각 단계에서 사용할 데이터 샘플[row]의 비율 (Default=1.0)\n", + " * 증가할수록 과적합방지 & 성능적 단점\n", + " * `colsample_bytree` : 각 트리를 학습할 때 사용할 feature[column]의 비율 (Default=1.0)\n", + " * 증가할수록 성능/비용 증가 & 과적합 위험\n", + " * gamma : 노드 분할시 필요한 최소 손실 감소량 (Default=0)\n", + " * 0인 경우, Loss감소가 없더라도 성능에 부정적이지 않는다면 분할\n", + " * 증가할수록 노드 분할을 엄격히 수행(과적합 방지, 성능 하향)\n", + "\n", + "### 하이퍼 파라미터 - LGBM(Light GBM)\n", + "\n", + "* 주요 설정값\n", + " * 타 모델과 유사\n", + " * `n_estimators`[Default=100], `learning_rate`[Default=0.1], `subsample`(bagging_fraction, bagging) [Default=1.0], \n", + " * Feature_fraction(`colsample_bytree`)[Default=1.0], `max_depth`[Default=-1]\n", + " * Num_leaves : 하나의 트리가 가질 수 있는 최대의 (분할이 끝난)리프 노드 수(Default=31)\n", + " * 값이 증가할수록 트리가 복잡 & 과적합 & 메모리 사용 증가\n", + " * Min_data_in_leaf(min_child_samples) : 리프 노드의 최소 샘플 수 (Default=20)\n", + "\n", + "### 하이퍼 파라미터 - Catboost\n", + "\n", + "* 주요 설정값\n", + " * 타 모델과 유사\n", + " * n_estimators[Default=100], learning_rate[Default=0.03], subsample[Default=None], Feature_fraction[Default=None]\n", + " * `Depth` : 트리의 최대 깂이 (Default=-1)\n", + " * `Bagging_temperature` : 샘플링의 무작위성을 제어 (Default=1.0)\n", + " * 높을수록 샘플링이 다양해지며 일반화 성능 향상 (속도 느려짐)\n", + " * `L2_leaf_reg` : 리프 노드의 가중치에 부여하는 패널티의 정도 (Default=3)\n", + " * 클수록 과적합 방지\n", + " " + ] + }, + { + "cell_type": "markdown", + "id": "c7545351", + "metadata": {}, + "source": [ + "## optuna\n", + "\n", + "### optuna란\n", + "\n", + "* 기존의 Tuning Tools\n", + " * Grid Search : 오래걸리지만 Grid에 적절한 값이 없으면 최적값을 찾지 못함\n", + " * Random Search : 위의 Grid Search 문제를 해결해도 최적값을 꼭 찾지 못함\n", + "* optuna vs hyperopt\n", + " * 25년 1월 기준, github star 기준으로 optuna 11.3k > hyperopt 7.3k\n", + "* optuna\n", + " * 정의한 목적함수를 기반으로 최적화를 쉽게 진행\n", + " * 베이지안 최적화와 유사한 TPE알고리즘을 사용한 탐색을 지행(그 외의 다양한 전략도 커스터마이징 가능)\n", + "\n", + "### optuna 구성요소와 작동방식\n", + "\n", + "* 구성요소\n", + " * Study : 최적화 과정 전체를 관리. 각 하이퍼 파라미터 탐색(Trial)의 결과를 저장\n", + " * Search Space : 하이퍼 파라미터의 범위 지정\n", + " * Objective function : 목적함수. 모델을 학습시키고 평가지표를 반환\n", + " * Trial : 하이퍼파라미터의 조합을 나타내는 단위 (1번의 Trial로 목적함수를 실행)\n", + "* 작동방식\n", + " * Search Space(탐색공간)에서 int/float/categorical 등을 정의\n", + " * Objective function(목적함수)를 실행해 모델학습하고 성능지표 반환\n", + " * 정의된 조합을 효율적으로 선택해, 반복적으로(Trial) 성능 개선\n", + " * 최적의 하이퍼파라미터와 정보를 Study객체에 저장\n", + "* 참고\n", + " * Search Space에서 정의한 범위에서 조합하므로, 그 밖의 범위에서 최적해가 있으면 찾을 수 없음" + ] + }, + { + "cell_type": "markdown", + "id": "bbc98d08", + "metadata": {}, + "source": [ + "## optuna 실습\n", + "\n", + "### 데이터 로딩" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e39f0450", + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.model_selection import train_test_split\n", + "from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrix, roc_auc_score, log_loss\n", + "import xgboost as xgb\n", + "import pandas as pd\n", + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "import time\n", + "\n", + "data = pd.read_csv(\"data_preprocessed.csv\")" + ] + }, + { + "cell_type": "markdown", + "id": "8316ab3d", + "metadata": {}, + "source": [ + "### LGBM Vanila모델" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "41a19793", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "정확도 : 0.703\n", + "AUC : 0.7578\n", + "Vanilla LGBM의 CF :\n", + "[[39914 16640]\n", + " [ 1629 3320]]\n" + ] + } + ], + "source": [ + "from lightgbm import LGBMClassifier\n", + "from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrix, roc_auc_score\n", + "\n", + "X = data.drop(columns=[\"TARGET\"])\n", + "y = data[\"TARGET\"]\n", + "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n", + "\n", + "# Full Model\n", + "full_model = LGBMClassifier(class_weight=\"balanced\", random_state=42)\n", + "full_model.fit(X_train, y_train)\n", + "\n", + "# Full Pred\n", + "y_pred_full = full_model.predict(X_test)\n", + "y_proba_full = full_model.predict_proba(X_test)\n", + "\n", + "# Full Results\n", + "accuracy_full = accuracy_score(y_test, y_pred_full)\n", + "auc_full = roc_auc_score(y_test, y_proba_full[:, 1])\n", + "cf_full = confusion_matrix(y_test, y_pred_full)\n", + "\n", + "print(f'정확도 : {round(accuracy_full,4)}')\n", + "print(f'AUC : {round(auc_full,4)}')\n", + "print(f'''Vanilla LGBM의 CF :\n", + "{cf_full}''')" + ] + }, + { + "cell_type": "markdown", + "id": "a7d8ae93", + "metadata": {}, + "source": [ + "### LGBM with optuna\n", + "\n", + "* 최적의 하이퍼파라미터 조합 찾기" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6be8bc0c", + "metadata": {}, + "outputs": [], + "source": [ + "import optuna\n", + "from lightgbm import LGBMClassifier, early_stopping\n", + "\n", + "# 데이터 분할\n", + "X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.1, random_state=42)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4ccd9dd6", + "metadata": {}, + "outputs": [], + "source": [ + "# 목적 함수 정의\n", + "def objective(trial):\n", + " # LGBMClassifier 하이퍼파라미터 설정\n", + " param = {\n", + " 'learning_rate': trial.suggest_float('learning_rate', 0.01, 0.2, log = True), # default = 0.1\n", + " 'num_leaves': trial.suggest_int('num_leaves', 20, 100), # default = 31\n", + " # 'max_depth': trial.suggest_int('max_depth', -1, 50), # default = -1\n", + " 'n_estimators' : trial.suggest_int('n_estimators', 100, 2000), # default = 100\n", + " 'subsample': trial.suggest_float('subsample', 0.8, 1.0), # default = 1.0\n", + " 'min_child_samples' : trial.suggest_int('min_data_in_leaf', 1, 100), # default = 20\n", + " 'colsample_bytree': trial.suggest_float('colsample_bytree', 0.8, 1.0), # default = 1.0\n", + " }\n", + "\n", + " # LGBMClassifier 모델 생성 및 학습\n", + " model = LGBMClassifier(**param, n_jobs=-1,\n", + " class_weight='balanced',\n", + " random_state=42,\n", + " # device_type='gpu', colab환경에서 정상적으로 gpu가 활용이 안되네요...이거 해결만 온종일 할 거 같아서 일단 cpu기준으로 학습합니다.\n", + " # https://stackoverflow.com/questions/75981883/can-not-use-lightgbm-gpu-in-colab-lightgbmerror-no-opencl-device-found\n", + " )\n", + " model.fit(X_train, y_train,\n", + " eval_set=[(X_val, y_val)],\n", + " eval_metric='auc',\n", + " callbacks=[early_stopping(stopping_rounds=50, verbose=False)]\n", + " )\n", + "\n", + " # 검증 데이터에 대한 예측\n", + " proba = model.predict(X_val)\n", + "\n", + " # 정확도 계산\n", + " roc_auc = roc_auc_score(y_val, proba)\n", + "\n", + " return roc_auc\n", + "\n", + "# Optuna 튜닝 실행\n", + "study = optuna.create_study(direction='maximize') # 높아질수록 좋은 roc_auc_score에 대해 maximize\n", + "study.optimize(objective, n_trials=50)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ef20e0d", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'learning_rate': 0.02642230824686883, 'num_leaves': 34, 'n_estimators': 864, 'subsample': 0.8342595493628282, 'min_data_in_leaf': 60, 'colsample_bytree': 0.8360285193709883}\n" + ] + } + ], + "source": [ + "# 최적 파라미터 출력\n", + "best_params = study.best_trial.params\n", + "print(best_params)" + ] + }, + { + "cell_type": "markdown", + "id": "f539575a", + "metadata": {}, + "source": [ + "* 찾은 조합을 활용한 final model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "55b5a6aa", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60\n", + "[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60\n", + "정확도 : 0.7229\n", + "AUC : 0.7602\n", + "Optuna LGBM의 CF :\n", + "[[41245 15309]\n", + " [ 1735 3214]]\n" + ] + } + ], + "source": [ + "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n", + "\n", + "final_model = LGBMClassifier(**best_params,\n", + " n_jobs=-1,\n", + " class_weight='balanced',\n", + " random_state=42,)\n", + "final_model.fit(X_train, y_train)\n", + "\n", + "# optuna Pred\n", + "y_pred_final = final_model.predict(X_test)\n", + "y_proba_final = final_model.predict_proba(X_test)\n", + "\n", + "# optuna Results\n", + "accuracy_final = accuracy_score(y_test, y_pred_final)\n", + "auc_final = roc_auc_score(y_test, y_proba_final[:, 1])\n", + "cf_final = confusion_matrix(y_test, y_pred_final)\n", + "\n", + "print(f'정확도 : {round(accuracy_final,4)}')\n", + "print(f'AUC : {round(auc_final,4)}')\n", + "print(f'''Optuna LGBM의 CF :\n", + "{cf_final}''')" + ] + }, + { + "cell_type": "markdown", + "id": "495aee0c", + "metadata": {}, + "source": [ + "### LGBM 결과비교 (vanila모델 vs optuna)\n", + "\n", + "* LGBM Vanila모델\n", + " * 정확도 : 0.703\n", + " * AUC : 0.7578\n", + " * Vanilla LGBM의 CF :\n", + "[[39914 16640]\n", + " [ 1629 3320]]\n", + "\n", + "* LGBM with optuna\n", + " * 정확도 : 0.7229\n", + " * AUC : 0.7602\n", + " * Optuna LGBM의 CF :\n", + "[[41245 15309]\n", + " [ 1735 3214]]" + ] + }, + { + "cell_type": "markdown", + "id": "8d617c93", + "metadata": {}, + "source": [ + "## Autogluon\n", + "\n", + "### Autogluon이란\n", + "\n", + "* Autogluon : optuna와 비슷한 방식으로 최적화하며, 최소한의 코드\n", + "\n", + "### Autogluon구성요소와 설정값\n", + "\n", + "* 구성요소\n", + " * `TablePredictor` : (분류/회귀에 사용) Tabular데이터 처리\n", + " * `TimeSeriesPredictor` : 시계열 데이터 예측\n", + " * `TextPredictor` : (연관성 분석 등) 자연어 처리\n", + " * `ImagePredictor` : 이미지 처리\n", + "* 설정값\n", + " * `Time_limit` : 학습 제한시간(Default=None)\n", + " * `Presets` : 사전 설정된 학습 전략 (best/high/good/medium quality. medium이 기본값)\n", + " * `hyperparameters` : (dict) 사용할 모델의 하이퍼파라미터들\n", + " * `Auto_stack` : 배깅 및 스택 앙상블링을 자동으로 활용할지 여부(Default=False)\n", + " * True(더 오래/정확히 학습)인 경우, `num_bag_fold` & `num_stack_levels` 자동 설정\n", + " * `num_bag_fold` : 배깅에 사용되는 폴드 수. 성능을 높이고 싶다면 5~10 사이를 권장(10이하의 값을 가짐)\n", + " * `num_stack_levels` : 스택에 사용되는 스태킹 레벨 수. 성능을 높이고 싶다면 2~3 사이를 권장(3이하의 값을 가짐)\n", + "* Leader board" + ] + }, + { + "cell_type": "markdown", + "id": "8e87ed90", + "metadata": {}, + "source": [ + "## Autogluon실습\n", + "\n", + "### 하단 코드의 실행내역 이해하기 \n", + "\n", + "* 실행내역 중 일부만 발췌함\n", + "```\n", + "# 0,1로 추정되는 값 bool 변환\n", + "Stage 1 Generators:\n", + " Fitting AsTypeFeatureGenerator...\n", + " Note: Converting 48 features to boolean dtype as they only contain 2 unique values.\n", + "\n", + "# Null값 처리\n", + "Stage 2 Generators:\n", + " Fitting FillNaFeatureGenerator...\n", + "\n", + "# 그대로 사용할 값 처리(변환X)\n", + "Stage 3 Generators:\n", + " Fitting IdentityFeatureGenerator...\n", + "\n", + "# Unique값 처리\n", + "Stage 4 Generators:\n", + " Fitting DropUniqueFeatureGenerator...\n", + "\n", + "# 중복값 처리\n", + "Stage 5 Generators:\n", + " Fitting DropDuplicatesFeatureGenerator...\n", + "\n", + "# Feature 처리(앞서 전처리한 데이터이기는 하나, autogluon이 판단하여 추가 처리)\n", + "Unused Original Features (Count: 4): ['FLAG_DOCUMENT_4', 'FLAG_DOCUMENT_7', 'FLAG_DOCUMENT_10', 'FLAG_DOCUMENT_12']\n", + " These features were not used to generate any of the output features. Add a feature generator compatible with these features to utilize them.\n", + " Features can also be unused if they carry very little information, such as being categorical but having almost entirely unique values or being duplicates of other features.\n", + " These features do not need to be present at inference time.\n", + " ('int', []) : 4 | ['FLAG_DOCUMENT_4', 'FLAG_DOCUMENT_7', 'FLAG_DOCUMENT_10', 'FLAG_DOCUMENT_12']\n", + "\n", + "# 사용할 모델\n", + "User-specified model hyperparameters to be fit:\n", + "{\n", + "\t'NN_TORCH': [{}],\n", + "\t'GBM': [{'extra_trees': True, 'ag_args': {'name_suffix': 'XT'}}, {}, {'learning_rate': 0.03, 'num_leaves': 128, 'feature_fraction': 0.9, 'min_data_in_leaf': 3, 'ag_args': {'name_suffix': 'Large', 'priority': 0, 'hyperparameter_tune_kwargs': None}}],\n", + "\t'CAT': [{}],\n", + "\t'XGB': [{}],\n", + "\t'FASTAI': [{}],\n", + "\t'RF': [{'criterion': 'gini', 'ag_args': {'name_suffix': 'Gini', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'entropy', 'ag_args': {'name_suffix': 'Entr', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'squared_error', 'ag_args': {'name_suffix': 'MSE', 'problem_types': ['regression', 'quantile']}}],\n", + "\t'XT': [{'criterion': 'gini', 'ag_args': {'name_suffix': 'Gini', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'entropy', 'ag_args': {'name_suffix': 'Entr', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'squared_error', 'ag_args': {'name_suffix': 'MSE', 'problem_types': ['regression', 'quantile']}}],\n", + "\t'KNN': [{'weights': 'uniform', 'ag_args': {'name_suffix': 'Unif'}}, {'weights': 'distance', 'ag_args': {'name_suffix': 'Dist'}}],\n", + "}\n", + "\n", + "# sequential하게 trial 진행\n", + "Fitting 13 L1 models, fit_strategy=\"sequential\" ...\n", + "Fitting model: KNeighborsUnif ... Training model for up to 3595.93s of the 3595.92s of remaining time.\n", + "\t0.5271\t = Validation score (roc_auc)\n", + "\t1.62s\t = Training runtime\n", + "\t50.83s\t = Validation runtime\n", + "Fitting model: KNeighborsDist ... Training model for up to 3543.23s of the 3543.23s of remaining time.\n", + "\t0.5298\t = Validation score (roc_auc)\n", + "\t0.63s\t = Training runtime\n", + "\t49.39s\t = Validation runtime\n", + "...\n", + "\n", + "# 실행시간 및 best model\n", + "AutoGluon training complete, total runtime = 1561.52s ... Best model: WeightedEnsemble_L2 | Estimated inference throughput: 4907.4 rows/s (24601 batch size)\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "36c13813", + "metadata": {}, + "outputs": [], + "source": [ + "from autogluon.tabular import TabularPredictor\n", + "from sklearn.model_selection import train_test_split\n", + "from sklearn.metrics import roc_auc_score\n", + "import pandas as pd\n", + "\n", + "path = 'autogluon_results' # 저장할 경로로\n", + "\n", + "# 데이터 분할 (이전 코드에서 X_train, y_train, X_val, y_val, X_test, y_test를 준비했다고 가정)\n", + "\n", + "# 데이터프레임으로 변환\n", + "train_data = X_train\n", + "train_data['target'] = y_train\n", + "\n", + "val_data = X_val\n", + "val_data['target'] = y_val\n", + "\n", + "# AutoGluon 학습\n", + "predictor = TabularPredictor(label='target', eval_metric='roc_auc', verbosity=2, path=path)\n", + "predictor.fit(train_data, tuning_data=val_data, time_limit=3600) # 시간 제한 설정 (1시간)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1a0754be", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "정확도 : 0.9199\n", + "AUC : 0.761\n", + "autogluon LGBM의 CF : \n", + "[[56492 62]\n", + " [ 4864 85]]\n" + ] + } + ], + "source": [ + "# 예측 및 평가\n", + "test_data = X_test\n", + "test_data['target'] = y_test\n", + "\n", + "y_pred_proba_ag = predictor.predict_proba(test_data)\n", + "y_pred_ag = predictor.predict(test_data)\n", + "\n", + "# optuna Results\n", + "accuracy_ag = accuracy_score(y_test, y_pred_ag)\n", + "auc_ag = roc_auc_score(y_test, y_pred_proba_ag.iloc[:, 1])\n", + "cf_ag = confusion_matrix(y_test, y_pred_ag)\n", + "\n", + "print(f'정확도 : {round(accuracy_ag,4)}')\n", + "print(f'AUC : {round(auc_ag,4)}')\n", + "print(f'''autogluon LGBM의 CF :\n", + "{cf_ag}''')" + ] + }, + { + "cell_type": "markdown", + "id": "31c97ed3", + "metadata": {}, + "source": [ + "### 결과 비교하기 (optuna vs autogluon)\n", + "\n", + "* LGBM with optuna\n", + " * 정확도 : 0.7229\n", + " * AUC : 0.7602\n", + " * Optuna LGBM의 CF :\n", + " [[41245 15309]\n", + " [ 1735 3214]]\n", + "\n", + "* Autogluon\n", + " * 정확도 : 0.9199\n", + " * AUC : 0.761\n", + " * autogluon LGBM의 CF : \n", + "[[56492 62]\n", + " [ 4864 85]]" + ] + }, + { + "cell_type": "markdown", + "id": "5779a4c3", + "metadata": {}, + "source": [ + "### Autogluon leaderboard\n", + "\n", + "* Leaderboard : Autogluon이 학습했던 모델들의 Score확인 가능\n", + " * 각 모델의 score나 time 등을 확인 가능" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dfc43063", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " model score_val eval_metric pred_time_val fit_time \\\n", + "0 WeightedEnsemble_L2 0.754022 roc_auc 5.013031 663.924621 \n", + "1 CatBoost 0.753185 roc_auc 0.050006 108.363011 \n", + "2 LightGBM 0.750591 roc_auc 0.188981 20.328846 \n", + "3 XGBoost 0.749953 roc_auc 0.238468 74.874593 \n", + "4 LightGBMXT 0.749184 roc_auc 0.669894 49.791667 \n", + "5 LightGBMLarge 0.748378 roc_auc 0.653803 39.147023 \n", + "6 NeuralNetTorch 0.740312 roc_auc 0.203650 140.524567 \n", + "7 NeuralNetFastAI 0.737314 roc_auc 0.325043 136.662297 \n", + "8 RandomForestEntr 0.731284 roc_auc 1.649255 304.078007 \n", + "9 ExtraTreesEntr 0.722843 roc_auc 1.512995 135.927222 \n", + "10 ExtraTreesGini 0.722248 roc_auc 1.357874 120.171119 \n", + "11 RandomForestGini 0.720812 roc_auc 2.502675 288.912408 \n", + "12 KNeighborsDist 0.529842 roc_auc 49.391568 0.634750 \n", + "13 KNeighborsUnif 0.527099 roc_auc 50.827020 1.620968 \n", + "\n", + " pred_time_val_marginal fit_time_marginal stack_level can_infer \\\n", + "0 0.005133 1.482976 2 True \n", + "1 0.050006 108.363011 1 True \n", + "2 0.188981 20.328846 1 True \n", + "3 0.238468 74.874593 1 True \n", + "4 0.669894 49.791667 1 True \n", + "5 0.653803 39.147023 1 True \n", + "6 0.203650 140.524567 1 True \n", + "7 0.325043 136.662297 1 True \n", + "8 1.649255 304.078007 1 True \n", + "9 1.512995 135.927222 1 True \n", + "10 1.357874 120.171119 1 True \n", + "11 2.502675 288.912408 1 True \n", + "12 49.391568 0.634750 1 True \n", + "13 50.827020 1.620968 1 True \n", + "\n", + " fit_order \n", + "0 14 \n", + "1 7 \n", + "2 4 \n", + "3 11 \n", + "4 3 \n", + "5 13 \n", + "6 12 \n", + "7 10 \n", + "8 6 \n", + "9 9 \n", + "10 8 \n", + "11 5 \n", + "12 2 \n", + "13 1 \n" + ] + } + ], + "source": [ + "# 최적 모델 요약 출력\n", + "print(predictor.leaderboard())" + ] + }, + { + "cell_type": "markdown", + "id": "9f8703b1", + "metadata": {}, + "source": [ + "## paramater를 직접 지정하기\n", + "- Autogluon은 이렇게 기본으로 사용해도 훌륭한 성능을 보여줍니다. 자동으로 파라미터를 지정하고, 앙상블까지 해주죠.\n", + "- 이런 베이스 모델을 생성한 뒤에는, 쓸모 없거나 너무 무거운 모델은 제거하고 다시 학습시킬 수도 있습니다.\n", + "- LGBM, XGB, CAT 세 개가 성능도 괜찮고 학습시간도 짧네요. 이거 세 개만 써봅시다.\n", + "- 위에서 Optuna에서 찾은 파라미터를 여기서 사용하실 수도 있겠죠?\n", + "\n", + "### Autogluon 하이퍼파라미터 설정\n", + "\n", + "* 베이스 모델을 생성한 뒤, 쓸모 없거나 너무 무거운 모델을 제거하고 학습 가능(Leaderboard로 확인)\n", + "* 위의 Leaderboard 결과에서, LGBM/XGB/CAT을 가지고 하이퍼파라미터를 설정" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d133882c", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Warning: path already exists! This predictor may overwrite an existing predictor! path=\"drive/MyDrive/Metacode/Week6/selected_model\"\n", + "Verbosity: 2 (Standard Logging)\n", + "=================== System Info ===================\n", + "AutoGluon Version: 1.2\n", + "Python Version: 3.11.11\n", + "Operating System: Linux\n", + "Platform Machine: x86_64\n", + "Platform Version: #1 SMP PREEMPT_DYNAMIC Thu Jun 27 21:05:47 UTC 2024\n", + "CPU Count: 2\n", + "Memory Avail: 8.70 GB / 12.67 GB (68.6%)\n", + "Disk Space Avail: 0.50 GB / 15.00 GB (3.3%)\n", + "\tWARNING: Available disk space is low and there is a risk that AutoGluon will run out of disk during fit, causing an exception. \n", + "\tWe recommend a minimum available disk space of 10 GB, and large datasets may require more.\n", + "===================================================\n", + "No presets specified! To achieve strong results with AutoGluon, it is recommended to use the available presets. Defaulting to `'medium'`...\n", + "\tRecommended Presets (For more details refer to https://auto.gluon.ai/stable/tutorials/tabular/tabular-essentials.html#presets):\n", + "\tpresets='experimental' : New in v1.2: Pre-trained foundation model + parallel fits. The absolute best accuracy without consideration for inference speed. Does not support GPU.\n", + "\tpresets='best' : Maximize accuracy. Recommended for most users. Use in competitions and benchmarks.\n", + "\tpresets='high' : Strong accuracy with fast inference speed.\n", + "\tpresets='good' : Good accuracy with very fast inference speed.\n", + "\tpresets='medium' : Fast training time, ideal for initial prototyping.\n", + "Beginning AutoGluon training ... Time limit = 3600s\n", + "AutoGluon will save models to \"/content/drive/MyDrive/Metacode/Week6/selected_model\"\n", + "Train Data Rows: 246008\n", + "Train Data Columns: 131\n", + "Tuning Data Rows: 24601\n", + "Tuning Data Columns: 131\n", + "Label Column: target\n", + "AutoGluon infers your prediction problem is: 'binary' (because only two unique label-values observed).\n", + "\t2 unique label values: [0, 1]\n", + "\tIf 'binary' is not the correct problem_type, please manually specify the problem_type parameter during Predictor init (You may specify problem_type as one of: ['binary', 'multiclass', 'regression', 'quantile'])\n", + "Problem Type: binary\n", + "Preprocessing data ...\n", + "Selected class <--> label mapping: class 1 = 1, class 0 = 0\n", + "Using Feature Generators to preprocess the data ...\n", + "Fitting AutoMLPipelineFeatureGenerator...\n", + "\tAvailable Memory: 8776.27 MB\n", + "\tTrain Data (Original) Memory Usage: 270.46 MB (3.1% of available memory)\n", + "\tInferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.\n", + "\tStage 1 Generators:\n", + "\t\tFitting AsTypeFeatureGenerator...\n", + "\t\t\tNote: Converting 48 features to boolean dtype as they only contain 2 unique values.\n", + "\tStage 2 Generators:\n", + "\t\tFitting FillNaFeatureGenerator...\n", + "\tStage 3 Generators:\n", + "\t\tFitting IdentityFeatureGenerator...\n", + "\tStage 4 Generators:\n", + "\t\tFitting DropUniqueFeatureGenerator...\n", + "\tStage 5 Generators:\n", + "\t\tFitting DropDuplicatesFeatureGenerator...\n", + "\tUnused Original Features (Count: 6): ['FLAG_DOCUMENT_4', 'FLAG_DOCUMENT_7', 'FLAG_DOCUMENT_10', 'FLAG_DOCUMENT_12', 'FLAG_DOCUMENT_17', 'CODE_GENDER_XNA']\n", + "\t\tThese features were not used to generate any of the output features. Add a feature generator compatible with these features to utilize them.\n", + "\t\tFeatures can also be unused if they carry very little information, such as being categorical but having almost entirely unique values or being duplicates of other features.\n", + "\t\tThese features do not need to be present at inference time.\n", + "\t\t('float', []) : 1 | ['CODE_GENDER_XNA']\n", + "\t\t('int', []) : 5 | ['FLAG_DOCUMENT_4', 'FLAG_DOCUMENT_7', 'FLAG_DOCUMENT_10', 'FLAG_DOCUMENT_12', 'FLAG_DOCUMENT_17']\n", + "\tTypes of features in original data (raw dtype, special dtypes):\n", + "\t\t('float', []) : 88 | ['AMT_INCOME_TOTAL', 'AMT_CREDIT', 'AMT_ANNUITY', 'AMT_GOODS_PRICE', 'REGION_POPULATION_RELATIVE', ...]\n", + "\t\t('int', []) : 37 | ['SK_ID_CURR', 'CNT_CHILDREN', 'DAYS_BIRTH', 'DAYS_ID_PUBLISH', 'FLAG_MOBIL', ...]\n", + "\tTypes of features in processed data (raw dtype, special dtypes):\n", + "\t\t('float', []) : 69 | ['AMT_INCOME_TOTAL', 'AMT_CREDIT', 'AMT_ANNUITY', 'AMT_GOODS_PRICE', 'REGION_POPULATION_RELATIVE', ...]\n", + "\t\t('int', []) : 14 | ['SK_ID_CURR', 'CNT_CHILDREN', 'DAYS_BIRTH', 'DAYS_ID_PUBLISH', 'HOUR_APPR_PROCESS_START', ...]\n", + "\t\t('int', ['bool']) : 42 | ['FLAG_MOBIL', 'FLAG_EMP_PHONE', 'FLAG_WORK_PHONE', 'FLAG_CONT_MOBILE', 'FLAG_PHONE', ...]\n", + "\t4.1s = Fit runtime\n", + "\t125 features in original data used to generate 125 features in processed data.\n", + "\tTrain Data (Processed) Memory Usage: 182.20 MB (2.1% of available memory)\n", + "Data preprocessing and feature engineering runtime = 4.62s ...\n", + "AutoGluon will gauge predictive performance using evaluation metric: 'roc_auc'\n", + "\tThis metric expects predicted probabilities rather than predicted class labels, so you'll need to use predict_proba() instead of predict()\n", + "\tTo change this, specify the eval_metric parameter of Predictor()\n", + "User-specified model hyperparameters to be fit:\n", + "{\n", + "\t'GBM': [{'learning_rate': 0.02642230824686883, 'num_leaves': 34, 'n_estimators': 864, 'subsample': 0.8342595493628282, 'min_data_in_leaf': 60, 'colsample_bytree': 0.8360285193709883, 'n_jobs': -1, 'class_weight': 'balanced'}],\n", + "\t'CAT': [{'scale_pos_weight': 11.377138257194607}],\n", + "\t'XGB': [{'scale_pos_weight': 11.377138257194607}],\n", + "}\n", + "Fitting 3 L1 models, fit_strategy=\"sequential\" ...\n", + "Fitting model: LightGBM ... Training model for up to 3595.38s of the 3595.38s of remaining time.\n", + "/usr/local/lib/python3.11/dist-packages/lightgbm/engine.py:204: UserWarning: Found `n_estimators` in params. Will use it instead of argument\n", + " _log_warning(f\"Found `{alias}` in params. Will use it instead of argument\")\n", + "\t0.8437\t = Validation score (roc_auc)\n", + "\t93.76s\t = Training runtime\n", + "\t2.52s\t = Validation runtime\n", + "Fitting model: CatBoost ... Training model for up to 3498.78s of the 3498.77s of remaining time.\n", + "\t0.9877\t = Validation score (roc_auc)\n", + "\t1941.71s\t = Training runtime\n", + "\t0.38s\t = Validation runtime\n", + "Fitting model: XGBoost ... Training model for up to 1556.46s of the 1556.46s of remaining time.\n", + "\t0.9982\t = Validation score (roc_auc)\n", + "\t1556.76s\t = Training runtime\n", + "\t11.44s\t = Validation runtime\n", + "Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.00s of the -11.97s of remaining time.\n", + "\tEnsemble Weights: {'XGBoost': 1.0}\n", + "\t0.9982\t = Validation score (roc_auc)\n", + "\t0.35s\t = Training runtime\n", + "\t0.0s\t = Validation runtime\n", + "AutoGluon training complete, total runtime = 3614.24s ... Best model: WeightedEnsemble_L2 | Estimated inference throughput: 2149.4 rows/s (24601 batch size)\n", + "TabularPredictor saved. To load, use: predictor = TabularPredictor.load(\"/content/drive/MyDrive/Metacode/Week6/selected_model\")\n" + ] + }, + { + "data": { + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from autogluon.tabular import TabularPredictor\n", + "from sklearn.model_selection import train_test_split\n", + "from sklearn.metrics import roc_auc_score\n", + "import pandas as pd\n", + "import os\n", + "\n", + "from autogluon.tabular import TabularPredictor\n", + "from sklearn.model_selection import train_test_split\n", + "from sklearn.metrics import roc_auc_score\n", + "import pandas as pd\n", + "\n", + "path = 'drive/MyDrive/Metacode/Week6/selected_model'\n", + "os.makedirs(path, exist_ok=True)\n", + "\n", + "train_data = X_train\n", + "train_data['target'] = y_train\n", + "\n", + "val_data = X_val\n", + "val_data['target'] = y_val\n", + "\n", + "scale_pos_weight = len(y_train[y_train == 0]) / len(y_train[y_train == 1]) # 불균형 데이터 가중치 설정\n", + "\n", + "# 모델별 하이퍼파라미터 지정\n", + "custom_hyperparameters = {\n", + " 'GBM': {\n", + " 'learning_rate': 0.02642230824686883,\n", + " 'num_leaves': 34,\n", + " 'n_estimators': 864,\n", + " 'subsample': 0.8342595493628282,\n", + " 'min_data_in_leaf': 60,\n", + " 'colsample_bytree': 0.8360285193709883,\n", + " 'n_jobs':-1,\n", + " 'class_weight':'balanced'\n", + " },\n", + " 'CAT': {\n", + " 'scale_pos_weight': scale_pos_weight\n", + " },\n", + " 'XGB': {\n", + " 'scale_pos_weight': scale_pos_weight\n", + " }\n", + "}\n", + "\n", + "# AutoGluon 학습\n", + "predictor = TabularPredictor(label='target', eval_metric='roc_auc', verbosity=2, path=path)\n", + "predictor.fit(\n", + " train_data,\n", + " tuning_data=val_data,\n", + " time_limit=3600,\n", + " hyperparameters=custom_hyperparameters\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "58b942ee", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "정확도 : 0.8675\n", + "AUC : 0.7107\n", + "autogluon CF :\n", + "[[51892 4662]\n", + " [ 3486 1463]]\n" + ] + } + ], + "source": [ + "# 예측 및 평가\n", + "test_data = X_test\n", + "test_data['target'] = y_test\n", + "\n", + "y_pred_proba_ag = predictor.predict_proba(test_data)\n", + "y_pred_ag = predictor.predict(test_data)\n", + "\n", + "# optuna Results\n", + "accuracy_ag = accuracy_score(y_test, y_pred_ag)\n", + "auc_ag = roc_auc_score(y_test, y_pred_proba_ag.iloc[:, 1])\n", + "cf_ag = confusion_matrix(y_test, y_pred_ag)\n", + "\n", + "print(f'정확도 : {round(accuracy_ag,4)}')\n", + "print(f'AUC : {round(auc_ag,4)}')\n", + "print(f'''autogluon CF :\n", + "{cf_ag}''')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8025e543", + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"predictor\",\n \"rows\": 4,\n \"fields\": [\n {\n \"column\": \"model\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 4,\n \"samples\": [\n \"WeightedEnsemble_L2\",\n \"LightGBM\",\n \"XGBoost\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"score_val\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.0756612918828574,\n \"min\": 0.843690128482574,\n \"max\": 0.9981761541420456,\n \"num_unique_values\": 3,\n \"samples\": [\n 0.9981761541420456,\n 0.9877217974956707,\n 0.843690128482574\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"eval_metric\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"roc_auc\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pred_time_val\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 5.833938564108144,\n \"min\": 0.3849499225616455,\n \"max\": 11.445684909820557,\n \"num_unique_values\": 4,\n \"samples\": [\n 11.445684909820557\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fit_time\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 816.1285596253546,\n \"min\": 93.75901937484741,\n \"max\": 1941.7070152759552,\n \"num_unique_values\": 4,\n \"samples\": [\n 1557.1112411022186\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pred_time_val_marginal\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 5.351480742903106,\n \"min\": 0.00453639030456543,\n \"max\": 11.441148519515991,\n \"num_unique_values\": 4,\n \"samples\": [\n 0.00453639030456543\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fit_time_marginal\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 995.9715188903901,\n \"min\": 0.3473982810974121,\n \"max\": 1941.7070152759552,\n \"num_unique_values\": 4,\n \"samples\": [\n 0.3473982810974121\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"stack_level\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 1,\n \"max\": 2,\n \"num_unique_values\": 2,\n \"samples\": [\n 2\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"can_infer\",\n \"properties\": {\n \"dtype\": \"boolean\",\n \"num_unique_values\": 1,\n \"samples\": [\n true\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fit_order\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 4,\n \"num_unique_values\": 4,\n \"samples\": [\n 4\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
modelscore_valeval_metricpred_time_valfit_timepred_time_val_marginalfit_time_marginalstack_levelcan_inferfit_order
0XGBoost0.998176roc_auc11.4411491556.76384311.4411491556.7638431True3
1WeightedEnsemble_L20.998176roc_auc11.4456851557.1112410.0045360.3473982True4
2CatBoost0.987722roc_auc0.3849501941.7070150.3849501941.7070151True2
3LightGBM0.843690roc_auc2.51918993.7590192.51918993.7590191True1
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "\n", + "
\n", + "
\n" + ], + "text/plain": [ + " model score_val eval_metric pred_time_val fit_time \\\n", + "0 XGBoost 0.998176 roc_auc 11.441149 1556.763843 \n", + "1 WeightedEnsemble_L2 0.998176 roc_auc 11.445685 1557.111241 \n", + "2 CatBoost 0.987722 roc_auc 0.384950 1941.707015 \n", + "3 LightGBM 0.843690 roc_auc 2.519189 93.759019 \n", + "\n", + " pred_time_val_marginal fit_time_marginal stack_level can_infer \\\n", + "0 11.441149 1556.763843 1 True \n", + "1 0.004536 0.347398 2 True \n", + "2 0.384950 1941.707015 1 True \n", + "3 2.519189 93.759019 1 True \n", + "\n", + " fit_order \n", + "0 3 \n", + "1 4 \n", + "2 2 \n", + "3 1 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "predictor.leaderboard()" + ] + }, + { + "cell_type": "markdown", + "id": "75c19626", + "metadata": {}, + "source": [ + "### 결과 비교하기 (optuna vs autogluon)\n", + "\n", + "* Autogluon기본\n", + " * 정확도 : 0.9199\n", + " * AUC : 0.761\n", + " * autogluon LGBM의 CF : \n", + "[[56492 62]\n", + " [ 4864 85]]\n", + "\n", + "* Autogluon 일부 모델만 추려낸 것 (낮아짐)\n", + " * 정확도 : 0.8675\n", + " * AUC : 0.7107\n", + " * autogluon CF :\n", + "[[51892 4662]\n", + " [ 3486 1463]]" + ] + }, + { + "cell_type": "markdown", + "id": "23309b50", + "metadata": {}, + "source": [ + "### 기타\n", + "\n", + "* 다른 AutoML로 pycaret도 있으나, Autogluon이 더 좋음\n", + " * 코드 한줄정도로 구현이 가능하지만, Autogluon성능이 더 좋았음\n", + "* 실무적으로는, **medium quality세팅으로 리더보드를 보면서, feature engineering(파생변수 생성 등)을 진행하게 됨**\n", + " * 성능의 영향은 데이터(GIGO). 하이퍼파라미터 튜닝 등 보다는 **데이터에 집중**\n", + " * EDA를 하다보면, 추가 할법한 (파생)변수가 보이기도 함\n", + " * SHAP를 확인하다보면 넣으면 좋을 것 같은 변수가 보이기도 함" + ] + }, + { + "cell_type": "markdown", + "id": "b8ea13f6", + "metadata": {}, + "source": [ + "# cuML 도커에서 설치하기\n", + "- https://rapids.ai/#quick-start" + ] + }, + { + "cell_type": "markdown", + "id": "7fee6a38", + "metadata": {}, + "source": [ + "# 과제 : 최종과제 준비하기\n", + "\n", + "* 구성 샘플\n", + " * 과제 목표\n", + " * 학습데이터\n", + " * EDA\n", + " * SHAP\n", + " * 학습모델\n", + " * 메트릭\n", + " * 리더보드\n", + " * appendix(hyper)\n", + "* 인당 15분 발표 + 5분 질문\n", + "* 데이콘에 결과 제출해보기 (+private score공유)\n", + "* 데이콘 우승자 발표 참고하면 좋음" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.4" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}