Kuyini Okungavamile Kwezibalo
Kuzibalo, idatha iwumthombo oyinhloko wokuqonda izimo: ukuziphatha kwabathengi, imiphumela yokuhlolwa, impilo yesiguli, ikhwalithi yokukhiqiza, ngisho nemikhuba yezomnotho. Kodwa-ke, akuwona wonke amaphuzu edatha "aziphatha" njengeningi. Ngezinye izikhathi sihlangana nenani elilodwa noma ngaphezulu elibonakala lihluke kakhulu kulo lonke isethi yedatha. Lawa ma-outliers aziwa ngokuthi ama-outliers. Ukuqonda ama-outliers kubalulekile ngoba angashintsha iziphetho zokuhlaziya, athonye amamodeli okubikezela, futhi abonise ngisho nemicimbi ebalulekile okufanele ihlolwe.
Ukuqonda Izinto Ezingaphandle
Kalula nje, i-outlier iyinani lokubona noma ledatha elihlukile kakhulu kuningi ledatha. Ama-Outliers angaba phezulu (phezulu kakhulu) noma aphansi (aphansi kakhulu) kunephethini evamile. Isibonelo, uma iningi lamaphuzu okuhlolwa kwabafundi ewela ebangeni lama-60-90, futhi amaphuzu elilodwa lama-5 noma ayi-100 ehluka kakhulu, lelo phuzu kufanele lisolwe njengelingaphandle.
Kubalulekile ukugcizelela: i-outlier ayisho njalo "iphutha." I-outlier imane ikhombisa ukuthi inani alivamile uma liqhathaniswa nesethi yedatha. I-Outliers ingavela emaphutheni okufaka, amathuluzi okulinganisa anephutha, noma imane ibonise izehlakalo ezingavamile kodwa ezibalulekile emhlabeni wangempela.
Isibonelo esilula
Cabanga ngemininingwane yemali engenayo yanyanga zonke (ngezigidi zama-rupiah) yabantu abayi-10:
5, 5, 6, 6, 6, 7, 7, 7, 8, 50
Lapha, inombolo engu-50 ivelele kakhulu uma iqhathaniswa nezinye. Ingabe u-50 uyiphutha? Kungaba ukufunda okungalungile (kufanele kube u-5,0), kodwa futhi kungaba yiqiniso ngoba umuntu ungumnikazi webhizinisi elikhulu. Kunoma ikuphi, u-50 usalokhu engavamile—okuhlukile ukuthi siwuphatha kanjani ekuhlaziyeni.
Kungani Kuvela Izinto Ezingaphandle?
Kunezimbangela eziningana ezivamile zokuvela kwezinto ezingaphandle:
1. Iphutha lokulinganisa noma ithuluzi
Isibonelo, inzwa yokushisa ingase ifunde amanani aphakeme ngezinye izikhathi ngenxa yokuphazamiseka.
2. Amaphutha ekurekhodeni noma ekufakeni idatha
Izibonelo zakudala: ukuthayipha u-1000 esikhundleni sika-100, noma amayunithi angalungile (cm vs m).
3. Ukwehlukahluka kwemvelo
Ezweni langempela, kunezimo ezingavamile kodwa ezingokoqobo: ukwanda kokuthengisa ngenxa yokukhushulwa okukhulu, isiguli siba nempendulo engavamile kumuthi, noma umsubathi ubeka irekhodi elibi kakhulu.
4. Izinguquko enqubweni noma ezimweni
Isibonelo, ifektri ihlangabezana nokwehluleka komshini ngosuku oluthile, okwenza inani lemikhiqizo enephutha lande kakhulu.
5. Imiphakathi exubile
Isethi yedatha ingaba namaqembu amaningana ahlukene ahlanganisiwe. Isibonelo, ukuphakama kwabafundi besikole samabanga aphakathi nabasekolishi kuxubile; amanye amanani "aphezulu" angase avele hhayi ngenxa yezimo ezingavamile, kodwa ngoba amaqembu ahluke ngempela.
Umthelela Wezinto Ezingaphandle Ekuhlaziyweni Kwezibalo
Izinto ezingaphandle zibalulekile ngoba zingathinta kakhulu imiphumela yokuhlaziywa, ikakhulukazi ezindleleni ezizwela amanani aphakeme kakhulu.
1. Kuthinta isilinganiso (isilinganiso)
Isilinganiso "sidonswa" kalula yinani elibi kakhulu. Esibonelweni semali engenayo esingenhla, isilinganiso singakhushulwa kakhulu ngenani elingu-50, yize iningi licishe libe ngu-5–8.
2. Kuthinta ukuphambuka okujwayelekile kanye nokwehluka
Ngenxa yokuthi ukubalwa kokwehluka kuhilela umehluko oyisikwele kusuka ku-mean, ama-outliers angakhulisa ukwehluka kanye nokuphambuka okujwayelekile, okwenza kubonakale sengathi idatha "isakazeke" kakhulu kunalokho eyikho ngempela.
3. Amamodeli okubuyela emuva okuphazamisayo kanye nokufunda komshini
Ekubuyeleni emuva okuqondile, ama-outliers angasonta umugqa wokubuyela emuva, okwenza izibikezelo zibe zimbi kakhulu kuningi lwedatha. Kwezinye izindlela zokwenza izinto, ama-outliers angabangela ukuthi imodeli ilingane kakhulu noma ithinte amapharamitha okuqeqesha.
4. Ukuhlolwa kwe-hypothesis yethonya
Ama-Outliers angaphula izinkolelo zokujwayelekile kanye nokulingana kokuhlukahluka okuvame ukusetshenziswa ezivivinyweni ze-parametric, ukuze iziphetho zezibalo zibe nobandlululo.
Kodwa-ke, izinto ezingaphandle nazo zingaba izimpawu ezibalulekile. Ekutholakaleni kokukhwabanisa, ukuthengiselana okungaphandle yilokho kanye esikufunayo. Ekunakekelweni kwezempilo, imiphumela yelabhorethri ehluke kakhulu ingabonisa isimo sezokwelapha esibucayi.
Indlela Yokuthola Izinto Ezingaphandle
Ayikho indlela eyodwa "efanele". Ukutholwa kwangaphandle kuvame ukuncika kumongo, uhlobo lwedatha, kanye nezinhloso zokuhlaziya. Nazi ezinye izindlela ezivamile:
1. Ukubona ngeso lengqondo: Ibhokisi lebhokisi kanye ne-Scatterplot
– Ama-boxplot athandwa kakhulu ngokubona ama-outliers. E-boxplot, ama-outliers avame ukuphawulwa njengamaphuzu awela ngaphandle kwebhokisi lama-whisker.
– Ama-Scatterplot asiza ukubona izinto ezingaphandle kobudlelwano phakathi kwezinto ezimbili eziguquguqukayo, isibonelo isisindo vs ukuphakama.
Ukubona ngeso lengqondo kuyasiza njengesinyathelo sokuqala ngoba kuyashesha futhi kuyashesha.
2. Indlela ye-IQR (Interquartile Range)
Indlela ye-IQR ivame ukusetshenziselwa idatha eguquguqukayo eyodwa (i-univariate).
– Bala u-Q1 (ikota 1) kanye no-Q3 (ikota 3)
– IQR = Q3 − Q1
– Umkhawulo ophansi = Q1 − 1,5 × IQR
– Umkhawulo ophezulu = Q3 + 1,5 × IQR
Amanani angaphandle kwale mingcele avame ukubhekwa njengangaphandle. Le ndlela iqinile ngoba ayithinteki kakhulu yimanani aqine kakhulu.
3. Isikolo sika-Z (ngokusekelwe ekuphambukeni okumaphakathi kanye nokujwayelekile)
I-Z-score ilinganisa ukuthi inani likude kangakanani nesilinganiso kumayunithi okuphambuka okujwayelekile.
– z = (x − isilinganiso) / sd
Amanani ane-|z| > 3 (ngezinye izikhathi > 2,5) avame ukubhekwa njengangabalulekile.
Ubuthakathaka: uma idatha isivele iqukethe izinto ezinkulu ezingaphandle, i-mean kanye ne-sd ziyathinteka ukuze ukutholwa kungabi nembambo kakhulu.
4. Izindlela ezisekelwe kumodeli kanye nezindlela eziningi
Kudatha eguquguqukayo eminingi, ama-outliers awabonakali njalo kukholomu eyodwa kuphela, kodwa ngokuhlanganiswa kweziguquguquko eziningana.
– Ibanga le-Mahalanobis livame ukusetshenziswa ukuthola ama-multivariate outliers.
– Ekufundeni komshini, kunezindlela ezifana ne-Isolation Forest, i-Local Outlier Factor (LOF), noma i-One-Class SVM.
Le ndlela ifanelekela amasethi edatha amakhulu nayinkimbinkimbi, isibonelo ukutholwa kwezinkinga zokuthengiselana kwezezimali.
Yini Okufanele Uyenze Uma Uthola Umuntu Ongajwayelekile?
Inkambo engcono kakhulu yokwenza akusikho ukuzisusa nje. Ngokuvamile, inqubo yokusingatha engaphandle ihilela izinyathelo eziningana:
1. Ukuqinisekiswa kwedatha
– Hlola ukuthi akukho yini okufakiwe okungalungile, okukopishiwe, noma amayunithi angalungile.
– Qhathanisa nomthombo wedatha wokuqala (isib. amafomu, izingodo zezinzwa, noma amarekhodi enziwe ngesandla).
2. Qonda umongo
– Ingabe amanani aqatha anengqondo kulesi sizinda?
– Isibonelo, izinga lokushisa lomzimba womuntu elingu-50°C cishe alilungile; kodwa imali engenayo eyizigidi ezingu-50 ingaba yinto efanelekile.
3. Nquma inhloso yokuhlaziywa
– Uma umgomo uwukuqonda ukuziphatha “okuvamile”, izinto ezingaphandle kungadingeka ziphathwe ukuze kuvinjelwe ukuthi zingabusi.
– Uma umgomo uwukuthola izehlakalo ezingavamile (ukukhwabanisa, ukwehluleka komshini), izinto ezingaphandle yizona ezigxilwe kakhulu.
4. Khetha isu lokuphatha
Ezinye izinketho ezivamile:
– Ukususwa: kwenziwa uma okungaphandle kufakazelwe ukuthi kuyiphutha futhi akumeleli.
– Ukuguqulwa kwedatha: isibonelo ukuguqulwa kwelogi kwedatha egobile.
– Ukuhlanganisa/ukubeka uphawu: ukukhawulela amanani adlulele kuma-percentile athile (isb. p1 kanye ne-p99).
– Sebenzisa izindlela eziqinile: i-median, i-IQR, i-robust regression, noma amamodeli amelana ne-outlier.
– Ukuhlaziywa okuhlukile: ngezinye izikhathi kufaneleka kakhulu ukuhlaziya izinto ezingaphandle njengezimo ezikhethekile.
Okubalulekile, izinqumo kumele zibhalwe phansi ukuze ukuhlaziya kube sobala futhi kube nomthwalo wemfanelo.
Izinto Ezingaphandle: Inkinga Noma Ulwazi Olubalulekile?
Ama-Outliers avame ukubhekwa “njengomsindo” ngoba angaphambukisa izifinyezo zezibalo. Kodwa-ke, ezimweni eziningi, ama-outliers ayisango lokuqonda okusha: ukuba khona kwengxenye yamakhasimende esezingeni eliphezulu, isimo sesiguli esidinga ukunakwa, isigaba esisha enkambisweni yokukhiqiza, noma ukukhwabanisa okungenzeka. Ngakho-ke, ama-outliers kufanele aphathwe njengento edinga ukuphenywa, hhayi ukuxoshwa ngokuzenzakalelayo.
Isiphetho
I-outlier kuzibalo yinani elihluka kakhulu kuphethini evamile yedatha. Ama-outliers angavela ngenxa yamaphutha, ukuhlukahluka kwemvelo, izinguquko zenqubo, noma umehluko weqembu ngaphakathi kwesethi yedatha. Umthelela wawo ungaba mkhulu kuma-mean, ukuhlukahluka, ukuhlolwa kwezibalo, kanye namamodeli okubikezela. Ukutholwa kwe-outlier kungafinyelelwa ngokubona ngeso lengqondo, izindlela ze-IQR, izikolo ze-z, ngisho nezindlela ze-multivariate kanye nokufunda komshini. Ukuphatha kumele kucabangele umongo kanye nezinhloso: ukuqinisekisa, ukuqonda izimbangela, bese ukhetha isu njengokususa, ukuguqula, ukunciphisa amanani, noma ukusebenzisa izindlela eziqinile.
Ngokuqonda okufanele, izinto ezingaphandle azizona nje "izinombolo ezingavamile," kodwa ziyizinto ezibalulekile zomkhuba wezibalo ezingathuthukisa ikhwalithi yokuhlaziywa nezinqumo eziqhutshwa idatha.
