Page 40 - Fister jr., Iztok, Andrej Brodnik, Matjaž Krnc and Iztok Fister (eds.). StuCoSReC. Proceedings of the 2019 6th Student Computer Science Research Conference. Koper: University of Primorska Press, 2019
P. 40
le 3 shows all the parameter values used by PSO algo- Table 5: Detected significant statistical differences
rithms. Some parameters are used for all PSO algorithms, with the Wilcoxon test with qα = 0.05 on Iris dataset
while some are algorithm specific. If a parameter is not used
for a specific algorithm the symbol “/“ is used. KM PSO CLPSO OVCPSO MPSO CPSO MCUPSO
From Table 4, we can see that the best classifiers are K- KM ∞+ + + ++ +
means and original PSO algorithm, because they got the
best minimum error rates. The CLPSO algorithm has the PSO /∞ ∼ + ∼∼ ∼
best mean value and the MPSO algorithm has the smallest
standard deviation. One major setback of the OVCPSO al- CLPSO / / ∞ ∼ +∼ +
gorithm was its worst found centers for clusters, that’s clas-
sification accuracy was only 29%. Friedman test gave the OVCPSO / / / ∞ ++ +
Table 4: Basic statistics of used algorithms for 51 MPSO // / / ∞∼ ∼
runs on Iris dataset
CPSO // / / /∞ ∼
KM PSO CLPSO OVCPSO MPSO CPSO MCUPSO
mean 0.0588 0.0553 0.0544 0.0644 0.0505 0.0579 0.0562 MCUPSO / / / / // ∞
std 0.0376 0.0305 0.0356 0.0954 0.0274 0.0424 0.0380
min 0.0000 0.0000 0.0222 0.0222 0.0222 0.0222 0.0222 obtained good results, it had the best median value and
median 0.0444 0.0444 0.0444 0.0444 0.0444 0.0444 0.0444 the best accuracy in the worst run in all of the 51 runs.
max 0.1777 0.1333 0.2000 0.7111 0.1555 0.2444 0.1777 Friedman test gave the statistics value of 174.63646 with
p-value of 4.66982e−35. For qα value of 0.05 we can say
statistic value of 104.81 with p-value of 2.48215e−20. For that the Friedman test rejected the H0 hypothesis. We can
qα value of 0.05 we can say based on the Friedman test that see from the Figure 7, that the MCUPSO algorithm ob-
used algorithms work significantly different. Mean ranks of tained the smallest rank, because of the average standard
algorithms from Figure 6 fit the results in the Table 4. Fig- deviation and smallest error rate on the worst run of the
ure 6 shows that the MCUPSO algorithm holds the smallest algorithm. On the Figure 7, we can see that MCUPSO,
rank, despite not having the best accuracy. We can see that
Table 6: Basic statistics of used algorithms for 51
7 runs on BCW dataset
6 KM PSO CLPSO OVCPSO MPSO CPSO MCUPSO
mean 0.1606 0.1552 0.1643 0.1479 0.1598 0.1672 0.1536
5 std 0.0768 0.0686 0.0779 0.0697 0.0728 0.0892 0.0723
min 0.0818 0.0818 0.0760 0.0818 0.0818 0.0877 0.0877
4 median 0.1345 0.1345 0.1403 0.1345 0.1345 0.1345 0.1169
max 0.3567 0.3684 0.3567 0.3567 0.3567 0.3684 0.3333
3
Mean rank 7
Mean rankKM PSO CLPSOAOlgVoCrPitShOmMPSO CPSO MCUPSO
6
Figure 6: Friedman mean ranks and Nemenyi post-
hoc test with qα = 0.05 for PSO algorithms and the 5
K-means algorithm on Iris dataset
4
Nemenyi test implies that MCUPSO is not significantly dif-
ferent from CPSO, MPSO, CLPSO and PSO algorithms. 3
Based on results in Table 5 we can reject the the hypothesis
of Nemenyi test. The Wilcoxon test detected insignificant 2
difference only between MCUPSO, CPSO, MPSO and PSO KM PSO CLPSOAOlgVoCrPitShOmMPSO CPSO MCUPSO
algorithms. Form the Wilcoxon test we can observe that
the MCUPSO algorithm is significantly different compared Figure 7: Friedman mean ranks and the Nemenyi
to OVCPSO, CLPSO and K-means algorithms. MCUPSO, post-hoc test with qα = 0.05 for PSO algorithms and
CPSO and MPSO algorithms performed the best based on the K-means algorithm on BCW dataset
Friedman mean rank and the Wilcoxon post-hoc test.
MPSO, CPSO and PSO algorithms do not show significant
From the Table 6, we can observe that the best accuracy ob- differences based on the Nemenyi test, but the Wilcoxon
tained the CLPSO algorithm, the best mean value obtained test found insignificant differences only between MCUPSO,
the OVCPSO algorithm, while the PSO algorithm recorded CPSO and MPSO. If we check the PSO algorithm com-
the smallest standard deviation. The MCUPSO algorithm pared to MCUPSO, MPSO and CPSO algorithms, then
we can observe that the Wilcoxon test detected significant
differences only between PSO and MCUPSO algorithms.
MPSO, CPSO, MCUPSO algorithms based on Friedman
mean rank and the Wilcoxon post-hoc test work the best
for this dataset.
StuCoSReC Proceedings of the 2019 6th Student Computer Science Research Conference 40
Koper, Slovenia, 10 October
rithms. Some parameters are used for all PSO algorithms, with the Wilcoxon test with qα = 0.05 on Iris dataset
while some are algorithm specific. If a parameter is not used
for a specific algorithm the symbol “/“ is used. KM PSO CLPSO OVCPSO MPSO CPSO MCUPSO
From Table 4, we can see that the best classifiers are K- KM ∞+ + + ++ +
means and original PSO algorithm, because they got the
best minimum error rates. The CLPSO algorithm has the PSO /∞ ∼ + ∼∼ ∼
best mean value and the MPSO algorithm has the smallest
standard deviation. One major setback of the OVCPSO al- CLPSO / / ∞ ∼ +∼ +
gorithm was its worst found centers for clusters, that’s clas-
sification accuracy was only 29%. Friedman test gave the OVCPSO / / / ∞ ++ +
Table 4: Basic statistics of used algorithms for 51 MPSO // / / ∞∼ ∼
runs on Iris dataset
CPSO // / / /∞ ∼
KM PSO CLPSO OVCPSO MPSO CPSO MCUPSO
mean 0.0588 0.0553 0.0544 0.0644 0.0505 0.0579 0.0562 MCUPSO / / / / // ∞
std 0.0376 0.0305 0.0356 0.0954 0.0274 0.0424 0.0380
min 0.0000 0.0000 0.0222 0.0222 0.0222 0.0222 0.0222 obtained good results, it had the best median value and
median 0.0444 0.0444 0.0444 0.0444 0.0444 0.0444 0.0444 the best accuracy in the worst run in all of the 51 runs.
max 0.1777 0.1333 0.2000 0.7111 0.1555 0.2444 0.1777 Friedman test gave the statistics value of 174.63646 with
p-value of 4.66982e−35. For qα value of 0.05 we can say
statistic value of 104.81 with p-value of 2.48215e−20. For that the Friedman test rejected the H0 hypothesis. We can
qα value of 0.05 we can say based on the Friedman test that see from the Figure 7, that the MCUPSO algorithm ob-
used algorithms work significantly different. Mean ranks of tained the smallest rank, because of the average standard
algorithms from Figure 6 fit the results in the Table 4. Fig- deviation and smallest error rate on the worst run of the
ure 6 shows that the MCUPSO algorithm holds the smallest algorithm. On the Figure 7, we can see that MCUPSO,
rank, despite not having the best accuracy. We can see that
Table 6: Basic statistics of used algorithms for 51
7 runs on BCW dataset
6 KM PSO CLPSO OVCPSO MPSO CPSO MCUPSO
mean 0.1606 0.1552 0.1643 0.1479 0.1598 0.1672 0.1536
5 std 0.0768 0.0686 0.0779 0.0697 0.0728 0.0892 0.0723
min 0.0818 0.0818 0.0760 0.0818 0.0818 0.0877 0.0877
4 median 0.1345 0.1345 0.1403 0.1345 0.1345 0.1345 0.1169
max 0.3567 0.3684 0.3567 0.3567 0.3567 0.3684 0.3333
3
Mean rank 7
Mean rankKM PSO CLPSOAOlgVoCrPitShOmMPSO CPSO MCUPSO
6
Figure 6: Friedman mean ranks and Nemenyi post-
hoc test with qα = 0.05 for PSO algorithms and the 5
K-means algorithm on Iris dataset
4
Nemenyi test implies that MCUPSO is not significantly dif-
ferent from CPSO, MPSO, CLPSO and PSO algorithms. 3
Based on results in Table 5 we can reject the the hypothesis
of Nemenyi test. The Wilcoxon test detected insignificant 2
difference only between MCUPSO, CPSO, MPSO and PSO KM PSO CLPSOAOlgVoCrPitShOmMPSO CPSO MCUPSO
algorithms. Form the Wilcoxon test we can observe that
the MCUPSO algorithm is significantly different compared Figure 7: Friedman mean ranks and the Nemenyi
to OVCPSO, CLPSO and K-means algorithms. MCUPSO, post-hoc test with qα = 0.05 for PSO algorithms and
CPSO and MPSO algorithms performed the best based on the K-means algorithm on BCW dataset
Friedman mean rank and the Wilcoxon post-hoc test.
MPSO, CPSO and PSO algorithms do not show significant
From the Table 6, we can observe that the best accuracy ob- differences based on the Nemenyi test, but the Wilcoxon
tained the CLPSO algorithm, the best mean value obtained test found insignificant differences only between MCUPSO,
the OVCPSO algorithm, while the PSO algorithm recorded CPSO and MPSO. If we check the PSO algorithm com-
the smallest standard deviation. The MCUPSO algorithm pared to MCUPSO, MPSO and CPSO algorithms, then
we can observe that the Wilcoxon test detected significant
differences only between PSO and MCUPSO algorithms.
MPSO, CPSO, MCUPSO algorithms based on Friedman
mean rank and the Wilcoxon post-hoc test work the best
for this dataset.
StuCoSReC Proceedings of the 2019 6th Student Computer Science Research Conference 40
Koper, Slovenia, 10 October
