2: Confusion Matrix Heat Map between Open Source and Commercial Tools 2) was computed showing the various associations between Open Source and Commercial tools.įig. The lowest associations were found between SAS base and KNIME (0.51), SAS base and RapidMiner (0.52), and KNIME and RapidMiner (0.56).įig 1: Association Matrix Heat Map for top 10 most popular data mining toolsĪ similar heat map (Fig. We also note strong association between Excel and SQL, and Tableau and SQL. Spark and Hadoop have the highest association with a lift=3.31, followed by Spark and Python (lift=2.05). Lift 1.2 the square is reddish, if less than 0.8, bluish, else grey. Lift=1 if X & Y appear with frequency expected if they are independent, and Lift (X&Y) > 1 indicates that X&Y appear together more than expected if they were independent, Where pct(X) is the percent of users who selected X. Lift (X & Y) = pct (X & Y) / ( pct (X) * pct (Y) )

There are many ways to measure how significant is associations between two nominal or binary features, eg chi-square or T-test, but we use a simple measure we call 'Lift', defined as We used a version of Apriori algorithm to analyze the results. At the bottom of this post there is a link to download the anonymized dataset.

The dataset consisted of 2759 votes, each for one or more tools. We took anonymized data from the results of the 2015 KDnuggets Data Mining Software Poll, and performed association analysis the top 20 tools.

Best data mining software