Benchmarking Recommender Algorithms for Business Intelligence Consultancy

Abstract

Recommender Systems are popularly used to give useful suggestions to the users in various business domains and the effectiveness of the recommender system depends upon the performance of the underlying algorithm. A common method for evaluating the performance of recommender algorithms is to benchmark them by calculating performance metrics. However, a benchmarking for recommender algorithms in the domain of Business Intelligence (BI) consultancy has not yet been performed. The BI consultants use their expertise and experience to provide professional advice or recommendations to their customers for effective and efficient decision-making. The goal of this research is to evaluate performance metrics of different recommender algorithm(s) that can be used in a recommender assistant for a BI consultancy firm to predict KPI recommendations. In this research, recommender algorithms based on traditional (collaborative-filtering), graph-based and Case-based reasoning recommender systems were compared by performing experiments to verify if recommender algorithm(s) exist that give more effective recommendations than the BI consultants. The experiments were carried out using a controlled experiment methodology similar to what Text REtrieval Conference (TREC) uses in the field of Information Retrieval and evaluated using the metrics used in TREC.