HamidReza Mahmoodi; Nazila Mehrabi; Azra Daei
Abstract
IntroductionPerformance evaluation systematically investigates a subject to improve program effectiveness using appropriate, ethical, feasible, and precise methods (Tootanchi et al., 2006). It measures outcomes against indicators to evaluate goal achievement, efficiency, resource effectiveness, process ...
Read More
IntroductionPerformance evaluation systematically investigates a subject to improve program effectiveness using appropriate, ethical, feasible, and precise methods (Tootanchi et al., 2006). It measures outcomes against indicators to evaluate goal achievement, efficiency, resource effectiveness, process quality, and program execution (Parker, 2000; Gholami & Noralizadeh, 2002). Knowledge, a vital organizational asset, enhances competitiveness by facilitating decision-making and performance improvement. As society shifts towards information-driven environments, knowledge technologies become essential, leveraging AI to solve complex issues and enhance decision-making. Effective implementation of these technologies provides competitive advantages by efficiently storing, protecting, processing, and utilizing knowledge, contributing to sustained performance, growth, and innovation. They offer benefits like increased accessibility, cost reduction, time savings, improved communication, innovation, enhanced data storage, reliability, and swift knowledge transfer (Arab-Mazari Zadeh et al., 2007). Thus, evaluating knowledge technology performance is crucial to ensure quality, customer satisfaction, and informed decision-making. Without it, organizations risk inefficiency and resource wastage. This study aims to identify and rank key components for evaluating knowledge technologies to ensure effective assessment and utilization.Literature ReviewHamidizadeh (2016) found a significant positive correlation between expert decision systems and decision-making efficiency, including improved speed, reduced interdepartmental information gaps, and lower organizational costs. Musivand et al. (2015) discovered that knowledge management systems enhance job quality in Iran's Ministry of Sports and Youth by positively impacting knowledge utilization, management, creation, storage, and organization. Naqib et al. (2013) identified the customer aspect as the most influential in knowledge management systems using a balanced scorecard model, with the financial aspect being the most affected. Fazli and Aghshalouei (2008) recommended a hybrid model for assessing decision-making units' performance. Latifi and Mousavi (2008) highlighted four key processes—identification and creation, registration and maintenance, sharing, and internalization—as crucial for effective knowledge management in Iranian software companies. Samimi and Aghaei (2005) proposed a performance evaluation model for knowledge management systems, emphasizing its role in system efficiency enhancement. Internationally, Kumar (2018) emphasized the critical role of knowledge technology in organizational knowledge management, particularly in data accessibility and user services. Mysore et al. (2018) highlighted digital tools like BIM and IoT in the construction industry. Simon and Georgi (2017) developed a framework integrating knowledge search behaviors and tools for asynchronous environments. Kumar et al. (2016) found that organizational culture and leadership, especially democratic styles, significantly influence knowledge absorption, with soft factors outweighing hard factors. Ngugi et al. (2016) demonstrated that knowledge technology positively impacts small enterprises' growth in Nairobi by facilitating skill transfer and process improvement. Milton et al. (1999) highlighted knowledge technology's role in supporting key knowledge management activities such as personalization, innovation, and monitoring. These studies collectively underscore the vital role of knowledge technologies and management systems in improving organizational efficiency, decision-making, and overall performance across various sectors.Methodology This applied research utilized a mixed exploratory approach, employing Delphi and Analytic Hierarchy Process (AHP) methods. Data was collected through library-documentary and field methods. The Delphi phase involved an open and closed questionnaire, the latter based on the open questionnaire findings, encompassing 10 components and 39 questions on a Likert scale. The study population included 18 purposively sampled faculty members from Tehran's public universities, with theoretical saturation determining the sample size. The subsequent AHP-designed questionnaire was distributed among 60 academic members and PhD students, achieving a numeric saturation with consistent mean values indicating data adequacy. Validity and reliability were ensured via consistency rates below 0.1 and analyzed using MAX QDA, SPSS 25, and Expert Choice 11 software.ResultsThe dual Delphi rounds in this study reached consensus among panel members, starting with 73 initial codes refined to 38 unique codes across 10 indices: financial costs, system quality, system infrastructure, technology and knowledge service quality, knowledge technology architecture, user interface, user satisfaction, value of results, perceived benefits, and up-to-dateness. A Likert-scale questionnaire in the second round confirmed all 38 components. Table 4 indicates that the financial component, with a weight of 0.178, significantly influences knowledge technology performance evaluation. The consistency rate of 0.09 ensures the reliability and stability of the findings. Other components, ranked by weight, include system quality (0.156), system infrastructure (0.154), user interface (0.125), and others, down to perceived benefits and up-to-dateness (0.038).Discussion Evaluating the performance of knowledge technologies in knowledge-based organizations is vital for identifying learning pathways and creating competitive advantages. Organizations require tools to enhance performance and continuously assess the effectiveness of their knowledge technologies, addressing strengths, weaknesses, opportunities, and threats. This research identified ten key components for performance evaluation: financial, system quality, system infrastructure, knowledge technology service quality, knowledge technology architecture, user interface, user satisfaction, value of results, perceived benefits, and up-to-dateness. The financial component, deemed most critical, includes startup costs, infrastructure and equipment costs, human resources training costs, and AI processing costs. System quality, ranked second, involves flexibility, effectiveness, use of expert systems, ease of access, and support for open access. System infrastructure, third in importance, covers physical and electronic spaces, application modernization, and elimination of outdated infrastructure. User interface, ranked fourth, focuses on usability, accessibility, user-friendliness, and visual appeal. Knowledge technology service quality, sixth in rank, includes information processing quality, metadata management, content volume, and content quality. User satisfaction, seventh, involves automated knowledge management, system efficiency, prediction of user needs, and satisfaction with system effectiveness. Value of results, eighth, includes continuous improvement, enhanced decision-making through AI, alignment of results with needs, and reliability. Perceived benefits and up-to-dateness, both ranked ninth, cover monitoring performance changes, goal achievement assessment, opportunities for new knowledge creation, and performance improvements. These evaluations highlight the operational quality within organizations and the challenges in successful knowledge management implementation. Previous research, such as Hamidizadeh (2016) and Musivand et al. (2015), supports these findings, emphasizing the role of knowledge technologies in decision-making and operational efficiency. International studies also affirm their importance in service delivery, knowledge structuring, and performance enhancement. Thus, evaluating knowledge technologies using these key components is essential for effective utilization and to avoid resource wastage and operational inefficiencies.ConclusionIn the Delphi phase, 10 indicators including the financial component, system quality, system infrastructure, knowledge technology service quality, knowledge technology architecture, user interface, user satisfaction about the system, the value of work results, perceived benefits and benefits from the system, up-to-dateness and 39 items were identified, Each of these indicators also has its own sub-indicators, whose degree of importance has also been examined. Based on the hierarchical analysis method, the financial component is in the first degree, the system quality component is in the second degree, the system infrastructure component is in the third degree, the user interface component is in the fourth degree, the knowledge technology architecture component is in the fifth degree, and the knowledge technology service quality component is in the sixth degree. The user satisfaction component about the system was ranked seventh, the value component of the results obtained from the card was ranked eighth, and the perceived benefits and benefits of the system and the component of being up-to-date were ranked ninth.Acknowledgments The authors consider it necessary to acknowledge and thank all the loved ones who helped us in this research.
