Using Importance-Performance Analysis and Decision Making Trial and Evaluation Laboratory to Enhance Order-Winner Criteria ~ A Study of Computer Industry

INTRODUCTION

After the rise of different quality management systems in the 80s, we tend to find Customer Satisfaction as the main assessment index for organizational performance in both theoretical and empirical studies. Thus, research on the importance and satisfaction of service quality characteristics has become popular. Generally speaking, the tool commonly used in this kind of research is Importance-Performance Analysis (IPA). This method directly analyzes the advantages and disadvantages of organizations from market surveys. Martilla and James (1977) initially developed market strategies using IPA, which have also been applied to varied industries for nearly 30 years. For example, in a recent study, Levenburg and Magal (2005) applied IPA to the construction of e-commerce strategies and resource distribution. Zhang and Chow (2004) applied IPA to improve tourism service quality. Matzler et al. (2003) applied IPA to bank service quality improvement and strategic development. Aigbedo and Parameswaran (2004) improved food service quality in schools using IPA. Matzler et al. (2004) used the automobile industry as an example and re-considered the application of IPA. Matzler et al. (2005) applied IPA to modern management and an orientation study of the tools. Huang et al. (2006) explored satisfactory service quality using intermediate and long trip passengers from high schools using IPA. Tonge and Moore (2007) assessed the perception quality of coastal Marine-Park visitors using IPA and Gap Analysis and managed environmental protection more effectively. Lee et al. (2008) used a modified IPA model to assess supplier quality performance. Martilla and James (1977) emphasized the advantages of IPA: low cost, useful application, better focus and strategic suggestions which were the main reason that the method was generally accepted and applied.

The basic concept of this method is to probe into the customers cognitive importance of quality characteristics through market research and the customers actual satisfaction with the quality characteristic after receiving the services. Using two-dimensional Importance and Performance matrix analysis, they allocated the quality characteristics into four categories according to importance and performance for the organizations market strategies. Of course, in the history of IPA development, many scholars tried to compare the effectiveness of different IPA models and its framework.

For example, Oh (2001), who re-studied the concept and methods of the past IPA researches provided future research directions and also reorganized ten conclusions for effectively and reliably using IPA. Bacon (2003) compared several different IPA methods using 15 pieces of data to find the most effective one. Fontenot et al. (2005) compared four models that were commonly used for studying Customer Satisfaction and demonstrated that when constructing the action project, multi-variance sequence, technique was the priority.

Recently, many scholars tried to modify the traditional IPA model. Through a bank service quality empirical study, Matzler et al. (2003) indicated that Customer Satisfaction exhibited a linear quality characteristic structure and demonstrated that the traditional IPA model would lead to wrong organizational decision-making. Sampson and Showalter (1999) demonstrated the negative correlation between importance and performance. They suggested that importance should not be revealed using point estimation. Instead, it should be shown using a causal performance function. Yavas and Shemwell (2001) modified IPA by multiplying the gap between the company’s performance and that of rivals using relative importance. They demonstrated the application using the medical industry as an example. In Customer Satisfaction results on the quality characteristics of outdoor recreational facilities, Tarrant and Smith (2002) modified IPA using means and standard deviation to supplement the disadvantage of point estimation in the traditional means.

The aforementioned scholars’ researches on IPA made great contributions. However, traditional IPA presupposes that explicit customer response data is used for assessing the importance and performance of quality characteristics and that each quality characteristic is an independent variable. Under this supposition, when the quality characteristic has explicit data and causation, traditional IPA cannot correctly show the importance and priority of improvements. This study proposes a new methodology for decision analysis that identifies the core order-winner criteria problems using a modified IPA and multiple regression analysis applied to define the implicit importance of quality characteristics and overall satisfaction. Gap Analysis (GA) is used for the performance of quality characteristics. The decision making trial and evaluation laboratory (DEMATEL) is used to determine the correlation and interrelationship among the quality characteristics. This methodology better defines the importance and priority over quality characteristic improvements and solves the practically complicated, contradictory problems using the least resources. This study performs empirical research into the Taiwanese computer industry and explains the application and benefits of the modified IPA and DEMATEL.

IPA models review: IPA was proposed by Martilla and James (1977) and used to develop effective market strategies. The organizations probed into the customers’ acceptance of specific quality characteristics using market research and they encountered two problems: (1) they only surveyed the acceptance of one aspect of the quality characteristics (importance or performance); (2) according to the research finding, they could not find the empirical importance and influence through the coefficient of determination in the statistical analysis. Thus, IPA revealed a perfect solution for the above problems. With such simple data, the organizations could analyze and study four different types of quality characteristics and develop a strategy and action plans with regard to the quality characteristic in each quadrant. The IPA questionnaire design tended to invite the respondents to answer the following two questions related to quality characteristics: (1) How important is the quality characteristic for you? (2) How is the organization’s performance with regard to this quality characteristic?

IPA constructs a two-dimensional matrix by collecting the above two responses. With regard to the central tendency of importance and performance, it divides the quality characteristics into four quadrants by treating the median as the matrix split value. Some scholars replaced the median with the means, which becomes the main statistics in the IPA. The traditional IPA model reveals the states of the quality characteristics using a two-dimensional graph used to explain and judge the strategies. We can respectively define four matrix quadrants: (1) Concentrate here: the customers suggest that the importance of the product or service quality characteristic is high; however, the organizational performance is low. (2) Keep up the good work: the customers suggest that the importance of the product or service quality characteristic is high and the organizational performance is also high. (3) Low priority: organizational performance on the product service quality characteristic is low and the customers’ cognitive importance is also low. (4) Possible overkill: organizational performance on the product or service quality characteristic is high; however, the customers’ cognitive importance is low.

Although IPA is easy to use and explain there are still some latent problems. After Martilla and James (1977) proposed IPA, many well-known scholars proposed modified models. Yavas and Shemwell (2001) integrated the relative importance as a weighting index to replace the importance of the vertical axis and used the relative performance. In other words, they compared and calculated the difference between the performances of the organization and its’ rivals and multiplied it by importance to replace performance in the original horizontal axis. In addition, they redefined four strategies of importance-performance into a two-dimensional matrix. Tarrant and Smith (2002) modified IPA using the standard deviation of the means of the quality characteristics. This method successfully assessed the dispersion of the average importance and performance. Martilla and James (1977) suggested that importance and performance were independent variables. However, the dynamic model proposed by Sampson and Showalter (1999) demonstrated that the change in organizational performance would further influence the customers’ cognitive importance. The main reason was in that the correlation coefficient between importance and performance was not zero. Based on this argument, when the organizational performance was higher, the gap between importance and performance would be immediately reduced. Likewise, the study of Matzler et al. (2003) indicated that organizational overall satisfaction was the function of quality characteristic performance. The study of Matzler et al. (2004) indicated that importance of a quality characteristic was the function of quality characteristic performance. Matzler and Sauerwein (2002) regarded importance as the function of performance. Therefore, they regarded performance as independent variables and overall satisfaction as dependent variables for Multiple Regression Analysis. In the solution of regression formula, they treated regression factor as the importance of quality characteristics to reconstruct importance and performance to analyze matrix. Deng (2007) and Deng et al. (2008a, b) provided a revised IPA that integrated partial correlation analysis and natural logarithmic transformation for measuring the importance of attributes. Their papers provide a modified IPA and makes extended use of IPA in assessing a supplier’s quality performance.

Although the above researches and findings contributed much to IPA theory and application, research on how the correlation between quality characteristics influence the conclusion has not been processed yet. To know such influence, this study takes the implicit importance from Matzler and Sauerwein (2002) as the satisfaction function of the quality characteristics and adopts gap analysis and IPA from Fontenot et al. (2005) to create a more reasonable IPA for analysis. This study takes into account of the correlation and interrelationship among quality characteristics to modify the importance and priority of improvements of quality characteristics, aiming to avoid wrong decisions, to improve quality characteristics and rationalize resource allocation.

PROPOSED MODEL

After the IPA literature review this study presents a more reasonable, modified IPA. As to the causation of quality characteristics that have not been analyzed, This research uses DEMATEL to identify the causation and interrelationship between the quality characteristics and adjusts their priority to determine the core problems and avoid the huge losses arising from wrong decisions. The proposed method also improves the quality characteristics and rationalizes resource allocation. The modified IPA and DEMATEL used in this study involve five subjects: (1) determining the gap between the importance and performance of each quality characteristic and estimating organizational performance; (2) multiple regression analysis is used to forge a functional relationship for the performance and overall satisfaction of each quality characteristic. The regression coefficient from this relationship is the estimate of importance for each quality characteristic; (3) implicit importance is shown on Y axis with the performance gap on the X axis of the IPA matrix; (4) DEMATEL is employed for the causation and interrelationship; (5) IPA and DEMATEL are combined for new decisions and improvements.

Modified IPA model: In customer satisfaction, the key is whether customers think what is important is recognized by the organization and the organization produces goods that customers want. Therefore, it is necessary to assess both the customers recognition and satisfaction for the goods and services. A common method for this assessment is the gap analysis developed by Parasuraman et al. (1985). Gap analysis is based on the difference between the customers recognition and perception. Traditional IPA lacks clear prioritization of improvements in quality characteristics, which can be complemented by the gap analysis for it can prioritize in a simpler and more reasonable way the improvements and resource allocation of quality characteristics of organizational goods and services (Fontenot et al., 2005).

In researches by Matzler and Sauerwein (2002) and Matzler et al. (2004), the IPA model has two types of measurements for the importance of quality characteristics. The first is customers disclose how important they think a given quality characteristic is, called the explicit importance. The second is how important that quality characteristic is for customers using the multiple regression equation, called the implicit importance. With performance as the independent variable and overall satisfaction as the dependent variable in the multiple regression equation, Matzler and Sauerwein (2002) proved that it is the implicit importance, not the customers self presentation that serves as the satisfaction function of a quality characteristic. Therefore, the multiple regression equation is a better measurement of the customers’ perception of the quality characteristics. The multiple regression equation with performance as the independent variable and overall satisfaction as the dependent variable is shown below:

(1)

In this equation:

In Matzler and Sauerwein (2002) the regression coefficient β_i of the equation stands for the performance influence of the quality characteristic on overall satisfaction. Yet, this regression coefficient β_i in the empirical research is negative, so it is reasonable to make |β_i| an estimate of the implicit importance. In the study by Hansen and Bush (1999), IPA was generally acknowledged a simple and effective tool for acquiring the greatest customer satisfaction using efficient allocation of limited resources. Hence, the traditional importance-performance matrix in this study was changed to one with the regression coefficient, not importance, as the Y axis and the gap analysis for quality characteristics, not performance, as the X axis. The modified IPA is shown in Fig. 1. The concentration trend in the form of an average is used to create four quadrants in the importance-performance matrix. The definition and strategy in those four quadrants are the same as those defined by Martilla and James (1977). Therefore, the new matrix is still easily explained and analyzed.

Quality characteristics are usually improved under the supposition that they are independent and can be improved according to IPA matrix shown in Fig. 1. However, when there is causation among the quality characteristics and one improved characteristic will definitely change the other relevant characteristics, quality characteristics with greater influence should be considered first. Knowing how such causation works on decision making and improvements is critical. The two sections below examine decision making and improvements based on a combination of the DEMATEL and IPA results.

Decision making trial and evaluation laboratory: The Decision Making Trial and Evaluation Laboratory (DEMATEL) was developed by Battelle Memorial Institute of the Geneva Research Center (Gabus and Fontela, 1973; Fontela and Gabus, 1976).

Fig. 1:	Modified IPA map

DEMATEL was used to solve sophisticated problems such as race, famine, environmental protection, energy and others (Fontela and Gabus, 1976). Recently Japan, Korea and Taiwan use it comprehensively in other domains. Hajime et al. (2005) integrated QFD, TRIZ and DEMATEL to settle disputes over new product designs. Nanayo and Toshiaki (2002) adopted a modified DEMATEL in an integral evaluation of the medical system. Kenichi and Yoshihiro (2002) introduced DEMATEL into an analysis of the functions and ineffectiveness of a snow melting system. Kim (2006) integrated PCA, AHP and DEMATEL to evaluate the impacts on the livestock industry and agricultural information. Wu and Lee (2007) used Fuzzy DEMATEL to discuss questions about the core capability of a manager. Lin and Wu (2008) applied Fuzzy DEMATEL to group decisions. DEMATEL has been utilized in many circles. DEMATEL aims to present the direct/indirect causation and degree of influence of quality characteristics by comparing their interrelationships in the matrix. The causation and influence on the matrix are shown as a reference when one makes decisions. In other words, DEMATEL can turn a complicated system into clear cause and effect and quantify how one quality characteristic affects another in identifying core problems and improvements. Here is a brief explanation of the DEMATEL framework and arithmetic.

Definition of quality characteristics and creation of scale of measurement: From the literature review brainstorming or specialists comments, this study lists and defines the quality characteristics of a certain complicated system and assumes that n number of quality characteristics exist. Next a scale of measurement for pair-wise comparison between the characteristics and their causation is created.

This scale has four measurement levels from 0, 1, 2, to 3, representing no influence low influence high influence and great influence (Lin and Wu, 2008). The measurement scale has six levels from 0, 1, 2, 3, 4 to 5, representing no influence extremely low influence low influence moderate influence high influence and great influence (Kim, 2006). In Huang et al. (2006, 2007) the scale has 11 levels from 0 to 10, representing no influence to great influence. Thus, the scale has no specific limitation or rules.

Direct-relation matrix: If the number of quality characteristics is n, when this study makes pair-wise comparison between the causation and quality characteristic influence, the result will be an nxn direct-relation matrix X, in which x_ij stands for how i quality characteristic influences j quality characteristic and quality characteristics x_ii at the corner are set to 0.

A sign matrix S is created to show whether the influence is positive or negative, shown as + and -.

Normalized direct-relation matrix: The normalized direct-relation matrix has two types of arithmetic. In the studies of Wu and Lee (2007), Lin and Wu (2008), Kim (2006) and Seyed-Hosseini et al. (2006), the normalized standard is row vector and the maximum, whereas Tzeng et al. (2007) used the column/row vector and the maximum.

As:

(2)

In formulas 2 and 3, when the direct-relation matrix X is multiplied by λ, the normalized direct-relation matrix is N.

(3)

Under the DEMATEL supposition, the sum of at least one i row must be in line with formula (4) and Lin and Wu (2008) concluded that nearly all cases can satisfy this formula.

(4)

Hence, this study derives a sub-stochastic matrix from the normalized direct-relation matrix N and an absorbing-state Markov chain matrix. According to Papoulis and Pillai (2002):

(5)

In this, O is the null matrix and I is the identity matrix.

Direct/indirect relation matrix: Since the normalized direct-relation matrix N contains properties of formula (5), indirect/direct-relation matrix, or total-relation matrix, can be derived from formula (6) (Huang et al., 2007). Further, the indirect relation matrix H, or total-indirect-relation matrix, can be derived from formula (7) (Lin and Wu, 2008).

(6)

(7)

Suppose that t_ij is the quality characteristic for the total-relation matrix T and i, j = 1, 2,..., n, then formulas 8 and 9 can be used to determine the sum of values in the row and column of the total-relation matrix T. Also, D_i is the sum of i row, which means quality characteristic i can affect the sum of other quality characteristics; R_j is the sum of j row, which means quality characteristic i is affected by the sum of other quality characteristics. D_i and R_j of the total-relation matrix T include direct/indirect influences.

(8)

(9)

Causal diagram: when D_k + R_k is prominent and k = i = j = 1, 2,..., n, this shows how the quality characteristic is influenced by or affects others and how prominent quality characteristic k is in all problems. When D_k-R_k is the relation, this shows the variation in the quality characteristic being influenced by or affecting others and whether quality characteristic k is more like a cause or a result. If the result is positive, the quality characteristic is more like a cause, otherwise it is a result. This causal diagram has (D + R) as x axis and (D - R) as y axis. This diagram is equal to 2D diagram of sing matrix s. With a causation diagram and the position of a quality characteristic, a decision maker can know if a quality characteristic is a cause or a result and how it is affected or affects others. According to the result, the decision maker can make plans to solve his/her problems.

This study determines the coordinate value (D_k + R_k, D_k-R_k) of each quality characteristic and marks the value on the causal diagram. When D_k-R_k is positive, quality characteristic k is a cause; when D_k-R_k is negative, quality characteristic k is a result. When the sum of D_k + R_k is bigger, a quality characteristic affects or is affected by others more greatly. According to the causal diagram, when D_k-R_k is negative and D_k + R_k is of small value, quality characteristic k is more independent, which means that k is influenced by only a few factors. When D_k-R_k is positive and D_k + R_k has a small value, quality characteristic k is also independent, which means that k influences only a few quality characteristics. When D_k-R_k is positive and D_k + R_k has a large value, quality characteristic k is a cause of a core problem and should be solved first. Therefore, the decision maker can know if a quality characteristic is a cause or a result and how it is affected or affects others by the causal diagram and make plans to solve his/her problems.

Combination of IPA and DEMATEL: Traditionally people use the IPA matrix in Fig. 1 to discern quality characteristics that should be improved one at a time. Yet, if causation lies among several quality characteristics, i.e., one quality characteristic can influence others, quality characteristics with great influence should be addressed first. To know the causation pattern of quality characteristics, it is good idea to use a 2D causal diagram charted in accordance with the DEMATEL result and with four quadrants that are demarcated by the average of each axis. Quadrant 1 has high prominence and high relation, showing the quality characteristics in this area are core problems and affect others. They should be addressed first. Quadrant 2 has low prominence and high relation, showing the quality characteristics in this area are highly independent and affect only a few quality characteristics. Quadrant 3 has low prominence and low relation, showing the quality characteristics in this area are highly independent and affected by only a few quality characteristics. Quadrant 4 has high prominence and low relation, showing the quality characteristics in this area are core problems and affected only by other quality characteristics. In all, the quality characteristics in quadrants 1, 2 and 4 of the causal diagram can sway the IPA matrix result. Quadrant 3 with low prominence and low relation already fulfills the IPA model supposition, so the quality characteristics in this area can be improved according to IPA strategies. The combination of IPA and DEMATEL has two categories and four policies, shown below:

Quality characteristic with high prominence and relation:

Quality characteristic with high prominence and low relation:

In accordance with its management and analysis an organization can lower down influence to a certain extent below which all quality characteristics are not influential, i.e., of no causation. This is in favor of simpler management, analysis and decision.