Subscribe Now Subscribe Today
Research Article
 

Using Fuzzy Group Analytical Network Process and ISO 9126 Quality Model in Software Selection: A case study in E-Learning Systems



Leila Etaati, Soheil Sadi-Nezhad and Ahmad Makue
 
Facebook Twitter Digg Reddit Linkedin StumbleUpon E-mail
ABSTRACT

ISO 9126 quality model is an assessment tool for determining software quality which has main attributes and sub attributes. In addition, there are some interrelations between ISO 9126 quality factors that must be considered during assessment. In this study, we present a Fuzzy Group Analytical Network Process (FGANP) model for evaluating software packages in which some dependencies between criteria are considered. This ANP model is based on the criteria and sub-criteria which were introduced by previous researchers. Also, interrelations between the factors are considered from their suggestions. Therefore, for deriving preference ratio among decision makers and to reach a global argument among them, we use Group Fuzzy Preference Programming (GFPP). Finally, we show the general assessment techniques and some obtained results using our model to evaluate the three existing E-learning software which are employed in some universities.

Services
Related Articles in ASCI
Similar Articles in this Journal
Search in Google Scholar
View Citation
Report Citation

 
  How to cite this article:

Leila Etaati, Soheil Sadi-Nezhad and Ahmad Makue, 2011. Using Fuzzy Group Analytical Network Process and ISO 9126 Quality Model in Software Selection: A case study in E-Learning Systems. Journal of Applied Sciences, 11: 96-103.

DOI: 10.3923/jas.2011.96.103

URL: https://scialert.net/abstract/?doi=jas.2011.96.103
 
Received: August 26, 2010; Accepted: October 09, 2010; Published: November 10, 2010



INTRODUCTION

Using computer package in various industries has been increased recently. Many companies use complicated soft wares to reduce cost of their firms and improve the quality of their works and products. Although, Software companies have been produced variety of packages to respond the existing demands. The cost of these packages is very high, so selecting appreciated software is a vital issue in many companies. For example, which strategy must be considered during selection of ERP package? Therefore, selecting a software package that meets the requirements of a company is a difficult task. This has led researchers to investigate better ways of evaluating and selecting software packages (Jadhav and Sonar, 2009). Hence, for each field of industry that uses special software package there are several attempts in software assessment that have been done by many researchers.

In accounting software, Arditi reviewed essential criteria to select suitable accounting software and finally proposed a list of popular packages on the US market and classified them by their respective characteristics (Arditi and Sandeep, 1991). In another research, Adhikari et al. (2004) reported a survey on process of selection and design of international accounting software and investigated the relationships among firm characteristics.

Another software package that mostly has been considered in various researches is Enterprise Resource Planning (ERP). Commercial packages of ERP could not provide a unique business model for every process of all industry (Teltumbde, 2000). Thus, no single ERP Software can meet all company business requirements (Wei et al., 2005). In 2000, Teltimbde used Nominal Group Technique (NGT) and the evaluation methodology adopting the Analytical Hierarchy Process (AHP) for evaluating ERP projects (Teltumbde, 2000).

Wei et al. (2005) proposed a comprehensive ERP system selection framework in which the objective hierarchy is constructed and the appropriate attributes are specified to provide detailed guidance for ERP system evaluation.

In the field of E-learning systems selection, there are many attempts that have been done during more that 10 years. Soong et al. (2001) identified some Critical Success Factors (CSF) for favorable usage of on-line course resources. Wang (2003) used a survey, generating items, collecting data and validating the multiple-item scale for developing a comprehensive model and tools for measuring learner satisfaction with asynchronous E-learning systems. Also, many other researchers used MCDM approach for evaluating E-learning systems. AHP proposed by Saaty (1980), has been widely used among other MCDM techniques for bench marking of E-learning systems (Shee and Wang, 2008; Tzeng et al., 2005; Chao and Chen, 2009; Colace et al., 2006; Mahdavi et al., 2008). Moreover, Jadhav and Sonar (2009) reviewed more than 50 papers including software package selection.

This review study investigated methodologies for selecting software packages, software evaluation techniques, software evaluation criteria and systems that support decision makers in evaluating software packages (Jadhav and Sonar, 2009).

In addition, there are several attempts to identify quality attributes. The ISO-IEC 9126-1 standard provides a hierarchical quality model composed of characteristics and sub characteristics in terms of internal quality, external quality and quality in use (ISO/IEC 9126-1, 2001; Jung, 2007). Many researchers used this model for evaluating software package. Bevan (1999, 2001) described the new framework for software product quality developed for ISO/IEC 9126-1 (2001). Suryn and Laverdiere (2007) presented a solution for methodically evaluating the internal quality of a soft switch. In his research he used measurements of internal quality attributes of software which has been defined by the technical report ISO/IEC 9126-3. Zulzalil et al. (2008), proposed a method to investigate and identify the relationships between ISO-IEC 9126 quality criteria in the development of web-based applications.

In this study, we used some identified relationships between ISO-IEC 9126 that have been demonstrated by prior researchers and created a network structure instead of existing ISO-IEC hierarchical structure quality model. Moreover, we used Fuzzy Group Analytical Network Process (FGANP) for evaluating software package.

SOFTWARE QUALITY CRITERIA

Definition: A lot of work has been done since 1976 by a number of researchers to design a software quality framework. According to ISO 9126-1 (ISO/IEC 9126-1, 2001) quality is defined as a set of features and characteristics of product or service that bears on its ability to satisfy the stated or implied needs. A quality model is defined as set of characteristics and the relationships between them, which provides basis for specifying quality requirements and evaluating the quality (ISO/IEC 9126-1, 2001).

Pioneer software quality models: McCall et al. (1977) proposed a model for software quality. Firstly, he started with 55 quality characteristics which had an important influence on Software quality. After that, McCall reduced the number of characteristics to eleven. Another model that was developed by Boehm and In (1996), added seven characteristics to McCall’s model. Moreover, his model included some considerations about evaluation of software products with respect to utility of the program. However, Boehm’s model was same as McCall’s model in which it represented a hierarchical structure of characteristics and both researchers (McCall and Boehme) included user’s needs. Perry (1987) had shown some interrelationships between Boehme’s criteria which was mostly used in later research. In another research, Dromey’s model demonstrated some relationship between the attributes and the sub-attributes of software quality (Dormey, 1995).

By increasing the number of quality models, the need for a sole standard model was felt. As a result, ISO institute started to develop ISO/IEC 9126 in 1985.It was a part of ISO 9000 as a vital standard for quality assurance. This model has a hierarchical tree structure that contains main characteristic and sub-characteristic. It has 6 main criteria including functionality, reliability, usability, efficiency, maintainability and portability. These criteria have many sub-criteria. Also it provides a framework which can consider some trade-offs between software product specifications (Behkamal et al., 2009) (Fig. 1).

ISO 9126 identifies the internal characteristics and external quality characteristics of a software product and contains various characters. However, it has some disadvantages abut showing very clearly how these aspects can be measured and the proposed criteria are so general. Suryn and Laverdiere (2007) analyzed architecture of a soft switch in the telecom industry in order to identify internal quality characteristics, sub characteristics and related measures critical to its functioning as well as a help to measure them. They have been used ISO/IEC 9126-3 as the fundamental reference for software product quality measurement and evaluation. In another research done by Wolfgang AHP is used to evaluate three different AHP software by considering ISO/IEC 9126 criteria (Ossadnik and Lange, 1999). Bee Chua and Laurel Evelyn Dyson used ISO/IEC as a useful evaluation tool for assessing E-learning soft wares (Chua and Dyson, 2004). In another research on B2B(business to business) application that has been done by Behkamal and Kahani, they Used ISO/IEC criteria as a well known quality model in software quality and customized it by some special characteristic of B2B application (Behkamal et al., 2009).

Fig. 1: ISO/IEC 9126 software quality structure

Fig. 2: Existing relationship among dimension of software quality

Fig. 3: ISO/IEC 9126 Network structure

Interrelation between ISO-IEC9126 criteria and ANP model: As mentioned before, Boehm has been presented a quality model which had a hierarchical structure, but this model did not consider any interrelationship between criteria. However these criteria may have some effects on each other. Perry (1987) showed that there was some interrelationship between quality factors. Perry labeled them as inverse, neutral and direct. Many researchers used Perry’s model as a useful subject. Zulzalil et al. (2008) presented a method to investigate and identify the relationships of quality criteria in the development of web-based applications and conducted this research. They used Experienced-based approach and an online survey to gather the intended relationships; in this case they used ISO/IEC characteristic of software quality and used predefined relationships among these criteria that has been proofed by Perry. Henningsson and Wohlin (2002) conducted a research based on relations between software quality attributes in their research. They used two different methods to identify these relationships. First, they review some literature on this subject and in second approach they conducted interview survey with people from industry. Finally they made a table which shows a summary of relationships ( Boehm and In, 1996). By considering Perry, McCall, Boehm, Knnet, Henningsson and Hazura-Zulzalil researches, Fig. 2 shows a summery of software quality relationships which have been addressed in former papers.

Figure 2 shows that there are some relations between 4 main criteria. For example, efficiency and reliability have effect on maintainability and usability. In this case we did not consider positive or negative impact which was in Perry’s model. So, by considering this similarity between criteria we can found some interrelations between ISO/IEC 9126 main criteria which have been shown in Fig. 3 (Zulzalil et al., 2008).

Hence, during evaluation of software package by ISO/IEC 9126 criteria, relationships between main criteria have been considered. As a result every assessment process in any kind of software package like accounting software, B2B, AHP and others, can consider these relations while using ISO/IEC as a reference model. However, we can not use AHP application in this situation. So, we apply ANP (Analytical Network Process) technique which has been proposed by Saaty (1980,1996).

ANP literature: ANP developed by Thomas Saaty provides a way to input judgments and measurements to derive ratio scale priorities for the distribution of influence among the criteria and groups of criteria in the decision (Saaty, 1996, 2003). The well-known AHP is a special case of the ANP. Both the AHP and the ANP derive ratio scale priorities by making paired comparisons of elements on a common property or criterion (Soheil et al., 2010). Many decision-making problems cannot be structured hierarchically because they have some dependency among their criteria. Hence, structuring a problem with functional dependencies that allows feedback among clusters is considered to be a network system. Over the years, ANP, a comprehensive multi-purpose decision method, has been widely used in solving many complicated decision-making problems. In study by Meade and Sarki on introducing a decision methodology and structure for manufacturing (and organizational) agility improvement, they used ANP to evaluate logistic strategies and to improve production speed (Meade and Sarkis, 1998). Moreover, Yuksel used ANP to demonstrate a process for quantitative SWOT analysis that can be performed even when there is dependence among strategic factors (Yuksel and Dagdeviren, 2007). Mikhailov and Singh (2003) represented a fuzzy ANP application to develop a decision support System. Mulebeke and Li (2006) applied analytical network process on multi attribute strategic decision making approach to help the selection of appropriate software to suit the product development process of a particular product. Soheil et al. (2010) proposed an ANP model for evaluation E-learning system; they used Fuzzy Group Analytical Network Process in evaluation process.

Fuzzy group ANP application

Step1: Creating network structure: These relations can be obtained through brainstorming by experts or studying previous researches. In this study, we created a network structure based on previous researches
Step 2: Pair wise comparison: Pairs of decision elements at each level are compared with respect to their importance towards their control criteria. There are some scales to determine the importance of one criterion or alternative on others.

Table 1: Linguistic value look-up table

Also, for helping the decision makers to enhance their decisions and improving their accuracy we used linguistic variables method as is shown in Table 1 (Cheng et al., 1999):

(1)

As a result, for deriving fuzzy comparision matrix we created matrix of pair wise comparisons (Eq. 1); where, = (1/uijk, 1/mijk, 1/lijk) and = {lijk, mijk, uijk}, also lijk, and uijk are the lower and the upper bounds Representing the scope of the fuzziness of the fuzzy number and mijk is core of the fuzzy number, corresponding to the maximum degree of membership, equal to one. By using concept of α-level sets or α-cuts (Mikhailov and Singh, 2003), for a given α-level, the interval version of the Fuzzy Preference Programming (FPP) method tries to find a crisp priority vector W = (w1, w2,..., w)T, which satisfies approximately all interval constraints: with solving the weighted GFPP model, the crisp priority vector W and the objective function C which is an indicator that measures the overall consistency of the judgments are obtained:

S.t.

(2)

where, Σni=1 wi>0; i = 1,…,n ; λk>0; k = 1,..., r; q = 1,... 2 m.

When, the interval judgments are consistent, the maximum value of C is greater or equal to one. For inconsistent judgments, C takes a value between one and zero that depends on the degree of inconsistency and the values of the tolerance parameters dq. In addition lk demonstrates the weight of decision makers. Here, we have some pair wise matrices which are enforced by GFPP model.

Fig. 4: GFPP process

Step 3: Super matrix formation: This model has six criteria; hence, for comparing the impact of each pair of elements on the main goal, we used questionnaires which have been filled out by decision makers. Furthermore, for each criterion and its sub criteria, we defined some questionnaires for reflecting the affection of criteria on each criterion. In addition, there are more relations that show the interrelationship between criteria. At last, we considered a super matrix which has been shown in Eq. 3

(3)

where, w12 is a vector that contains comparision of criteria respect to goal, w22 is a vector that represents the inner dependence of the ISO/IEC criteria, w32 is a vector that contains comparision of sub criteria respect to main criteria and w43 is a vector that illistrates comparision of Alternatives to sub crieria.

Step 4: Determine final assessment, at last, final step is to apply the ANP to matrix operations in order to determine the overall priorities of the alternative, Fig. 4 shows this algorithm

CASE STUDY

In this case, we evaluated some E-learning systems by considering both ISO attributes and of E-leaning success factors. In order to address the problem, our Evaluation process contains five steps as follows:

Step 1: Choosing ISO quality model as a basis and adding some interrelations between its criteria that has been mentioned in previous section. Hence, the ISO 9126 model was chosen as a basis
Step 2: Identifying quality characteristics. At first, success factors of E-learning applications are identified through studying E-learning literature. After this, by comparing E-learning success factors with quality factors of ISO 9126 model, the attributes that do not exist in this model are added in the appropriate level in the Network structure of the model
Step 3: We asked from some experts in field of E-learning systems to judge about these factors and assign weights to them
Step 4: Applying GFPP Analytical Network Process on three existing E-learning system in Iran

Step 1 has been widely explained in earlier. In Step 2, for evaluating an E-learning system, many factors must be considered during the assessment process which affect such system (Soheil et al., 2010). In addition, the medium of E-learning is the world wide web and most of E-learning systems transaction does perform on the basis of web. Hence, E-learning quality is related to the quality of the web pages and web services (Behkamal et al., 2009). Moreover, there are several studies in field of determining E-learning success factor (Colace et al., 2006; Chao and Chen, 2009; Tzeng et al., 2005).

Among these researches, Wang (2003) identified 4 main criteria and 12 sub-criteria for them. Also, in 2008 Wang proposed an AHP application based on these factors. Table 2 shows these criteria with their sub criteria.

However, only one of the main criteria in Wang’s model; learner interface; has the same sub criteria in comparison to usability in ISO/IEC 9126 standard (ease of use, user friendly, operational stability and ease of understanding) (Soheil et al., 2010).

Table 2: Wang’s E-learning factors [34]

So, we added other mentioned criteria such as learning community, System content and Personalization of Wang model to ISO software quality model in order to evaluate E-learning systems. At last, our final model consists of 9 main criteria (contains Wang factors’ and ISO criteria) and 29 sub criteria. Secondly, we conducted a survey to found out pair wise comparisons between criteria, sub criteria and alternative using a Standard AHP approach, thus we evaluated an AHP model for selection of the most appropriate E-learning Software. In following Step, we considered existing relationships in ISO model.

In addition, In December 2009 we evaluated three E-learning systems in Iran that have been used in different well-known university, namely, Iran University of Science and Technology (IUST), AmirKabir University of Technology (AUT), Virtual University of shiraz (SVU) which all of them located in Tehran. All of these university provided high standard E-learning systems for student we have captured some information about their performance by conducting some interviews with students and staffs. This process continue for about one month until January 2010.

AT this step, a survey was carried out. Data were gathered from students, the managers of these systems and also some software developers who were involved in developing E-learning systems. To do this, we conduct another pair wise comparison between criteria. Also, for helping the decision makers to decide between pairs of elements and improving their accuracy, we used linguistic variable weight method that is shown in Table 1.

To identify global priorities in E-learning system with its interdependence, we created a super matrix as mentioned in earlier, Eq. 3. Finally, GFPP method has been applied on each comparison matrix and, final weight of each sub criteria and criteria have been shown in Table 3. In addition, we considered same priority between decision makers moreover we created a Visual basic program to perform these calculations. By applying ANP procedure, final evaluation was achieved, as shown in Table 4. Finally, SVU has the best output in our assessment process.

Table 3: Weight of criteria and sub criteria

Table 4: Final evaluation result

CONCLUSION

The need for evaluating Software package introduces the need to use various methodologies and factors for assessment. In this study we propose a MCDM approach by using ANP and GFPP to evaluate Software system. In this approach we considered some interrelations between ISO-IEC 9126 quality model that mentioned in prior researches. There can be other interrelation between criteria that were not considered in this study. Also, for future researches we can consider priorities of decision makers. Also, we assessed three E-learning systems in Iran that the result shows that SVU University is the best E-learning system among others.

REFERENCES
1:  Jadhav, A.S. and R.M. Sonar, 2009. Evaluating and selecting software packages: A review. Inform. Software Technol. J., 51: 555-563.
CrossRef  |  

2:  Arditi, D. and S. Sandeep, 1991. Selection criteria for commercially available software in construction accounting. Int. J. Project Manage., 9: 39-44.
CrossRef  |  

3:  Adhikari, A., M.I. Lebow and H. Zhang, 2004. Firm characteristics and selection of international accounting software. J. Int. Account. Audit. Taxat., 13: 53-69.
CrossRef  |  

4:  Boehm, B. and H. In, 1996. Identifying quality-requirement conflicts. IEEE Software, 13: 25-35.
Direct Link  |  

5:  Behkamal, B., M. Kahani and M.K. Akbari, 2009. Customizing ISO 9126 quality model for evaluation of B2B applications. Inform. Software Technol., 51: 599-609.
CrossRef  |  

6:  Bevan, N., 2001. International standards for HCI and usability. Int. J. Hum. Comput. Stud., 55: 533-552.
Direct Link  |  

7:  Bevan, N., 1999. Quality in use: Meeting user needs for quality. J. Syst. Software, 49: 89-96.
CrossRef  |  

8:  Soong, M.H.B., H.C. Chan, B.C. Chua and K.F. Loh, 2001. Critical success factors for on-line course resources. Comput. Educ., 36: 101-120.
CrossRef  |  Direct Link  |  

9:  Chua, B.B. and L.E. Dyson, 2004. Applying the ISO 9126 model to the evaluation of an e-learning system. Proceedings of the 21st ASCILITE Conference, Dec. 5-8, Perth, Western Australia, pp: 184-190.

10:  Cheng, C.H., K.L. Yang and C.L. Hwang, 1999. Evaluating attack helicopters by AHP based on linguistic variable Weight. Eur. J. Operational Res., 116: 423-435.
Direct Link  |  

11:  Colace, F., M. De Santo and A. Pietrosanto, 2006. Evaluation models for e-learning platform: An AHP approach. Proceedings of the 36th Annual Frontiers in Education Conference, Oct. 27-31, San Diego, CA, pp: 1-6.

12:  Dormey, G.R., 1995. A model for software product quality. IEEE Trans. Software Eng., 21: 146-162.
Direct Link  |  

13:  Shee, D.Y. and Y.S. Wang, 2008. Multi-criteria evaluation of the web-based E-learning system: A methodology based on learner satisfaction and its applications. J. Comput. Educ., 50: 894-905.
CrossRef  |  

14:  Henningsson, K. and C. Wohlin, 2002. Understanding the relations between software quality attributes: A survey approach. Proceedings 12th International Conference for Software Quality, Oct. 02, Ottawa, Canada, pp: 1-13.

15:  ISO/IEC 9126-1, 2001. Software Engineering Product Quality-Part 1: Quality Model. 1st Edn., International Organization for Standardization, USA.

16:  Jung, H.W., 2007. Validating the external quality subcharacteristics of software products according to ISO/IEC 9126. J. Comput. Standards Interfaces, 29: 653-661.
CrossRef  |  

17:  Meade, L. and J. Sarkis, 1998. Strategic analysis of logistics and supply chain management systems using the analytical network process. Trans. Res. Part E: Logist. Trans. Rev., 34: 201-215.
CrossRef  |  

18:  McCall, J.A., P.G. Richards and G.F. Walters, 1977. Factors in Software Quality. Vol. 1, NTIS, Springfield, VA.

19:  Mikhailov, L. and M.G. Singh, 2003. Fuzzy analytic network process and its application to the development of decision support systems. IEEE Trans. Syst. Man Cybernetics-Part C: Appl. Rev., 33: 33-41.
CrossRef  |  

20:  Mulebeke, J.A.M. and Z. Li, 2006. Analytical network process for software selection in product development: A case study. J. Eng. Manage., 23: 337-352.
Direct Link  |  

21:  Ossadnik, W. and O. Lange, 1999. Theory and methodology AHP based evaluation of AHP-software. Eur. J. Operat. Res., 118: 578-588.
CrossRef  |  

22:  Perry, W.E., 1987. Effective Methods for EDI Quality Assurance. Prentice-Hall, New Jersey, USA.

23:  Chao, R.J. and Y.H. Chen, 2009. Evaluation of the criteria and effectiveness of distance E-learning with consistent fuzzy preference relations. Exp. Syst. Appl. Int. J., 36: 10657-10662.
Direct Link  |  

24:  Soheil, S.N., E. Leila and M. Ahmad, 2010. A fuzzy ANP model for evaluating e-learning platform. Proceeding of the 23rd International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems IEA/AIE, June 1-4, Berlin, Heidelberg, pp: 254-263.

25:  Saaty, T.L., 1980. The Analytical Hierarchy Process. RWS Publication, Pittsburgh, PA.

26:  Saaty, T.L., 1996. The Analytic Network Process: Decision Making with Dependence and Feedback. RWS Publications, Pittsburgh, PA.

27:  Suryn, W. and F. Laverdiere, 2007. The applicability analysis of ISO/IEC 9126 series of standards in telecom sector: The example of soft switch category of Software. Software Qual. J., 15: 417-448.
Direct Link  |  

28:  Saaty, T.L., 2003. Decision making in complex environments. Instructions on How to use the Super Decisions Software.

29:  Tzeng, G.H., Y.P. Ou Yang, C.T. Lin and C.B. Chen, 2005. Hierarchical MADM with fuzzy integral for evaluating enterprise intranet web sites. Inform. Sci., 169: 409-426.
CrossRef  |  

30:  Teltumbde, A., 2000. A framework for evaluating ERP projects. Int. J. Prod. Res., 38: 4507-4520.
Direct Link  |  

31:  Wei, C.C., C.F. Chien and M.J.J. Wang, 2005. An AHP-based approach to ERP system selection. Int. J. Prod. Econ., 96: 47-62.
CrossRef  |  Direct Link  |  

32:  Wang, Y.S., 2003. Assessment of learner satisfaction with asynchronous electronic learning systems. Inform. Manage., 41: 75-86.
CrossRef  |  

33:  Yuksel, I. and M. Dagdeviren, 2007. Using the Analytic Network Process (ANP) in a SWOT analysis. A case study for a textile firm. J. Inform. Sci., 177: 3364-3382.
CrossRef  |  

34:  Zulzalil, H., A.A. Abd Ghani, M.H. Selamat and R. Mahmod, 2008. A case study to identify quality attributes relationships for web based applications. Int. J. Comput. Sci. Network Security, 8: 215-220.
Direct Link  |  

35:  Mahdavi, I., H. Fazlollahtabar, A. Heidarzade, N. Mahdavi-Amiri and Y.I. Rooshan, 2008. A heuristic methodology for multi-criteria evaluation of web-based E-learning systems based on user satisfaction. J. Applied Sci., 7: 4603-4609.
CrossRef  |  Direct Link  |  

©  2021 Science Alert. All Rights Reserved