Abstract: As the top level of management and IS managers spent too much of their budget in IT development, the assessment of WBIS is greatly needed in order to justify the IT investment in terms of its impacts on employee performance. In this concern, it is important to highlight that a very little research has been conducted to assess the two critical factors including interaction design quality and contextual performance. For this purpose, this study is primarily concerned with the development and validation of new multi-dimensional instrument and model for assessing the effectiveness of WBMIS. The effectiveness is assessed based on employee performance: Task performance and contextual performance. A sample of a total of 384 UNRWA respondents was gathered. The results of validity and reliability analysis provide enough confidence to proceed toward validating the developed assessment model. The results indicate that there is a positive significant relationship between information, service, communication quality, user satisfaction and effectiveness. However, the results showed that interface design and system quality have insignificant relationship with satisfaction. The theoretical and practical implications and future point of research are discussed. Finally, the empirically validated instrument and model would be helpful and contribute to the IS empirical studies.
INTRODUCTION
Recently, the Web-based Management Information System (WBMIS) is mainly concerned with supporting management functions including decision making in order to provide an effective management for organizations at different levels.
Concerning the adoption of the WBMIS, many Information System (IS) researchers suggested that the use of WBMIS leads to many important advantages such as the performance increase, improvement of service quality, cost reduction and other related features that are actually gained from the employment of internet technology (Bayo-Moriones and Lera-Lopez, 2007; De Toni and Zanutto, 2006). Furthermore, the deployment of WBMIS in the international organizations such as UNRWA, has a significant influence on the level of organization productivity, effectiveness as well as its image (AlHendawi and Baharudin, 2013; Gichoya, 2005; Panigyrakis and Chatzipanagiotou, 2006; Al-Adaileh, 2009).
Although, the organizations spent too much cost in IT investment, organizations has adopted web based management information system in order to integrate the significant advantages of internet-based systems with organization functions, which leads to significant contributions to the organization effectiveness and performance (Love and Irani, 2004; Wang et al., 2007). Since employee task and contextual performance is considered as one of the strategic objectives for any organization, it is important to assess the impact of WBMIS on the employee performance in order to achieve the organization’s objectives as a whole.
However, based on literature review the IS researchers mentioned that the existing assessment models (D and M92, D and M03) are incomplete to be fit to solve any research problem because the model variables do not cover important fields such as interaction design and the contextual performance of employees (Abugabah et al., 2009; Alsabawy et al., 2011; Gorla et al., 2010; Abugabah and Sanzogni, 2010; Halonen et al., 2010; Hosnavi and Ramezan, 2010; Rahim, 2008; Petter et al., 2008). This means the factors included within D and M models are not enough to identify many problems such as the current research problem and consequently, there is a need for developing a more comprehensive model and thus, there is essentiality to develop an instrument in order to validate this proposed model. Therefore, this study takes the initiative to develop the D and M evaluation models based on the software engineering standards including interaction design (Zhang et al., 2004). Also, it is a leading to assess the effects of quality factors on the contextual or the interpersonal skills of Web users.
To fill the gaps in literature and related empirical studies, this study is primarily concerned with the literature review to conceptualize and explore the relationships among study variables and this is explained in the section two.
TERMINOLOGICAL FOUNDATIONS
This section briefly explains what has been written about the study variable: effectiveness, information quality, system quality, service quality, interaction design quality and user satisfaction. At the end of the present section, the concepts of study’s factors are summarized inside Table 1.
Effectiveness: Concerning effectiveness, Thong and Yap (1996) and DeLone and McLean (1992) suggested that although the effectiveness is a difficult variable to be measured based on an accurate measure, IS effectiveness is considered the ultimate outcome of IS research (i.e., the IS dependent variable). Despite the IS research are not agreed with regard a singular standard definition for IS effectiveness (Hamilton and Chervany, 1981; Delone and McLean, 1992; Thong and Yap, 1996), Hamilton and Chervany (1981) defined the IS effectiveness as the extent to which IS actually adds value towards achieving organizational goals. In this context Grover et al. (1996) also identifies the effective system as a value-added system which positively affects the user behavior. This means it improves user communication, flexibility, productivity and information management. In efforts to conceptualize the IS effectiveness, it is highly important to mention that many researchers highlighted that employee portal as a type WBMIS has a positive impacts on the organizations in several fields such as employee productivity and communication (Tojib et al., 2006). To comprehensively cover the user/employee performance, there is an essentiality to consider the both professional and interpersonal skills.Thus, within the context of this study, the IS effectiveness is conceptualized as the employee performance including task and contextual (interpersonal) performance as an organization objective (Ling, 2003).
User satisfaction: Considering user satisfaction, Torkzadeh and Doll (1999) identified an equivalent term to the user satisfaction called customer satisfaction. They define it from the cognitive perception as the degree to which the system assists users to contribute values to internal and external customers, where many of IS researchers considered the users or the organization’s employees the internal customers (Lai, 2006; Chang and Huang, 2010). Recently, it is found that the IS research (such as Wang et al., 2007; Sugianto and Tojib, 2006; Tojib et al., 2006; Muylle et al., 2004; Delone and Mclean, 2003) have special focus on the satisfaction with E-commerce or web based environment because most of the firms adopt the e-management and web based data processing in their daily work (Panigyrakis and Chatzipanagiotou, 2006).
| Table 1: | Summary of the concepts of study factors |
In the context of web satisfaction and E-commerce, Delone and McLean (2003) considered user satisfaction as one of the most important measures for assessing the effectiveness of the E-Commerce system. Sugianto and Tojib (2006) defined the user satisfaction with portal system as an affective attitude of the users towards the portal system that they interact with directly. With respect to satisfaction with e-learning system, Wang et al. (2007) considered user satisfaction as one of the important measures for assessing e-learning system. Regarding the dimensions of user satisfaction, many researchers (such as, Williams and Anderson, 1991; Locke, 1976) highlighted that satisfaction consists of two main components or dimensions including cognitive component (i.e., behavioral actions or thinking) and affective component (i.e., emotional attitude). With regard to the definition of user satisfaction in this study, the concept of (Lin, 2003) is adapted.
Information quality: Many IS researchers mentioned some concepts regarding the information quality in order to develop its measures in the light of these concepts (Petter et al., 2008; Schaupp et al., 2006; Delone and McLean, 2003; Delone and McLean, 1992). However, IS researchers (for example, Gable et al., 2008; Lin and Lee, 2006; Yang et al., 2005; Sedera et al., 2004; McKinney et al., 2002; Seddon and Kiew, 1994) were mainly concerned with the measures that can assess the quality of information regardless to the concept of information quality. Concerning the concept of information quality, (Gorla et al., 2010; Delone and McLean, 2003, Delone and McLean, 1992) mentioned that information quality is one of the most important measures for examining the overall IS success. Also, Petter et al. (2008) highlighted that information quality is concerned with the system outputs. In this context, Schaupp et al. (2006) defined the information quality as a well-known success factor which is essential to be used when assessing the overall success, especially the context of web based systems. Within the context of this study, the definition of Delone and McLean (2003) is suggested for adaptation as the most appropriate definition.
System quality: The system quality manly focuses on the performance properties of the system. Generally, IS studies such as (Alsabawy et al., 2011; Tella, 2011; Gable et al., 2008) did not consider the conceptual definition of system quality, they primarily focused on the measures of system quality. However, some of these studies such as (Gable et al., 2003; Seddon, 1997) just mentioned that system quality is one of the IS effectiveness/success dimensions. Other research mentioned that the system quality is typically focused on technical success or system performance characteristics (Delone and McLean, 2003; Petter et al., 2008).
Interaction design quality: The WBMIS is designed to browse information as well as organization managerial functions where the applications should be integrated together within Web information system (Daniel and Ward, 2005; Chan and Liu, 2007). Joinson (2003) suggested that the interactivity of the system positively affects the interaction among employees which in turn leads to positive impact upon the satisfaction of users within organization. Therefore, the interaction design could be one of the critical factor in the assessment of IS effectiveness because enabling such a type of interaction through employing the communication tools can enhance and affect the quality of interaction and knowledge extraction/sharing among the organization staff and others (Benbya et al., 2004; Kim et al., 2002). Considering interaction, interactive portal is one of the most commonly terms that could be used to denote the use of web site (Jarvenpaa, 1999). The interaction design has turned into a discipline that has not only concerned with system development, but also with product design and development (Edeholt and Lowgren, 2003). Similarly, Holmlid (2009) pointed out that the interaction design became one of the significant user-centered design disciplines. Based on Cooper et al. (2007), the interaction term is established by Bill Moggridge and Bill Verplankin the mid-1980s. Verplank mentioned that interaction design term is adapted from the user interface design as computer science term into industrial design profession term. While, Cooper et al. (2007) mentioned that interaction design focuses on both form design and behavioral actions. Furthermore, it is explained that interaction design is related to the industrial design of software products (Moggridge, 2007). Albrecht et al. (2005) defined the interactivity on the internet as the degree to which the organizations participate in online exchange with others regardless to the restrictions of time and distance. The study of Lawson-Body and Limayem (2004) mentioned that communication systems and collaboration are considered as interaction dimensions. Accordingly, within the context of this research, the interaction design quality is conceptualized as a two dimensional factor including User Interface Quality (UIQ) and communication quality (ComQ).
Service quality: There is interchangeability in using the terms e-service quality and web service quality (Udo et al., 2010). In addition, Delone and McLean (2003) mentioned that information system quality consists of the service quality as a third critical dimension affecting the success or the effectiveness of e-commerce system which is working via internet environment. Service quality, as defined by Delone and McLean (2003), is the overall support that is provided by service provider including IS department, a new organization unit or outsourcing services providers. While Petter et al. (2008) defined the service quality as the quality of the support provided by IS department and IT support personnel to the users of the system. Regarding the definition of service quality, the concept of Petter et al. (2008) is adapted as it is consistent with the context of the research. Table 1 reveals the summary of the concepts of the study factors towards the development of theoretical model where these definitions are adapted from AlHendawi and Baharudin (2013).
REVIEW OF RELATED WORK AND HYPOTHESES DEVELOPMENT
Based on extensive literature review, the relationships among the study factors including quality factors, user satisfaction and effectiveness of WBMIS are discussed in the following whereas; the symbol (→) denotes the relationship between two variables:
| • | Relationship between system quality and user satisfaction (SQ→US ): Leclercq (2007) found the relationship between system quality and user satisfaction is not sufficient. However, according to literature review, most of IS researchers highlighted that there is a positive significant relationship between SQ and US (such as, McGill and Klobas, 2005; Almutairi and Subramanian, 2005; Wixom and Todd, 2005; Seddon and Kiew, 1996; Seddon and Yip, 1992). So, it is clearly seen from the review that the relationship is already existed in most of studies. To ensure the existence of the relationship between system quality and user satisfaction, the following hypothesis is proposed to be examined: |
| • | H1: The system quality has a positive effect on the user satisfaction with WBIS |
| • | Relationship between Information Quality and User Satisfaction (IQ→US): Concerning the relationship between Information Quality (IQ) and User Satisfaction (US), many researchers (such as, Chiu et al., 2007; Halawi et al., 2007; Kulkarni et al., 2006; Wixom and Todd, 2005; Rai et al., 2002; Seddon and Kiew, 1996) have found a positive relationship between information quality and user satisfaction. Therefore, the positive relationship is considered and thus, the hypothesis assuming the relationship between information quality and user satisfaction is proposed to be tested: |
| • | H2: The information quality has a positive effect on the user satisfaction with WBIS |
| • | Relationship between Service Quality and User Satisfaction (SERQ→US): Many research test the relationship between service quality and user satisfaction and they found the relationship between service quality and user satisfaction was significant. It was found that there was a relationship between service quality in terms of the responsiveness of IS support staff and user satisfaction (Shaw et al., 2002). Yang and Fang (2004) mentioned that there is a significant relationship between online service quality and customer (i.e., users) satisfaction. Also, Leclercq (2007) mentioned that there is relationship between IS support functions and user satisfaction. Similarly, Halawi et al. (2007) found a significant effect for service quality on user satisfaction in a knowledge management system. Thus, within the context of this research, it is important to test the relationship between service quality and user satisfaction: |
| • | H3: The service quality has a positive effect on the user satisfaction with WBIS |
| • | Relationship between Interaction Design Quality and User Satisfaction (IDQ→US): Cooper et al. (2007) highlighted that interaction design is not only concerned with the form design but it also keep highly focus to behavior where behavioral actions are affected by attitude (Davis, 1989; Doll and Torkzadeh, 1991). He also mentioned that interaction design is considered as one of the factors that target the desires and the satisfaction of users. Lawson-Body et al. (2010) highlighted that the level of interactivity has a moderation role in the relationship between some independent factors and customer loyalty (i.e., dependent variable), therefore it affects the user satisfaction which is one of the determinant for effectiveness. Joinson (2003) mentioned that the interaction between employees influences the satisfaction of users within organization. Some IS researchers such as Urbach et al. (2009) find a positive influence for collaboration quality and user satisfaction. Since, the interaction design is conceptualized as a two dimensional variable, there is a need to examine the relationship between the 2 dimensions of interaction design (user interface quality and communication tools quality). Accordingly, within the context of the research, the following two hypotheses (one for each dimension) are proposed: |
| • | H4: The user interface quality has a positive effect on the user satisfaction with WBIS | |
| • | H5: The communication quality has a positive effect on the user satisfaction with WBIS |
| • | Relationship user satisfaction and effectiveness (US→effectiveness): With regard to user satisfaction, it functions the mediating role between the independent quality factors (including information, system, interaction design and decision support) and the dependent variable WBMIS effectiveness. For this reason, it is important to ensure the significance of relationship between US and effectiveness. Based on the literature review, Law and Ngai (2007) found a strong relationship between user satisfaction and organizational performance. The worth mentioning, the majority of IS studies that assessed the relationship between US and the contributions of IS through exploring the effect of user satisfaction on performance (such as, McGill et al., 2003), individual job (such as, Torkzadeh and Doll, 1999), productivity and effectiveness (for example, Halawi et al., 2007; McGill and Klobas, 2005; Rai et al., 2002). They actually found a positive significant relationship between user satisfaction and effectiveness. Since the effectiveness is conceptualized in terms of task and contextual performance, there is a need to test two hypotheses. Accordingly, the following hypotheses are proposed to be examined: |
| • | H6: The user satisfaction has a positive effect on task performance | |
| • | H7: The user satisfaction has a positive effect on contextual performance |
PROPOSED MODEL
Based on literature review, there is a need to develop existing Delone and McLean models in order to fill the incompleteness gap and shortage in the model variables to proceed towards the assessment of the WBMIS effectiveness. To this end, a new mode is developed where includes the user satisfaction as a mediator. This model is developed based on based on D and M03. Figure 1 shows the proposed theoretical model.
Figure 1 illustrates the relationships between the variables of the proposed new model. These variables are the Independent Variables (IVs) including five quality factors, Mediating Variable (MV) which is user satisfaction and the Dependent Variables (DVs) which are task performance and contextual performance. It is obvious that there are: (1) Associations between quality factors and user satisfaction as attitude, (2) Associations between user satisfaction and WBMIS effectiveness. Concerning the WBMIS effectiveness, it is conceptualized and proposed to be assessed in term of employee performance in order to be consistent with the strategic objectives of the development plan in UNRWA. This plan focuses on the development of employee performance in both sides task performance (Professional skills) and contextual performance (interpersonal skills). Regarding the components of user satisfaction, it is divided into two dimensions including overall satisfaction and internal user satisfaction.
| Fig. 1: | Proposed assessment model for measuring WBMIS effectiveness |
However, while most of IS researchers deal with overall satisfaction as a representative of user satisfaction (such as, Urbach et al., 2009; Petter et al., 2008; Tojib et al., 2008; Sugianto et al., 2007; Halawi et al., 2007; Gable et al., 2008). Therefore, this model seems to be more comprehensive in filling literature gap regarding user satisfaction as it deals with the two parts of satisfaction (i.e., affective and cognitive parts). To fill the incompleteness gap of D and M03 assessment model, a 2-dimensional independent factor called interaction design quality is added as one of the quality factors. Second, the dependent variable (i.e., WBMIS) is conceptualized in terms of employee performance. To comprehensively cover the performance, the proposed dependent variable is conceptualized to include not only the task performance but also the contextual performance. However, IS researchers have just defined the performance as the perceived usefulness or task performance only.
RESEARCH METHODOLOGY
This section presents the employed methodology in order to acquire empirical evidence on the relationship between variables identified in previous section. It also outlines generation of scale items, sampling methods, procedures and techniques of data collection, in addition to statistical techniques for data analysis.
Generation of scale items: An extensive literature review was conducted to ensure that a comprehensive list of items was generated to assess the study variables included in the proposed model. Based on this review, the researcher generated an initial scale of seventy one items, where these items are adapted from standard scales. Table 2 reveals the sources of the adapted scale’s items.
Initial instrument: In order to proceed towards the validation of the model, there is an essentiality for developing an instrument (i.e., questionnaire) to include the seventy one items, which are generated based on literature survey. The questionnaire has sex dimensions: Four quality factors as independent variables, user satisfaction as a mediator and effectiveness as a dependent variable. Additionally, the questionnaire was developed using a seven point Likert-type scale ranging from 1 = strongly disagree to 7 = strongly agree. However, this instrument should be tested before the data collection process and therefore, a pilot study was conducted and a total of 39 questionnaires are collected.
Validity and reliability of initial instrument: To ensure validity, the questionnaire of 71 items is sent to 13 academics, system analysts and practitioners who worked in the closely related field. As a result of validity, they recommended to eliminate 15 items because of Redundancy and overlapping between questionnaire dimensions. Second, few items are rephrased to ensure easier understanding for respondents. Considering the criterion validity, the corrected item-to-total correlations are used to delete the garbage items (Cronbach, 1951). In such validity, items with item-to-total correlations less than 0.4 is deleted (Wang et al., 2007). After that the reliability of each dimension is assessed by Cronbach Alph test. As a result of item-to-total correlation and reliability tests, no item was deleted because all items with corrected item-to-total correlations greater than 0.4 and also, the reliability of questionnaire dimensions is mostly greater than 0.7. Table 3 shows the item-total correlation and the cronbach alpha of each dimension.
Based on the reliability analysis, the item to correlation values are greater than 0.6 and also the reliability for each dimension is greater than 0.7. The values indicate that the questionnaire’s items not only have good correlation but also have a considerably high reliability. Accordingly, this instrument has enough validity and reliability to be used in the data collection process.
Sample and procedure: It is important to highlight three important points: One of the research objectives is to generalize the findings, every population element (i.e., employees) has an equal opportunity of being chosen and there is homogeneity among the population elements.
| Table 2: | Adaptation of dependent variable scale |
| Table 3: | Summary of results from item-to-total correlation and reliability test |
| Adapted from Alhendawi and Baharudin (2013) | |
This means that the simple random sampling is the most appropriate approach towards obtaining a representative sample (Sekaran and Roger, 2010; Patten, 2004). The whole population consists of approximately 12000 employees working at Palestine-Gaza. According to Krejcie and Morgan (1970), there is a need to collect 375 questionnaire. Regarding the usable questionnaires, 384 subjects can be considered for the analysis purposes.
GOODNESS OF THE INSTRUMENT MEASURES
Goodness of measures is a procedure for assessing the instrument measures and scales before data analysis (Nunnally and Bernstein, 1994). To ensure goodness of measures, factor analysis, convergent validity and reliability for each scale were applied.
Factor analysis: The factor analysis is mainly used to decide regarding the basic structure of the study variables in the analysis (Hair et al., 2010). Therefore, an exploratory factor analysis was conducted to test the factor structure of the 56-item instrument. The population sample (384 subjects) was tested using principal components factor analysis as the extraction method and varimax as the orthogonal rotation method. According to Hair et al. (2010), to enhance the convergent and discriminant validity, five commonly used conditions are applied: (1) Eigen value greater than or equal to 1, (2) Factor loading of each item is greater than or equal 0.5, otherwise item is deleted, (3) The item with factor loading greater than 0.5 on two or more factors also it is deleted, (4) A simple factor structure and (5) Exclusion of single item factors from the standpoint of parsimony. Based on the proposed theoretical model, the study factors are actually nine. Five independent variables: System quality, information quality, service quality, interaction design (2-dimensional: user interface, communication tools). Two variables included with in user satisfaction as a mediator (overall satisfaction and internal user satisfaction), in addition to two dependent variable (Task performance and contextual performance). Table 4, 5 and 6 reveals the results of factor analysis.
Based on the results of factor analysis shown in Tables 4, 5 and 6, it is clearly seen that (1) without determining the number of factors, there are 8 factors with eigen values greater than 1 as follows: 5 independent factors, one mediator factor and two dependent factors, (2) There is a need to merge the two dimensions of user satisfaction into one dimension, (3) The total of 10 items: c5, c7, d1, d6, f9, f10, a7, b1, b2, b3 should be eliminated before analysis because the factor loading of these items does not meet the commonly assumed conditions, (4) One item named f5 is shifted from communication quality factor to information quality factor, whereas; this item seems to be more closely to information quality than user interface components because the item-to-total correlations are greater than 0.6 (Wang et al., 2007) and (5) The interaction design quality is classified into two factors which is totally compatible with conceptualization in previous section.
| Table 4: | Results of factor analysis for independent variables |
| **Total variance explained = 64.98% | |
To conclude, the final structure of the study instrument has 8 components with 47 items. Therefore, the theoretical model should be restructured in order to obtain the final theoretical model’s structure.
Based on Fig. 2, generally the framework of the study remained unmodified while the only change is regarding user satisfaction. The proposed two dimensions of user satisfaction (Dim1, Dim2) should be merged together in order to be consistent with the results of factor analysis.
| Table 5: | Results of factor analysis for mediation variable |
| **Total variance explained = 63.506% | |
| Table 6: | Results of factor analysis for dependent variables |
| **Total variance explained = 66.16% | |
The interaction design factor remains as it is proposed in Fig. 1. For clarity, the two dimensions including interaction design and employee performance are separated because each dimension is practically considered as a separate factor.
Convergent validity: After identifying the factor structure of the instrument scale using factor analysis, convergent validity is used to assess the validity of the generated structure. Regarding the convergent validity, it checks whether the correlations among the factor’s measures differ from zero or not. To this end, Pearson correlation matrix is employed to investigate the correlations between measures of each factor; whereas the correlations are as follows: Task performance = 0.493; contextual performance = 0.434; system quality = 0.288; information quality = 0.496; service quality =0.404; interaction design quality = 0.164 and user satisfaction = 0.225. Since these correlations are significantly higher than zero and (sig. value<0.01) and also the AVEs values are greater than 0.5 (Hair et al., 2010), then these scales have enough convergent validity to proceed towards the Reliability analysis (Table 7) .
Reliability analysis: Using this sample of 384 respondents, the reliability (alpha) for the instrument scales: Task performance, contextual performance, system quality, information quality, service quality, interaction quality (user interface quality, communication quality) and user satisfaction are : 0.930, 0.905, 0.859, 0.850, 0.885, 0.777, 0.820 and 0.917, respectively. In addition, the corrected item-to-total correlations for all items are approximately greater than or equal to 0.6 (Wang et al., 2007), which in turn give an additional evidence for the validity of instrument scales. Table 7 shows the AVEs values and the reliability of all study scales.
| Fig. 2: | Final assessment model after factor analysis |
| Table 7: | AVEs values and Cronbach’s alpha of study’s scales |
TESTING HYPOTHESES
After validating the scales and conducting goodness of measures, the hypotheses of this study should be checked in order to justify the relationships among the theoretical model variables. Practically, the multiple regression analysis is employed in order to check the significance of quality factors as predictors for user satisfaction (i.e., there is a need for are 5 hypotheses), as well as it checks whether the user satisfaction a predictor for task performance and contextual performance (i.e., there are 2 hypotheses). Table 8 demonstrates the results of multiple regression analysis of the relationships between quality factors and user satisfaction.
Regarding the results of regression analysis of user satisfaction on quality factors, Table 8 demonstrates the degree to which the independent variables contributed to the dependent variable (i.e., user satisfaction). It presents the â-values which indicate how much each independent variable contributes on predicting dependent variable. It is clearly seen that information, service and communication quality have a positive significant relationship with user satisfaction at β-values (0.360, 0.325 and 0.165, respectively) and high significance level (sig.<0.01). In contrast the relationship between system quality and interface quality are insignificantly related with user satisfaction. Moreover, the multiple regression analysis showed that the above five quality factors predict a percentage of 62% of total variance explained in user satisfaction. More details regarding the multiple regression analysis are shown in appendix A.
Concerning the regression analysis of effectiveness (task and contextual performance) on user satisfaction, Table 9 shows the result of the regression analysis.
Based on Table 9, the user satisfaction is significantly predicts the effectiveness including task and contextual performance. The contribution of user satisfaction to contextual performance is higher than that of the task performance. Moreover, the relationships between user satisfaction, task performance and contextual performance are highly significant at (β-value = 0.652, sig. = 0.000) and (β-value = 0.708, sig. = 0.000), respectively.
| Table 8: | Regression analysis of user satisfaction on quality factors |
| Table 9: | Regression analysis of effectiveness on user satisfaction |
| Table 10: | The summary of hypotheses testing |
| *Means not supported | |
Regarding the total variance explained, user satisfaction predicts 42.5 and 50.2% of variance in task performance and contextual performance, respectively.
Accordingly, the hypotheses (H2, H3, H5, H6, H7) are supported while the remaining hypotheses (H1, H4) are not supported and the results of hypotheses testing are explained in Table 10.
DISCUSSION
As this study aims at developing a new model for assessing WBMIS effectiveness, it is mainly concerned with the development of a multidimensional instrument as well as exploring the significance of the proposed theoretical model.
The findings of the factor analysis indicate that 8 dimensions (system quality, information quality, service quality, interface quality, communication quality, user satisfaction, task performance and contextual performance) have a sense in assessing the WBIS effectiveness. Also, the factor analysis emphasized the uni-dimensionality of user satisfaction as the cognitive and affective parts are found considerably related. Moreover, the results are consistent with conceptualization of interaction design quality as the factor analysis shows that there is essentiality to separate the variable into two factors (i.e., two dimensions).
The results of multiple regression analysis revealed that the quality factors predict a percent age of 62% of total variance in the user satisfaction which means that the proposed quality factors have a considerable contribution to user satisfaction. Further, the three quality factors (information quality, service quality and communication quality) have a significant positive relationship with user satisfaction. This is in line with most of previous studies such as (Lin, 2010; Petter and McLean, 2009; Petter et al., 2008; Wang et al., 2007) These significant relationships indicate that information content, IT support and the availability of communication and feedback tools have a positive effect on the user perception towards increasing their satisfaction. However, two quality factors including system quality and interface design are insignificantly related to user satisfaction which is consistent with some previous studies such as (Liu et al, 2009; Wu and Wang, 2006). The insignificance of these relationships showed that system appearance, graphic and technical characteristics of system use have no effect on the user perception as these characteristic may have no important role in developing the professional or interpersonal skills.
It is worth mentioning that the findings of this study emphasized the role of user satisfaction as a predictor of employee performance in addition to the contextual performance; whereas there is no published study discussed the contextual performance as a dependent variable in the field of information management and quality. As another significant issue, it can be concluded that the findings are in agreement with the theoretical findings of (AlHendawi and Baharudin, 2013).
Finally, as the model successfully applied on UNRWA, it would be adoptable and it is suggested to assess the proposed model in different organization or context.
IMPLICATIONS
The developed model and instrument reveal how the quality factors influence the effectiveness of WBMIS which in turn leads to the following advantages: (1) As the study focuses on the assessment of effectiveness, it should be in the highest priority and large interest for the top management level in the organizations; (2) It would assist specialists, practitioners and decision makers to assess how much the system improves the contextual and task performance; (3) It helps to examine the level of user satisfaction with the system which has a considerably effect on the level of employee performance. (4) It would stimulate the IS empirical research for how systems are developed and assessed, (5) It would have a positive impact on the cost management and human resources investment and (6) It would assist in evaluating and improving the behavior and attitudes of system users and therefore, the user loyalty and initiatives are increased. Accordingly, So, the research could have a positive impact on the organization effectiveness as a whole.
The findings of the study which has a sample of 384 respondents indicate that the adapted instrument has sufficient validity and reliability to be considered in any related future research. Also, the findings provide adequate confidence to the researchers to conclude that the eight dimensions make sense in terms of three parties, namely, quality factors, user satisfaction and user performance. The results emphasize that users’ responses can be considered as a way of evaluating the effects of web-based information systems.
Appendix A: The multiple regression analysis among study variables
| 1. Regression of quality factors on user satisfaction | |
| aPredictors: (Constant), ComQ_Mean, UIQ_Mean, IQ_Mean, SERQ_Mean, SQ_Mean. bDependent variable: Sat_Mean | |
| 2. Regression analysis of user satisfaction on contextual performance |
| aPredictors: (Constant), Sat_Mean. bDependent variable: CP_Mean |
| 3. Regression analysis of user satisfaction on task performance |
| aPredictors: (Constant), Sat_Mean. bDependent variable: TP_Mean |
Based on these responses, the analysis and findings indicate that there are highly significant relationships between the three parties of the theoretical model: quality factors, user satisfaction and effectiveness of the system.
Since, the proposed model is successfully applied on one of the biggest international organizations such as UNRWA, it is expected that this model would help in the developing WBMIS towards enhancing the employee’s performance, practices and perception.
As UNRWA plays a semi governmental role in providing the basic and critical services such as education, health and relief, it seems that this study would contribute to the governmental and public sectors organizations. Additionally, it would contribute the empirical studies in the fields of IS, human computer interaction and user behavior.
CONCLUSION
This study is mainly concerned with conceptualizing and developing a new assessment model for evaluating the effectiveness of WBMIS in the international organizations such as UNRWA. The present research is considered as one of the fewest focusing on the extent to which quality factors: System quality, information quality, service quality, user interface quality and communication quality contribute to employee performance with a key role of user satisfaction. Assessing the impacts of quality factors brings the need for developing a multidimensional instrument and consequently, a new instrument has developed and adequately examined through testing its validity, reliability and factor analysis. The sample of this study is a large scale sample including the responses of 384 UNRWA employees working at different positions and departments. As a result of data analysis, the developed instrument has enough validity and reliability to be used for measuring the study variables. Additionally, it is found that the quality factors: Information quality, service quality and communication quality have significant relationships with user satisfaction and also the user satisfaction is significantly related to the effectiveness or employee performance (sig.<0.01). However, it is also found that the system quality and interface quality have no significant effect on the user satisfaction. Therefore, the proposed model is adoptable, promising and it would contribute to the empirical studies in the fields of IS development and user behavior.
ACKNOWLEDGMENTS
This research has been supported by partially from the Short Term Research Grant No: 304/PKOMP/6311028 and RUI Grant No. 1001/PKOMP/811251 from the Universiti Sains Malaysia. Any opinions, findings and conclusions or recommendations expressed in this material are those of authors and do not necessarily reflect the views of UNRWA.