Abstract: QoS evaluation is a process of determining the attributes of the investigated object and quantifying the objective value or subject utility of these attributes. The current research on QoS evaluation is limited to specific domains or specific services. In the research, the QoS evaluation indexes for network services are given firstly. Then a multi-grades evaluation model based on D-S evidence theory to evaluate the running quality of network services is provided, which can achieve variable granularity evaluations at different levels such as users, services and network systems. The mechanism integrates service management, policy management and network management together to provide QoS guaranteed services. A fuzzy quality evaluation method for internet services is designed based on the D-S evidence theory. Examples show that the method can combine the subjective evaluation with objective evaluation and reduce subjectivity during evaluation process.
INTRODUCTION
Network management usually refers to monitoring and controlling the operation of target network to provide effective and reliable services. Today the concept of service management is proposed so as to focus on managing the network services (Danfeng and Fangchun, 2007). The basic purpose of service management is to provide differentiated services and quality guarantee for the Internet application. The New Generation Operation System and Software (NGOSS) is the evidence of service management transformation from network oriented towards user oriented.
Then, the problem of QoS evaluation is the challenge for this purpose. That is, how to evaluate the QoS provided by the service management system? It includes various aspects, such as the evaluation index, evaluation flow and evaluation model. Jun et al. (2006) proposed a novel approach to output-based speed quality evaluation using fuzzy directed graph support vector machine. Ruo-Ying et al. (2004) presented a service evaluation system to support multiple service classification for NGI IP-VPN. Bordbar et al. (2005) proposed an evaluation framework and experiment validations for the service running on the wireless network. Tarmizi et al. (2006) focused on QoS evaluation of different TCP congestion control algorithm using NS2. Current researches on quality evaluation, however, are based on traditional QoS control algorithm and evaluation of network layer. Although some of them are designed to support service level evaluation, they are limited to specific domain or specific service. So far there has been no general service evaluation model and method.
In this research, we first give the QoS evaluation index and general evaluation flow based on the existed work of service evaluation (Jing-Lun and Jun, 2006; Zhenyu et al., 2007) and the characteristic of service management in computer networks (Pavlou, 2006). Then a multi-grades Internet QoS management and evaluation model are proposed, which integrate the network management, policy management and service management so as to provide QoS guaranteed services. Finally, we adopt the fuzzy evaluation concept and design a fuzzy service evaluation method based on the D-S evidence theory. This method can provide variable granularity quality evaluation, like the evaluation of a single service performed by a single user, the evaluation of a service performed by all users or the evaluation of all services used by a single user. The comprehensive evaluation for integrated services can be obtained finally.
QoS EVALUATION INDEX
QoS evaluation is a process of determining the attributes of the investigated object and quantifying the objective value or subject utility of these attributes. The origin of QoS evaluation began with the concept of Customer Perceived QoS (Jing-Lun and Jun, 2006), which is proposed by Christian Gronroos in the 1980s. Besides the subjective evaluation of users, the evaluation of network QoS refers to various objective evaluation indexes. Because users cannot give out the definite expectation of specific QoS metrics, this research adopts the fuzzy matrix to quantify the users` expectation. QoS requirement is another consideration. A well service management policy must satisfy the user`s application demands, such as bandwidth, delay, price and so on. Furthermore, network management is not sufficient to use a single or several QoS metrics to evaluate the network service quality. In a comprehensive evaluation, evaluation from user and system perspective is required because it is a multiple to multiple relationship between users and network services. Figure 1 shows the corresponding relationship between them.
Users with common demands are called a class of users. Service management system is aim to provide them the same service quality. However, users of different classes should get differentiate services according to the class of them. The parameter of Service Difference (SD) in used to describe the difference.
QoS space: Suppose the set of network services is S. The set of QoS parameters is P = P1∪P2. P1 is the set of parameters which take a value more high, the customer is more satisfied, such as bandwidth. Contrarily, P2 is the set of parameters which take a value more low, the customer is more satisfied, such as delay, response time and price. There is a function f: S → P 1(s)∪f2(s), where f(s)⊆P1, f2(s)⊆P2 and f1(s)∩f2(s) = Φ.
User class and QoS mapping: L is the set of user classes. 1x D2…x D|f(s)|. map(l,s) is described to establish the mapping between user class and QoS parameters. |f(s)| is the number of parameters to s. Di is the range of parameter i(1≤i≤|f(s)|). D1x D2x …x D|f(s)| is the cartesian product of all parameters to s. The function defines the range of QoS parameters corresponding to a special user class.
User subjective satisfaction: Suppose the set of users is U. ∀sεS, ∀u,u εU. The user subjective satisfaction of u to s is defined as a fuzzy matrix
rij is the membership degree of parameter i belonging to evaluation grade j, which is given by user u. Service s has p parameters.
| Fig. 1: | Corresponding relationship between users and services |
i is a parameter of s. User` subjective opinion to each parameter is defined as m grades.
Service difference (SD): ∀sεS, ∀u,u εU,
The actual QoS of user u obtained is
Similarly, Promised Service Difference (PSD) is defined as the difference between various uses classes.
THE MULTI-GRADES QUALITY EVALUATION MODEL FOR NETWORK SERVICES
Considering the characteristic of network service, we propose a variable granularity QoS management and evaluation model, which contains SLA negotiation module, QoS guarantee module, QoS evaluation module and physical network module. Figure 2 shows the relationship between these four modules. The labeled digitals in Fig. 2 denote the system flow.
One of the key issues for service management is to provide differentiated service according to the service class of users. The existed QoS evaluation frameworks are IntServ, DiffServ and MPLS proposed by IETF.
| Fig. 2: | Internet QoS management and evaluation model |
However, because of the complexity of protocols and best-effort of network actuality, existing technologies cannot provide the performance guarantee and differentiated services. In addition, most of the network services are provided by dedicated application servers, such as FTP server, Web server and so on. In order to provide the guaranteed quality of service, the application servers should be capable of offering the claimed services and its QoS. Caviglione and Tomaso de Cola (2004) proposed a FTP solution with clients` bandwidth request. Since the self-defined interaction protocol and format of PDU are designed for a specific service, the proposed solution lacks the extensibility. Considering the properties of policy management (Supadulchai and Aagesen, 2007), we propose a framework consisting of service management module, policy management module and network management module to provide the general QoS assurance. Policy based management can control the use of bandwidth with the help of predefined policy on the basis of dynamic factors (such as timing, priority and network status). Its distinct character is to set and implement personalized policy according to clients` requirements, which is coherent with dynamic and adaptable QoS management (Aib and Boutaba, 2007). The service management module can interact with the policy management module according to the users` QoS requirements. In this way, the specific QoS control unit will be activated, like FTP-QoS-Control unit, Email-QoS-Control unit or WWW-QoS-Control unit. It can be considered as the control information residing on specific application servers.
QoS management and evaluation flow: The process flow of service management and quality evaluation can be generalized as follows.
| • | Users and the service provider negotiate and sign the SLA. |
| • | The service manager negotiates with policy server according to the submitted SLA by user. Then, the policy server selects a proper service management policy and the specific service control module will be activated. |
| • | The network manager transforms the high layer service management policy into executable configuration scripts on network devices. |
| • | The network devices submit the specific operation index to network manager. |
| • | The network manager submits the obtained network management index to the service manager. These indexes are tightly coupled with specific network services. |
| • | The service manager module establishes mapping relationship between the specific network management parameters and different network services. Then the QoS evaluation indexes can be obtained, including users` subjective evaluation and the objective evaluation metrics. |
| • | The QoS evaluation module computes the overall service evaluation based on the information from service management. And the computed information is fed back to service management module. |
| • | The policy server adjusts the management policy based on the feedback of QoS or notifies users to re-negotiate SLA with service provider. |
THE FUZZY QUALITY EVALUATION ARITHMETICS BASED ON D-S EVIDENCE THEORY
D-S Evidence theory is also referred to Dempster-Shafer theory or Belief Function theory (Timothy, 2001). The Advantage of D-S theory is that it is based on fuzzy measure. This method is intuitive and easy to understand. During the QoS evaluation processes, there does exist uncertainty and inaccuracy. For example, the customers may use satisfaction or dissatisfaction to evaluate a service instead of accurate parameters. Then we find that QoS evaluation is suitable to use D-S theory for users` subjective evaluation. Similarly, the large granularity evaluation formed from fine granularity evaluation is also suitable to use D-S theory. In view of evaluation granularity, users` evaluation to a specific service is the minimum. If we regard these users as various information source, each user has an evaluation to a special service. Then the comprehensive evaluation by all users can be obtained through D-S theory. In the same way, the services can also be regarded as information source. The acquisition of general QoS evaluation to the particular user is as same as the above mentioned.
User satisfaction to specific service: User u has to assign weights for the attributes of service s as vector E(u,s) = (e1, e2, …, ep). R(u,s) is the fuzzy evaluation matrix. The degree vector of membership which represents user`s subjective evaluation is defined as (β1, β2, …, βm) through E composed with R. The vector (β1, β2, …, βm) is called user satisfaction specifically referred to the evaluation provided by a user to a specific service, namely β(u,s).
Fuzzy evaluation of specific service quality: The quality evaluation of specific service is obtained by fusion of all users satisfaction. Furthermore, the evaluation of Service Difference (SD) and Service Fairness (SF) also needed because there are a lot of consumers using a same service. Maybe they are due to same grades, maybe due to various grades.
Integrated QoS evaluation of specific service: ∀sεS,
Us≤U. u`, u⊂Us. Us is a set of
users that using service s. The subjective satisfaction of u to s is β(u,s)
= (β1, β2, …, βm). The
subjective satisfaction of u` to s is β(u`,s) = (β`1,
β`2, …, β`m). β12(s)
= (β121, β122, …,
β12m) is the satisfaction fused β(u,s)
and β(u`,s) through D-S theory. β12i(s)
= β`i. βi/(1-K) (Timothy, 2001) (1),
Evaluation of service difference: ∀sεS, ∀l,kεL. The Promised Service Difference (PSD) between class l and class k is d(l, k, s) = map(l, s) -map(k, s). The actual service difference between class l and class k is the average value of all uses` Service Difference (SD), namely e(l, k, s). Then the evaluation of service difference is defined as:
(2) |
If its value trends to zero, it shows that the system can provide differentiate service according to the service class of users.
Evaluation of service fairness: ∀sεS. Uls⊆Uls. Uls is the set of consumers using service s and belonging to class l. s. The actual QoS of user u obtained is γ(u, s). Then evaluation of Service Equitableness is defined as
|
(3) |
The more its value closes to zero, the better the fairness is.
General QoS evaluation to a particular user: ∀uεU. Su⊆S. Su is the set of services used by user u. The satisfaction of u to s is β(u, s) = (β1, β2, …, βm). The satisfaction of u to s` is β(u,s`) = (β`1, β`2, …, β`m). Then we can get general QoS evaluation to u through using D-S theory stage by stage, namely β(U, Sall).
The comprehensive evaluation by all users to all services: In the same way, we can get the comprehensive evaluation by all users to all services based on individual evaluation of specific service quality, namely β(Uall, Sall).
| Table 1: | Corresponding relationship between user class and QoS |
EXAMPLES ANALYSIS
The validity and feasibility of these methods are proved though simulation experiment based on OPNET11.5. We design a simple experimental procedure that there are only the two typical services such as FTP and email. Suppose each service has three QoS parameters. There are three user classes, as to normal user, intermediate user and advanced user. Table 1 shows the relationship between user class and QoS parameters according to SLA. User A is a normal user and used email service. User B is an advanced user and used FTP and email services.
Calculation of β(s, u): The user` subjective opinion to service is defined as {excellent, good, bad}. The user satisfaction matrix of user A to email service is
The user satisfaction matrix of user B to FTP service is
The user satisfaction matrix of user B to email service is
The weights of services attributes assigned by users are E(A, 2) = (0.15, 0.67, 0.18), E(B,1) = 0.45, 0.37, 0.18 and E(B, 2) = (0.25, 0.67, 0.08). Then we can obtain the user satisfaction to specific service, respectively.
Calculation of β(Uall, S): For there are only two users, so β(Uall, S) = βAB(s). If the number of user and service increased, several times combinations needs to be done. According to Eq. 1,
βAB(s) = (0.503, 0.391, 0.106). In the same way we can obtain β(U, Sall) and β(Uall, Sall).
Evaluation of service difference between user A and user B: The promised QoS of user A to email service is α(A, 2) = (2, 10, 300). The promissory QoS of user B to email service is α(B, 2) = (10, 200, 3). The actual QoS of user A is γ(A, 2) = (1, 5, 400). The actual QoS of user B is γ(B, 2) = (9, 100, 96). According to Eq. 2, we can get the following result.
Then the Evaluation of Service Difference between user A and user B equal to
The result shows that the service difference is up to the requirement.
Evaluation of service fairness for the same user class: In the above experiment there are no same class uses, so we need not to calculate the evaluation of service fairness.
CONCLUSION AND PROSPECT
A good quality evaluation mechanism can not only simplify network management, but also increase the efficiency and reliability of service management. In this research, a variable granularity and fuzzy quality evaluation mechanism to the network services are given. First of all, policy based management bridges the service evaluation module, service management module and network management module. It is good at managing the global network status and service performance. Also, it is possible to adjust the management policy based on the result of service evaluation so as to provide better service quality. Secondly, the QoS evaluation arithmetic integrates the subjective and objective attributes. It makes the evaluation result more creditable. It is easy to achieve changeable granularity QoS evaluation from user, service to network system based on D-S theory. In the future work, we will study on the optimization and generalization of the service evaluation algorithms.
ACKNOWLEDGMENTS
This research is supported by the National Natural Science Foundation of China (No. 90304006 and No. 60673170).