End-to-End Baseline File Transfer Performance Testbed

RESULTS

Results are based from the performance metrics which are throughput and datagram loss. MRTG is the main tool used to measure the throughput for the baseline performance test while Jperf was used for UDP datagram loss test.

Table 2:	Throughput for IPv4 and Ipv6

Throughput: There are not much difference between IPv4 and IPv6 maximum and average throughput for baseline test scenario. The maximum throughput cannot reach 100% due to Transmission Control Protocol (TCP) overhead during file transfer process.

Table 2 shows the detail throughput result for IPv4 and IPv6 performance test. The table clearly reveals that smaller file size will result in lower maximum and average throughput. This is because there is a lot of small size files required to be transferred in the 8 h window test period.

Fig. 4:	Detail real-time throughput graph captured for small size file (a) 100 MB and (b) 1000 MB

Table 3:	Jitter and datagram loss corresponding to bandwidth size for both IPv4 and IPv6 UDP test

The result is lower when the tool collects the throughput data at the beginning and the end of each file transfer session for each small files.

Downward spikes in Fig. 4a and b occur at the beginning and end session of file transfer process and it occur more often for smaller size file. From Fig. 4a, we can see that 8 sec are required to transfer a 100 MB file via 100 Mbps link, while from Fig. 4b shows that it took around 1 min 30 sec to transfer 1000 MB file.

Datagram loss: Datagram loss was measured by using Jperf UDP test. Less than 0.3% packet loss was achieved for this baseline test because the test was carried out in controlled environment, the configuration was done properly, a factory crimped network cable was used and there is no other unwanted application running in the background process during the test on both hosts. Datagram loss for IPv6 is higher than IPv4, but the overall maximum datagram loss for IPv6 still below 0.3% which is very low. The minimum percentage datagram loss for IPv6 test is 0.062% at 50 Mbps UDP bandwidth, while the minimum percentage datagram loss for IPv4 is 0.000% at 30 Mbps UDP bandwidth.

Table 3 shows all jitters or delay variations for IPv4 are 0.000 msec and for IPv6 the jitters are little bit higher than IPv4 which are 0.073 and 0.043 m sec at 20 and 40 Mbps UDP bandwidth. Meanwhile the average difference between IPv4 and IPv6 datagram loss is 0.155%.

DISCUSSION

Theoretically IPv4 has less advantage as compared to IPv6 (Sailan et al., 2009). Through product information gathering process, there is inconsistency between theoretical IPv6 performances against actual IPv6 performance. For example, a dual stack IPv4 and IPv6 capable router have higher IPv4 packet forwarding performance as compared to IPv6 (Cisco Systems Inc., 2010b). This situation is the main reason to carry out this study and another three scenarios test (testbed with network switch, router and live network experiment) to compare the actual network performance for native IPv4 and IPv6 address configuration. 3Com Multi Services Router was chosen to test out for the next study because it has the highest score from IPv6 support product comparison done before (Sailan et al., 2009).

Throughput is one of the performance metrics proposed for the three test scenarios. Proposed tool to measure the throughput is Open Source based tool which is MRTG. It is because MRTG is one of the network management tools widely accepted by researchers and Information Technology (IT) professionals. Another Open Source based throughput measurement tool proposed is Jperf. The second metric is datagram loss. Datagram loss is also an important metric to be measured in network performance monitoring and analysis. High datagram loss will definitely degrade the network performance. Datagram loss can be affected by bad or broken network cable, duplex mismatch and signal degradation.

Other related study done by Yunos et al. (2010) used a virtual machine to simulate the test scenario and benchmark performance of both IPv4 and IPv6. The test was carried out for only 100 sec for every test scenario. The study revealed that IPv6 performance is better than IPv4 in most of the test except in TCP transfer, but the improvement is not much. The authors also revealed that performance in packet transfer for TCP transfer, UDP transfer and bandwidth was improved for Multi Protocol Label Switching (MPLS) tunnel test. In the end the authors concluded that migration from IPv4 to IPv6 will not bring big impact on the network performance itself.

Another study related to file transfer was done by Xiuduan and Veeraraghavan (2009). The study focused on how to select appropriate values for operational parameters in hybrid architecture files transfer. Formal method was used to construct the model. The authors revealed that admission-control algorithm could be used to optimize the file transfer performance in hybrid network.

A study on aggressive block reordering for large file transfers was done last year by Anastasiadis et al. (2009). The study focused on how aggressive block reordering can be used as a technique to optimize the file cache system of content or file sharing server. An algorithm was constructed and evaluated by using prototype with FreeBSD environment. The authors concluded that block reordering can improve server cache performance, doubles the server network throughput and reduces the required disk bandwidth.

A comparison study of VoIP performance on IPv6 and IPv4 networks was done by Yasinovskyy et al. (2009). The authors used softphone and bare PCS to do the experiment. The result showed that VoIP performance varies when the same softphone running on different operating systems, but there is no significant performance difference between IPv4 and IPv6 test.

There is also another related study by Kolahi et al. (2009) on performance analysis of IPv4 and IPv6 on Windows Vista and Windows XP over fast ethernet in peer-peer LAN. The metrics chosen for this study are throughput and Round Trip Time (RTT), while the data size used for the test are range from 128 to 1408 bytes. Results from the test showed that throughput is higher on Wondows Vista compared to Windows XP and there is also no significant difference on RTT and throughput regardless of IP version.

All of the test as mentioned above are related to how to improve the network performance and used throughput as the main metric. The difference is the method used to do the test. Data or file size used for the test is smaller compared to this study. The time taken to do the test for each file size are 8 h and the results for UDP datagram loss was given of up to 3 decimal point for this study. This baseline test setup was designed to get accurate comparative result of End-to-End (E2E) of IPv4 an IPv6 network performance under controlled network environment. Metrics and tools configuration for the test setup also determine the accuracy of the measurement process. For example, network throughput will be measured by using network management tool MRTG for 5 min throughput collection interval, while Windows network monitoring graph was used for 1 sec throughput interval monitoring. Every file transferring process for each file size was tested for 8 h in order to get accurate average and maximum throughput result. If the file transfers process was done for one time (non-repetitive), MRTG might not be able to produce the throughput graph, because non repetitive transfer for 100 MB file over 100 Mbps link will take less than 2 min, while MRTG was configured to collect data for every 5 min. Five minutes instead of 1 min data collection interval was configured due to reduce processing load of both hosts, reduce SNMP packet, error, datagram loss and hence to increase accuracy level of the test. Jperf was used during the UDP datagram test with maximum UDP bandwidth of 50 Mbps to imitate actual live network bandwidth which normally monopolized by TCP traffic. Jperf and device manager tools were used to monitor and ensure the packet loss will be kept to low level or less than 0.3% so that the result for the test is accurate.

CONCLUSION

There are a lot IPv6 advantages which should result in better throughput performance as compared to IPv4. This study pointed out the inconsistency between theoretical and actual IPv6 performance, principal used to design the accurate network performance test scenarios, accurate metrics and measurement tools and less than 0.3% level of datagram loss to make sure accurate measurement result were proposed. We use Windows Vista operating system and use open source tools such as MRTG, Jperf, Filezilla for FTP and Wamp for web server. Result from the test shows that, there is no significant difference in file transfer performance between IPv4 and IPv6 for baseline test. Small size test file will result in lower maximum and average throughput of both IPv4 and IPv6. During the tools evaluation process it was discovered that average throughput is higher when the tool reach steady state condition. Finally, it was found out that the actual IPv4 and IPv6 maximum throughput for 100 Mbps link will not reach the maximum 100 Mbps due to TCP overhead during file transfer process.

As for future research direction, this study had embarked on second and third test scenario which involved a testbed with network switch and router for IPv4 and IPv6 end to end network performance test. Once all data from tested scenarios have been collected and analyzed, detailed characteristic will be applied in the next simulation process. After that, simulation model will be designed based on detail information gathered from tested scenarios. Then, simulation result will be compared with tested scenarios’ result. Lastly accurate model and simulation process will be used for network extrapolation or network performance prediction in a large scale implementation.

ACKNOWLEDGMENTS

This research is funded by Universiti Kebangsaan Malaysia and Public Service Department of Malaysia. The research group is known as Network Management Group of Computer Science Department, Universiti Kebangsaan Malaysia. Please visit our website at http://www.ftsm.ukm.my/network for further detail.