• 6
name

A PHP Error was encountered

Severity: Notice

Message: Undefined index: userid

Filename: views/question.php

Line Number: 191

Backtrace:

File: /home/prodcxja/public_html/questions/application/views/question.php
Line: 191
Function: _error_handler

File: /home/prodcxja/public_html/questions/application/controllers/Questions.php
Line: 433
Function: view

File: /home/prodcxja/public_html/questions/index.php
Line: 315
Function: require_once

Update: Should I only set jumbo frame to server and file server, not client?

If so, is there any impact on communication between server and client?


I am running some performance test for our product.

Currently all the testing related machines(servers, file servers, clients, db) are on a 10G network connected by a powerful Dell OpenManage Switch.

We are using iscsi for the file server. We have a cluster server that contains several nodes.

The performance test I am running basically is to simulate the following scenario: 1. client machine will create a large number of threads to send http request to the server. 2. Based on the different type of requests, server needs to get some data from file server which is shared by all the other server nodes.

The test results is: Without jumbo frames, MTU 1500, server CPU 70%, and avg response time for the http request is 1 second.

With jumbo frames, MTU 9000, server CPU 20%, and avg response time for the http request is 5 seconds.

We have configured jumbo frames on all machines, and changed TCP settings.

Any ideas?

      • 2
    • But if I set client to 1500, and server to 9000, then server will still send 9000 to the client. Will that be a problem?
      • 1
    • Only turn on jumbo frames for the iSCSI interfaces. You have dedicated interfaces for that, correct?
    • Jumbo frames should only be used for hosts accessing the iSCSI storage device, not from a client to the server fronting said storage.

Well:

  • Bigger frames = more data on each package = your CPU works less to send data (it has a smaller number of packages per second), but takes longer to assemble each payload (more latency).
  • Smaller frames = less data on each package = your cpu works more to send data (more packages per second), but takes less time to assemble each payload (less latency).
  • 3
Reply Report
    • Add to that apossible windows firewall.... can have significant impact on CPU usage... per packet.
      • 2
    • And you need to add CPU load on the switches as well. Also less interupts need to be serviced for a the volume of data - I guess that really comes down to less CPU required to do the work though, depending on how you look at it.
    • Just imagine that a lot of data must be put together to assemble a Jumbo package before it leaving the interface.
      • 2
    • But if you think one level upper, for application level, they still need to get a complete http request. Maybe one request can be encapsulated into one 9000 frame, and into several 1500 frams, but the total time should be the same from sending a request to receiving the request.

I have been trying to read up and understand more about the impact of utilising Jumbo frames, and why it still hasn't become mainstream after more than a decade. This paper hints on the real world problems faced by Jumbo frame sizes, preventing it from achieving more than the bulk-file-transfer scenario.

http://www.chelsio.com/jumbo_enet_frames.html

Summary of issues contributing to high-latency delays

  1. Delayed pipelining across transmission mediums
  2. Small transmit/receive buffers causing dropped packets = retransmit
  3. Larger packet size = higher chance of collision = retransmit
  4. Lower CRC quality at greater payload lengths = corrupt packets = retransmit
  5. End-to-end path MTU discovery, both ways
  • 1
Reply Report

Your "server" should have one interface for iSCSI (this interface is the disk bus!), and one interface for clients to access the server on. The iSCSI interface should be MTU9000, the client access interface should be have a default MTU. Also, your switch needs to be configured to support both jumbo frames and flow control, or performance will suck.

  • 0
Reply Report
      • 2
    • This isn't to say jumbo frames may not help your clients as well, but right now your disk bus is competing for bandwidth with the client communication. You may end up using jumbo frames on both NICs, but you definitely need two.
    • It is confusing. You said one interface for iscsi, which is a disk bus, and this interface should be MTU9000. Two questions: 1) If I understand correctly, the iscsi interface should connect to the NIC. Why it is a disk card? 2) How to set MTU9000 to a disk card?
      • 1
    • iSCSI is block level storage over TCP/IP. The users do not talk to the iSCSI storage device the server does, they talk to your server, which in turn talks to the device. Would you run really long SATA cables from your server to every desktop? NO.
      • 2
    • But the server has one NIC. It need to use this NIC to talk to file server and client. I can only set one MTU value for this NIC
      • 2
    • Then I suggest you add another one if this is meant to be a permanent solution and if you want ideal performance.

Trending Tags